Two new Hager–Zhang iterative schemes with improved parameter choices for monotone nonlinear systems and their applications in compressed sensing

Open Access

Issue		RAIRO-Oper. Res. Volume 56, Number 1, January-February 2022


Page(s)		239 - 273
DOI		https://doi.org/10.1051/ro/2021190
Published online		07 February 2022

H. Abdullahi, A.S. Halilu and M.Y. Waziri, A modified conjugate gradient method via a double direction approach for solving large-scale symmetric nonlinear equations. J. Numer. Math. Stoch. 10 (2018) 32–44. [MathSciNet] [Google Scholar]
A.B. Abubakar, P. Kumam, H. Mohammad, A.M. Awwal and K. Sitthithakerngkiet, A modified Fletcher-Reeves conjugate gradient method for monotone nonlinear equations with some applications. Mathematics 7 (2019) 745. [CrossRef] [Google Scholar]
N. Andrei, Open problems in conjugate gradient algorithms for unconstrained optimization. Bull. Malays. Math. Sci. Soc. 34 (2011) 319–330. [MathSciNet] [Google Scholar]
N. Andrei, Accelerated adaptive Perry conjugate gradient algorithms based on the self-scaling BFGS update. J. Comput. Appl. Math. 325 (2017) 149–164. [CrossRef] [MathSciNet] [Google Scholar]
N. Andrei, A note on memory-less SR1 and memory-less BFGS methods for large-scale unconstrained optimization. Numer. Algor. (2021). DOI: 10.1007/s11075-021-01185-8. [Google Scholar]
M.R. Arazm, S. Babaie-Kafaki and R. Ghanbari, An extended Dai-Liao conjugate gradient method with global convergence for nonconvex functions. Glasnik Matematic 52 (2017) 361–375. [CrossRef] [Google Scholar]
S. Babaie-Kafaki and R. Ghanbari, A descent family of Dai-Liao conjugate gradient methods. Optim. Methods Softw. 29 (2013) 583–591. [Google Scholar]
S. Babaie-Kafaki and R. Ghanbari, The Dai-Liao nonlinear conjugate gradient method with optimal parameter choices. Eur. J. Oper. Res. 234 (2014) 625–630. [Google Scholar]
S. Babaie-Kafaki and R. Ghanbari, Two optimal Dai-Liao conjugate gradient methods. Optimization 64 (2015) 2277–2287. [Google Scholar]
S. Babaie-Kafaki, R. Ghanbari and N. Mahdavi-Amiri, Two new conjugate gradient methods based on modified secant equations. J. Comput. Appl. Math. 234 (2010) 1374–1386. [MathSciNet] [Google Scholar]
A. Beck and M. Teboulle, A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imag. Sci. 2 (2009) 183–202. [CrossRef] [Google Scholar]
C.G. Broyden, A class of methods for solving nonlinear simultaneous equations. Math. Comput. 19 (1965) 577–593. [CrossRef] [Google Scholar]
W. Cheng, A PRP type method for systems of monotone equations. Math. Comput Modelling 50 (2009) 15–20. [CrossRef] [MathSciNet] [Google Scholar]
Y.H. Dai and L.Z. Liao, New conjugacy conditions and related nonlinear conjugate gradient methods. Appl. Math. Optim. 43 (2001) 87–101. [Google Scholar]
Y.H. Dai and Y.X. Yuan, Nonlinear Conjugate Gradient Methods. Shanghai Scientific and Technical Publishers, Shanghai (2000). [Google Scholar]
Z. Dai, X. Chen and F. Wen, A modified Perry’s conjugate gradient method-based derivative-free method for solving large-scale nonlinear monotone equation. Appl. Math. Comput. 270 (2015) 378–386. [MathSciNet] [Google Scholar]
M.K. Dauda, M. Mamat, M.A. Mohamed and M.Y. Waziri, Improved quasi-Newton method via SR1 update for solving symmetric systems of nonlinear equations. Malay. J. Fund. Appl. Sci. 15 (2019) 117–120. [CrossRef] [Google Scholar]
Y. Ding, Y. Xiao and J. Li, A class of conjugate gradient methods for convex constrained monotone equations. Optimization 66 (2017) 2309–2328. [Google Scholar]
E.D. Dolan and J.J. Moré, Benchmarking optimization software with performance profiles. Math. Program. 91 (2002) 201–213. [Google Scholar]
M. Elad, Sparse and Redundant Representations: From Theory to Applications in Signal and Image Processing. Springer Science and Business Media, LCC, Berlin (2010). [CrossRef] [Google Scholar]
M. Fatemi, An optimal parameter for Dai-Liao family of conjugate gradient methods. J. Optim. Theory Appl. 169 (2016) 587–605. [CrossRef] [MathSciNet] [Google Scholar]
J.A. Fessler, Model-based image reconstruction for MRI. IEEE Signal Process. Mag. 27 (2010) 81–89. [CrossRef] [PubMed] [Google Scholar]
M.A.T. Figueiredo and R. Nowak, An EM algorithm for wavelet-based image restoration. IEEE Trans. Image Process. 12 (2003) 906–916. [CrossRef] [MathSciNet] [PubMed] [Google Scholar]
M. Figueiredo, R. Nowak and S.J. Wright, Gradient projection for sparse reconstruction, application to compressed sensing and other inverse problems. In: IEEE J-STSP. IEEE Press, Piscataway, NJ (2007) 586–597. [Google Scholar]
J.A. Ford and I.A. Moghrabi, Multi-step quasi-Newton methods for optimization. J. Comput. Appl. Math. 50 (1994) 305–323. [MathSciNet] [Google Scholar]
J.A. Ford, Y. Narushima and H. Yabe, Multi-step nonlinear conjugate gradient methods for unconstrained minimization. Comput. Optim. Appl. 40 (2008) 191–216. [CrossRef] [MathSciNet] [Google Scholar]
L. Grippo, F. Lampariello and S. Lucidi, A nonmonotone line search technique for Newton’s method. SIAM J. Numer. Anal. 23 (1986) 707–716. [Google Scholar]
W.W. Hager and H. Zhang, A new conjugate gradient method with guaranteed descent and an efficient line search. SIAM J. Optim. 16 (2005) 170–192. [Google Scholar]
W.W. Hager and H. Zhang, A survey of nonlinear conjugate gradient methods. Pac. J. Optim. 2 (2006) 35–58. [MathSciNet] [Google Scholar]
W.W. Hager and H. Zhang, Algorithm 851: CG_Descent, a conjugate gradient method with guaranteed descent. ACM Trans. Math. Softw. 32 (2006) 113–137. [Google Scholar]
E.T. Hale, W. Yin and Y. Zhang, A fixed-point continuation method for l₁ regularized minimization with applications to compressed sensing. SIAM J. Optim. 19 (2008) 1107–1130. [CrossRef] [MathSciNet] [Google Scholar]
A.S. Halilu and M.Y. Waziri, An improved derivative-free method via double direction approach for solving systems of nonlinear equations. J. Ram. Mathl. Soc. 33 (2018) 75–89. [Google Scholar]
A.S. Halilu, A. Majumder, M.Y. Waziri and H. Abdullahi, Double direction and step length method for solving system of nonlinear equations. Eur. J. Mol. Clinl. Med. 7 (2020) 3899–3913. [Google Scholar]
A.S. Halilu, A. Majumder, M.Y. Waziri, A.M. Awwal and K. Ahmed, On solving double direction methods for convex constrained monotone nonlinear equations with image restoration. Comput. Appl. Math. 40 (2021) 1–27. [CrossRef] [Google Scholar]
A.S. Halilu, A. Majumder, M.Y. Waziri, K. Ahmed and A.M. Awwal, Motion control of the two joint planar robotic manipulators through accelerated Dai-Liao method for solving system of nonlinear equatiions. Eng. Comput. (2021). DOI: 10.1108/EC-06-2021-0317. [Google Scholar]
A.S. Halilu, A. Majumder, M.Y. Waziri and K. Ahmed, Signal recovery with convex constrained nonlinear monotone equations through conjugate gradient hybrid approach. Math. Comp. Simul. 187 (2021) 520–539. [CrossRef] [Google Scholar]
M.R. Hestenes and E.L. Stiefel, Methods of conjugate gradients for solving linear systems. J. Res. Nat. Bur. Stand. 49 (1952) 409–436. [CrossRef] [Google Scholar]
G.A. Hively, On a class of nonlinear integral equations arising in transport theory. SIAM J. Math. Anal. 9 (1978) 787–792. [Google Scholar]
Y. Hu and Y. Wang, An efficient projected gradient method for convex constrained monotone equations with applications in compressive sensing. J. Appl. Math. Phys. 8 (2020) 983–998. [CrossRef] [Google Scholar]
Z. Khoshgam and A. Ashrafi, A new modidified scaled conjugate gradient method for large-scale unconstrained optimization with non-convex objective function. Optim. Methods Softw. 34 (2019) 783–796. [CrossRef] [MathSciNet] [Google Scholar]
M.S. Koorapetse and P. Kaelo, Globally convergent three-term conjugate gradient projection methods for solving nonlinear monotone equations. Arab. J. Math. 7 (2018) 289–301. [CrossRef] [MathSciNet] [Google Scholar]
W. La Cruz, A spectral algorithm for large-scale systems of nonlinear monotone equations. Numer. Algor. 76 (2017) 1109–1130. [CrossRef] [Google Scholar]
W. La Cruz, J.M. Martinez and M. Raydan, Spectral residual method without gradient information for solving large-scale nonlinear systems of equations: theory and experiments. Technical Report RT-04-08 (2004). [Google Scholar]
D.H. Li and M. Fukushima, A derivative-free line search and global convergence of Broyden-like method for nonlinear equations. Optim. Methods Softw. 13 (2000) 583–599. [Google Scholar]
D.H. Li and M. Fukushima, A globally and superlinearly convergent Gauss–Newton-based BFGS method for symmetric nonlinear equations. SIAM J. Numer. Anal. 37 (2000) 152–172. [Google Scholar]
J.K. Liu and S.J. Li, A projection method for convex constrained monotone nonlinear equations with applications. Comput. Math. Appl. 70 (2015) 2442–2453. [CrossRef] [MathSciNet] [Google Scholar]
D.Y. Liu and Y.F. Shang, A new Perry conjugate gradient method with the generalized conjugacy condition. In: Comput. Intel. Softw. Eng. (CiSE). 2010 International Conference on Issue (2010). [Google Scholar]
D.Y. Liu and G.Q. Xu, A Perry descent conjugate gradient method with restricted spectrum. Optimization Online, Nonlinear Optimization (unconstrained optimization) (2011) 1–19. [Google Scholar]
I.E. Livieris and P. Pintelas, Globally convergent modified Perrys conjugate gradient method. Appl. Math. Comput. 218 (2012) 9197–9207. [MathSciNet] [Google Scholar]
Y.B. Musa, M.Y. Waziri and A.S. Halilu, On computing the regularization parameter for the Levenberg–Marquardt method via the spectral radius approach to solving systems of nonlinear equations. J. Numer. Math. Stoch. 9 (2017) 80–94. [MathSciNet] [Google Scholar]
J. Nocedal and S.J. Wright, Numerical Optimization. Springer, New York (1999). [CrossRef] [Google Scholar]
J.M. Ortega and W.C. Rheinboldt, Iterative Solution of Nonlinear Equations in Several Variables. Academic Press, New York (1970). [Google Scholar]
J.S. Pang, Inexact Newton methods for the nonlinear complementarity problem. Math. Program. 36 (1986) 54–71. [CrossRef] [Google Scholar]
G. Peiting and H. Chuanjiang, A derivative-free three-term projection algorithm involving spectral quotient for solving nonlinear monotone equations. Optimization 67 (2018) 1631–1648. [CrossRef] [MathSciNet] [Google Scholar]
J.M. Perry, A class of conjugate gradient algorithms with a two step variable metric memory. Discussion paper 269, Center for Mathematical Studies in Economics and Management Science, Northwestern University, Chicago (1977). [Google Scholar]
A. Perry, A modified conjugate gradient algorithm. Oper. Res. Tech. Notes 26 (1978) 1073–1078. [CrossRef] [Google Scholar]
B.T. Polak, The conjugate gradient method in extreme problems. USSR Comput. Math. Math. Phys. 4 (1969) 94–112. [CrossRef] [Google Scholar]
E. Polak and G. Ribičre, Note sur la convergence de méthodes de directions conjuguées. Rev. Fr. Inf. Rech. Oper. 16 (1969) 35–43. [Google Scholar]
J.K. Romberg, Imaging via compressive sampling. IEEE Signal Process. Mag. 25 (2008) 14–20. [CrossRef] [Google Scholar]
J. Sabi’u, A. Shah and M.Y. Waziri, Two optimal Hager-Zhang conjugate gradient methods for solving monotone nonlinear equations. Appl. Numer. Math. 153 (2020) 217–233. [Google Scholar]
J. Sabi’u, A. Shah, M.Y. Waziri and K. Ahmed, Modified Hager-Zhang conjugate gradient methods via singular value analysis for solving monotone nonlinear equations with convex constraint. Int. J. Comput. Meth. 18 (2021) 2050043. [CrossRef] [Google Scholar]
D.F. Shanno, On the convergence of a new conjugate gradient algorithm. SIAM J. Numer. Anal. 15 (1978) 1247–1257. [CrossRef] [MathSciNet] [Google Scholar]
M.V. Solodov and A.N. Iusem, Newton-type methods with generalized distances for constrained optimization. Optimization 41 (1997) 257–278. [CrossRef] [MathSciNet] [Google Scholar]
M.V. Solodov and B.F. Svaiter, A globally convergent inexact Newton method for systems of monotone equations. In: Reformulation: Nonsmooth, Piecewise Smooth, Semismooth and Smoothing Methods. Springer (1998) 355–369. [CrossRef] [Google Scholar]
W. Sun and Y.X. Yuan, Optimization Theory and Methods: Nonlinear Programming. Springer, New York (2006). [Google Scholar]
S. Wang and H. Guan, A scaled conjugate gradient method for solving monotone nonlinear equations with convex constraints. J. Appl. Math. (2013). DOI: 10.1155/2013/286486. [Google Scholar]
D.S. Watkins, Fundamentals of Matrix Computations, 2nd edition. John Wiley and Sons, New York (2004). [Google Scholar]
M.Y. Waziri and K. Ahmed, Two descent Dai-Yuan conjugate gradient methods for systems of monotone nonlinear equations. J. Sci. Comput. 90 (2022) 1–53. [CrossRef] [Google Scholar]
M.Y. Waziri, W.J. Leong and M.A. Hassan, Jacobian free-diagonal Newton’s method for nonlinear systems with singular Jacobian. Malay. J. Math. Sci. 5 (2011) 241–255. [Google Scholar]
M.Y. Waziri, K. Ahmed and J. Sabi’u, A family of Hager-Zhang conjugate gradient methods for system of monotone nonlinear equations. Appl. Math. Comput. 361 (2019) 645–660. [MathSciNet] [Google Scholar]
M.Y. Waziri, K. Ahmed and J. Sabi’u, Descent Perry conjugate gradient methods for systems of monotone nonlinear equations. Numer. Algor. 85 (2020) 763–785. [Google Scholar]
M.Y. Waziri, K. Ahmed and J. Sabi’u, A Dai-Liao conjugate gradient method via modified secant equation for system of nonlinear equations. Arab. J. Math. 9 (2020) 443–457. [Google Scholar]
M.Y. Waziri, K. Ahmed, J. Sabi’uand A.S. Halilu, Enhanced Dai-Liao conjugate gradient methods for systems of monotone nonlinear equations. SeMA J. 78 (2020) 15–51. [Google Scholar]
M.Y. Waziri, Y.M. Kufena and A.S. Halilu, Derivative-free three-term spectral conjugate gradient method for symmetric nonlinear equations. Thai J. Math. 18 (2020) 1417–1431. [MathSciNet] [Google Scholar]
M.Y. Waziri, K. Ahmed, A.S. Halilu and A.M. Awwal, Modified Dai-Yuan iterative scheme for nonlinear systems and its application. Numer. Alg. Control Optim. (2021). DOI: 10.3934/naco.2021044. [Google Scholar]
M.Y. Waziri, H. Usman, A.S. Halilu and K. Ahmed, Modified matrix-free methods for solving systems of nonlinear equations. Optimization 70 (2021) 2321–2340. [CrossRef] [MathSciNet] [Google Scholar]
M.Y. Waziri, Y.M. Kufena and A.S. Halilu, Double direction three-term spectral conjugate gradient method for solving symmetric nonlinear equations. Res. Control Opt. 6 (2022) 1–11. [Google Scholar]
Y.H. Xiao and H. Zhu, A conjugate gradient method to solve convex constrained monotone equations with applications in compressive sensing. J. Math. Anal. Appl. 405 (2013) 310–319. [CrossRef] [MathSciNet] [Google Scholar]
Y.H. Xiao, Q.Y. Wang and Q.J. Hu, Non-smooth equations based method for l₁-norm problems with applications to compressed sensing. Nonlinear Anal. TMA 74 (2011) 3570–3577. [CrossRef] [Google Scholar]
J.C. Yang, J. Wright, T.S. Huang and Y. Ma, Image super-resolution via sparse representation. IEEE Trans. Image Process. 19 (2010) 2861–2873. [Google Scholar]
G. Yu, A derivative-free method for solving large-scale nonlinear systems of equations. J. Ind. Manag. Optim. 6 (2010) 149–160. [MathSciNet] [Google Scholar]
Z. Yu, J. Lina, J. Sun, Y. Xiao, L. Liu and Z. Li, Spectral gradient projection method for monotone nonlinear equations with convex constraints. Appl. Numer. Math. 59 (2009) 2416–2424. [CrossRef] [MathSciNet] [Google Scholar]
N. Yuan, A derivative-free projection method for solving convex constrained monotone equations. ScienceAsia 43 (2017) 195–200. [CrossRef] [Google Scholar]
G. Yuan, B. Wang and Z. Sheng, The Hager-Zhang conjugate gradient algorithm for large-scale nonlinear equations. Int. J. Comput. Math. 96 (2019) 1533–1547. [CrossRef] [MathSciNet] [Google Scholar]
L. Zhang and W. Zhou, Spectral gradient projection method for solving nonlinear monotone equations. J. Comput. Appl. Math. 196 (2006) 478–484. [Google Scholar]
Y.B. Zhao and D. Li, Monotonicity of fixed point and normal mappings associated with variational inequality and its application. SIAM J. Optim. 11 (2001) 962–973. [CrossRef] [MathSciNet] [Google Scholar]
W. Zhou and D. Li, Limited memory BFGS method for nonlinear monotone equations. J. Comput. Math. 25 (2007) 89–96. [MathSciNet] [Google Scholar]

Current usage metrics show cumulative count of Article Views (full-text article views including HTML views, PDF and ePub downloads, according to the available data) and Abstracts Views on Vision4Press platform.

Data correspond to usage on the plateform after 2015. The current usage metrics is available 48-96 hours after online publication and is updated daily on week days.

Initial download of the metrics may take a while.