Performance Assessment of Iterative Numerical Techniques for Nonlinear Equation Solving and Optimization Problems

Abstract

Nonlinear equations and optimization problems are central to engineering and scientific computing, yet they rarely admit closed-form solutions and must be solved by iterative numerical methods whose performance varies widely with problem structure. This study presents a systematic, controlled comparative assessment of seven prominent numerical techniques spanning four algorithmic families: bracketing methods (Bisection and Regula-Falsi), open methods (Newton-Raphson and Secant), fixed-point iteration with Aitken Δ² acceleration, and optimization methods (Golden Section Search, Newton's optimization, and Gradient Descent). Each method was implemented from first principles under a uniform experimental framework employing IEEE 754 double-precision arithmetic, a strict triple convergence criterion, and five complementary performance metrics. The methods were evaluated on six benchmark test functions chosen to span polynomial, transcendental, and stiff problem types, and on three real-world engineering case studies drawn from fluid mechanics (the Colebrook-White friction-factor equation), chemical engineering (Haber-Bosch ammonia synthesis equilibrium), and structural mechanics (two-bar truss optimization). The results confirm and quantify several fundamental principles. Newton-Raphson achieved quadratic convergence, reaching a tolerance of 10⁻⁸ in four to six iterations versus twenty-seven to twenty-eight for Bisection, while the Secant method delivered the highest efficiency index per function evaluation. A fundamental tension between convergence speed and robustness emerged consistently: the fastest methods were the least reliable from poor initial estimates, whereas the most robust methods converged slowly. Aitken acceleration reduced fixed-point iteration counts by roughly half at negligible cost. The engineering case studies validated these findings under practical conditions, with derivative-based methods yielding speedups of up to two orders of magnitude. The study concludes that no single method is universally optimal and offers an evidence-based decision framework, supported by a failure-mode analysis, to guide method selection according to problem smoothness, derivative availability, accuracy requirements, and the need for guaranteed convergence.