Inexact generalized Gauss--Newton--CG for Binary Cross-Entropy minimization

Jamhuri, Mohammad, Sari, Silvi Puspita, Amiroch, Siti, Juhari, Juhari and Fitria, Vivi Aida (2025) Inexact generalized Gauss--Newton--CG for Binary Cross-Entropy minimization. Jurnal Riset Mahasiswa Matematika, 5 (2). pp. 102-122. ISSN 2808-1552; E-ISSN 2808-4926

Preview

Text 27823.pdf - Published Version Available under License Creative Commons Attribution Share Alike. Download (2MB) | Preview

Abstract

Binary cross-entropy (BCE) minimization is a standard objective in probabilistic binary classification, yet practical training pipelines often rely on first-order methods whose performance can be sensitive to step-size choices and may require many iterations to reach low-loss solutions. This paper studies an inexact curvature-based solver that combines a (generalized) Gauss–Newton approximation with conjugate gradient (CG) inner iterations for minimizing the regularized BCE objective in full-batch logistic regression. At each outer iteration, the method computes a descent direction by approximately solving a damped Gauss–Newton system in a matrix-free manner via repeated products with X and X⊤, and terminates CG according to a relative-residual inexactness rule. Numerical experiments on three benchmark datasets show that the proposed Inexact GGN–CG can substantially reduce the number of outer iterations on smaller numerical data, while remaining competitive in predictive performance, and can improve both validation and test mean BCE on larger mixed-type data after one-hot encoding. In particular, on Adult Census Income the method achieves lower test mean BCE (0.3176 ± 0.0044) and higher F1-score (0.6623 ± 0.0066) than Adam and gradient descent under the same regularization-selection protocol, at the cost of additional CG work. These results highlight how damping and inexactness jointly govern the trade-off between curvature-solve effort, wall-clock time, and achieved BCE values in deterministic logistic-regression training.

Item Type:	Journal Article
Keywords:	generalized Gauss--Newton; conjugate gradient; inexact methods; binary cross-entropy; logistic regression; second-order optimization
Subjects:	01 MATHEMATICAL SCIENCES > 0102 Applied Mathematics > 010201 Approximation Theory and Asymptotic Methods 01 MATHEMATICAL SCIENCES > 0104 Statistics > 010401 Applied Statistics 01 MATHEMATICAL SCIENCES > 0102 Applied Mathematics 01 MATHEMATICAL SCIENCES > 0104 Statistics
Divisions:	Faculty of Mathematics and Sciences > Department of Mathematics
Depositing User:	Juhari Juhari
Date Deposited:	10 Jul 2026 14:34

Downloads

Origin of downloads

Actions (login required)

View Item