A Damped Hessian-Free Newton–Conjugate Gradient Method for Weighted Multiclass Neural Classification

Irawan, Andy, Abidin, Zainal ORCID: https://orcid.org/0000-0002-9261-4952 and Mohammad, Jamhuri (2026) A Damped Hessian-Free Newton–Conjugate Gradient Method for Weighted Multiclass Neural Classification. CAUCHY – Jurnal Matematika Murni dan Aplikasi, 11 (1). pp. 820-841. ISSN p-ISSN: 2086-0382; e-ISSN: 2477-3344

Preview

Text CHAUCHY-A Damped Hessian-Free Newton–Conjugate Gradient.pdf Download (653kB) | Preview

Abstract

This study presents a deterministic damped Hessian-free Newton–CG method for weighted
multiclass neural classification. The method is built from a weighted categorical cross-entropy
objective, a damped local quadratic model, and a matrix-free curvature representation through
Hessian–vector products. The search direction is computed by an inexact conjugate gradient
solve, while Armijo backtracking and adaptive damping are used to improve stability. The
method is implemented for the classification of academic predicate categories using preprocessed
student data with mixed categorical and numerical features. Its numerical behavior is
compared with SGD with momentum, RMSProp, and Adam under the same loss, initialization,
and network architecture. The proposed method is computationally feasible, attains
the best overall weighted test-set performance among the compared methods, and exhibits a
distinct optimization trajectory driven by curvature-informed updates. These results show
that a damped Hessian-free formulation provides a mathematically transparent, reproducible,
and practically competitive framework for second-order optimization in multiclass neural
classification.

Item Type:	Journal Article
Keywords:	conjugate gradient; Hessian-free optimization; multiclass classification; neural networks; second-order methods
Subjects:	08 INFORMATION AND COMPUTING SCIENCES > 0802 Computation Theory and Mathematics > 080299 Computation Theory and Mathematics not elsewhere classified
Divisions:	Graduate Schools > Magister Programme > Graduate School of Informatics Engineering
Depositing User:	Andy Irawan
Date Deposited:	13 Jul 2026 09:05

Downloads

Origin of downloads

Actions (login required)

View Item