A damped Hessian-free newton--conjugate gradient method for weighted multiclass neural classification

Irawan, Andy, Abidin, Zainal ORCID: https://orcid.org/0000-0002-9261-4952 and Jamhuri, Mohammad (2026) A damped Hessian-free newton--conjugate gradient method for weighted multiclass neural classification. CAUCHY : Jurnal Matematika Murni dan Aplikasi, 11 (1). pp. 820-841. ISSN 2086-0382; e-ISSN 2477-3344

Preview

Text 27962.pdf - Published Version Available under License Creative Commons Attribution Share Alike. Download (653kB) | Preview

Abstract

This study presents a deterministic damped Hessian-free Newton–CG method for weighted multiclass neural classification. The method is built from a weighted categorical cross-entropy objective, a damped local quadratic model, and a matrix-free curvature representation through Hessian–vector products. The search direction is computed by an inexact conjugate gradient solve, while Armijo backtracking and adaptive damping are used to improve stability. The method is implemented for the classification of academic predicate categories using preprocessed student data with mixed categorical and numerical features. Its numerical behavior is compared with SGD with momentum, RMSProp, and Adam under the same loss, initialization, and network architecture. The proposed method is computationally feasible, attains the best overall weighted test-set performance among the compared methods, and exhibits a distinct optimization trajectory driven by curvature-informed updates. These results show that a damped Hessian-free formulation provides a mathematically transparent, reproducible, and practically competitive framework for second-order optimization in multiclass neural classification.

Item Type:	Journal Article
Keywords:	conjugate gradient; Hessian-free optimization; multiclass classification; neural networks; second-order methods
Subjects:	08 INFORMATION AND COMPUTING SCIENCES > 0802 Computation Theory and Mathematics > 080299 Computation Theory and Mathematics not elsewhere classified
Divisions:	Graduate Schools > Magister Programme > Graduate School of Informatics Engineering
Depositing User:	Andy Irawan
Date Deposited:	13 Jul 2026 09:05

Downloads

Origin of downloads

Actions (login required)

View Item