A Comparative Study of Machine Learning Algorithms for Detecting Heart Disease

Abstract

Cardiovascular diseases remain a leading cause of mortality worldwide. Identifying underlying clinical phenotypes early, such as distinct categories of chest pain, is vital for diagnostic triage and downstream medical decision-making. This study evaluates the performance of four prominent machine learning algorithms – gradient boosting (GB), random forest, multinomial logistic regression, and support vector machines (SVM) – in classifying four distinct chest pain types (cp = 0, 1, 2, 3) utilizing a clinical dataset of 180 patient records. The models were trained using a stratified 80/20 train-test split combined with 5-fold cross-validation for hyperparameter optimization. Evaluation was conducted through rigorous classification metrics: Accuracy, Macro $F_1$-score, Weighted $F_1$-score, Cohen’s Kappa, and receiver operating characteristic area under the curve (ROC-AUC). The empirical findings reveal that GB achieves the highest overall performance with an accuracy of 76.1% and an ROC-AUC of 0.912. Post hoc statistical evaluation using Nemenyi and McNemar tests ($\alpha = 0.01$) confirmed that GB and random forest demonstrate significantly higher predictive power than multinomial logistic regression and SVM. However, multinomial logistic regression remains clinically invaluable, offering explicit Odds Ratios that identify “oldpeak” (exercise-induced ST depression), “thalach” (maximum heart rate), and “age” as the most statistically significant clinical predictors. Across all models, classification performance degraded when detecting Class 3 chest pain, though GB retained the highest relative recall for this minority category.