Evaluasi dan Interpretabilitas Model Machine learning untuk Prediksi Diabetes dengan Nested cross-validation dan SHAP

Errie Tri Armawan; Riana Safitri; Lutvi Riyandari

doi:10.55382/jurnalpustakaai.v6i1.1751

Authors

Errie Tri Armawan STMIK Widya Utama Purwokerto
Riana Safitri STMIK Widya Utama Purwokerto
Lutvi Riyandari STMIK Widya Utama Purwokerto

DOI:

https://doi.org/10.55382/jurnalpustakaai.v6i1.1751

Keywords:

Machine Learning, Prediksi Diabetes, Nested Cross-Validation, SHAP, Interpretab

Abstract

Penelitian ini mengevaluasi dan membandingkan kinerja tiga algoritma machine learning Logistic Regression, Random Forest, dan XGBoost untuk prediksi diabetes menggunakan nested cross-validation (5-fold outer, 3-fold inner) dengan pipeline preprocessing terintegrasi untuk mencegah data leakage. Dataset yang digunakan adalah Pima Indians Diabetes Dataset (n = 768). Uji Friedman (p = 0,819) mengonfirmasi bahwa ketiga model tidak berbeda secara statistik, sehingga Logistic Regression dipilih berdasarkan prinsip parsimoni dengan stabilitas tertinggi (AUC-ROC 72,3% ± 1,6% pada nested cross-validation) dan precision 74,8% ± 6,8%. Pada data uji independen, model mencapai akurasi 69,5%, AUC-ROC 81,4%, dan PR-AUC 65,9%. Analisis SHAP mengidentifikasi Glucose, BMI, dan DiabetesPedigreeFunction sebagai tiga prediktor teratas, yang selaras dengan kriteria diagnostik dan faktor risiko dalam pedoman American Diabetes Association dan World Health Organization. Kesesuaian ini membuktikan bahwa model mempelajari pola yang bermakna secara klinis, bukan sekadar korelasi statistik.

Downloads

Download data is not yet available.

References

Erlin, Yulvia Nora Marlim, Junadhi, Laili Suryati, and Nova Agustina, “Deteksi Dini Penyakit Diabetes Menggunakan Machine Learning dengan Algoritma Logistic Regression,” J. Nas. Tek. Elektro dan Teknol. Inf., vol. 11, no. 2, pp. 88–96, 2022, doi: 10.22146/jnteti.v11i2.3586.

A. Pramudyantoro, E. Utami, and D. Ariatmanto, “Penggabungan K-Nearest Neighbors Dan Lightgbm Untuk Prediksi Diabetes Pada Dataset Pima Indians: Menggunakan Pendekatan Exploratory Data Analysis,” JIPI (Jurnal Ilm. Penelit. dan Pembelajaran Inform., vol. 9, no. 3, pp. 1133–1144, 2024, doi: 10.29100/jipi.v9i3.4966.

E. Safitri, D. Rofianto, N. Purwati, H. Kurniawan, and S. Karnila, “Prediksi Penyakit Diabetes Melitus Menggunakan Algoritma Machine Learning,” J. Sist. dan Teknol. Inf., vol. 12, no. 4, pp. 760–766, 2024, doi: 10.26418/justin.v12i4.84620.

A. Setiawan, “E ISSN?: 2809-4069 Pengembangan Sistem Prediksi Risiko Diabetes Menggunakan Algoritma Support Vector Machine ( SVM ),” vol. 5, no. 3, pp. 567–572, 2025, doi: https://doi.org/10.55382/jurnalpustakaai.v5i3.1437.

S. Arti and E. Suherlan, “E ISSN?: 2809-4069 Evaluasi Kinerja Machine Learning dalam Memprediksi Kemampuan Adaptasi Mahasiswa pada Lingkungan Pembelajaran Daring,” vol. 5, no. 1, pp. 50–57, 2025, doi: https://doi.org/10.55382/jurnalpustakaai.v5i1.901.

International Diabetes Federation, “IDF Diabetes Atlas 11th Edition,” Brussels, Belgium, 2025.

World Health Organization, “Diabetes (Fact Sheet),” Geneva, 2024.

V. Chang, J. Bailey, Q. A. Xu, and Z. Sun, “Pima Indians diabetes mellitus classification based on machine learning (ML) algorithms,” Neural Comput. Appl., vol. 35, no. 22, pp. 16157–16173, 2023, doi: 10.1007/s00521-022-07049-z.

S. Kapoor and A. Narayanan, “Leakage and the reproducibility crisis in machine-learning-based science,” Patterns, vol. 4, no. 9, p. 100804, 2023, doi: 10.1016/j.patter.2023.100804.

B. Bischl et al., “Hyperparameter optimization: Foundations, algorithms, best practices, and open challenges,” Wiley Interdiscip. Rev. Data Min. Knowl. Discov., vol. 13, no. 2, pp. 1–43, 2023, doi: 10.1002/widm.1484.

R. Hasan, V. Dattana, S. Mahmood, and S. Hussain, “Towards Transparent Diabetes Prediction: Combining AutoML and Explainable AI for Improved Clinical Insights,” Inf., vol. 16, no. 1, 2025, doi: 10.3390/info16010007.

R. Hendawi, J. Li, and S. Roy, “A Mobile App That Addresses Interpretability Challenges in Machine Learning–Based Diabetes Predictions: Survey-Based User Study,” JMIR Form. Res., vol. 7, no. 1, pp. 1–18, 2023, doi: 10.2196/50328.

M. M. Islam, H. R. Rifat, M. S. Bin Shahid, A. Akhter, M. A. Uddin, and K. M. M. Uddin, “Explainable Machine Learning for Efficient Diabetes Prediction Using Hyperparameter Tuning, SHAP Analysis, Partial Dependency, and LIME,” Eng. Reports, vol. 7, no. 1, 2025, doi: 10.1002/eng2.13080.

P. Netayawijit, W. Chansanam, and K. Sorn-In, “Interpretable Machine Learning Framework for Diabetes Prediction: Integrating SMOTE Balancing with SHAP Explainability for Clinical Decision Support,” Healthc., vol. 13, no. 20, pp. 1–26, 2025, doi: 10.3390/healthcare13202588.

M. Kutlu, T. B. Donmez, and C. Freeman, “Machine Learning Interpretability in Diabetes Risk Assessment: A SHAP Analysis,” Comput. Electron. Med., vol. 1, no. 1, pp. 34–44, 2024, doi: 10.69882/adba.cem.2024075.

S. M. Lundberg and S.-I. Lee, “A Unified Approach to Interpreting Model Predictions,” in Advances in Neural Information Processing Systems, Curran Associates, Inc., 2017, pp. 4765–4774. [Online]. Available: https://papers.nips.cc/paper/7062-a-unified-approach-to-interpreting-model-predictions.pdf

F. Pedregosa et al., “Scikit-Learn Classifier Tuning from Complex Training Sets,” J. Mach. Learn. Res., vol. 12, pp. 2825–2830, 2011, [Online]. Available: https://www.jmlr.org/papers/volume12/pedregosa11a/pedregosa11a.pdf

World Health Organization, Classification of diabetes mellitus. Geneva, 2019. [Online]. Available: https://apps.who.int/iris/bitstream/handle/10665/325158/9789241515702-eng.pdf

J. W. Smith, J. E. Everhart, W. C. Dickson, W. C. Knowler, and R. S. Johannes, “Using the ADAP learning algorithm to forecast the onset of diabetes mellitus,” Proc. Annu. Symp. Comput. Appl. Med. Care, pp. 261–265, 1988, [Online]. Available: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2245318/

A. Altamimi et al., “An automated approach to predict diabetic patients using KNN imputation and effective data mining techniques,” BMC Med. Res. Methodol., vol. 24, no. 1, 2024, doi: 10.1186/s12874-024-02324-0.

E. Richardson, R. Trevizani, J. A. Greenbaum, H. Carter, M. Nielsen, and B. Peters, “The receiver operating characteristic curve accurately assesses imbalanced datasets,” Patterns, vol. 5, no. 6, p. 100994, 2024, doi: 10.1016/j.patter.2024.100994.

T. Kosolwattana, C. Liu, R. Hu, S. Han, H. Chen, and Y. Lin, “A self-inspected adaptive SMOTE algorithm (SASMOTE) for highly imbalanced data classification in healthcare,” BioData Min., vol. 16, no. 1, pp. 1–14, 2023, doi: 10.1186/s13040-023-00330-4.

S. Gholampour, “Impact of Nature of Medical Data on Machine and Deep Learning for Imbalanced Datasets: Clinical Validity of SMOTE Is Questionable,” Mach. Learn. Knowl. Extr., vol. 6, no. 2, pp. 827–841, 2024, doi: 10.3390/make6020039.

American Diabetes Association Professional Practice Committee, “2. Diagnosis and Classification of Diabetes: Standards of Care in Diabetes—2024,” Diabetes Care, vol. 47, no. Supplement_1, pp. S20–S42, 2024, doi: 10.2337/dc25-S002.

S. Ali et al., “Explainable Artificial Intelligence (XAI): What we know and what is left to attain Trustworthy Artificial Intelligence,” Inf. Fusion, vol. 99, no. April, p. 101805, 2023, doi: 10.1016/j.inffus.2023.101805.