Analisis Performa Class Weight Dan Focal Loss Pada Model Indobert Untuk Klasifikasi Teks Depresi Berbahasa Indonesia
DOI:
https://doi.org/10.30865/jurikom.v13i2.9620Keywords:
Text Classification, Depression Indication, IndoBERT, Class Weight, Focal Loss, Class Imbalance.Abstract
The development of depression detection can be done using exploration of social media content. However, the classification of depression indicative texts faces a major challenge in the form of class distribution imbalances, which can degrade the model's generalization capabilities. This study aims to analyze how the method of overcoming class imbalance affects the performance of the IndoBERT model in the classification of Indonesian depression indication texts by emphasizing the analysis of training stability based on the dynamics of training loss and validation loss. The dataset used consists of 3,863 data, data that has gone through the process of cleaning, removing duplicate data, tokenization, encoding, and dividing data into stratification into training data, validation data, and test data. The IndoBERT-base-p1 model was fine-tuned using three training scenarios, namely baseline, class weight, and focal loss with an early stopping mechanism based on validation loss. The test results showed that the baseline IndoBERT scenario produced an accuracy of 77.52%, a weighted precision of 0.7752, a weighted recall of 0.7752, a weighted F1-score of 0.7737, and a ROC-AUC of 0.8528 with a relatively stable training pattern. The class weight method produced an accuracy of 74.68%, a weighted F1-score of 0.7467, and a ROC-AUC of 0.8342 which showed an increase in class discrimination ability but accompanied by a decrease in overall accuracy. Meanwhile, the focal loss method produced an accuracy of 72.87%, a weighted F1-score of 0.7291, and a ROC-AUC of 0.8188 with more balanced training characteristics than the weight class. The findings suggest that handling classroom imbalances does not necessarily improve global performance, so model evaluations need to consider a balance between accuracy, sensitivity, and stability of training.
References
[1] M. Y. Dhinora and E. Mailoa, “Analisa Tweet Mahasiswa untuk Deteksi Gejala Depresi dengan Penerapan Natural Language Processing,” Jurnal Indonesia : Manajemen Informatika dan Komunikasi, vol. 6, no. 2, pp. 1193–1211, 2025, doi: 10.63447/jimik.v6i2.1405.
[2] G. F. Situmorang and R. Purba, “Deteksi Potensi Depresi dari Unggahan Media Sosial X Menggunakan IndoBERT,” Building of Informatics, Technology and Science (BITS), vol. 6, no. 2, pp. 649–661, Sep. 2024, doi: 10.47065/bits.v6i2.5496.
[3] A. G. S. Moeslim, E. Firmansyah, and B. Sutara, “Perbandingan Kinerja XGBoost dan IndoBERT untuk Klasifikasi Teks Kesehatan Bahasa Indonesia,” Data Sciences Indonesia (DSI), vol. 5, no. 2, pp. 62–72, Dec. 2025, doi: 10.47709/dsi.v5i2.7281.
[4] M. B. M. Amin et al., “Deteksi Spam Berbahasa Indonesia Berbasis Teks Menggunakan Model Bert,” Jurnal Teknologi Informasi dan Ilmu Komputer, vol. 11, no. 6, pp. 1291–1302, Dec. 2024, doi: 10.25126/jtiik.2024118121.
[5] M. Fitri, J. S. Nurkhotimah, F. N. Ihsan, K. Amaliah, and D. Rofianto, “Model Ensembel untuk Deteksi Depresi di Twitter Berbahasa Indonesia,” Jurnal Sifo Mikroskil, vol. 26, no. 2, Oct. 2025, doi: 10.55601/jsm.v26i2.1812.
[6] C. Sintiya, G. H. Hutagaol, D. Bate`e, and S. Irviantina, “Evaluasi Teknik Resampling untuk Class Balancing dalam Analisis Sentimen Kesehatan Mental Berbasis Bi-LSTM,” Jurnal Sifo Mikroskil, vol. 26, no. 2, pp. 257–274, Oct. 2025, doi: 10.55601/jsm.v26i2.1799.
[7] H. F. Karim and A. P. Wibowo, “Kinerja Metode Fine-Tuning IndoBERT untuk Klasifikasi Emosi Multi-Kelas pada Teks Informal Bahasa Indonesia,” Bulletin of Computer Science Research, vol. 6, no. 1, pp. 63–74, 2025, doi: 10.47065/bulletincsr.v6i1.850.
[8] E. P. Adamansyah and A. Yudhistira, “Evaluasi Opini Publik di Media Sosial X terhadap Kebijakan Pajak Pertambahan Nilai 12% di Indonesia Menggunakan Naive Bayes dan Decision Tree,” Jurnal Pendidikan dan Teknologi Indonesia, vol. 5, no. 3, pp. 831–843, Mar. 2025, doi: 10.52436/1.jpti.710.
[9] A. A. Pangestu and A. R. Purnajaya, “Perbandingan Performa Arsitektur Machine Learning untuk Deteksi Dini Depresi Berbasis Natural Language Processing dalam Bahasa Indonesia,” J-SIGN (Journal of Informatics, Information System, and Artificial Intelligence), vol. 3, no. 2, pp. 93–104, Oct. 2025, doi: 10.24815/j-sign.v3i2.49873.
[10] Z. Darojah, R. Susetyoko, N. Ramadijanti, P. Elektronika Negeri Surabaya, and P. Korespondensi, “Strategi Penanganan Imbalance Class Pada Model Klasifikasi Penerimaan Kartu Indonesia Pintar Kuliah Berbasis Neural Network Menggunakan Kombinasi SMOTE dan ENN,” Jurnal Teknologi Informasi dan Ilmu Komputer (JTIIK), vol. 10, no. 2, pp. 457–466, 2023, doi: 10.25126/jtiik.2023106480.
[11] I. A. Rahma and L. H. Suadaa, “Penerapan Text Augmentation untuk Mengatasi Data yang Tidak Seimbang pada Klasifikasi Teks Berbahasa Indonesia,” Jurnal Teknologi Informasi dan Ilmu Komputer, vol. 10, no. 6, pp. 1329–1340, Dec. 2023, doi: 10.25126/jtiik.2023107325.
[12] M. Fajar Ramadhan, Febriyanti Panjaitan, Winarnie, Hery Oktafiandi, and Yohanes, “Analisis Sentimen Kutipan Media Sosial Berbahasa Indonesia Menggunakan Convolutional Neural Network,” Jurnal Komputer, Informasi dan Teknologi, vol. 5, no. 2, pp. 17–26, Dec. 2025, doi: 10.53697/jkomitek.v6i1.3627.
[13] E. Sutoyo and M. C. Permana, “Enhancing telemedicine service quality through sentiment analysis of user review dataset in Indonesia,” Data Brief, vol. 61, p. 111878, Aug. 2025, doi: 10.1016/j.dib.2025.111878.
[14] T. Jia, R. de Vries, Z. Kapelan, T. H. M. van Emmerik, and R. Taormina, “Detecting floating litter in freshwater bodies with semi-supervised deep learning,” Water Res., vol. 266, p. 122405, Nov. 2024, doi: 10.1016/j.watres.2024.122405.
[15] M. J. Faisti, R. Hadapiningradja Kusumodestoni, G. Wahyu, and N. Wibowo, “Mental Health Classification Using Naïve Bayes and Random Forest Algorithms,” Journal of Applied Informatics and Computing (JAIC), vol. 9, no. 4, pp. 1740–1750, 2025, [Online]. Available: http://jurnal.polibatam.ac.id/index.php/JAIC
[16] R. Oktafiani, A. Hermawan, and D. Avianto, “Pengaruh Komposisi Split Data Terhadap Performa Klasifikasi Penyakit Kanker Payudara Menggunakan Algoritma Machine Learning,” Jurnal Sains dan Informatika, pp. 19–28, Jun. 2023, doi: 10.34128/jsi.v9i1.622.
[17] M. R. Manoppo et al., “Analisis Sentimen Publik Di Media Sosial Terhadap Kenaikan PPN 12% di Indonesia Menggunakan IndoBERT,” Jurnal Kecerdasan Buatan dan Teknologi Informasi, vol. 4, no. 2, pp. 152–163, May 2025, doi: 10.69916/jkbti.v4i2.322.
[18] I. Mursidah, R. Sanjaya, B. Yulianto, D. Sweetania, and P. Sularsih, “Klasifikasi Sentimen Google Play Store Aplikasi ChatGPT Berbahasa Indonesia Berbasis IndoBERT,” Jurnal Minfo Polgan, vol. 14, no. 2, pp. 3349–3359, Dec. 2025, doi: 10.33395/jmp.v14i2.15751.
[19] A. F. Hidayatullah, R. A. Apong, D. T. C. Lai, and A. Qazi, “Corpus creation and language identification for code-mixed Indonesian-Javanese-English Tweets,” PeerJ Comput. Sci., vol. 9, p. e1312, Jun. 2023, doi: 10.7717/peerj-cs.1312.
[20] N. N. A. Aryanti and O. Suria, “Analisis Sentimen Terhadap Pemutusan Hubungan Kerja di Indonesia : Komparasi IndoBERT dengan SVM, Random Forest, dan Decision Tree Dengan Optimasi TF - IDF,” Rabit : Jurnal Teknologi dan Sistem Informasi Univrab, vol. 10, no. 2, pp. 1158–1176, Jul. 2025, doi: 10.36341/rabit.v10i2.6364.
[21] F. Indriani, R. A. Nugroho, M. R. Faisal, and D. Kartini, “Comparative Evaluation of IndoBERT, IndoBERTweet, and mBERT for Multilabel Student Feedback Classification,” Jurnal RESTI, vol. 8, no. 6, pp. 748–757, Dec. 2024, doi: 10.29207/resti.v8i6.6100.
[22] D. Ramadhan Alfinsyah and B. Pilu Hartato, “Evaluating the Impact of Random Over Sampling on IndoBERT Performance for Indonesian Sentiment Analysis,” Journal of Applied Informatics and Computing (JAIC), vol. 9, no. 6, pp. 3270–3282, 2025, [Online]. Available: http://jurnal.polibatam.ac.id/index.php/JAIC
[23] Yohanes Anton Nugroho and Hotma Antoni Hutahaean, “Pengembangan Model Deep Learning LSTM dan CNN untuk Peramalan Penjualan Sepeda Motor di Indonesia,” Jupiter: Publikasi Ilmu Keteknikan Industri, Teknik Elektro dan Informatika, vol. 3, no. 2, pp. 94–104, Mar. 2025, doi: 10.61132/jupiter.v3i2.795.
[24] M. N. Razali, N. Arbaiy, P.-C. Lin, and S. Ismail, “Optimizing Multiclass Classification Using Convolutional Neural Networks with Class Weights and Early Stopping for Imbalanced Datasets,” Electronics (Basel)., vol. 14, no. 4, p. 705, Feb. 2025, doi: 10.3390/electronics14040705.
[25] A. I. Faiq, M. R. S. Sanjaya, D. R. Indah, and E. L. Ruskan, “Hybrid Fine-Tuning IndoBERT dan Ensemble TF-IDF Logistic Regression Untuk Analisis Sentimen Ulasan Aplikasi Zalora,” Rabit : Jurnal Teknologi dan Sistem Informasi Univrab, vol. 11, no. 1, pp. 324–336, Jan. 2026, doi: 10.36341/rabit.v11i1.6924.
[26] A. Kunaefi, Z. Abidin, and R. Kusumawati, “Klasifikasi Berita Hoaks Bahasa Indonesia Menggunakan IndoBERT Fine - Tuning Dengan Pendekatan Focal Loss Pada Data Tidak Seimbang,” JIPI (Jurnal Ilmiah Penelitian dan Pembelajaran Informatika), vol. 10, no. 2, pp. 1706–1714, May 2025, doi: 10.29100/jipi.v10i2.7811.
[27] F. Alzami et al., “Bayesian-Optimized CLAHE for Enhanced Drowsiness Detection in Low-Light Conditions Using Time-Distributed MobileNetV2-GRU Architecture,” Statistics, Optimization and Information Computing, vol. 15, no. 1, pp. 274–294, Dec. 2025, doi: 10.19139/soic-2310-5070-3024.
[28] V. W. Handayani, M. S. Margareth Amiatun Ruth, R. Rulaningtyas, M. R. Caesarardhi, B. A. Yudhantorro, and A. Yudianto, “Development and evaluation of a convolutional neural network model for sex prediction using cephalometric radiographs and cranial photographs,” BMC Med. Imaging, vol. 25, no. 1, p. 348, Aug. 2025, doi: 10.1186/s12880-025-01892-x.
