Performance Comparison Between ResNet50 and MobileNetV2 for Indonesian Sign Language Classification

Feriska Putri Daviana; Aryanti Aryanti; Nurhajar Anugraha

doi:10.30865/jurikom.v12i3.8667

Authors

Feriska Putri Daviana Politeknik Negeri Sriwijaya, Palembang
Aryanti Aryanti Politeknik Negeri Sriwijaya, Palembang
Nurhajar Anugraha Politeknik Negeri Sriwijaya, Palembang

DOI:

https://doi.org/10.30865/jurikom.v12i3.8667

Keywords:

Indonesian Sign Language, Image Classification, Convolutional Neural Network, ResNet50, MobileNetV2

Abstract

Hearing impairment was considered a significant barrier to understanding verbal communication. Therefore, an alternative communication medium in the form of sign language was required to bridge interactions between Deaf and hearing individuals. One of the sign languages used in Indonesia was the Indonesian Sign Language (BISINDO). The advancement of deep learning technology provided a great opportunity to develop an effective and accurate BISINDO alphabet classification system. This research was conducted to evaluate and compare the performance of two Convolutional Neural Network (CNN) architectures, namely ResNet50 and MobileNetV2, in classifying BISINDO alphabet images consisting of 26 classes from A to Z. Model training wa carried out over 100 epochs and was analyzed using metrics such as training and validation accuracy, precision, recall, F1-score, and confusion matrix. The training process used a dataset that was divided into 80% training data and 20% validation data, and include image preprocessing steps such as resizing and rescaling. The evaluation results showed that ResNet50 achieved 86.42% training accuracy and 98.64% validation accuracy with 98.80% precision, 98.69% recall, 98.57% F1-score, and 31 misclassifications. In contrast, MobileNetV2 showed superior performance with 99.99% training accuracy, 99.65% validation accuracy, 99.69% precision, 99.65% recall, 99.61% F1-score, and only 8 misclassifications. Based on these results, MobileNetV2 was recommended as a more effective and efficient architecture for BISINDO alphabet image classification compared to ResNet50.

References

A. S. Nugraheni, A. P. Husain, and H. Unayah, “Optimalisasi Penggunaan Bahasa Isyarat Dengan Sibi Dan Bisindo Pada Mahasiswa Difabel Tunarungu Di Prodi Pgmi Uin Sunan Kalijaga,” J. Holistika, vol. 5, no. 1, p. 28, 2023, doi: 10.24853/holistika.5.1.28-33.

R. Mandasari and S. Winduwati, “Upaya Public Relations Pusbisindo dalam Mengampanyekan Penggunaan Bahasa Isyarat Indonesia di Kalangan Masyarakat,” Prologia, vol. 6, no. 2, pp. 355–361, 2022, doi: 10.24912/pr.v6i2.15572.

N. Tri et al., Deep Learning : Teori , Algoritma , dan Aplikasi, no. March. 2025.

Naufal MSesilia ShaniaJessica Millenia et al., “Analisis Perbandingan Algoritma Klasifikasi MLP dan CNN pada Dataset American Sign Language,” vol. 1, no. 10, 2021, doi: https://doi.org/10.29207/resti.v5i3.3009.

M. E. Al Rivan and A. G. Riyadi, “Perbandingan Arsitektur LeNet dan AlexNet Pada Metode Convolutional Neural Network Untuk Pengenalan American Sign Language,” J. Komput. Terap., vol. 7, no. 1, pp. 53–61, 2021, doi: 10.35143/jkt.v7i1.4489.

O. D. Nurhayati, D. Eridani, and M. H. Tsalavin, “Sistem Isyarat Bahasa Indonesia (SIBI) Metode Convolutional Neural Network Sequential secara Real Time,” J. Teknol. Inf. dan Ilmu Komput., vol. 9, no. 4, pp. 819–828, 2022, doi: 10.25126/jtiik.2022944787.

Y. Brianorman and R. Munir, “Perbandingan Pre-Trained CNN: Klasifikasi Pengenalan Bahasa Isyarat Huruf Hijaiyah,” J. Sist. Info. Bisnis, vol. 13, no. 1, pp. 52–59, 2023, doi: 10.21456/vol13iss1pp52-59.

I. J. Thira, D. Riana, A. N. Ilhami, B. R. S. Dwinanda, and H. Choerunisya, “Pengenalan Alfabet Sistem Isyarat Bahasa Indonesia (SIBI) Menggunakan Convolutional Neural Network,” J. Algoritm., vol. 20, no. 2, pp. 421–432, 2023, doi: 10.33364/algoritma/v.20-2.1480.

A. Victor Ikechukwu, S. Murali, R. Deepu, and R. C. Shivamurthy, “ResNet-50 vs VGG-19 vs training from scratch: A comparative analysis of the segmentation and classification of Pneumonia from chest X-ray images,” Glob. Transitions Proc., vol. 2, no. 2, pp. 375–381, 2021, doi: 10.1016/j.gltp.2021.08.027.

Faiz Nashrullah, Suryo Adhi Wibowo, and Gelar Budiman, “The Investigation of Epoch Parameters in ResNet-50 Architecture for Pornographic Classification,” J. Comput. Electron. Telecommun., vol. 1, no. 1, pp. 1–8, 2020, doi: 10.52435/complete.v1i1.51.

K. Dong, C. Zhou, Y. Ruan, and Y. Li, “MobileNetV2 Model for Image Classification,” Proc. - 2020 2nd Int. Conf. Inf. Technol. Comput. Appl. ITCA 2020, pp. 476–480, 2020, doi: 10.1109/ITCA52113.2020.00106.

G. W. Intyanto, “Klasifikasi Citra Bunga dengan Menggunakan Deep Learning: CNN (Convolution Neural Network),” J. Arus Elektro Indones., vol. 7, no. 3, p. 80, 2021, doi: 10.19184/jaei.v7i3.28141.

S. SATRIA, Sumijan, and Billy Hendrik, “Implementasi Convolutional Neural Netowork Untuk Klasifikasi Citra KTP-El,” J. CoSciTech (Computer Sci. Inf. Technol., vol. 5, no. 1, pp. 169–176, 2024, doi: 10.37859/coscitech.v5i1.6708.

G. Gumelar, Q. Ain, R. Marsuciati, S. Agustanti Bambang, A. Sunyoto, and M. Syukri Mustafa, “Kombinasi Algoritma Sampling dengan Algoritma Klasifikasi untuk Meningkatkan Performa Klasifikasi Dataset Imbalance,” SISFOTEK Sist. Inf. dan Teknol., pp. 250–255, 2021.

S. Andika Maulana, S. Husna Batubara, Y. Permata Putri Pasaribu, H. Syahputra, and F. Ramadhani, “Deteksi Burung Menggunakan Convolutional Neural Network (Cnn) Dengan Model Arsitektur Mobilenetv2,” JATI (Jurnal Mhs. Tek. Inform., vol. 8, no. 4, pp. 6108–6114, 2024, doi: 10.36040/jati.v8i4.10126.

D. Martomanggolo, “Perbandingan Convolutional Neural Network pada Transfer Learning Method untuk Mengklasifikasikan Sel Darah Putih,” Ultim. J. Tek. Inform., vol. 13, no. 1, p. 51, 2021.

A. Derry, M. Krzywinski, and N. Altman, “Convolutional neural networks,” Nat. Methods, vol. 20, no. 9, pp. 1269–1270, 2023, doi: 10.1038/s41592-023-01973-1.

T. Berliani, E. Rahardja, and L. Septiana, “Perbandingan Kemampuan Klasifikasi Citra X-ray Paru-paru menggunakan Transfer Learning ResNet-50 dan VGG-16,” J. Med. Heal., vol. 5, no. 2, pp. 123–135, 2023, doi: 10.28932/jmh.v5i2.6116.

B. Wijayanto, R. M. Mahendra, M. I. Salam, and T. Informatika, “Identifikasi Jenis Ikan Cupang Menggunakan Metode CNN Dengan Arsitektur MobileNetV2 Berbasis Mobile,” vol. 4, pp. 519–525, 2025.

D. Kurniawan, D. Ariatmanto, M. T. Informatika, and U. A. Yogyakarta, “Identifikasi varietas bibit durian menggunakan mobilenetv2 berdasarkan gambar daun,” vol. 7, no. 2, pp. 231–240, 2024.

M. Heydarian, T. E. Doyle, and R. Samavi, “MLCM: Multi-Label Confusion Matrix,” IEEE Access, vol. 10, pp. 19083–19095, 2022, doi: 10.1109/ACCESS.2022.3151048.

M. F. Naufal and S. F. Kusuma, “Comparison Analysis of Machine Learning and Deep Learning Algorithms for Image Classification of Indonesian Language Signing Systems (Sibi),” Jtiik, vol. 10, no. 4, pp. 873–882, 2023, doi: 10.25126/jtiik.2023106828.