Penerapan Model Hybrid Convolutional Neural Network dan Long Short-Term Memory untuk Pengenalan Real-Time Sistem Isyarat Bahasa Indonesia (SIBI)
Abstract
The Indonesian Sign Language System (SIBI) is an essential means of communication for the deaf and speech-impaired community in Indonesia. However, the limited public understanding of SIBI often hinders effective communication. This study develops a real-time SIBI sign recognition model to facilitate effective communication for the deaf and speech-impaired in Indonesia. The proposed method integrates a hybrid CNN-LSTM model to process the spatial and temporal information from the data. The study evaluates the model's performance on 25 types of SIBI signs. The dataset used consists of image sequences captured in real-time. Training is conducted with various parameters, including batch size, learning rate, and epochs. Model evaluation is carried out using accuracy, precision, recall, and f1-score metrics. The training and validation results show an increase in accuracy with the number of epochs: 87% at 10 epochs, 93% at 25 epochs, and 100% at 50 epochs. In real-time detection tests, the model with the image sequence dataset accurately detected SIBI signs in environments and with objects consistent with the dataset. The real-time detection program generates SIBI sign predictions in text form and sentences. The output of this research is efficient and accurate SIBI sign recognition technology. This research is expected to facilitate more effective communication for the deaf and speech-impaired community in Indonesia.
Keywords
Full Text:
PDFReferences
F. Iffah and Y. Fitri Yasni, “Manusia Sebagai Makhluk Sosial,†2022.
U. Mahadi, “Komunikasi yang Baik,†J. Public Policy Adm. Silampari, vol. 2, no. 2, pp. 80–90, 2021.
A. S. Nugraheni, A. P. Husain, and H. Unayah, “Optimalisasi Penggunaan Bahasa Isyarat Dengan Sibi Dan Bisindo Pada Mahasiswa Difabel Tunarungu Di Prodi Pgmi Uin Sunan Kalijaga,†J. Holistika, vol. 5, no. 1, p. 28, 2023, doi: 10.24853/holistika.5.1.28-33.
A. Pratiwi and A. Amri, “Penggunaan Sistem Isyarat Bahasa Indonesia (SIBI) sebagai media komunikasi (studi pada siswa tunarungu di SLB),†J. Ilm. Mhs. FISIP Unsyiah, vol. 4, no. 3, pp. 1–12, 2019, [Online]. Available: www.jim.unsyiah.ac.id/FISIP
Z. Zhang, P. Cui, and W. Zhu, “Deep Learning on Graphs: A Survey,†IEEE Trans. Knowl. Data Eng., vol. 34, no. 1, pp. 249–270, 2022, doi: 10.1109/TKDE.2020.2981333.
G. Gumelar, H. Hafiar, and P. Subekti, “Bahasa Isyarat Indonesia Sebagai Budaya Tuli Melalui Pemaknaan Anggota Gerakan Untuk Kesejahteraan Tuna Rungu,†INFORMASI, vol. 48, no. 1, p. 65, Jul. 2018, doi: 10.21831/informasi.v48i1.17727.
K. Xia, J. Huang, and H. Wang, “LSTM-CNN Architecture for Human Activity Recognition,†IEEE Access, vol. 8, pp. 56855–56866, 2020, doi: 10.1109/ACCESS.2020.2982225.
H. M. Putri, F. Fadlisyah, and W. Fuadi, “Pendeteksian Bahasa Isyarat Indonesia Secara Real-Time Menggunakan Long Short-Term Memory (Lstm),†J. Teknol. Terap. Sains 4.0, vol. 3, no. 1, p. 663, 2022, doi: 10.29103/tts.v3i1.6853.
G. Khartheesvar, M. Kumar, A. K. Yadav, and D. Yadav, “Automatic Indian sign language recognition using MediaPipe holistic and LSTM network,†Multimed. Tools Appl., 2023, doi: 10.1007/s11042-023-17361-y.
P. Sinha, D. Kumar, and A. Prakash, “Real Time Sign Language Prediction Using Cnn and Lstm,†no. 04, pp. 7304–7316, 2023.
F. Marpaung, F. Aulia, and R. C. Nabila, Computer Vision Dan Pengolahan Citra Digital. 2022.
Y. Vita Via, W. S.J. Saputra, M. Idham Fachrurrozi, E. Yulia Puspaningrum, F. Tri Anggraeny, and S. Rohman Nudin, “Object Localization and Detecting Alphabet in Sign Language BISINDO Using Convolution Neural Network,†Rom. J. Ofapplied Sci. Technol., vol. XVI, no. 1, pp. 143–149, 2023.
K. R. Nur Manab, E. P. Mandyartha, and A. M. Rizki, “Rancang Bangun Sistem Deteksi Huruf Rusia Berbasis Web Flask,†Pros. Semin. Nas. Inform. Bela Negara, vol. 2, pp. 156–160, 2021, doi: 10.33005/santika.v2i0.108.
L. Alzubaidi et al., Review of deep learning: concepts, CNN architectures, challenges, applications, future directions, vol. 8, no. 1. Springer International Publishing, 2021. doi: 10.1186/s40537-021-00444-8.
M. I. Choldun R and K. Surendro, “Klasifikasi Penelitian Dalam Deep Learning,†Improv. J. Ilm. Manaj. Inform., vol. 10, no. 1, pp. 25–33, 2018.
E. Rasywir, R. Sinaga, and Y. Pratama, “Evaluasi Pembangunan Sistem Pakar Penyakit Tanaman Sawit dengan Metode Deep Neural Network (DNN),†J. Media …, vol. 4, no. 5, pp. 1206–1215, 2020, doi: 10.30865/mib.v4i4.2518.
A. Roihan, P. A. Sunarya, and A. S. Rafika, “Pemanfaatan Machine Learning dalam Berbagai Bidang: Review paper,†IJCIT (Indonesian J. Comput. Inf. Technol., vol. 5, no. 1, pp. 75–82, 2020, doi: 10.31294/ijcit.v5i1.7951.
R. R. Pratama, “Analisis Model Machine Learning Terhadap Pengenalan Aktifitas Manusia,†MATRIK J. Manajemen, Tek. Inform. dan Rekayasa Komput., vol. 19, no. 2, pp. 302–311, 2020, doi: 10.30812/matrik.v19i2.688.
I. W. Widya, I. Gede, and A. Wibawa, “Klasifikasi Bentuk Wajah Manusia Menggunakan Metode Convolutional Neural Network (CNN),†vol. 1, no. November, pp. 373–378, 2022.
M. R. Alwanda, R. P. K. Ramadhan, and D. Alamsyah, “Implementasi Metode Convolutional Neural Network Menggunakan Arsitektur LeNet-5 untuk Pengenalan Doodle,†J. Algoritm., vol. 1, no. 1, pp. 45–56, 2020, doi: 10.35957/algoritme.v1i1.434.
A. Sherstinsky, “Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) network,†Phys. D Nonlinear Phenom., vol. 404, p. 132306, 2020, doi: https://doi.org/10.1016/j.physd.2019.132306.
M. W. P. Aldi, Jondri, and A. Aditsania, “Analisis dan Implementasi Long Short Term Memory Neural Network untuk Prediksi Harga Bitcoin,†e-Proceeding Eng. Vol.5 No.2, vol. 5, no. 2, pp. 3548–3555, 2018.
DOI: https://doi.org/10.30865/mib.v8i3.7837
Refbacks
- There are currently no refbacks.
Copyright (c) 2024 JURNAL MEDIA INFORMATIKA BUDIDARMA
This work is licensed under a Creative Commons Attribution 4.0 International License.
JURNAL MEDIA INFORMATIKA BUDIDARMA
Universitas Budi Darma
Secretariat: Sisingamangaraja No. 338 Telp 061-7875998
Email: mib.stmikbd@gmail.com
This work is licensed under a Creative Commons Attribution 4.0 International License.