Sistem Deteksi Kosakata Bahasa Isyarat Secara Real Time dengan Tensorflow Menggunakan Metode Convolutional Neural Network

Authors

  • Vincent Surya Widjaya Universitas Pradita, Banten
  • Ito Wasito Universitas Pradita, Banten

DOI:

https://doi.org/10.30865/mib.v8i3.7714

Keywords:

Sign language, Convolutional Neural Networks, Detection, Deep Learning, Tensorflow

Abstract

Most people are not used to communicating via sign language because sign language is not a mandatory language to learn so very few people understand how to communicate using sign language. This problem for people with special needs to interact and communicate with other people, especially those who do not understand how to communicate using sign language. Convolutional Neural Network is the method that will be used in this research because this method is one of the deep learning methods that currently provides the best results in object detection. The Convolutional Neural Network method is able to imitate human abilities in the image recognition system of the human visual cortex, so that it is able to process visual information. This research used Tensorflow to develop various models as well as work related to other statistical analysis. The evaluation metrics results obtained from this research show precision of 82.99%, recall of 84.42% and f1-score of 83.68%. The value of average precision is 0.830% and average recall is 0.844%. There is also a loss value produced by this model of 0.039065. For accuracy results from real time sign language detection, the accuracy value was 94.64%.

References

M. A. Pratama, M. R. Erfit, N. M. Farhani, I. A. Hartono, and M. Maryamah, “Klasifikasi Abjad SIBI (Sistem Bahasa Isyarat Indonesia) menggunakan Mediapipe dengan metode Deep Learning,†Semin. Nas. Sains Data, pp. 134–141, 2023.

R. Agung Firmansyah, Y. Agung Prabowo, and Zakaria, “Rancang Bangun Flex Sensor Gloves untuk Penerjemah Bahasa Isyarat Menggunakan K-Nearest Neighbors,†Semin. Nas. Sains dan Teknol. Terap., p. 361, 2019.

A. R. Syah, “APLIKASI PENERJEMAH BAHASA ISYARAT MENGGUNAKAN METODE K-NN (K-NEARST NEIGHBOUR ),†J. Teknol. Pint., vol. 2, no. 4, pp. 1–12, 2022.

I. N. T. A. Putra, K. S. Kartini, Y. K. Suyitno, I. M. Sugiarta, and N. K. E. Puspita, “Penerapan Library Tensorflow, Cvzone, dan Numpy pada Sistem Deteksi Bahasa Isyarat Secara Real Time,†J. Krisnadana, vol. 2, no. 3, pp. 412–423, 2023, doi: 10.58982/krisnadana.v2i3.335.

A. A. Alhamdani, “Penerapan Deep Learning dengan menggunakan Algoritma Convolutional Neural Network (CNN) untuk Gesture Recognition,†J. Softw. Eng. Inf. Commun. Technol., vol. 2, no. 1, pp. 78–82, 2021.

L. Arisandi and B. Satya, “Sistem Klarifikasi Bahasa Isyarat Indonesia (Bisindo) Dengan Menggunakan Algoritma Convolutional Neural Network,†J. Sist. Cerdas, vol. 5, no. 3, pp. 135–146, 2022, doi: 10.37396/jsc.v5i3.262.

M. Sholawati, K. Auliasari, and F. Ariwibisono, “Pengembangan Aplikasi Pengenalan Bahasa Isyarat Abjad Sibi Menggunakan Metode Convolutional Neural Network (CNN),†JATI (Jurnal Mhs. Tek. Inform., vol. 6, no. 1, pp. 134–144, 2022, doi: 10.36040/jati.v6i1.4507.

R. H. Alfikri, M. S. Utomo, H. Februariyanti, and E. Nurwahyudi, “Pembangunan Aplikasi Penerjemah Bahasa Isyarat Dengan Metode CNN Berbasis Android,†J. Teknoinfo, vol. 16, no. 2, p. 183, 2022, doi: 10.33365/jti.v16i2.1752.

R. I. Borman, B. Priyopradono, and A. R. Syah, “Klasifikasi Objek Kode Tangan pada Pengenalan Isyarat Alphabet Bahasa Isyarat Indonesia (BISINDO),†Semin. Nas. Inform. dan Apl., pp. 1–4, 2018.

N. I. Pratiwi, I. Widaningrum, and D. Mustikasari, “Perancangan Sistem Deteksi Isyarat BISINDO Dengan Metode Adaptive Neuro-Fuzzy Inference System (ANFIS),†KomtekInfo, vol. 6, no. 1, pp. 50–61, 2019, doi: 10.29165/komtekinfo.v6i1.232.

I. Amri, “IMPLEMENTASI ALGORITMA CONVOLUTIONAL NEURAL NETWORK UNTUK MENERJEMAHKAN BAHASA ISYARAT,†Kohesi J. Multidisiplin Saintek, vol. 2, no. 9, pp. 70–87, 2024.

R. Aryanto, M. Alfan Rosid, and S. Busono, “Penerapan Deep Learning untuk Pengenalan Tulisan Tangan Bahasa Aksara Lota Ende dengan Menggunakan Metode Convolutional Neural Networks,†J. Inf. dan Teknol., vol. 5, no. 1, pp. 258–264, 2023, doi: 10.37034/jidt.v5i1.313.

S. Apendi, C. Setianingsih, and M. W. Paryasto, “Deteksi Bahasa Isyarat Sistem Isyarat Bahasa Indonesia Menggunakan Metode Single Shot Multibox Detector | Apendi | eProceedings of Engineering,†e-Proceeding od Eng., vol. 10, no. 1, pp. 249–255, 2023, [Online]. Available: https://openlibrarypublications.telkomuniversity.ac.id/index.php/engineering/article/view/19322

A. Pratiwi and A. Fauzi, “IMPLEMENTATION OF DEEP LEARNING ON FLOWER CLASSIFICATION USING IMPLEMENTASI DEEP LEARNING PADA PENGKLASIFIKASIAN BUNGA,†J. Tek. Inform., vol. 5, no. 2, pp. 487–495, 2024.

I. B. A. Peling, I. M. P. A. Ariawan, and G. B. Subiksa, “Deteksi Bahasa Isyarat Menggunakan Tensorflow Lite dan American Sign Language (ASL),†J. Krisnadana, vol. 3, no. 2, pp. 90–100, 2024, doi: 10.58982/krisnadana.v3i2.534.

R. M. Yusup et al., “PENDETEKSIAN OBJEK MENGGUNAKAN OPENCV DAN METODE YOLOv4-TINY UNTUK MEMBANTU TUNANETRA,†J. Comput. Sci. Inf. Technol., vol. 1, no. 2, pp. 59–68, 2024.

E. Sentosa, D. I. Mulyana, A. F. Cahyana, and N. G. Pramuditasari, “Implementasi Image Classification Pada Batik Motif Bali Dengan Data Augmentation dan Convolutional Neural Network,†J. Pendidik. Tambusai, vol. 6, no. 1, pp. 1451–1463, 2022.

F. Charli, H. Syaputra, M. Akbar, S. Sauda, and F. Panjaitan, “Implementasi Metode Faster Region Convolutional Neural Network (Faster R-CNN) Untuk Pengenalan Jenis Burung Lovebird,†J. Inf. Technol. Ampera, vol. 1, no. 3, pp. 185–197, 2020, doi: 10.51519/journalita.volume1.isssue3.year2020.page185-197.

M. Rio Akbar, “Perancangan Komik Bisindo Tentang Belajar Berhitung Untuk Anak Usia Dini,†J. Sains Inform. Terap., vol. 1, no. 1, pp. 45–51, 2022, doi: 10.62357/jsit.v1i1.52.

R. Fatmawati, R. Asmara, Y. R. Prayogi, and R. Y. Hakkun, “Aplikasi Pembelajaran Sistem Isyarat Bahasa Indonesia (SIBI) Berbasis Voice Menggunakan OpenSIBI,†Technomedia J., vol. 7, no. 1, pp. 22–39, 2022, doi: 10.33050/tmj.v7i1.1690.

Downloads

Published

2024-07-26

Issue

Vol. 8 No. 3 (2024): Juli 2024

Section

Articles

License

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License

Authors who publish with this journal agree to the following terms:

  1. Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under Creative Commons Attribution 4.0 International License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
  2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
  3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (Refer to The Effect of Open Access).