Sistem Deteksi Kosakata Bahasa Isyarat Secara Real Time dengan Tensorflow Menggunakan Metode Convolutional Neural Network

 (*)Vincent Surya Widjaya Mail (Universitas Pradita, Banten, Indonesia)
 Ito Wasito (Universitas Pradita, Banten, Indonesia)

(*) Corresponding Author

Submitted: May 15, 2024; Published: July 26, 2024

Abstract

Most people are not used to communicating via sign language because sign language is not a mandatory language to learn so very few people understand how to communicate using sign language. This problem for people with special needs to interact and communicate with other people, especially those who do not understand how to communicate using sign language. Convolutional Neural Network is the method that will be used in this research because this method is one of the deep learning methods that currently provides the best results in object detection. The Convolutional Neural Network method is able to imitate human abilities in the image recognition system of the human visual cortex, so that it is able to process visual information. This research used Tensorflow to develop various models as well as work related to other statistical analysis. The evaluation metrics results obtained from this research show precision of 82.99%, recall of 84.42% and f1-score of 83.68%. The value of average precision is 0.830% and average recall is 0.844%. There is also a loss value produced by this model of 0.039065. For accuracy results from real time sign language detection, the accuracy value was 94.64%.

Keywords


Sign language; Convolutional Neural Networks; Detection; Deep Learning; Tensorflow

Full Text:

PDF


Article Metrics

Abstract view : 172 times
PDF - 120 times

References

M. A. Pratama, M. R. Erfit, N. M. Farhani, I. A. Hartono, and M. Maryamah, “Klasifikasi Abjad SIBI (Sistem Bahasa Isyarat Indonesia) menggunakan Mediapipe dengan metode Deep Learning,” Semin. Nas. Sains Data, pp. 134–141, 2023.

R. Agung Firmansyah, Y. Agung Prabowo, and Zakaria, “Rancang Bangun Flex Sensor Gloves untuk Penerjemah Bahasa Isyarat Menggunakan K-Nearest Neighbors,” Semin. Nas. Sains dan Teknol. Terap., p. 361, 2019.

A. R. Syah, “APLIKASI PENERJEMAH BAHASA ISYARAT MENGGUNAKAN METODE K-NN (K-NEARST NEIGHBOUR ),” J. Teknol. Pint., vol. 2, no. 4, pp. 1–12, 2022.

I. N. T. A. Putra, K. S. Kartini, Y. K. Suyitno, I. M. Sugiarta, and N. K. E. Puspita, “Penerapan Library Tensorflow, Cvzone, dan Numpy pada Sistem Deteksi Bahasa Isyarat Secara Real Time,” J. Krisnadana, vol. 2, no. 3, pp. 412–423, 2023, doi: 10.58982/krisnadana.v2i3.335.

A. A. Alhamdani, “Penerapan Deep Learning dengan menggunakan Algoritma Convolutional Neural Network (CNN) untuk Gesture Recognition,” J. Softw. Eng. Inf. Commun. Technol., vol. 2, no. 1, pp. 78–82, 2021.

L. Arisandi and B. Satya, “Sistem Klarifikasi Bahasa Isyarat Indonesia (Bisindo) Dengan Menggunakan Algoritma Convolutional Neural Network,” J. Sist. Cerdas, vol. 5, no. 3, pp. 135–146, 2022, doi: 10.37396/jsc.v5i3.262.

M. Sholawati, K. Auliasari, and F. Ariwibisono, “Pengembangan Aplikasi Pengenalan Bahasa Isyarat Abjad Sibi Menggunakan Metode Convolutional Neural Network (CNN),” JATI (Jurnal Mhs. Tek. Inform., vol. 6, no. 1, pp. 134–144, 2022, doi: 10.36040/jati.v6i1.4507.

R. H. Alfikri, M. S. Utomo, H. Februariyanti, and E. Nurwahyudi, “Pembangunan Aplikasi Penerjemah Bahasa Isyarat Dengan Metode CNN Berbasis Android,” J. Teknoinfo, vol. 16, no. 2, p. 183, 2022, doi: 10.33365/jti.v16i2.1752.

R. I. Borman, B. Priyopradono, and A. R. Syah, “Klasifikasi Objek Kode Tangan pada Pengenalan Isyarat Alphabet Bahasa Isyarat Indonesia (BISINDO),” Semin. Nas. Inform. dan Apl., pp. 1–4, 2018.

N. I. Pratiwi, I. Widaningrum, and D. Mustikasari, “Perancangan Sistem Deteksi Isyarat BISINDO Dengan Metode Adaptive Neuro-Fuzzy Inference System (ANFIS),” KomtekInfo, vol. 6, no. 1, pp. 50–61, 2019, doi: 10.29165/komtekinfo.v6i1.232.

I. Amri, “IMPLEMENTASI ALGORITMA CONVOLUTIONAL NEURAL NETWORK UNTUK MENERJEMAHKAN BAHASA ISYARAT,” Kohesi J. Multidisiplin Saintek, vol. 2, no. 9, pp. 70–87, 2024.

R. Aryanto, M. Alfan Rosid, and S. Busono, “Penerapan Deep Learning untuk Pengenalan Tulisan Tangan Bahasa Aksara Lota Ende dengan Menggunakan Metode Convolutional Neural Networks,” J. Inf. dan Teknol., vol. 5, no. 1, pp. 258–264, 2023, doi: 10.37034/jidt.v5i1.313.

S. Apendi, C. Setianingsih, and M. W. Paryasto, “Deteksi Bahasa Isyarat Sistem Isyarat Bahasa Indonesia Menggunakan Metode Single Shot Multibox Detector | Apendi | eProceedings of Engineering,” e-Proceeding od Eng., vol. 10, no. 1, pp. 249–255, 2023, [Online]. Available: https://openlibrarypublications.telkomuniversity.ac.id/index.php/engineering/article/view/19322

A. Pratiwi and A. Fauzi, “IMPLEMENTATION OF DEEP LEARNING ON FLOWER CLASSIFICATION USING IMPLEMENTASI DEEP LEARNING PADA PENGKLASIFIKASIAN BUNGA,” J. Tek. Inform., vol. 5, no. 2, pp. 487–495, 2024.

I. B. A. Peling, I. M. P. A. Ariawan, and G. B. Subiksa, “Deteksi Bahasa Isyarat Menggunakan Tensorflow Lite dan American Sign Language (ASL),” J. Krisnadana, vol. 3, no. 2, pp. 90–100, 2024, doi: 10.58982/krisnadana.v3i2.534.

R. M. Yusup et al., “PENDETEKSIAN OBJEK MENGGUNAKAN OPENCV DAN METODE YOLOv4-TINY UNTUK MEMBANTU TUNANETRA,” J. Comput. Sci. Inf. Technol., vol. 1, no. 2, pp. 59–68, 2024.

E. Sentosa, D. I. Mulyana, A. F. Cahyana, and N. G. Pramuditasari, “Implementasi Image Classification Pada Batik Motif Bali Dengan Data Augmentation dan Convolutional Neural Network,” J. Pendidik. Tambusai, vol. 6, no. 1, pp. 1451–1463, 2022.

F. Charli, H. Syaputra, M. Akbar, S. Sauda, and F. Panjaitan, “Implementasi Metode Faster Region Convolutional Neural Network (Faster R-CNN) Untuk Pengenalan Jenis Burung Lovebird,” J. Inf. Technol. Ampera, vol. 1, no. 3, pp. 185–197, 2020, doi: 10.51519/journalita.volume1.isssue3.year2020.page185-197.

M. Rio Akbar, “Perancangan Komik Bisindo Tentang Belajar Berhitung Untuk Anak Usia Dini,” J. Sains Inform. Terap., vol. 1, no. 1, pp. 45–51, 2022, doi: 10.62357/jsit.v1i1.52.

R. Fatmawati, R. Asmara, Y. R. Prayogi, and R. Y. Hakkun, “Aplikasi Pembelajaran Sistem Isyarat Bahasa Indonesia (SIBI) Berbasis Voice Menggunakan OpenSIBI,” Technomedia J., vol. 7, no. 1, pp. 22–39, 2022, doi: 10.33050/tmj.v7i1.1690.

Refbacks

  • There are currently no refbacks.


Copyright (c) 2024 JURNAL MEDIA INFORMATIKA BUDIDARMA

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.



JURNAL MEDIA INFORMATIKA BUDIDARMA
STMIK Budi Darma
Secretariat: Sisingamangaraja No. 338 Telp 061-7875998
Email: mib.stmikbd@gmail.com

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.