Facial Expression Recognition inCovid-19 Pandemic EraUsing 2 Stage Convolutional Neural Network

 (*)Kevin Kurniawan Mail (Bina Nusantara University, Jakarta, Indonesia)
 Lili Ayu Wulandhari (Bina Nusantara University, Jakarta, Indonesia)
 Antonius Rildo Pramudya Gondosiswojo (Bina Nusantara University, Jakarta, Indonesia)

(*) Corresponding Author

Submitted: September 7, 2023; Published: September 29, 2023

Abstract

In the Covid-19 pandemic era, the use of face mask has become mandatory for all citizens to prevent the spread of the virus. This regulation has becomes a big problem for the Facial Expression Recognitions (FER) applications because face mask cover more than halfof human faces. Starting from this problem, this experiment attemps to produce a robust network which can perform well in both conditions: recognizing expressions with and without a face mask. Dataset used in this experiment is FER2013, with a preprocessing step to produce a FER2013 masked. This research uses 2-stage network, where the first network is used to recognize whether the subject is wearing a mask or not, and then the second network is used to recognize the expression based on the result from the first stage. The network in this experiment is based on Convolutional Neural Network (CNN), with Imagenet as our pretrainedmodel and EfficientNet as our architecture model.Ourproposed model has shown quite good performance in recognizing expressions, even when the data consists of subjects who use and do not use masks with an accuracy of 57.55%.

Keywords


Pandemic Era; Covid-19; Facial Expression Recognition; Face Mask; Occlusion; Convolutional Neural Network

Full Text:

PDF


Article Metrics

Abstract view : 175 times
PDF - 25 times

References

P. Saxena, “Real-time emotion recognition: Potential use cases and challenges,” indiaai.gov.in, 2021. .

Tinaliah and T. Elizabeth, “Penerapan Convolutional Neural Network Untuk Klasifikasi Citra Ekspresi Wajah Manusia Pada MMA Facial Expression Dataset,” J. Tek. Inform. dan Sist. Inf., vol. 8, no. 4, pp. 2051–2059, 2021, doi: 10.35957/jatisi.v8i4.1437.

S. Minaee, M. Minaei, and A. Abdolrashidi, “Deep-emotion: Facial expression recognition using attentional convolutional network,” Sensors, vol. 21, no. 9, pp. 1–16, 2021, doi: 10.3390/s21093046.

S. T. Zulkarnain and N. Suciati, “Modified Local Ternary Pattern With Convolutional Neural Network for Face Expression Recognition,” JUTI J. Ilm. Teknol. Inf., vol. 19, no. 1, p. 10, 2021, doi: 10.12962/j24068535.v19i1.a1031.

R. Septian, D. I. Saputra, and S. Sambasri, “Klasifikasi Emosi Menggunakan Convolutional Neural Networks,” 2019, doi: 10.13733/j.jcam.issn.2095-5553.2022.11.026.

L. Pham, T. H. Vu, and T. A. Tran, “Facial expression recognition using residual masking network,” Proc. - Int. Conf. Pattern Recognit., pp. 4513–4519, 2020, doi: 10.1109/ICPR48806.2021.9411919.

Syahromi, “Facial Expression Recognition Menggunakan Algoritma Mtcnn (Multi-Task Cascaded Convolutional Neural Network) Dan Vgg16,” Apl. dan Anal. Lit. Fasilkom UI, pp. 4–25, 2022.

Ministry of Health, “Peta Sebaran,” Https://Covid19.Go.Id/Peta-Sebaran, p. https://covid19.go.id/peta-sebaran, 2022.

Haeruddin, Herman, and Patrick Pratama Hendri, “PENGEMBANGAN APLIKASI EMOTION RECOGNITION DAN FACIAL RECOGNITION MENGGUNAKAN ALGORITMA LOCAL BINARY PATTERN HISTOGRAM (LBPH) DAN CONVOLUTIONAL NEURAL NETWORK (CNN),” J. Teknol. Terpadu, vol. 9, no. 1, pp. 49–55, 2023.

B. Yang, J. Wu, and G. Hattori, “Facial Expression Recognition with the advent of face masks,” ACM Int. Conf. Proceeding Ser., pp. 335–337, 2020, doi: 10.1145/3428361.3432075.

G. Castellano, B. De Carolis, and N. Macchiarulo, “Automatic emotion recognition from facial expressions when wearing a mask,” ACM Int. Conf. Proceeding Ser., 2021, doi: 10.1145/3464385.3464730.

A. Anwar and A. Raychowdhury, “Masked Face Recognition for Secure Authentication,” pp. 1–8, 2020.

A. Lioga Seandrio, A. Hendrianto Pratomo, and M. Y. Florestiyanto, “Implementation of Convolutional Neural Network (CNN) in Facial Expression Recognition,” Telemat. J. Inform. dan Teknol. Inf., vol. 18, no. 2, pp. 211–221, 2021, doi: 10.31515/telematika.v18i2.4823.

M. Fakhrudin and A. Fatoni, “Pengaruh Risiko Spesifik Bank dan Good Corporate Governance Terhadap Stabilitas Bank Pembiayaan Rakyat Syariah,” vol. 4, no. 1, pp. 116–127, 2023, doi: 10.47065/jtear.v4i1.734.

D. Alamsyah and D. Pratama, “Implementasi Convolutional Neural Networks (CNN) untuk Klasifikasi Ekspresi Citra Wajah pada FER-2013 Dataset,” J. Teknol. Inf., vol. 4, no. 2, pp. 350–355, 2020, doi: 10.36294/jurti.v4i2.1714.

Lia Farokhah, “Implementasi Convolutional Neural Network untuk Klasifikasi Variasi Intensitas Emosi pada Dynamic Image Sequence,” J. RESTI (Rekayasa Sist. dan Teknol. Informasi), vol. 4, no. 6, pp. 1070–1076, 2020, doi: 10.29207/resti.v4i6.2644.

S. Sriyati, A. Setyanto, and E. E. Luthfi, “Literature Review: Pengenalan Wajah Menggunakan Algoritma Convolutional Neural Network,” J. Teknol. Inf. dan Komun., vol. 8, no. 2, 2020, doi: 10.30646/tikomsin.v8i2.463.

P. Musa, W. K. Anam, S. B. Musa, W. Aryunani, R. Senjaya, and P. Sularsih, “Pembelajaran Mendalam Pengklasifikasi Ekspresi Wajah Manusia dengan Model Arsitektur Xception pada Metode Convolutional Neural Network,” Rekayasa, vol. 16, no. 1, pp. 65–73, 2023, doi: 10.21107/rekayasa.v16i1.16974.

Y. Fajriyanti, A. H. Rahmah, S. Ulfa, and E. Hadiyanti, “Analisis Motivasi Kerja Generasi Z Yang Dipengaruhi Oleh Lingkungan Kerja dan Komitmen Kerja,” vol. 4, no. 1, pp. 107–115, 2023, doi: 10.47065/jtear.v4i1.808.

S. P. Ristiawanto, B. Irawan, and C. Setianingsih, “PENGENALAN EKSPRESI WAJAH BERBASIS CONVOLUTIONAL NEURAL NETWORK MENGGUNAKAN ARSITEKTUR RESIDUAL NETWORK-50,” e-Proceeding Eng., vol. 8, no. 5, pp. 6442–6454, 2021.

P. P. Kusdiananggalih, E. Rachmawat, and Risnandar, “Pengenalan Ekspresi Wajah Menggunakan Deep Convolutional Neural Network,” J. Tugas Akhir Fak. Inform. e-Proceeding, vol. 8, no. 2, p. 75, 2021, doi: 10.35200/explore.v11i2.478.

J. Omar, N. H. Shabrina, A. N. Bhakti, and A. Patria, “Emotion Recognition Using Convolutional Neural Network (CNN),” Ultim. Comput. J. Sist. Komput., vol. 13, no. 1, 2021, doi: 10.1088/1742-6596/1962/1/012040.

Bila bermanfaat silahkan share artikel ini

Berikan Komentar Anda terhadap artikel Facial Expression Recognition inCovid-19 Pandemic EraUsing 2 Stage Convolutional Neural Network

Refbacks

  • There are currently no refbacks.


Copyright (c) 2023 Kevin Kurniawan, Lili Ayu Wulandhari, Antonius Rildo Pramudya Gondosiswojo

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.

Jurnal Sistem Komputer dan Informatika (JSON)
Dikelola oleh STMIK Budi Darma
Sekretariat : Jln. Sisingamangaraja No. 338 Telp 061-7875998
email : jurnal.json@gmail.com


Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.