Optimization of Sentiment Analysis Classification of ChatGPT on Big Data Twitter in Indonesia using BERT

Frans Mikael Sinaga, Ronsen Purba, Sio Jurnalis Pipin, Wulan Sri Lestari, Sunaryo Winardi

Abstract


This research is grounded in the emergence of ChatGPT technology, supported by prior and similar studies. The urgency of the issue is highlighted by previous research indicating non-convergent classification outcomes in LSTM (Long Short-Term Memory) methods due to suboptimal hyperparameter settings and limitations in understanding text data within Big Data. The presence of ChatGPT technology brings both benefits and potential misuse, such as copyright infringement, unauthorized news extraction, and violations of accountability principles. Understanding public sentiment towards the presence of ChatGPT technology is crucial. The research aims to implement the BERT (Bidirectional Encoder Representations from Transformers) method to achieve accurate and convergent sentiment analysis classification. This study involves data preprocessing stages using Natural Language Processing (NLP) techniques. Text data, already vectorized, is classified using BERT to determine public sentiment (positive, negative, neutral) towards ChatGPT technology, ensuring greater accuracy, convergence, and contextual relevance. Performance testing of the BERT model is conducted using a Confusion Matrix. With parameters set to Max Sequence Length = 128 and Batch Size = 16, the highest classification accuracy achieved is 93.4%.

Keywords


Sentiment Analysis; ChatGPT; BigData; BERT

Full Text:

PDF

References


L. Marron, “Exploring the Potential of ChatGPT 3.5 in Higher Education,†2023, pp. 326–349. doi: 10.4018/978-1-6684-8292-6.ch017.

F. W. Putra, I. B. Rangka, S. Aminah, and M. H. R. Aditama, “ChatGPT in the higher education environment: perspectives from the theory of high order thinking skills,†J Public Health (Bangkok), vol. 45, no. 4, pp. e840–e841, Nov. 2023, doi: 10.1093/pubmed/fdad120.

T. Adiguzel, M. H. Kaya, and F. K. Cansu, “Revolutionizing education with AI: Exploring the transformative potential of ChatGPT,†Contemp Educ Technol, vol. 15, no. 3, p. ep429, Jul. 2023, doi: 10.30935/cedtech/13152.

F. M. Sinaga, S. Winardi, S. J. Pipin, and M. N. Hakim, “Sentiment Analysis Classification of ChatGPT on Twitter Big Data in Indonesia Using Fast R-CNN,†JURNAL MEDIA INFORMATIKA BUDIDARMA, vol. 7, no. 4, pp. 2137–2148, 2023.

H. Rehioui and A. Idrissi, “New Clustering Algorithms for Twitter Sentiment Analysis,†IEEE Syst J, vol. 14, no. 1, pp. 530–537, Mar. 2020, doi: 10.1109/JSYST.2019.2912759.

S. J. Pipin and H. Kurniawan, “Analisis Sentimen Kebijakan MBKM Berdasarkan Opini Masyarakat di Twitter Menggunakan LSTM,†Jurnal SIFO Mikroskil, vol. 23, no. 2, pp. 197–208, 2022, doi: https://doi.org/10.55601/jsm.v23i2.900.

M. Pota, M. Ventura, R. Catelli, and M. Esposito, “An Effective BERT-Based Pipeline for Twitter Sentiment Analysis: A Case Study in Italian,†Sensors, vol. 21, no. 1, p. 133, Dec. 2020, doi: 10.3390/s21010133.

Y. Akbar and T. Sugiharto, “Analisis Sentimen Pengguna Twitter di Indonesia Terhadap ChatGPT Menggunakan Algoritma C4.5 dan Naïve Bayes,†Jurnal Sains dan Teknologi, vol. 5, no. 1, pp. 115–122, 2023, doi: 10.55338/saintek.v4i3.1368.

T. Tang, X. Tang, and T. Yuan, “Fine-Tuning BERT for Multi-Label Sentiment Analysis in Unbalanced Code-Switching Text,†IEEE Access, vol. 8, pp. 193248–193256, 2020, doi: 10.1109/ACCESS.2020.3030468.

M. Pota, M. Ventura, H. Fujita, and M. Esposito, “Multilingual evaluation of pre-processing for BERT-based sentiment analysis of tweets,†Expert Syst Appl, vol. 181, p. 115119, Nov. 2021, doi: 10.1016/j.eswa.2021.115119.

A. Bello, S.-C. Ng, and M.-F. Leung, “A BERT Framework to Sentiment Analysis of Tweets,†Sensors, vol. 23, no. 1, p. 506, Jan. 2023, doi: 10.3390/s23010506.

A. Zhao and Y. Yu, “Knowledge-enabled BERT for aspect-based sentiment analysis,†Knowl Based Syst, vol. 227, p. 107220, Sep. 2021, doi: 10.1016/j.knosys.2021.107220.

A. M. Bagirov, R. M. Aliguliyev, and N. Sultanova, “Finding compact and well-separated clusters: Clustering using silhouette coefficients,†Pattern Recognit, vol. 135, p. 109144, Mar. 2023, doi: 10.1016/j.patcog.2022.109144.

K. Chen, Y. Zhou, and F. Dai, “A LSTM-based method for stock returns prediction : A case study of China stock market,†May 2015, [Online]. Available: http://arxiv.org/abs/1506.00019

L. N. Pradany and C. Fatichah, “ANALISA SENTIMEN KEBIJAKAN PEMERINTAH PADA KONTEN TWITTER BERBAHASA INDONESIA MENGGUNAKAN SVM DAN K-MEDOID CLUSTERINGâ€.

F. Mikael Sinaga, P. Sirait, and A. Halim, “Optimization of SV-kNNC using Silhouette Coefficient and LMKNN for Stock Price Prediction,†in 2020 3rd International Seminar on Research of Information Technology and Intelligent Systems (ISRITI), IEEE, Dec. 2020, pp. 326–331. doi: 10.1109/ISRITI51436.2020.9315516.

L. Yang, Y. Li, J. Wang, and R. S. Sherratt, “Sentiment Analysis for E-Commerce Product Reviews in Chinese Based on Sentiment Lexicon and Deep Learning,†IEEE, vol. 8, pp. 23522–23530, 2020, doi: 10.1109/ACCESS.2020.2969854.

F. M. Sinaga, S. Winardi, and Gunawan, “3SV-KNNC Optimization using SVR and LMKNN for Stock Price Prediction,†in 2022 IEEE International Conference of Computer Science and Information Technology (ICOSNIKOM), IEEE, Oct. 2022, pp. 1–6. doi: 10.1109/ICOSNIKOM56551.2022.10034892.

M. P. Geetha and D. Karthika Renuka, “Improving the performance of aspect based sentiment analysis using fine-tuned Bert Base Uncased model,†International Journal of Intelligent Networks, vol. 2, pp. 64–69, 2021, doi: 10.1016/j.ijin.2021.06.005.

F. M. Sinaga, S. J. Pipin, S. Winardi, K. M. Tarigan, and A. P. Brahmana, “Analyzing Sentiment with Self-Organizing Map and Long Short-Term Memory Algorithms,†MATRIK : Jurnal Manajemen, Teknik Informatika dan Rekayasa Komputer, vol. 23, no. 1, pp. 131–142, Nov. 2023, doi: 10.30812/matrik.v23i1.3332.




DOI: https://doi.org/10.30865/mib.v8i3.7861

Refbacks

  • There are currently no refbacks.


Copyright (c) 2024 JURNAL MEDIA INFORMATIKA BUDIDARMA

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.


JURNAL MEDIA INFORMATIKA BUDIDARMA
Universitas Budi Darma
Secretariat: Sisingamangaraja No. 338 Telp 061-7875998
Email: mib.stmikbd@gmail.com

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.