Optimization of Sentiment Analysis Classification of ChatGPT on Big Data Twitter in Indonesia using BERT

Authors

  • Frans Mikael Sinaga Universitas Mikroskil, Medan
  • Ronsen Purba Universitas Mikroskil, Medan
  • Sio Jurnalis Pipin Universitas Mikroskil, Medan http://orcid.org/0009-0004-6703-6662
  • Wulan Sri Lestari Universitas Mikroskil, Medan
  • Sunaryo Winardi Universitas Mikroskil, Medan

DOI:

https://doi.org/10.30865/mib.v8i3.7861

Keywords:

Sentiment Analysis, ChatGPT, BigData, BERT

Abstract

This research is grounded in the emergence of ChatGPT technology, supported by prior and similar studies. The urgency of the issue is highlighted by previous research indicating non-convergent classification outcomes in LSTM (Long Short-Term Memory) methods due to suboptimal hyperparameter settings and limitations in understanding text data within Big Data. The presence of ChatGPT technology brings both benefits and potential misuse, such as copyright infringement, unauthorized news extraction, and violations of accountability principles. Understanding public sentiment towards the presence of ChatGPT technology is crucial. The research aims to implement the BERT (Bidirectional Encoder Representations from Transformers) method to achieve accurate and convergent sentiment analysis classification. This study involves data preprocessing stages using Natural Language Processing (NLP) techniques. Text data, already vectorized, is classified using BERT to determine public sentiment (positive, negative, neutral) towards ChatGPT technology, ensuring greater accuracy, convergence, and contextual relevance. Performance testing of the BERT model is conducted using a Confusion Matrix. With parameters set to Max Sequence Length = 128 and Batch Size = 16, the highest classification accuracy achieved is 93.4%.

Author Biography

Sio Jurnalis Pipin, Universitas Mikroskil, Medan

Department of information technology

References

L. Marron, “Exploring the Potential of ChatGPT 3.5 in Higher Education,†2023, pp. 326–349. doi: 10.4018/978-1-6684-8292-6.ch017.

F. W. Putra, I. B. Rangka, S. Aminah, and M. H. R. Aditama, “ChatGPT in the higher education environment: perspectives from the theory of high order thinking skills,†J Public Health (Bangkok), vol. 45, no. 4, pp. e840–e841, Nov. 2023, doi: 10.1093/pubmed/fdad120.

T. Adiguzel, M. H. Kaya, and F. K. Cansu, “Revolutionizing education with AI: Exploring the transformative potential of ChatGPT,†Contemp Educ Technol, vol. 15, no. 3, p. ep429, Jul. 2023, doi: 10.30935/cedtech/13152.

F. M. Sinaga, S. Winardi, S. J. Pipin, and M. N. Hakim, “Sentiment Analysis Classification of ChatGPT on Twitter Big Data in Indonesia Using Fast R-CNN,†JURNAL MEDIA INFORMATIKA BUDIDARMA, vol. 7, no. 4, pp. 2137–2148, 2023.

H. Rehioui and A. Idrissi, “New Clustering Algorithms for Twitter Sentiment Analysis,†IEEE Syst J, vol. 14, no. 1, pp. 530–537, Mar. 2020, doi: 10.1109/JSYST.2019.2912759.

S. J. Pipin and H. Kurniawan, “Analisis Sentimen Kebijakan MBKM Berdasarkan Opini Masyarakat di Twitter Menggunakan LSTM,†Jurnal SIFO Mikroskil, vol. 23, no. 2, pp. 197–208, 2022, doi: https://doi.org/10.55601/jsm.v23i2.900.

M. Pota, M. Ventura, R. Catelli, and M. Esposito, “An Effective BERT-Based Pipeline for Twitter Sentiment Analysis: A Case Study in Italian,†Sensors, vol. 21, no. 1, p. 133, Dec. 2020, doi: 10.3390/s21010133.

Y. Akbar and T. Sugiharto, “Analisis Sentimen Pengguna Twitter di Indonesia Terhadap ChatGPT Menggunakan Algoritma C4.5 dan Naïve Bayes,†Jurnal Sains dan Teknologi, vol. 5, no. 1, pp. 115–122, 2023, doi: 10.55338/saintek.v4i3.1368.

T. Tang, X. Tang, and T. Yuan, “Fine-Tuning BERT for Multi-Label Sentiment Analysis in Unbalanced Code-Switching Text,†IEEE Access, vol. 8, pp. 193248–193256, 2020, doi: 10.1109/ACCESS.2020.3030468.

M. Pota, M. Ventura, H. Fujita, and M. Esposito, “Multilingual evaluation of pre-processing for BERT-based sentiment analysis of tweets,†Expert Syst Appl, vol. 181, p. 115119, Nov. 2021, doi: 10.1016/j.eswa.2021.115119.

A. Bello, S.-C. Ng, and M.-F. Leung, “A BERT Framework to Sentiment Analysis of Tweets,†Sensors, vol. 23, no. 1, p. 506, Jan. 2023, doi: 10.3390/s23010506.

A. Zhao and Y. Yu, “Knowledge-enabled BERT for aspect-based sentiment analysis,†Knowl Based Syst, vol. 227, p. 107220, Sep. 2021, doi: 10.1016/j.knosys.2021.107220.

A. M. Bagirov, R. M. Aliguliyev, and N. Sultanova, “Finding compact and well-separated clusters: Clustering using silhouette coefficients,†Pattern Recognit, vol. 135, p. 109144, Mar. 2023, doi: 10.1016/j.patcog.2022.109144.

K. Chen, Y. Zhou, and F. Dai, “A LSTM-based method for stock returns prediction : A case study of China stock market,†May 2015, [Online]. Available: http://arxiv.org/abs/1506.00019

L. N. Pradany and C. Fatichah, “ANALISA SENTIMEN KEBIJAKAN PEMERINTAH PADA KONTEN TWITTER BERBAHASA INDONESIA MENGGUNAKAN SVM DAN K-MEDOID CLUSTERINGâ€.

F. Mikael Sinaga, P. Sirait, and A. Halim, “Optimization of SV-kNNC using Silhouette Coefficient and LMKNN for Stock Price Prediction,†in 2020 3rd International Seminar on Research of Information Technology and Intelligent Systems (ISRITI), IEEE, Dec. 2020, pp. 326–331. doi: 10.1109/ISRITI51436.2020.9315516.

L. Yang, Y. Li, J. Wang, and R. S. Sherratt, “Sentiment Analysis for E-Commerce Product Reviews in Chinese Based on Sentiment Lexicon and Deep Learning,†IEEE, vol. 8, pp. 23522–23530, 2020, doi: 10.1109/ACCESS.2020.2969854.

F. M. Sinaga, S. Winardi, and Gunawan, “3SV-KNNC Optimization using SVR and LMKNN for Stock Price Prediction,†in 2022 IEEE International Conference of Computer Science and Information Technology (ICOSNIKOM), IEEE, Oct. 2022, pp. 1–6. doi: 10.1109/ICOSNIKOM56551.2022.10034892.

M. P. Geetha and D. Karthika Renuka, “Improving the performance of aspect based sentiment analysis using fine-tuned Bert Base Uncased model,†International Journal of Intelligent Networks, vol. 2, pp. 64–69, 2021, doi: 10.1016/j.ijin.2021.06.005.

F. M. Sinaga, S. J. Pipin, S. Winardi, K. M. Tarigan, and A. P. Brahmana, “Analyzing Sentiment with Self-Organizing Map and Long Short-Term Memory Algorithms,†MATRIK : Jurnal Manajemen, Teknik Informatika dan Rekayasa Komputer, vol. 23, no. 1, pp. 131–142, Nov. 2023, doi: 10.30812/matrik.v23i1.3332.

Downloads

Published

2024-07-27

Issue

Vol. 8 No. 3 (2024): Juli 2024

Section

Articles

License

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License

Authors who publish with this journal agree to the following terms:

  1. Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under Creative Commons Attribution 4.0 International License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
  2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
  3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (Refer to The Effect of Open Access).