Implementasi Algoritma Transformers BART dan Penggunaan Metode Optimasi Adam Untuk Klasifikasi Judul Berita Palsu
DOI:
https://doi.org/10.30865/mib.v8i3.7852Keywords:
Bart, Adam, Text, Classification, TransformersAbstract
Classification is a process of identifying new data provided based on validation of previous data. One classification process that can be used is fake news classification. The classification process requires as little time as possible to get maximum results, so a faster method is needed to classify news. The BART algorithm can be a method that can be used to carry out classification and use Adam optimization to improve the performance of the algorithm. The aim of this research is to classify fake news, whether the BART algorithm and Adam optimization are able to provide good results and to label whether the news is fake or not. The results of this process are based on the use of a dataset of 65% for training, 30% for validation, and 5% to produce 2 BART models. With the additional use of Adam optimization and several other parameters for the training process, the first model was able to provide accuracy performance of 92.88%, training loss reached 12.2%, and validation loss reached 28.4% and the second model produced an accuracy of 92.63 %, training loss 15% and validation loss reaching 20.2%. In the first model, it can predict 105 data labeled negative and 1306 positive data. Meanwhile, the second model was able to predict 128 data labeled negative and 1283 positive data.
References
R. Yunanto, A. P. Purfini, dan A. Prabuwisesa, “Jurnal Manajemen Informatika (JAMIKA) Survei Literatur: Deteksi Berita Palsu Menggunakan Pendekatan Deep Learning,†Jurnal Manajemen Informatika (JAMIKA), vol. 11, no. 2, hlm. 118–130, 2021, doi: 10.34010/jamika.v11i2.493.
T. A. Roshinta, E. Kumala, dan I. F. Dinata, “Sistem Deteksi Berita Hoax Berbahasa Indonesia Bidang Kesehatan,†remik, vol. 7, no. 2, hlm. 1167–1173, Apr 2023, doi: 10.33395/remik.v7i2.12369.
M. Lewis dkk., “BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension,†ArXiv, Okt 2019, [Daring]. Tersedia pada: http://arxiv.org/abs/1910.13461
Y. Yu dkk., “Diverse Image Inpainting with Bidirectional and Autoregressive Transformers,†ArXiv, Apr 2021, [Daring]. Tersedia pada: http://arxiv.org/abs/2104.12335
D. P. Kingma dan J. Ba, “Adam: A Method for Stochastic Optimization,†ArXiv, Des 2014, [Daring]. Tersedia pada: http://arxiv.org/abs/1412.6980
R. K. Putri dan M. Athoillah, “SUPPORT VECTOR MACHINE UNTUK IDENTIFIKASI BERITA HOAX TERKAIT VIRUS CORONA (COVID-19),†JPIT, vol. 6, no. 3, hlm. 162–167, 2021, doi: 10.30591/jpit.v6i3.
W. Afandi, S. N. Saputro, A. M. Kusumaningrum, H. Ardiansyah, M. H. Kafabi, dan S. Sudianto, “Klasifikasi Judul Berita Clickbait menggunakan RNN-LSTM,†JPIT, vol. 7, no. 2, hlm. 85–89, 2022, doi: 10.30591/jpit.v7i2.
J. Khatib Sulaiman, A. Fikri Hanif, T. Bayu Sasongko, A. Dwi Laksito, dan U. Amikom Yogyakarta, “Perbandingan Kinerja LSTM, Bi-LSTM, dan GRU pada Klasifikasi Judul Berita Clickbait,†Indonesian Journal of Computer Science Attribution, vol. 12, no. 4, hlm. 2136–2150, 2023, doi: 10.33022/ijcs.v12i4.3281.
T. H. C. Chiang, C. S. Liao, dan W. C. Wang, “Investigating the Difference of Fake News Source Credibility Recognition between ANN and BERT Algorithms in Artificial Intelligence,†Applied Sciences (Switzerland), vol. 12, no. 15, Agu 2022, doi: 10.3390/app12157725.
H. Najadat, M. Tawalbeh, dan R. Awawdeh, “Fake news detection for Arabic headlines-articles news data using deep learning,†International Journal of Electrical and Computer Engineering, vol. 12, no. 4, hlm. 3951–3959, Agu 2022, doi: 10.11591/ijece.v12i4.pp3951-3959.
I. Ketut, A. Enriko, F. Nizar Gustiyana, R. H. Putra, dan K. Kunci, “JURNAL MEDIA INFORMATIKA BUDIDARMA Komparasi Hasil Optimasi Pada Prediksi Harga Saham PT. Telkom Indonesia Menggunakan Algoritma Long Short Term Memory,†JURNAL MEDIA INFORMATIKA BUDIDARMA, 2023, doi: 10.30865/mib.v7i2.5822.
S. SHAHANE, “Fake News Classification.†Diakses: 11 Juni 2024. [Daring]. Tersedia pada: https://www.kaggle.com/datasets/saurabhshahane/fake-news-classification
T. Emmanuel, T. Maupong, D. Mpoeleng, T. Semong, B. Mphago, dan O. Tabona, “A survey on missing data in machine learning,†J Big Data, vol. 8, no. 1, Des 2021, doi: 10.1186/s40537-021-00516-9.
L. Hickman, S. Thapa, L. Tay, M. Cao, dan P. Srinivasan, “Text Preprocessing for Text Mining in Organizational Research: Review and Recommendations,†Organ Res Methods, vol. 25, no. 1, hlm. 114–146, Jan 2022, doi: 10.1177/1094428120971683.
M. Siino, I. Tinnirello, dan M. La Cascia, “Is text preprocessing still worth the time? A comparative survey on the influence of popular preprocessing methods on Transformers and traditional classifiers,†Inf Syst, vol. 121, Mar 2024, doi: 10.1016/j.is.2023.102342.
K. Maharana, S. Mondal, dan B. Nemade, “A review: Data pre-processing and data augmentation techniques,†Global Transitions Proceedings, vol. 3, no. 1, hlm. 91–99, Jun 2022, doi: 10.1016/j.gltp.2022.04.020.
R. Ghorbani dan R. Ghousi, “Comparing Different Resampling Methods in Predicting Students’ Performance Using Machine Learning Techniques,†IEEE Access, vol. 8, hlm. 67899–67911, 2020, doi: 10.1109/ACCESS.2020.2986809.
J. Petrus, Ermatita, Sukemi, dan Erwin, “An adaptable sentence segmentation based on Indonesian rules,†IAES International Journal of Artificial Intelligence, vol. 12, no. 3, hlm. 1491–1499, Sep 2023, doi: 10.11591/ijai.v12.i3.pp1491-1499.
V. Bushaev, “Adam — latest trends in deep learning optimization.†Diakses: 27 Juni 2024. [Daring]. Tersedia pada: https://towardsdatascience.com/adam-latest-trends-in-deep-learning-optimization-6be9a291375c
J. Devlin, M.-W. Chang, K. Lee, dan K. Toutanova, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,†ArXiv, Okt 2018, [Daring]. Tersedia pada: http://arxiv.org/abs/1810.04805
A. R. Openai, K. N. Openai, T. S. Openai, dan I. S. Openai, “Improving Language Understanding by Generative Pre-Training.†Diakses: 13 Juni 2024. [Daring]. Tersedia pada: https://gluebenchmark.com/leaderboard
A. Vaswani dkk., “Attention Is All You Need,†ArXiv, Jun 2017, [Daring]. Tersedia pada: http://arxiv.org/abs/1706.03762
V. Dendi Yunanda dan N. Hendrastuty, “JURNAL MEDIA INFORMATIKA BUDIDARMA Perbandingan Kernel Polynomial dan RBF Pada Algoritma SVM Untuk Analisis Sentimen Skincare di Indonesia,†JURNAL MEDIA INFORMATIKA BUDIDARMA, vol. 8, no. 2, hlm. 726–735, 2024, doi: 10.30865/mib.v8i2.7425.
I. Verawati dan S. N. Jaelani, “JURNAL MEDIA INFORMATIKA BUDIDARMA Analisis Sentimen Pengguna Twitter Terhadap Bus Listrik Menggunakan Naïve Bayes,†JURNAL MEDIA INFORMATIKA BUDIDARMA, vol. 8, no. 2, hlm. 832–842, 2024, doi: 10.30865/mib.v8i2.7030.
V. Zouhar dkk., “A Formal Perspective on Byte-Pair Encoding,†ArXiv, Jun 2023, [Daring]. Tersedia pada: http://arxiv.org/abs/2306.16837
Downloads
Published
Issue
Section
License
This work is licensed under a Creative Commons Attribution 4.0 International License
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under Creative Commons Attribution 4.0 International License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (Refer to The Effect of Open Access).
