Implementasi Convolutional Neural Network dan Large Language Model untuk Klasifikasi dan Deskripsi Pengenalan Pola Crochet

Hernanda Lilih Kusumaningtyas; Dedi Gunawan

doi:10.30865/json.v7i4.9699

Authors

Hernanda Lilih Kusumaningtyas Universitas Muhammadiyah Surakarta
Dedi Gunawan Universitas Muhammadiyah Surakarta

DOI:

https://doi.org/10.30865/json.v7i4.9699

Keywords:

Pengenalan Pola, Convolutional Neural Network, Large Language Model, Computer Vision, Pengenalan Pola Crochet

Abstract

Crochet merupakan kerajinan tekstil dengan beragam pola yang sulit diidentifikasi secara manual karena memerlukan pengalaman dan waktu. Penelitian ini bertujuan mengembangkan sistem klasifikasi pola crochet berbasis Convolutional Neural Network (CNN) serta mengintegrasikannya dengan Large Language Model (LLM) untuk menghasilkan deskripsi pola secara otomatis dalam bahasa Indonesia. Dataset yang digunakan terdiri dari 830 citra yang dibagi ke dalam delapan kelas. Model CNN digunakan untuk ekstraksi fitur dan klasifikasi citra, kemudian hasil prediksi diteruskan ke LLM untuk menghasilkan deskripsi tekstual. Evaluasi dilakukan menggunakan metrik accuracy, precision, recall, F1-score, dan confusion matrix. Hasil pengujian menunjukkan bahwa model mencapai akurasi sebesar 93% dengan nilai macro average dan weighted average sebesar 0,93, yang mengindikasikan performa yang seimbang pada seluruh kelas, meskipun masih terdapat kesalahan pada pola dengan kemiripan tekstur. Kontribusi utama penelitian ini terletak pada integrasi CNN dan LLM dalam satu sistem multimodal yang tidak hanya menghasilkan klasifikasi, tetapi juga deskripsi pola secara otomatis. Sistem diimplementasikan dalam aplikasi berbasis web yang memungkinkan pengguna memperoleh hasil klasifikasi, confidence score, dan deskripsi secara langsung. Pendekatan ini diharapkan dapat meningkatkan efisiensi proses identifikasi pola crochet serta mendukung pengembangan sistem multimodal pada domain tekstil.

References

[1] E. Ji, “Introduction to the Application of Hand Crochet in Tweed Clothing — An Example of the Design of the ’ White Velvet Shiyun Shang ’,” J. Theory Pract. Humanit. Sci., vol. 1, no. 5, pp. 13–19, 2024, [Online]. Available: https://woodyinternational.com/index.php/jtphss/article/view/78

[2] H. Qu, Y. Zhou, P. Y. Mok, G. Flatz, and L. Li, “Efficient and Effective Detection of Repeated Pattern from Fronto-Parallel Images with Unknown Visual Contents,” Signals, vol. 6, no. 1, p. 4, 2025, doi: 10.3390/signals6010004.

[3] T. Mufizar, A. Sudiarjo, E. D. S. Mulyani, A. A. Wakih, M. A. Kasyfurrahman, and L. A. Mahbub, “Classification of Tasikmalaya batik motifs using convolutional neural networks,” IAES Int. J. Artif. Intell., vol. 14, no. 4, pp. 3287–3299, 2025, doi: 10.11591/ijai.v14.i4.pp3287-3299.

[4] X. Zhao, L. Wang, Y. Zhang, X. Han, M. Deveci, and M. Parmar, “A review of convolutional neural networks in computer vision,” Artif. Intell. Rev., vol. 57, no. 4, p. 99, 2024, doi: 10.1007/s10462-024-10721-6.

[5] M. M. Taye, “Theoretical Understanding of Convolutional Neural Network: Concepts, Architectures, Applications, Future Directions,” Computation, vol. 11, no. 3, p. 52, 2023, doi: 10.3390/computation11030052.

[6] I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning. MIT Press, 2016. [Online]. Available: http://www.deeplearningbook.org

[7] I. Hermawan, M. Agustin, and D. Arnaldy, “Rice Seedling Image Classification Using Light Convolutional Neural Network,” Khazanah Inform. J. Ilmu Komput. dan Inform., vol. 9, no. 2, pp. 111–119, 2023, doi: 10.23917/khif.v9i2.20401.

[8] P. Dabbo and F. Y. Bisilisin, “Klasifikasi Motif Kain Tenun Sabu Raijua Menggunakan Convolutional Neural Network (CNN) Berbasis Citra,” KETIK J. Inform., vol. 1, no. 6, pp. 11–18, 2024, doi: 10.70404/ketik.v1i06.88.

[9] H. Herwinsyah and D. Yuswanto Jaya, “Accuracy potential of the Convolutional Neural Network (CNN) in recognizing traditional clothing,” IT J. Res. Dev., vol. 8, no. 2, pp. 95–106, 2023, doi: 10.25299/itjrd.2023.12690.

[10] Y. Rizki, R. M. Taufiq, H. Mukhtar, and D. Putri, “Klasifikasi Pola Kain Tenun Melayu Menggunakan Faster R-CNN,” IT J. Res. Dev., vol. 5, no. 2, pp. 215–225, 2021, doi: 10.25299/itjrd.2021.vol5(2).5831.

[11] S. S. Mohammed, H. G. Clarke, and S. N. Mahmood, “A fully convolutional neural network for fast detection, classification, and segmentation of fabric defects,” Neural Comput. Appl., vol. 37, no. 28, pp. 23249–23272, 2025, doi: 10.1007/s00521-025-11495-w.

[12] L. Tan, Q. Fu, and J. Li, “An Improved Neural Network Model Based on DenseNet for Fabric Texture Recognition,” Sensors, vol. 24, no. 23, p. 7758, 2024, doi: 10.3390/s24237758.

[13] Z. Marciniak, P. Lertjaturaphat, and A. Bianchi, “StitchFlow: Enabling In-Situ Creative Explorations of Crochet Patterns with Stitch Tracking and Process Sharing,” in UIST 2025 - Proceedings of the 38th Annual ACM Symposium on User Interface Software and Technology, Association for Computing Machinery, 2025. doi: 10.1145/3746059.3747715.

[14] K. Seitz, P. Rein, J. Lincke, and R. Hirschfeld, “Digital Crochet: Toward a Visual Language for Pattern Description,” in Proceedings of the 2022 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software, Association for Computing Machinery, 2022, pp. 48–62. doi: 10.1145/3563835.3567657.

[15] G. Singh, “Vision–Language Foundation Models and Multimodal Large Language Models: A Comprehensive Survey of Architectures, Benchmarks, and Open Challenges,” 2026. doi: 10.20944/preprints202602.0467.v1.

[16] Z. Li, X. Wu, H. Du, F. Liu, H. Nghiem, and G. Shi, “A Survey of State of the Art Large Vision Language Models: Alignment, Benchmark, Evaluations and Challenges,” in 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2025, pp. 1578–1597. doi: 10.1109/CVPRW67362.2025.00147.

[17] J. Li, D. Li, C. Xiong, and S. Hoi, “BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation,” 2022, doi: 10.48550/arXiv.2201.12086.

[18] Z. Gan, L. Li, C. Li, L. Wang, Z. Liu, and J. Gao, “Vision-Language Pre-Training: Basics, Recent Advances, and Future Trends,” Found. Trends Comput. Graph. Vis., vol. 14, no. 3–4, pp. 163–352, 2022, doi: 10.1561/0600000105.

[19] J. Li, D. Li, S. Savarese, and S. Hoi, “BLIP-2: bootstrapping language-image pre-training with frozen image encoders and large language models,” in Proceedings of the 40th International Conference on Machine Learning, Honolulu, Hawaii, USA: JMLR.org, 2023, pp. 19730–19742. [Online]. Available: https://dl.acm.org/doi/10.5555/3618408.3619222

[20] V. R. Joseph, “Optimal ratio for data splitting,” Stat. Anal. Data Min. ASA Data Sci. J., vol. 15, no. 4, pp. 531–538, 2022, doi: 10.1002/sam.11583.

[21] I. Wirabowo and I. Susilawati, “Implementasi Convolution Neural Network (CNN) untuk Deteksi Penyakit pada Daun Jagung Berbasis Citra Digital,” J. Pustaka Data, vol. 5, no. 1, pp. 233–241, 2025, doi: 10.55382/jurnalpustakadata.v5i1.1046.

[22] H. Imaduddin, W. Widayat, and M. S. Adhantoro, “Transfer Learning Architectures for Breast Cancer Detection in Mammographic Image Data,” J. Theor. Appl. Inf. Technol., vol. 103, no. 20, pp. 8336–8347, 2025.

[23] K. Alomar, H. I. Aysel, and X. Cai, “Data Augmentation in Classification and Segmentation: A Survey and New Strategies,” J. Imaging, vol. 9, no. 2, p. 46, 2023, doi: 10.3390/jimaging9020046.

[24] M. H. Reza et al., “A comprehensive review of convolutional neural networks: foundations, enhancements and applications,” Neural Comput. Appl., vol. 38, no. 4, p. 56, 2026, doi: 10.1007/s00521-025-11827-w.

[25] A. Puspitasari, D. S. Salsabila, and D. Roliawati, “Penerapan ResNet50-CNN untuk Optimalisasi Klasifikasi pada Data Fashion,” Indones. J. Data Sci., vol. 3, no. 1, pp. 1–12, 2025, doi: 10.30989/ijds.v3i1.1533.

[26] D. R. Fauzi and G. A. Haqdu D, “Comparison of CNN Models Using EfficientNetB0, MobileNetV2, and ResNet50 for Traffic Density with Transfer Learning,” J. Intell. Syst. Technol. Informatics, vol. 1, no. 1, pp. 22–30, 2025, doi: 10.64878/jistics.v1i1.6.

[27] H. Imaduddin, F. Y. A’la, A. Fatmawati, and B. A. Hermansyah, “Comparison of transfer learning method for COVID-19 detection using convolution neural network,” Bull. Electr. Eng. Informatics, vol. 11, no. 2, pp. 1091–1099, 2022, doi: 10.11591/eei.v11i2.3525.

[28] R. M. Schmidt, F. Schneider, and P. Hennig, “Descending through a Crowded Valley - Benchmarking Deep Learning Optimizers,” in Proceedings of the 38th International Conference on Machine Learning, PMLR, 2021, pp. 9367–9376. [Online]. Available: https://proceedings.mlr.press/v139/schmidt21a.html

[29] I. Kandel and M. Castelli, “The effect of batch size on the generalizability of the convolutional neural networks on a histopathology dataset,” ICT Express, vol. 6, no. 4, pp. 312–315, 2020, doi: 10.1016/j.icte.2020.04.010.

[30] K. D. Reddy and A. Patil, “CXR-MultiTaskNet a unified deep learning framework for joint disease localization and classification in chest radiographs,” Sci. Rep., vol. 15, no. 1, p. 32022, 2025, doi: 10.1038/s41598-025-16669-z.

[31] A. Zheng, Evaluating Machine Learning Models. O’Reilly Media, 2015.

[32] I. D. Mienye, N. Jere, G. Obaido, O. O. Ogunruku, E. Esenogho, and C. Modisane, “Large language models: an overview of foundational architectures, recent trends, and a new taxonomy,” Discov. Appl. Sci., vol. 7, no. 9, p. 1027, 2025, doi: 10.1007/s42452-025-07668-w.

[33] J. Bai et al., “Qwen Technical Report,” 2023. doi: 10.48550/arXiv.2309.16609.