Edge-Based Person Detection Using MobileNetV2 on ESP32-CAM for Home Surveillance System

Adi Purnama; Indriani; Bagus Alit Prasetyo; Agitama Nugraha

doi:10.30865/json.v7i3.9619

Authors

Adi Purnama Universitas Widyatama
Indriani Universitas Widyatama
Bagus Alit Prasetyo Universitas Widyatama
Agitama Nugraha Universitas Widyatama

DOI:

https://doi.org/10.30865/json.v7i3.9619

Keywords:

MobileNetV2, ESP32-CAM, Person Detection, MQTT, Home Surveillance System

Abstract

Affordable home surveillance systems increasingly require on-device intelligence to mitigate privacy risks, network dependency, and response latency associated with cloud-based video analytics. This research develops an edge-based person detection system utilizing a quantized MobileNetV2 architecture (α = 0.1) deployed on an ESP32-CAM module, integrated with the Message Queuing Telemetry Transport (MQTT) protocol for real-time alert delivery. To construct the dataset, image data were collected via a motion-triggered acquisition setup employing a passive infrared (PIR) sensor, resulting in a total of 500 manually labeled images categorized into person and non-person classes. The model was trained in over 40 epochs with a learning rate of 0.001, utilizing data augmentation and INT8 quantization to optimize embedded deployment. Performance was evaluated using a 23% testing split (116 unseen images), yielding an overall accuracy of 78.45%. For the person class, the model achieved 90.00% precision, 77.78% recall, and an 83.44% F1-score. On-device deployment via TensorFlow Lite required a peak RAM of 485.4 KB and 102.1 KB of flash memory. The average inference time was recorded at 1018 ms per frame, which limits continuous high-framerate processing but remains feasible for basic surveillance at approximately 1 frame per second. Finally, MQTT communication via an EMQX broker successfully transmitted detection alerts and image links to a mobile application for real-time monitoring and storage.

References

L. Sugiharti, M. A. Esquivias, M. S. Shaari, L. Agustin, and H. Rohmawati, “Criminality and Income Inequality in Indonesia,” Soc. Sci., vol. 11, no. 3, p. 142, Mar. 2022, doi: 10.3390/socsci11030142.

I. Ikhsan and A. Amri, “Exploration of macroeconomic effects on criminality in Indonesia,” Cogent Soc. Sci., vol. 9, no. 1, Dec. 2023, doi: 10.1080/23311886.2023.2206678.

G. Vardakis, G. Hatzivasilis, E. Koutsaki, and N. Papadakis, “Review of Smart-Home Security Using the Internet of Things,” Aug. 22, 2024, Multidisciplinary Digital Publishing Institute. doi: 10.3390/electronics13163343.

A. A. Aldridge, “Examining the security essences of Internet of Things (IoT) devices in smart homes: challenges, vulnerabilities, and countermeasures,” Issues Inf. Syst., vol. 25, no. 1, pp. 279–292, Jan. 2024, doi: 10.48009/1_iis_2024_123.

I. Cahyo Utomo, “Evaluasi Kerentanan Keamanan Pada Perangkat Iot: Studi Kasus Pada Smart home,” Indones. J. Comput. Sci., vol. 13, no. 3, pp. 4611–4625, Jun. 2024, doi: 10.33022/ijcs.v13i3.3994.

A. C. Cob-Parro, C. Losada-Gutiérrez, M. Marrón-Romera, A. Gardel-Vicente, and I. Bravo-Muñoz, “Smart video surveillance system based on edge computing,” Sensors, vol. 21, no. 9, p. 2958, Apr. 2021, doi: 10.3390/s21092958.

I. Rodriguez-Conde, C. Campos, and F. Fdez-Riverola, “On-device object detection for more efficient and privacy-compliant visual perception in context-aware systems,” Oct. 02, 2021, Multidisciplinary Digital Publishing Institute. doi: 10.3390/app11199173.

K. Dong, C. Zhou, Y. Ruan, and Y. Li, “MobileNetV2 Model for Image Classification,” in Proceedings - 2020 2nd International Conference on Information Technology and Computer Application, ITCA 2020, Institute of Electrical and Electronics Engineers Inc., Dec. 2020, pp. 476–480. doi: 10.1109/ITCA52113.2020.00106.

T. Singh, “Optimizing Neural Networks for Real-Time Object Detection Using Edge Computing and AI,” Oct. 2024. doi: 10.2139/SSRN.5018415.

Y. Xu, T. M. Khan, Y. Song, and E. Meijering, “Edge deep learning in computer vision and medical diagnostics: a comprehensive survey,” Artif. Intell. Rev., vol. 58, no. 93, pp. 1–78, Jan. 2025, doi: 10.1007/s10462-024-11033-5.

R. Kaur and S. Singh, “A comprehensive review of object detection with deep learning,” Jan. 01, 2022, Academic Press. doi: 10.1016/j.dsp.2022.103812.

M. A. Abu Talib, S. Setumin, A. I. Che Ani, and S. J. Abu Bakar, “An Analysis of Lightweight Convolutional Neural Network Models for Image Classification Task on Edge Device,” in 15th IEEE International Conference on Control System, Computing and Engineering, ICCSCE 2025 - Conference Proceedings, Institute of Electrical and Electronics Engineers Inc., 2025, pp. 175–180. doi: 10.1109/ICCSCE65566.2025.11182642.

L. Zhao and L. Wang, “A new lightweight network based on MobileNetV3,” KSII Trans. Internet Inf. Syst., vol. 16, no. 1, pp. 1–15, 2022, doi: 10.3837/tiis.2022.01.001.

R. Kallimani, K. Pai, P. Raghuwanshi, S. Iyer, and O. L. A. López, “TinyML: Tools, applications, challenges, and future research directions,” Multimed. Tools Appl., vol. 83, no. 10, pp. 29015–29045, Sep. 2024, doi: 10.1007/s11042-023-16740-9.

Y. A. Soliman, A. S. Ghoneim, and M. M. Elkhouly, “A Comprehensive Systematic Review of TinyML for Person Detection Systems,” IAENG Int. J. Comput. Sci., vol. 52, no. 11, pp. 4074–4086, 2025.

Y. H. Chang, F. C. Wu, and H. W. Lin, “Design and Implementation of ESP32-Based Edge Computing for Object Detection,” Sensors, vol. 25, no. 6, p. 1656, 2025, doi: 10.3390/s25061656.

H. Lokhande and S. R. Ganorkar, “Object detection in video surveillance using MobileNetV2 on resource-constrained low-power edge devices,” Bull. Electr. Eng. Informatics, vol. 14, no. 1, pp. 357–365, Feb. 2025, doi: 10.11591/eei.v14i1.8131.

D. Dovhal, N. Myronova, and A. Parkhomenko, “Research on Object Recognition Approaches for Mobile Platforms with Limited Resources,” in CEUR Workshop Proceedings, 2025, pp. 142–154.

A. Musa, H. A. Kakudi, M. Hassan, M. Hamada, U. Umar, and M. L. Salisu, “Lightweight Deep Learning Models For Edge Devices—A Survey,” Int. J. Comput. Inf. Syst. Ind. Manag. Appl., vol. 17, pp. 189–206, Jan. 2025, doi: 10.70917/2025014.

K. Dokic, “Microcontrollers on the edge – is esp32 with camera ready for machine learning?,” in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Springer, 2020, pp. 213–220. doi: 10.1007/978-3-030-51935-3_23.

B. Motamedi and B. Villányi, “A Reliable Publish–Subscribe Mechanism for Internet of Things-Enabled Smart Greenhouses,” Appl. Sci., vol. 14, no. 15, p. 6407, Jul. 2024, doi: 10.3390/app14156407.

M. Colaço Júnior, R. Cruz, L. Araújo, A. Bliacheriene, and F. Nunes, “Evaluation of a process for the Experimental Development of Data Mining, AI and Data Science applications aligned with the Strategic Planning,” J. Inf. Syst. Technol. Manag., vol. 19, pp. 1–31, Nov. 2022, doi: 10.4301/s1807-1775202219018.

A. Ghanad, “An Overview of Quantitative Research Methods,” Int. J. Multidiscip. Res. Anal., vol. 06, no. 08, pp. 3794–3803, Aug. 2023, doi: 10.47191/ijmra/v6-i8-52.

J. W. . Creswell and J. D. Creswell, Research design : Qualitative, Quantitative, and Mixed Methods Approaches, 6th ed. SAGE Publications, Inc., 2022.

S. T. Nowroz, N. M. Saleh, S. Shakur, S. Banerjee, and F. Amsaad, “A Benchmark Reference for ESP32-CAM Module,” May 2025, Accessed: Jan. 13, 2026. [Online]. Available: https://arxiv.org/pdf/2505.24081

A. Atturoybi, M. F. Riadi, H. B. Jatiyoso, and A. Mardamsyah, “Development of PIR sensor-based security system and IoT-based esp-32 wrover cam module for monitoring military headquarters and vital objects,” J. Mandiri IT, vol. 14, no. 1, pp. 191–197, Jul. 2025, doi: 10.35335/MANDIRI.V14I1.437.

M. Kashyap, A. K. Dev, and V. Sharma, “Implementation and analysis of EMQX broker for MQTT protocol in the Internet of Things,” e-Prime - Adv. Electr. Eng. Electron. Energy, vol. 10, p. 100846, Dec. 2024, doi: 10.1016/j.prime.2024.100846.

I. D. Tirta, A. Wisaksono, A. Ahfas, and J. Jamaaluddin, “Home Surveillance Monitoring with Esp32-Cam and SD Card For Data Storage,” J. Comput. Networks, Archit. High Perform. Comput., vol. 6, no. 1, pp. 419–429, Jan. 2024, doi: 10.47709/cnahpc.v6i1.3498.

G. Mezzadri, T. Laloë, F. Mathy, and P. Reynaud-Bouret, “Hold-out strategy for selecting learning models: Application to categorization subjected to presentation orders,” J. Math. Psychol., vol. 109, p. 102691, Aug. 2022, doi: 10.1016/j.jmp.2022.102691.

S. Brockmann and T. Schlippe, “Optimizing Convolutional Neural Networks for Image Classification on Resource-Constrained Microcontroller Units,” Computers, vol. 13, no. 7, p. 173, Jul. 2024, doi: 10.3390/computers13070173.

M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L. C. Chen, “MobileNetV2: Inverted Residuals and Linear Bottlenecks,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2018, pp. 4510–4520. doi: 10.1109/CVPR.2018.00474.

Y. Zhou, S. Chen, Y. Wang, and W. Huan, “Review of research on lightweight convolutional neural networks,” in Proceedings of 2020 IEEE 5th Information Technology and Mechatronics Engineering Conference, ITOEC 2020, Institute of Electrical and Electronics Engineers Inc., Jun. 2020, pp. 1713–1720. doi: 10.1109/ITOEC49072.2020.9141847.

G. N. Mamtha, S. Sharma, and N. Sing, “Embedded Machine Learning with Tensorflow Lite Micro,” in 2023 International Conference on Power Energy, Environment and Intelligent Control, PEEIC 2023, Institute of Electrical and Electronics Engineers Inc., 2023, pp. 1480–1483. doi: 10.1109/PEEIC59336.2023.10451233.

K. T. M. Tran, A. X. Pham, N. P. Nguyen, and P. T. Dang, “Analysis and Performance Comparison of IoT Message Transfer Protocols Applying in Real Photovoltaic System,” Int. J. Networked Distrib. Comput., vol. 12, no. 1, pp. 131–143, Jun. 2024, doi: 10.1007/s44227-024-00021-4.

S. Raschka, Y. Liu, and V. Mirjalili, Machine learning with PyTorch and Scikit-Learn. Packt Publishing, Limited, 2022.