Augmented Reality in the Internet of Things (AR + IoT): A Review

 (*)Aulia Akhrian Syahidi Mail (Politeknik Negeri Banjarmasin, Banjarmasin, Indonesia)
 Kohei Arai (Saga University, Saga, Japan)
 Herman Tolle (Universitas Brawijaya, Malang, Indonesia)
 Ahmad Afif Supianto (Indonesian Institute of Sciences, Bandung, Indonesia)
 Kiyoshi Kiyokawa (Nara Institute of Science and Technology, Nara, Japan)

(*) Corresponding Author

Submitted: October 4, 2021; Published: November 30, 2021

Abstract

Augmented Reality (AR) and Internet of Things (IoT) devices can converge, we call it AR + IoT. The integration of AR and IoT has promising opportunities for future product research and development. IoT is a tool for communicating, monitoring, and remotely controlling with connected products, while AR is a means to see and find out what is happening with connected products based on visualization/interface based on additional information. The mapping review method was used to carry out this research. The result is that as many as 43 papers from 2014-2020 are the main targets of review. The largest publication database is in the IEEE Xplore Digital Library, the application dominates in the field of social life, the research objectives are varied, and the recommended future work is to maximize the potential for integration between AR and IoT or even with other promising devices. Then comfort, convenience, and user experience are also recommended in implementing AR + IoT

Keywords


Advanced Mobile Computing; Augmented Reality; Human-Computer Interaction; Internet of Things; Literature Review; Mapping Study; Trend and Challenges

Full Text:

PDF


Article Metrics

Abstract view : 1096 times
PDF - 639 times

References

Kompasiana, “Bagaimana Perkembangan IoT di Tahun 2020?,” 2019, Accessed on: March 20, 2020. [Online]. Available: https://www.kompasiana.com/idmetafora/5c91de4e3ba7f707c618c112/bagaimana-perkembangan-iot-di-tahun-2020?page=all.

I. U. Din, M. Guizani, S. Hassan, B.-S. Kim, M. K. Khan, M. Atiquzzaman, and S. H. Ahmed, “The Internet of Things: A Review of Enabled Technologies and Future Challenges,” IEEE Access, vol. 7, pp. 1-35, 2018.

H. Kato and M. Billinghurst, “Marker Tracking and HMD Calibration for a Video-Based Augmented Reality Conferencing System,” In Proc. of the 2nd IEEE and ACM IWAR, IEEE Xplore Digital Library, pp. 1-10, 1999.

A. A. Syahidi, H. Tolle, A. A. Supianto, and K. Arai, “BandoAR: Real-Time Text Based Detection System Using Augmented Reality for Media Translator Banjar Language to Indonesian with Smartphone,” In Proc. of the 5th IEEE ICETAS, IEEE Xplore Digital Library, pp. 1-6, 2018.

G. R. Shinde, P. S. Dhotre, P. N. Mahalle, and N. Dey, “Internet of Things Integrated Augmented Reality,” 2020, SpringerBriefs in Applied Sciences and Technology, Springer, Accessed on: July 15, 2020. [Online]. Available: https://link.springer.com/book/10.1007/978-981-15-6374-4.

S. Lanka, S. Ehsan, and A. Ehsan, “A Review of Research on Emerging technologies of the Internet of Things and Augmented Reality,” In Proc. of the Int. Conf. on I-SMAC, IEEE Xplore Digital Library, pp. 770-774, 2017.

A. Badouch, S.-D. Krit, M. Kabrane, and K. Karimi, “Augmented Reality Services Implemented within Smart Cities, based on an Internet of Things Infrastructure, Concepts, and Challenges: An Overview,” In Proc. of the 4th ICEMIS, ACM Digital Library, pp. 1-4, 2018.

D. Jo and G. J. Kim, “AR Enabled IoT for a Smart and Interactive Environment: A Survey and Future Directions,” Sensors - MDPI, vol. 19, no. 4330, pp. 1-17, 2019.

F. Elberzhager, A. Rosbach, R. Eschbachb, and J. Münch, “Reducing Test Effort: A Systematic Mapping Study on Existing Approaches,” Information and Software Technology - Elsevier, vol. 54, no. 10, pp. 1092-1106, 2012.

C. R. L. Neto, P. A. M. S. Neto, E. S. de Almeida, and S. R. L. Meira, “A Mapping Study on Software Product Lines Testing Tools,” In Proc. of the 24th Int. Conf. on SEKE, pp. 628-634, 2012.

D. Agrawal, S. B. Mane, A. Pacharne, and S. Tiwari, “IoT Based Augmented Reality System of Human Heart: An Android Application,” In Proc. of the 2nd ICOEI 2018, IEEE Xplore Digital Library, pp. 899-902, 2018.

H. Akasaki, F. Ishizawa, M. Sakamoto, and T. Nakajima, “Building a Platform Society Towards Sustainability Based on Internet-of-Things,” In Proc. of the Int. Conf. on DAPI, Springer International Publishing, pp. 235–251, 2017.

T. T Amici, P. H. Filho, and A. B. Campo, “Augmented Reality Applied to a Wireless Power Measurement System of an Industrial 4.0 Advanced Manufacturing Line,” In Proc. of the 13th IEEE Int. Conf. on INDUSCON, IEEE Xplore Digital Library, pp. 1402-1406, 2018.

M. A. Ankireddy, R. AV., R. G. M., and A. M., “Augmented Reality Rendered for IoT Applications,” In Proc. of the IEEE 16th INDICON, IEEE Xplore Digital Library, pp. 1-4, 2019.

S. Baskaran and H. K. Nagabushanam, “Relational Localization Based Augmented Reality Interface for IoT Applications,” In Proc. of the Int. Conf. on ICTC, IEEE Xplore Digital Library, pp. 103-106, 2018.

J. D. H. Bezerra and C. T. de Souza, “smAR2t: a Models at Runtime Architecture to Interact with the Web Of Things using Augmented Reality,” In Proc. of the SBES, ACM Digital Library, pp. 124-129, 2019.

F. Campana and F. Domınguez, “Ariotivity: Provisioning Smart Objects with Augmented Reality and Open Standards,” In Proc. of the IEEE 3rd ETCM, IEEE Xplore Digital Library, pp. 1-6, 2018.

Y. Cao, Z. Xu, F. Li, W. Zhong, K. Huo, and K. Ramani, “V.Ra: An In-Situ Visual Authoring System for Robot IoT Task Planning with Augmented Reality,” In Proc. of the CHI Conference, ACM Digital Library, pp. 1-6, 2019.

S. Chae, Y. Yang, H. Choi, I.-J. Kim, J. Byun, J. Jo, and T.-D. Han, “Smart Advisor: Real-Time Information Provider with Mobile Augmented Reality,” In Proc. of the IEEE ICCE, IEEE Xplore Digital Library, pp. 97-98, 2016.

I. Y. Chang, “Augmented Reality Interfaces for the Internet of Things,” In Proc. of the SIGGRAPH, ACM Digital Library, pp. 1-2, 2018.

P.-S. Chiu, J.-W. Chang, M.-C. Lee , C.-H. Chen, and D.-S. Lee, “Enabling Intelligent Environment by the Design of Emotionally Aware Virtual Assistant: A Case of Smart Campus,” IEEE Access - Special Section on Data Mining for Internet of Things, vol. 8, no. 2020, pp. 62032-62041, 2020.

H.-B. Choi, K.-W. Lim, and Y.-B. Ko, “Improved Virtual Anchor Selection for AR-assisted Sensor Positioning in Harsh Indoor Conditions,” In Proc. of the GIoTS, IEEE Xplore Digital Library, pp. 1-6, 2020.

A. Croatti and A. Ricci, “Mashing up the Physical and Augmented Reality: The Web of Augmented Things Idea,” In Proc. of the WoT Conference, ACM Digital Library, pp. 1-4, 2017.

G. White, C. Cabrera, A. Palade, and S. an Clarke, “Augmented Reality in IoT,” In Proc. of the ICSOC, International Publishing, pp. 149-160, 2019.

F. Ghorbani, M. Kia, M. Delrobaei, and Q. Rahman, “Evaluating the Possibility of Integrating Augmented Reality and Internet of Things Technologies to Help Patients with Alzheimer’s Disease,” In Proc. of the 26th National and 4th ICBME, IEEE Xplore Digital Library, pp. 139-144, 2019.

K. Huo, Y. Cao, S. H. Yoon, Z. Xu, G. Chen, and K. Ramani, “Scenariot: Spatially Mapping Smart Things Within Augmented Reality Scenes,” In Proc. of the CHI Conference, ACM Digital Library, pp. 1-13, 2018.

M. Jayaweera, I. Wijesooriya, D. Wijewardana, T. De Silva, and C. Gamage, “Enhanced Real-Time Machine Inspection with Mobile Augmented Reality for Maintenance and Repair,” In Proc. of the 2nd ACM/IEEE Int. Conf. on IoTDI, ACM Digital Library, pp. 287-288, 2017.

D. Jo and G. J. Kim, “ARIoT: Scalable Augmented Reality Framework for Interacting with Internet of Things Appliances Everywhere,” IEEE Transactions on Consumer Electronics, vol. 62, no. 3, pp. 334-340, 2016.

D. Jo and G. J. Kim, “In-Situ AR Manuals for IoT Appliances,” In Proc. of the IEEE ICCE, IEEE Xplore Digital Library, pp. 409-410, 2016.

D. Jo and G. J. Kim, “IoT + AR: Pervasive and Augmented Environments for “Digi‑Log” Shopping Experience,” Human-Centric Computing and Information Sciences, Springer International Publishing, vol. 9, no. 1, pp. 1-17, 2019.

J. A. Karasinski, R. Joyce, C. Carroll, J. Gale, and S. Hillenius, “An Augmented Reality/Internet of Things Prototype for Just-in-time Astronaut Training,” In Proc. of the Int. Conf. on VAMR, Springer International Publishing, pp. 248-260, 2017.

M. Khalil, C. Bergs, T. Papadopoulos, R. Wuchner, K.-U. Bletzinger, and M. Heizmann, “IIoT-based Fatigue Life Indication using Augmented Reality,” In Proc. of the IEEE 17th Int. Conf. on INDIN, IEEE Xplore Digital Library, pp. 749-751, 2019.

R. Khanna and V. M., “Augmented Reality Based IoT Controller,” In Proc. of the Int. Conf. on ViTECoN, IEEE Xplore Digital Library, pp. 1-5, 2019.

J. Lee, K. Lee, B. Nam, and Y. Wu, “IoT Platform-based iAR: a Prototype for Plant O&M Applications,” In Proc. of the IEEE ISMAR-Adjunct, IEEE Xplore Digital Library, pp. 149-150, 2016.

T. Leppanen, A. Heikkinen, A. Karhu, E. Harjula, J. Riekki, and T. Koskela, “Augmented Reality Web Applications with Mobile Agents in the Internet of Things,” In Proc. of the 8th Int. Conf. on NGMAST, IEEE Xplore Digital Library, pp. 54-59, 2014.

M. Makolkina, A. Koucheryavy, and A. Paramonov, “The Models of Moving Users and IoT Devices Density Investigation for Augmented Reality Applications,” In Proc. of the Internet of Things, Smart Spaces, and Next Generation Networks and Systems (ruSMART/NsCC/NEW2AN), Springer International Publishing, pp. 671–682, 2017.

A. Mishra, S. Karmakar, A. Bose, and A. Dutta, “Design and Development of IoT‑Based Latency‑Optimized Augmented Reality Framework in Home Automation and Telemetry for Smart Lifestyle,” Journal of Reliable Intelligent Environments - Springer International Publishing, pp. 1-19, 2020.

J. Monge and O. Postolache, “Augmented Reality and Smart Sensors for Physical Rehabilitation,” In Proc. of the Int. Conf. and Expos. on EPE, IEEE Xplore Digital Library, pp. 1010-1014, 2018.

M. Narsimlu, M. V. Naik, and M. R. Sree, “An Innovative Research on Augmented Reality to Control Status of Physical Light using MAYA as Part of IoT,” In Proc. of the Int. Conf. on SmartTechCon, IEEE Xplore Digital Library, pp. 1221-1227, 2017.

M. Novak, O. Wilfert, and T. Simicek, “Visible Light Communication Beacon System for Internet of Things,” In Proc.of the COMITE, IEEE Xplore Digital Library, pp. 1-5, 2017.

T. Ohta, R. Ito, and Y. Kakuda, “Design of a Node Status Visualizing Software Utilizing the AR Technology for Multihop Wireless Networks,” In Proc. of the IEEE 41st Annual COMPSAC, IEEE Xplore Digital Library, pp. 270-271, 2017.

Y. Park, S. Yun, and K. Kim, “When IoT met Augmented Reality: Visualizing the Source of the Wireless Signal in AR View,” In Proc. of the 17th Annual Int. Conf. on MobiSys, ACM Digital Library, pp. 117-129, 2019.

P. Phupattanasilp and S.-R. Tong, “Augmented Reality in the Integrative Internet of Things (AR-IoT): Application for Precision Farming,” Sustainability - MDPI, vol. 11, no. 2658, pp. 1-17, 2019.

B. Pokrić, S. Krčo, M. Pokrić, P. Knežević, and D. Jovanović, “Engaging Citizen Communities in Smart Cities using IoT, Serious Gaming, and Fast Markerless Augmented Reality,” In Proc. of the Int. Conf. on RIoT, IEEE Xplore Digital Library, pp. 1-6, 2015.

J. A. Purmaissur, P. Towakel, S. P. Guness, A. Seeam, and X. A. Bellekens, “Augmented-Reality Computer-Vision Assisted Disaggregated Energy Monitoring and IoT Control Platform,” In Proc. of the Int. Conf. on ICONIC, IEEE Xplore Digital Library, pp. 1-6, 2018.

J. Rambach, A. Pagani, and D. Stricker, “Augmented Things: Enhancing AR Applications Leveraging the Internet of Things and Universal 3D Object Tracking,” In Proc. of the IEEE ISMAR, IEEE Xplore Digital Library, pp. 103-108, 2017.

M. Alonso-Rosa, A. Gil-de-Castro, A. Moreno-Munoz, J. Garrido-Zafra, E. Gutierrez-Ballesteros, and E. Cañete-Carmona, “An IoT Based Mobile Augmented Reality Application for Energy Visualization in Buildings Environments,” Applied Science - MDPI, vol. 10, no. 600, pp. 1-15, 2020.

C. Shao, B. Islam, and S. Nirjon, “MARBLE: Mobile Augmented Reality Using a Distributed BLE Beacon Infrastructure,” In Proc. of the IEEE/ACM 3rd Int. Conf. on IoTDI, IEEE Xplore Digital Library, pp. 60-71, 2018.

H. Subakti and J.-R. Jiang, “Indoor Augmented Reality Using Deep Learning for Industry 4.0 Smart Factories,” In Proc. of the IEEE 42nd Annual COMPSAC, IEEE Xplore Digital Library, pp. 63-68, 2018.

Y. Sun, A. Armengol-Urpi, S. N. R. Kantareddy, J. Siegel, and S. Sarma, “MagicHand: Interact with IoT Devices in Augmented Reality Environment,” In Proc. of the IEEE Conf. on VR, IEEE Xplore Digital Library, pp. 1738-1743, 2019.

Z. Tan, H. Qu, J. Zhao, S. Zhou, and W. Wang, “UAV-aided Data Collection in Self Organized IoT Network for Social Augmented Reality,” In Proc. of the 29th ITNAC, IEEE Xplore Digital Library, pp. 1-7, 2020.

P. A. Vanin, A. S. Nesterov, and I. Y. Kholodilin, “Integration of IIoT and AR Technologies to Educational Process Through Laboratory Complex,” In Proc. of the GloSIC, IEEE Xplore Digital Library, pp. 1-6, 2018.

T. Zachariah and P. Dutta, “Browsing theWeb of Things in Mobile Augmented Reality,” In Proc. of the 20th Int. Workshop on HotMobile, ACM Digital Library, pp. 129-134, 2019.

S. Lukosch, M. Billinghurst, L. Alem, and K. Kiyokawa, “Collaboration in Augmented Reality,” Computer Supported Cooperative Work (CSCW), Springer Link, vol. 24, pp. 515-525, 2015.

Bila bermanfaat silahkan share artikel ini

Berikan Komentar Anda terhadap artikel Augmented Reality in the Internet of Things (AR + IoT): A Review

Refbacks

  • There are currently no refbacks.


Copyright (c) 2021 Aulia Akhrian Syahidi, Kohei Arai, Herman Tolle, Ahmad Afif Supianto, Kiyoshi Kiyokawa

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.

The IJICS (International Journal of Informatics and Computer Science)
Published by STMIK Budi Darma.
Jl. Sisingamangaraja No.338 Simpang Limun, Medan, North Sumatera
Email: ijics.stmikbudidarma@gmail.com

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.