Rekonstruksi Model 3D dari Set Citra Menggunakan Metode SFM-MVS dan Algoritma Poisson

Authors

  • Giri Hanbudi Universitas Widyatama, Bandung
  • Esa Fauzi Universitas Widyatama, Bandung

DOI:

https://doi.org/10.30865/mib.v6i3.4126

Keywords:

3D Reconstruction, Structure From Motion (SFM), Multi-View Stereo (MVS), Surface Reconstruction, Sparse Point Cloud, Dense Point Cloud

Abstract

The digital creative industry is growing massively and very fast. The use of 3D technology in various industrial fields is also in demand, for example in the manufacturing, film, and animation industries. In a certain scene, generally 3D generalists will create a 3D model that resembles a real model in the real world which is usually used as a property in the 3D scene such as accessories, furniture, wardrobes, and so on. The process of modeling 3D models manually by a 3D generalist is a long process and requires a long time in the process, where the process generally includes 3d layouting, 3d modeling, and 3d texturing. Through this 3D model reconstruction process, 3D models can be obtained quickly, cheaply, and efficiently. This process can be carried out through several stages using a set of images from a model to be reconstructed. Several approaches can be used in this 3D model reconstruction process. In this paper, we will use the Structure From Motion (SFM) and Multi-View Stereo (MVS) methods to obtain information on the sparse and dense point clouds in the image, followed by The Surface Reconstruction process using the Poisson Algorithm to obtain a triangle mesh from the actual shape of the object. The refinement process on the triangle mesh results is carried out in order to get maximum results. Through a combination of these methods, we managed to get a detailed and accurate 3D model with the real object being observed.

References

R. Jiao, “A partial steganography algorithm for 3D triangle mesh,†in 2017 International Conference on Machine Learning and Cybernetics (ICMLC), Ningbo, Jul. 2017, pp. 361–365. doi: 10.1109/ICMLC.2017.8108948.

Q. Zhang, P. An, S. Wang, X. Bai, and W. Zhang, “Image-based Space Object Reconstruction and Relative Motion Estimation using Incremental Structure from Motion,†in 2018 IEEE CSAA Guidance, Navigation and Control Conference (CGNCC), Xiamen, China, Aug. 2018, pp. 1–6. doi: 10.1109/GNCC42960.2018.9019205.

S. Bianco, G. Ciocca, and D. Marelli, “Evaluating the Performance of Structure from Motion Pipelines,†J. Imaging, vol. 4, no. 8, p. 98, Aug. 2018, doi: 10.3390/jimaging4080098.

H. Cui, S. Shen, X. Gao, and Z. Hu, “Batched Incremental Structure-from-Motion,†in 2017 International Conference on 3D Vision (3DV), Qingdao, Oct. 2017, pp. 205–214. doi: 10.1109/3DV.2017.00032.

Shuhan Shen, “Accurate Multiple View 3D Reconstruction Using Patch-Based Stereo for Large-Scale Scenes,†IEEE Trans. on Image Process., vol. 22, no. 5, pp. 1901–1914, May 2013, doi: 10.1109/TIP.2013.2237921.

Y. Furukawa and C. Hernández, Multi-view stereo: a tutorial. Boston Delft: Now, 2015.

Z. Shao, N. Yang, X. Xiao, L. Zhang, and Z. Peng, “A Multi-View Dense Point Cloud Generation Algorithm Based on Low-Altitude Remote Sensing Images,†Remote Sensing, vol. 8, no. 5, p. 381, May 2016, doi: 10.3390/rs8050381.

M. Berger, J. A. Levine, L. G. Nonato, G. Taubin, and C. T. Silva, “A benchmark for surface reconstruction,†ACM Trans. Graph., vol. 32, no. 2, pp. 1–17, Apr. 2013, doi: 10.1145/2451236.2451246.

F. Bernardini, J. Mittleman, H. Rushmeier, C. Silva, and G. Taubin, “The ball-pivoting algorithm for surface reconstruction,†IEEE Transactions on Visualization and Computer Graphics, vol. 5, no. 4, pp. 349–359, Oct. 1999, doi: 10.1109/2945.817351.

M. Kazhdan, M. Bolitho, and H. Hoppe, “Poisson surface reconstruction,†in Proceedings of the fourth Eurographics symposium on Geometry processing, Goslar, DEU, Jun. 2006, pp. 61–70.

Y. Cai, M. Cao, L. Li, and X. Liu, “An End-to-End Approach to Reconstructing 3D Model From Image Set,†IEEE Access, vol. 8, pp. 193268–193284, 2020, doi: 10.1109/ACCESS.2020.3032169.

J. Yang, C. Xiao, P. Wang, Y. Luo, and C. An, “Camera Array Calibration Using a Simple Checkerboard Pattern,†in 2019 IEEE International Conference on Signal, Information and Data Processing (ICSIDP), Chongqing, China, Dec. 2019, pp. 1–5. doi: 10.1109/ICSIDP47821.2019.9172948.

E.-S. Kim and S.-Y. Park, “Extrinsic calibration of a camera-LIDAR multi sensor system using a planar chessboard,†in 2019 Eleventh International Conference on Ubiquitous and Future Networks (ICUFN), Zagreb, Croatia, Jul. 2019, pp. 89–91. doi: 10.1109/ICUFN.2019.8806057.

Z. Cui, “Global Structure-from-Motion and Its Application,†Thesis, Simon Fraser University, Canada, 2017. Accessed: May 26, 2022. [Online]. Available: http://summit.sfu.ca/item/17568

A. Locher, M. Havlena, and L. Van Gool, “Progressive Structure from Motion,†in Computer Vision – ECCV 2018, vol. 11208, V. Ferrari, M. Hebert, C. Sminchisescu, and Y. Weiss, Eds. Cham: Springer International Publishing, 2018, pp. 22–38. doi: 10.1007/978-3-030-01225-0_2.

Y. Xue et al., “MVSCRF: Learning Multi-View Stereo With Conditional Random Fields,†in 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Korea (South), Oct. 2019, pp. 4311–4320. doi: 10.1109/ICCV.2019.00441.

J. Li, E. Li, Y. Chen, L. Xu, and Y. Zhang, “Bundled depth-map merging for multi-view stereo,†in 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Jun. 2010, pp. 2769–2776. doi: 10.1109/CVPR.2010.5540004.

L. Zhou, Z. Zhang, H. Jiang, H. Sun, H. Bao, and G. Zhang, “DP-MVS: Detail Preserving Multi-View Surface Reconstruction of Large-Scale Scenes,†Remote Sensing, vol. 13, no. 22, p. 4569, Nov. 2021, doi: 10.3390/rs13224569.

H. Song et al., “Integrated Quality Mesh Generation for Poisson Surface Reconstruction in HPC Applications,†in 2017 IEEE 19th International Conference on High Performance Computing and Communications; IEEE 15th International Conference on Smart City; IEEE 3rd International Conference on Data Science and Systems (HPCC/SmartCity/DSS), Bangkok, Dec. 2017, pp. 450–457. doi: 10.1109/HPCC-SmartCity-DSS.2017.59.

S. Zhu et al., “Very Large-Scale Global SfM by Distributed Motion Averaging,†in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, Jun. 2018, pp. 4568–4577. doi: 10.1109/CVPR.2018.00480.

F. Guo, J. Yang, Y. Chen, and B. Yao, “Research on image detection and matching based on SIFT features,†in 2018 3rd International Conference on Control and Robotics Engineering (ICCRE), Nagoya, Apr. 2018, pp. 130–134. doi: 10.1109/ICCRE.2018.8376448.

J. Feng, C. Ai, Z. An, Z. Zhou, and Y. Shi, “A Feature Detection and Matching Algorithm Based on Harris Algorithm,†in 2019 International Conference on Communications, Information System and Computer Engineering (CISCE), Haikou, China, Jul. 2019, pp. 616–621. doi: 10.1109/CISCE.2019.00144.

W. Wu and W. Liu, “An Optimized Method Based on RANSAC for Fundamental Matrix Estimation,†in 2018 IEEE 3rd International Conference on Signal and Image Processing (ICSIP), Shenzhen, Jul. 2018, pp. 372–376. doi: 10.1109/SIPROCESS.2018.8600471.

S. Mills, “Four- and Seven-Point Relative Camera Pose from Oriented Features,†in 2018 International Conference on 3D Vision (3DV), Verona, Sep. 2018, pp. 218–227. doi: 10.1109/3DV.2018.00034.

Y. Wang, M. Yu, G. Jiang, Z. Pan, and J. Lin, “Image Registration Algorithm Based on Convolutional Neural Network and Local Homography Transformation,†Applied Sciences, vol. 10, no. 3, p. 732, Jan. 2020, doi: 10.3390/app10030732.

M. Holia and Z. Shah, “Automatic Image Registration,†in Kalpa Publications in Engineering, Aug. 2017, vol. 1, pp. 402–411. doi: 10.29007/kqbg.

J. Chen, D. Wu, P. Song, F. Deng, Y. He, and S. Pang, “Multi-View Triangulation: Systematic Comparison and an Improved Method,†IEEE Access, vol. 8, pp. 21017–21027, 2020, doi: 10.1109/ACCESS.2020.2969082.

R. Zhang et al., “Distributed Very Large Scale Bundle Adjustment by Global Camera Consensus,†IEEE Trans. Pattern Anal. Mach. Intell., vol. 42, no. 2, pp. 291–303, Feb. 2020, doi: 10.1109/TPAMI.2018.2840719.

J. Liao, Y. Fu, Q. Yan, and C. Xiao, “Folding patch correspondence for multiview stereo,†Comput Anim Virtual Worlds, vol. 31, no. 4–5, Jul. 2020, doi: 10.1002/cav.1938.

Y. Furukawa and J. Ponce, “Accurate, Dense, and Robust Multiview Stereopsis,†IEEE transactions on pattern analysis and machine intelligence, vol. 32, pp. 1362–76, Aug. 2010, doi: 10.1109/TPAMI.2009.161.

N. Luo, L. Huang, Q. Wang, and G. Liu, “An Improved Algorithm Robust to Illumination Variations for Reconstructing Point Cloud Models from Images,†Remote Sensing, vol. 13, no. 4, p. 567, Feb. 2021, doi: 10.3390/rs13040567.

M. Kazhdan, M. Chuang, S. Rusinkiewicz, and H. Hoppe, “Poisson Surface Reconstruction with Envelope Constraints,†Computer Graphics Forum, vol. 39, no. 5, pp. 173–182, Aug. 2020, doi: 10.1111/cgf.14077.

Downloads

Published

2022-07-25

Issue

Vol. 6 No. 3 (2022): Juli 2022

Section

Articles

License

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License

Authors who publish with this journal agree to the following terms:

  1. Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under Creative Commons Attribution 4.0 International License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
  2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
  3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (Refer to The Effect of Open Access).