We present a novel solution to automatic semantic modeling of indoor scenes from a sparse set of low-quality RGB-D images. Such data presents challenges due to noise, low resolution, occlusion and missing depth information. We exploit the knowledge in a scene database containing 100s of indoor scenes with over 10,000 manually segmented and labeled mesh models of objects. In seconds, we output a visually plausible 3D scene, adapting these models and their parts to fit the input scans. Contextual relationships learned from the database are used to constrain reconstruction, ensuring semantic compatibility between both object models and parts. Small objects and objects with incomplete depth information which are
difficult to recover reliably are processed with a two-stage approach. Major objects are recognized first, providing a known scene structure. 2D contour-based model retrieval is then used to recover smaller objects. Evaluations using our own data and two public datasets show that our approach can model typical real-world indoor scenes efficiently and robustly.
Yp Cao · Tao Ju · J Xu · Sm Hu. Extracting Sharp Features from RGB-D Images: RGB-D Sharp Features. 2016.
Shuai Zheng · Jun Hong · Kang Zhang · Baotong Li · Xin Li. A multi-frame graph matching algorithm for low-bandwidth RGB-D SLAM. In Computer-Aided Design.Volume 78.Page 107-117.2016.
Boyang Gao · Fei Chen · Franceso Trapani · Mario Selvaggio · Darwin G Caldwell. Robust object localization based on error patterns learning for dexterous mobile manipulation. In International Conference on Advanced Robotics and Mechatronics.Page 213-218.2016.
Lebeda K, Hadfield S, Bowden R.. TMAGIC: A model-free 3D tracker. In IEEE Transactions on Image Processing.2017.
Lebeda K.. 2D and 3D tracking and modelling. 2016.
Xiao Y, Feng C, Taguchi Y, et al.. User-Guided Dimensional Analysis of Indoor Building Environments from Single Frames of RGB-D Sensors. In Journal of Computing in Civil Engineering.Volume 31.Issue 4.2017.
Kim H, de Campos T, Hilton A.. Room Layout Estimation with Object and Material Attributes Information using a Spherical Camera. In International Conference on 3D Vision .Page 519-527.2016.
Cao Y P, Ju T, Xu J, et al.. Extracting Sharp Features from RGB‐D Images. In Computer Graphics Forum.
Cheng M M, Hou Q B, Zhang S H, et al. Intelligent Visual Media Processing: When Graphics Meets Vision. In Journal of Computer Science and Technology.Volume 32.Issue 1.Page 110-121.2017.
Yang S, Xu J, Chen K, et al.. View suggestion for interactive segmentation of indoor scenes. In Computational Visual Media.Page 1-16.2017.
3D scene modeling has long been a fundamental problem in computer graphics and computer vision. With the popularity of consumer-level RGB-D cameras, there is a growing interest in digitizing real-world indoor 3D scenes. However, modeling indoor
3D scenes remains a challenging problem because of the complex structure of interior objects and poor quality of RGB-D data acquired by consumer-level sensors. Various methods have been proposed to tackle these challenges. In this survey, we provide an overview of recent advances in indoor scene modeling techniques, as well as public datasets and code libraries which can facilitate experiments and evaluation.