AIN2: Artificial Intelligence for Indoor Digital Twins

Details
Funded by: QNRF	Reference: NPRP15S-0405-210132
Start: 2023-07-09	Duration: 28 Months
Partners
Coordinator: HBKU	Qatar
Contractor: CRS4	Italy

Abstract

While the area of structured indoor reconstruction has witnessed substantial progress in the past decade, current solutions are mostly limited to relatively simple environments with constrained shapes and provide very limited opportunities for seamlessly going from captured data to a dynamically modifiable representation. In AIN2, we aim to substantially advance this research field by proposing specialized solutions for rapidly capturing, exploring, and visually editing indoor environment starting from panoramic images. The project will tackle fundamental problems in the field of reality-based indoor reconstruction, modeling, exploration, leading to new methods, algorithms, and reference implementations.

Publications

	[1] Giovanni Pintore, Marco Agus, and Enrico Gobbetti. Automatic 3D modeling and exploration of indoor structures from panoramic imagery. In SIGGRAPH Asia 2024 Courses (SA Courses '24), december 2024. ACM Press. DOI: 10.1145/3680532.3689580. To appear.
	[2] Giovanni Pintore, Marco Agus, Alberto Signoroni, and Enrico Gobbetti. DDD: Deep indoor panoramic Depth estimation with Density maps consistency. In STAG: Smart Tools and Applications in Graphics, november 2024. To appear.
	[3] Uzair Shah, Sara Jashari, Muhammad Tukur, Giovanni Pintore, Enrico Gobbetti, Jens Schneider, and Marco Agus. VISPI: Virtual Staging Pipeline for Single Indoor Panoramic Images. In STAG: Smart Tools and Applications in Graphics, november 2024. To appear.
	[4] Sara Jashari, Muhammad Tukur, Yehia Boraey, Uzair Shah, Mahmood Alzubaidi, Giovanni Pintore, Enrico Gobbetti, Alberto Jaspe-Villanueva, Jens Schneider, Noora Fetais, and Marco Agus. Evaluating AI-based static stereoscopic rendering of indoor panoramic scenes. In STAG: Smart Tools and Applications in Graphics, november 2024. To appear.
	[5] Muhammad Tukur, Yehia Boraey, Sara Jashari, Alberto Jaspe-Villanueva, Uzair Shah, Mahmood Alzubaidi, Giovanni Pintore, Enrico Gobbetti, Jens Schneider, and Marco Agus. Virtual Staging Technologies for the Metaverse. In Proc. IEEE iMeta, 2024.
	[6] Uzair Shah, Jens Schneider, Giovanni Pintore, Enrico Gobbetti, Mahmood Alzubaidi, Mowafa Househ, and Marco Agus. EleViT: exploiting element-wise products for designing efficient and lightweight vision transformers. In Proc. T4V - IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2024. To appear.
	[7] Uzair Shah, Muhammad Tukur, Mahmood Alzubaidi, Giovanni Pintore, Enrico Gobbetti, Mowafa Househ, Jens Schneider, and Marco Agus. MultiPanoWise: holistic deep architecture for multi-task dense prediction from a single panoramic image. In Proc. OmniCV - IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2024. To appear.
	[8] Giovanni Pintore, Alberto Jaspe-Villanueva, Markus Hadwiger, Jens Schneider, Marco Agus, Fabio Marton, Fabio Bettio, and Enrico Gobbetti. Deep synthesis and exploration of omnidirectional stereoscopic environments from a single surround-view panoramic image. Computers & Graphics, 119: 103907, March 2024. DOI: 10.1016/j.cag.2024.103907.
	[9] Muhammad Tukur, Atiq Ur Rehman, Giovanni Pintore, Enrico Gobbetti, Jens Schneider, and Marco Agus. PanoStyle: Semantic, Geometry-Aware and Shading Independent Photorealistic Style Transfer for Indoor Panoramic Scenes. In Proc. of the First Computer Vision Aided Architectural Design Workshop, International Conference of Computer Vision (ICCVW). Pages 1553-1564, October 2023.
	[10] Giovanni Pintore, Alberto Jaspe Villanueva, Markus Hadwiget, Enrico Gobbetti, Jens Schneider, and Marco Agus. PanoVerse: automatic generation of stereoscopic environments from single indoor panoramic images for Metaverse applications. In Proc. Web3D 2023 - 28th International ACM Conference on 3D Web Technology, October 2023. DOI: 10.1145/3611314.3615914. Honorable mention award in the best paper category at Web3D 2023.
	[11] Giovanni Pintore, Fabio Bettio, Marco Agus, and Enrico Gobbetti. Deep scene synthesis of Atlanta-world interiors from a single omnidirectional image. IEEE Transactions on Visualization and Computer Graphics, 29, November 2023. DOI: 10.1109/TVCG.2023.3320219. Proc. ISMAR.