Semi-Automatic Geographic Information System Framework for Creating Photo-Realistic Digital Twin Cities to Support Autonomous Driving Research

Xu, Haowen; Shao, Yunli; Chen, Jianfei; Wang, Chieh (Ross); Berres, Andy

doi:10.1177/03611981231205884

Semi-Automatic Geographic Information System Framework for Creating Photo-Realistic Digital Twin Cities to Support Autonomous Driving Research

Journal Article · Sun Nov 19 00:00:00 EST 2023 · Transportation Research Record: Journal of the Transportation Research Board

DOI:https://doi.org/10.1177/03611981231205884· OSTI ID:2279171

^[1]; ^[2]; ^[1]; ^[1]; ^[3]

Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)
University of Georgia, Athens, GA (United States)
National Renewable Energy Laboratory (NREL), Golden, CO (United States)

Digital twin cities are frequently used in vehicle and traffic simulations to render realistic on-road driving scenarios under various traffic and environmental conditions. These digital twins provide a high-fidelity replica of the physical world (e.g., buildings, roads, infrastructures, traffic) to create three-dimensional (3D) virtual-physical environments to support various emerging vehicle and transportation technologies such as connected and automated vehicles. These virtual environments provide a cost-effective digital proving ground to evaluate, validate, and test emerging technologies that include control algorithms, localization, perception, and sensors. Replicating a real-world traffic scenario in a digital twin using a traditional 3D modeling approach is a time-consuming and labor-intensive effort. Here this paper presents a semi-automated spatial framework to construct realistic 3D digital twin cities to support autonomous driving research using readily available geographic information system (GIS) data and 3D prefabricated (prefab) models. We start with a comprehensive review of geospatial data sources of essential digital entities required in a 3D digital twin city and present an integrated GIS-3D modeling pipeline using customized QGIS/GDAL and Blender scripting in Python. The pipeline outputs are realistic 3D digital twin cities compatible with common vehicle simulation software, such as CARLA and IPG CarMaker. The paper closes with a showcase to demonstrate the quality and usability of a digital twin city created to replicate the Shallowford Road corridor in Chattanooga in both Unity and Unreal engine-based virtual environment. The generated digital twin city can be applied to a hardware-in-the-loop simulation environment with an actual testing vehicle to facilitate autonomous driving research.

View Accepted Manuscript (DOE)

Research Organization:: National Renewable Energy Laboratory (NREL), Golden, CO (United States); Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)

Sponsoring Organization:: USDOE; USDOE Office of Energy Efficiency and Renewable Energy (EERE), Transportation Office. Vehicle Technologies Office

Grant/Contract Number:: AC05-00OR22725; AC36-08GO28308

OSTI ID:: 2279171

Alternate ID(s):: OSTI ID: 2371112

Report Number(s):: NREL/JA-5700-88473; MainId:89252; UUID:5c06c2e4-7957-4ac4-a903-64435d212c84; MainAdminId:71476

Journal Information:: Transportation Research Record: Journal of the Transportation Research Board, Journal Name: Transportation Research Record: Journal of the Transportation Research Board Journal Issue: 6 Vol. 2678; ISSN 0361-1981

Publisher:: SAGECopyright Statement

Country of Publication:: United States

Language:: English

References (63)

New Simulation Tools for Training and Testing Automated Vehicles Ma, Jiaqi; Schwarz, Chris; Wang, Ziran Road Vehicle Automation 7 https://doi.org/10.1007/978-3-030-52840-9_11	book	January 2020
AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles Shah, Shital; Dey, Debadeepta; Lovett, Chris Field and Service Robotics https://doi.org/10.1007/978-3-319-67361-5_40	book	November 2017
Urban 3D segmentation and modelling from street view images and LiDAR point clouds Babahajiani, Pouria; Fan, Lixin; Kämäräinen, Joni-Kristian Machine Vision and Applications, Vol. 28, Issue 7 https://doi.org/10.1007/s00138-017-0845-3	journal	May 2017
3D Urban Scene Modeling Integrating Recognition and Reconstruction Cornelis, Nico; Leibe, Bastian; Cornelis, Kurt International Journal of Computer Vision, Vol. 78, Issue 2-3 https://doi.org/10.1007/s11263-007-0081-9	journal	October 2007
A Novel Approach for Generation and Visualization of Virtual 3D City Model Using Open Source Libraries Singla, Jai G.; Padia, Kirti Journal of the Indian Society of Remote Sensing, Vol. 49, Issue 6 https://doi.org/10.1007/s12524-020-01191-8	journal	October 2020
A novel digital twin-centric approach for driver intention prediction and traffic congestion avoidance Kumar, Sathish A. P.; Madhumathi, R.; Chelliah, Pethuru Raj Journal of Reliable Intelligent Environments, Vol. 4, Issue 4 https://doi.org/10.1007/s40860-018-0069-y	journal	October 2018
Digital geoTwin Vienna: Towards a Digital Twin City as Geodata Hub Lehner, Hubert; Dorffner, Lionel PFG – Journal of Photogrammetry, Remote Sensing and Geoinformation Science, Vol. 88, Issue 1 https://doi.org/10.1007/s41064-020-00101-4	journal	February 2020
ArcGIS Urban: an application for plan assessment Soward, Ellen; Li, Jianling Computational Urban Science, Vol. 1, Issue 1 https://doi.org/10.1007/s43762-021-00016-9	journal	July 2021
A smart city case study of Singapore—Is Singapore truly smart? Cavada, Marianna; Tight, Miles R.; Rogers, Christopher D. F. Smart City Emergence https://doi.org/10.1016/B978-0-12-816169-2.00014-6	book	July 2019
The Shuttle Radar Topography Mission (SRTM): a breakthrough in remote sensing of topography van Zyl, Jakob J. Acta Astronautica, Vol. 48, Issue 5-12 https://doi.org/10.1016/S0094-5765(01)00020-0	journal	March 2001
A BIM-data mining integrated digital twin framework for advanced project management Pan, Yue; Zhang, Limao Automation in Construction, Vol. 124 https://doi.org/10.1016/j.autcon.2021.103564	journal	April 2021
3D geospatial visualizations: Animation and motion effects on spatial objects Evangelidis, Konstantinos; Papadopoulos, Theofilos; Papatheodorou, Konstantinos Computers & Geosciences, Vol. 111 https://doi.org/10.1016/j.cageo.2017.11.007	journal	February 2018
Close range photogrammetry for dynamically tracking drifted snow deposition Basnet, K.; Muste, M.; Constantinescu, G. Cold Regions Science and Technology, Vol. 121 https://doi.org/10.1016/j.coldregions.2015.08.013	journal	January 2016
Disaster City Digital Twin: A vision for integrating artificial and human intelligence for disaster management Fan, Chao; Zhang, Cheng; Yahja, Alex International Journal of Information Management, Vol. 56 https://doi.org/10.1016/j.ijinfomgt.2019.102049	journal	February 2021
Terrestrial laser scanning and close range photogrammetry for 3D archaeological documentation: the Upper Palaeolithic Cave of Parpalló as a case study Lerma, José Luis; Navarro, Santiago; Cabrelles, Miriam Journal of Archaeological Science, Vol. 37, Issue 3 https://doi.org/10.1016/j.jas.2009.10.011	journal	March 2010
Towards an effective framework for building smart cities: Lessons from Seoul and San Francisco Lee, Jung Hoon; Hancock, Marguerite Gong; Hu, Mei-Chih Technological Forecasting and Social Change, Vol. 89 https://doi.org/10.1016/j.techfore.2013.08.033	journal	November 2014
Evaluating connected and autonomous vehicles using a hardware-in-the-loop testbed and a living lab Shao, Yunli; Mohd Zulkefli, Mohd Azrin; Sun, Zongxuan Transportation Research Part C: Emerging Technologies, Vol. 102 https://doi.org/10.1016/j.trc.2019.03.010	journal	May 2019
Energy and mobility impacts of connected autonomous vehicles with co-optimization of speed and powertrain on mixed vehicle platoons Sun, Wenbo; Wang, Shian; Shao, Yunli Transportation Research Part C: Emerging Technologies, Vol. 142 https://doi.org/10.1016/j.trc.2022.103764	journal	September 2022
Digital Twins in the Intelligent Transport Systems Rudskoy, Andrey; Ilin, Igor; Prokhorov, Andrey Transportation Research Procedia, Vol. 54 https://doi.org/10.1016/j.trpro.2021.02.152	journal	January 2021
Topographic mapping from interferometric synthetic aperture radar observations Zebker, Howard A.; Goldstein, Richard M. Journal of Geophysical Research, Vol. 91, Issue B5 https://doi.org/10.1029/JB091iB05p04993	journal	January 1986
Smart City Digital Twin–Enabled Energy Management: Toward Real-Time Urban Building Energy Benchmarking Francisco, Abigail; Mohammadi, Neda; Taylor, John E. Journal of Management in Engineering, Vol. 36, Issue 2 https://doi.org/10.1061/(ASCE)ME.1943-5479.0000741	journal	March 2020
Smart Cities with Digital Twin Systems for Disaster Management Ford, David N.; Wolf, Charles M. Journal of Management in Engineering, Vol. 36, Issue 4 https://doi.org/10.1061/(ASCE)ME.1943-5479.0000779	journal	July 2020
Complete Streets: Promises and Proof Jordan, Samuel W.; Ivey, Stephanie Journal of Urban Planning and Development, Vol. 147, Issue 2 https://doi.org/10.1061/(ASCE)UP.1943-5444.0000684	journal	June 2021
Computer Vision-Enabled Smart Traffic Monitoring for Sustainable Transportation Management Shao, Yunli; Wang, Chieh (Ross); Berres, Andy International Conference on Transportation and Development 2022 https://doi.org/10.1061/9780784484319.004	conference	August 2022
Evaluating Connected and Automated Vehicles in Co-Simulation Environment of Traffic Microsimulation and Vehicle Dynamics Shao, Yunli; Chambon, Paul; Cook, Adian International Conference on Transportation and Development 2023 https://doi.org/10.1061/9780784484876.019	conference	June 2023
The Role of Smart City Characteristics in the Plans of Fifteen Cities Angelidou, Margarita Journal of Urban Technology, Vol. 24, Issue 4 https://doi.org/10.1080/10630732.2017.1348880	journal	August 2017
Building and exploiting a Digital Twin for the management of drinking water distribution networks Conejos Fuertes, P.; Martínez Alzamora, F.; Hervás Carot, M. Urban Water Journal, Vol. 17, Issue 8 https://doi.org/10.1080/1573062X.2020.1771382	journal	June 2020
Block-NeRF: Scalable Large Scene Neural View Synthesis Tancik, Matthew; Casser, Vincent; Yan, Xinchen 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/CVPR52688.2022.00807	conference	June 2022
A Digital Smart City for Emerging Mobility Systems Zayas, Raymond M.; Beaver, Logan E.; Chalaki, Behdad 2022 IEEE 2nd International Conference on Digital Twins and Parallel Intelligence (DTPI) https://doi.org/10.1109/DTPI55838.2022.9998963	conference	October 2022
Virtual Reality to Implement Driving Simulation for Combining CAN BUS and Automotive Sensors Tsai, Pu-Sheng; Wu, Ter-Feng; Hu, Nien-Tsu 2017 International Conference on Information, Communication and Engineering (ICICE) https://doi.org/10.1109/ICICE.2017.8479271	conference	November 2017
Characteristics of ASTER GDEM version 2 Tachikawa, Tetsushi; Hato, Masami; Kaku, Manabu 2011 IEEE International Geoscience and Remote Sensing Symposium https://doi.org/10.1109/IGARSS.2011.6050017	conference	July 2011
V2XSim: A V2X Simulator for Connected and Automated Vehicle Environment Simulation Zhang, Ethan; Masoud, Neda 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC) https://doi.org/10.1109/ITSC45102.2020.9294660	conference	September 2020
3D LIDAR point cloud based intersection recognition for autonomous driving Zhu, Quanwen; Chen, Long; Li, Qingquan 2012 IEEE Intelligent Vehicles Symposium https://doi.org/10.1109/IVS.2012.6232219	conference	June 2012
An Augmented Reality Environment for Connected and Automated Vehicle Testing and Evaluation ^* Feng, Yiheng; Yu, Chunhui; Xu, Shaobing 2018 IEEE Intelligent Vehicles Symposium (IV) https://doi.org/10.1109/IVS.2018.8500545	conference	June 2018
The Role of Digital Twins in Connected and Automated Vehicles Schwarz, Chris; Wang, Ziran IEEE Intelligent Transportation Systems Magazine, Vol. 14, Issue 6 https://doi.org/10.1109/MITS.2021.3129524	journal	November 2022
Simulating the Autonomous Future: A Look at Virtual Vehicle Environments and How to Validate Simulation Using Public Data Sets Deter, Dean; Wang, Chieh; Cook, Adian IEEE Signal Processing Magazine, Vol. 38, Issue 1 https://doi.org/10.1109/MSP.2020.2984428	journal	January 2021
Energy-Efficient Connected and Automated Vehicles: Real-Time Traffic Prediction-Enabled Co-Optimization of Vehicle Motion and Powertrain Operation Shao, Yunli; Sun, Zongxuan IEEE Vehicular Technology Magazine, Vol. 16, Issue 3 https://doi.org/10.1109/MVT.2021.3085999	journal	September 2021
Generation of three dimensional photo-realistic models from Lidar and image data Li-Chee-Ming, J.; Gumerov, D.; Ciobanu, T. 2009 IEEE Toronto International Conference Science and Technology for Humanity (TIC-STH) https://doi.org/10.1109/TIC-STH.2009.5444457	conference	September 2009
Smart Mobility in the Cloud: Enabling Real-Time Situational Awareness and Cyber-Physical Control Through a Digital Twin for Traffic Xu, Haowen; Berres, Andy; Yoginath, Srikanth B. IEEE Transactions on Intelligent Transportation Systems, Vol. 24, Issue 3 https://doi.org/10.1109/TITS.2022.3226746	journal	March 2023
Concepts for Cartography-Oriented Visualization of Virtual 3D City Models Semmo, Amir; Hildebrandt, Dieter; Trapp, Matthias Photogrammetrie - Fernerkundung - Geoinformation, Vol. 2012, Issue 4 https://doi.org/10.1127/1432-8364/2012/0131	journal	August 2012
Image-based street-side city modeling Xiao, Jianxiong; Fang, Tian; Zhao, Peng ACM SIGGRAPH Asia 2009 papers on - SIGGRAPH Asia '09 https://doi.org/10.1145/1661412.1618460	conference	January 2009
UNIGINE engine render Shergin, Denis ACM SIGGRAPH 2012 Computer Animation Festival https://doi.org/10.1145/2341836.2341908	conference	August 2012
Hardware-in-the-Loop Testing of Connected and Automated Vehicle Applications: A Use Case for Queue-Aware Signalized Intersection Approach and Departure Ma, Jiaqi; Zhou, Fang; Huang, Zhitong Transportation Research Record: Journal of the Transportation Research Board, Vol. 2672, Issue 22 https://doi.org/10.1177/0361198118793001	journal	September 2018
Using 3D Rule-Based Modeling to Interactively Visualize “Complete Streets” Design Scenarios Bejleri, Ilir; Noh, Soowoong; Bufkin, Jamie N. Transportation Research Record: Journal of the Transportation Research Board, Vol. 2675, Issue 10 https://doi.org/10.1177/0361198121999051	journal	March 2021
Developing a City-Level Digital Twin –Propositions and a Case Study Wan, L.; Nochta, T.; Schooling, J. M. International Conference on Smart Infrastructure and Construction 2019 (ICSIC) https://doi.org/10.1680/icsic.64669.187	conference	January 2019
Digital Twins for Cities: A State of the Art Review Ketzler, Bernd; Naserentin, Vasilis; Latino, Fabio Built Environment, Vol. 46, Issue 4 https://doi.org/10.2148/benv.46.4.547	journal	December 2020
Knowledge Discovery in Smart City Digital Twins Mohammadi, Neda; Taylor, John Proceedings of the Annual Hawaii International Conference on System Sciences https://doi.org/10.24251/HICSS.2020.204	conference	January 2020
Status report for the 3D Elevation Program, 2013-2014 Lukas, Vicki; Eldridge, Diane F.; Jason, Allyson L. https://doi.org/10.3133/ofr20151161	report	January 2015
What Makes a “Complete Street” Complete? Kingsbury, Kevin T.; Lowry, Michael B.; Dixon, Michael P. Transportation Research Record: Journal of the Transportation Research Board, Vol. 2245, Issue 1 https://doi.org/10.3141/2245-13	journal	January 2011
Cyber-Physical Systems Improving Building Energy Management: Digital Twin and Artificial Intelligence Agostinelli, Sofia; Cumo, Fabrizio; Guidi, Giambattista Energies, Vol. 14, Issue 8 https://doi.org/10.3390/en14082338	journal	April 2021
ASTER Global Digital Elevation Model (GDEM) and ASTER Global Water Body Dataset (ASTWBD) Abrams, Michael; Crippen, Robert; Fujisada, Hiroyuki Remote Sensing, Vol. 12, Issue 7 https://doi.org/10.3390/rs12071156	journal	April 2020
The New Landsat Collection-2 Digital Elevation Model Franks, Shannon; Storey, James; Rengarajan, Rajagopalan Remote Sensing, Vol. 12, Issue 23 https://doi.org/10.3390/rs12233909	journal	November 2020
A Geospatial Platform to Manage Large-Scale Individual Mobility for an Urban Digital Twin Platform Lee, Ahyun; Lee, Kang-Woo; Kim, Kyong-Ho Remote Sensing, Vol. 14, Issue 3 https://doi.org/10.3390/rs14030723	journal	February 2022
City Digital Twin Potentials: A Review and Research Agenda Shahat, Ehab; Hyun, Chang T.; Yeom, Chunho Sustainability, Vol. 13, Issue 6 https://doi.org/10.3390/su13063386	journal	March 2021
On the Reliable Generation of 3D City Models from Open Data Girindran, Renoy; Boyd, Doreen S.; Rosser, Julian Urban Science, Vol. 4, Issue 4 https://doi.org/10.3390/urbansci4040047	journal	September 2020
Real-Sim Interface: Enabling Multi-resolution Simulation and X-in-the-Loop Development for Connected and Automated Vehicles Shao, Yunli; Deter, Dean; Cook, Adian SAE International Journal of Connected and Automated Vehicles, Vol. 5, Issue 4 https://doi.org/10.4271/12-05-04-0026	journal	June 2022
Pre-Deployment Testing of Low Speed, Urban Road Autonomous Driving in a Simulated Environment Li, Xinchen; Arul Doss, Aravind Chandradoss; Aksun Guvenc, Bilin SAE International Journal of Advances and Current Practices in Mobility, Vol. 2, Issue 6 https://doi.org/10.4271/2020-01-0706	journal	April 2020
Virtual and augmented reality applications for environmental science education and training Sermet, Yusuf; Demir, Ibrahim New Perspectives on Virtual and Augmented Reality https://doi.org/10.4324/9781003001874-17	book	May 2020
Accuracy Assessment of 3d Models Generated from Google Street view Imagery Bruno, N.; Roncella, R. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Vol. XLII-2/W9 https://doi.org/10.5194/isprs-archives-XLII-2-W9-181-2019	journal	January 2019
Comparison Assessment of Digital 3d Models Obtained by Drone-Based Lidar and Drone Imagery Bouziani, M.; Amraoui, M.; Kellouch, S. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Vol. XLVI-4/W5-2021 https://doi.org/10.5194/isprs-archives-XLVI-4-W5-2021-113-2021	journal	December 2021
Virtual 3d city Modeling: Techniques and Applications Singh, S. P.; Jain, K.; Mandla, V. R. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Vol. XL-2/W2 https://doi.org/10.5194/isprsarchives-XL-2-W2-73-2013	journal	August 2013
Study on Construction of 3d Building Based on uav Images Xie, F.; Lin, Z.; Gui, D. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Vol. XXXIX-B1 https://doi.org/10.5194/isprsarchives-XXXIX-B1-469-2012	journal	July 2012
3D City Model Generator: The Application of Neuro-Fuzzy Systems in CAD Kobayashi, Yoshihiro ACADIA proceedings https://doi.org/10.52842/conf.acadia.2002.163	conference	January 2002

Similar Records

Performance Assessment of Prediction In Dynamic Environments (PRIDE) in Manufacturing Environments

Conference · Wed Dec 31 23:00:00 EST 2008 · OSTI ID:965324

Toward a Smart Metaverse City: Immersive Realism and 3D Visualization of Digital Twin Cities

Book · Mon May 01 00:00:00 EDT 2023 · OSTI ID:1995716

Related Subjects

33 ADVANCED PROPULSION SYSTEMS
97 MATHEMATICS AND COMPUTING
data and data science
geospatial data visualization
interactive visualization
visualization in transportation

Semi-Automatic Geographic Information System Framework for Creating Photo-Realistic Digital Twin Cities to Support Autonomous Driving Research

Citation Formats

References (63)

Similar Records

Related Subjects