Development of Multimodal Physical and Virtual Traffic Reality Simulation System

As urban traffic complexity increases, realistic multimodal simulation environments are essential for evaluating transportation safety and human behavior. This study introduces a novel multimodal, multi-participant co-simulation framework designed to comprehensively model interactions between drivers, bicyclists, and pedestrians. The framework integrates CARLA, a high-fidelity driving simulator, with PTV Vissim, a widely used microscopic traffic simulation tool. This integration was achieved through the development of custom scripts in Python and C++ that enable real-time data exchange and synchronization between the platforms. Additionally, physiological sensors, including heart rate monitors, electrodermal activity sensors, and EEG devices, were integrated using Lab Streaming Layer to capture physiological responses under different traffic conditions. Three experimental case studies validate the system’s capabilities. In the first, cyclists showed a significant rightward lane shift (from 0.94 m to 1.14 m, p<0.00001) and elevated heart rates (69.45 to 72.75 bpm, p<0.00001) in response to overtaking vehicles. In the second, pedestrians exhibited more conservative gap acceptance behavior at 50 mph vs. 30 mph (gap acceptance time: 3.70 vs. 3.18 s, p<0.00001), with corresponding increases in HR (3.54 bpm vs. 1.91 bpm post-event). In the third case study, mean vehicle speeds recorded during simulated driving were compared with real-world field data along urban corridors, demonstrating strong alignment and validating the system’s ability to reproduce realistic traffic conditions. These findings demonstrate the system’s effectiveness in capturing dynamic, real-time human responses and provide a foundation for advancing human-centered, multimodal traffic research.