AM-DIMPO: Action-Masked Diffusion-Implicit Policy Optimization for On-Ramp Merging Under Dense Traffic

Highway ramp merging requires autonomous vehicles to make safe and efficient decisions in dense mixed traffic, where strong vehicle interactions and rapidly changing acceptable gaps make the task particularly challenging. Existing reinforcement learning methods are often unimodal and overly conservative, while diffusion-based policies, despite their ability to generate multimodal actions, usually suffer from high inference latency and safety risks caused by unconstrained sampling. To address these issues, this paper proposes AM-DIMPO, an action-mask-guided safe diffusion-implicit policy optimization framework for ramp-merging tasks. The proposed method combines DDIM-based implicit sampling with a state-dependent continuous action mask to improve multimodal action generation efficiency while enhancing action feasibility. In addition, the mask correction signal is incorporated into policy learning to encourage the policy to generate actions closer to the safe feasible region. Experiments are conducted in a Gym-based ramp-merging simulator under both light-traffic and dense-traffic scenarios, where the proposed method is compared with classical reinforcement learning baselines, diffusion reinforcement learning baselines, and a safety-aware PPO baseline. The results show that, in dense traffic, AM-DIMPO achieves a merging success rate of 97.3%, an average speed of 16.27 m/s, and an inference latency of 68 ms; in light traffic, the success rate reaches 98.1%. Moreover, the proposed method maintains robust performance under the tested noisy-observation and reduced-visibility settings. Overall, AM-DIMPO achieves a favorable balance among empirical safety, traffic efficiency, robustness, and real-time inference performance in dense highway ramp-merging tasks.