Safety-critical scenarios are rare yet pivotal for evaluating and enhancing the robustness of autonomous driving systems. While existing methods generate safety-critical driving trajectories, simulations, or single-view videos, they fall short of meeting the demands of advanced end-to-end autonomous systems (E2E AD), which require real-world, multi-view video data. To bridge this gap, we introduce SafeMVDrive, the first framework designed to generate high-quality, safety-critical, multi-view driving videos grounded in real-world domains. SafeMVDrive strategically integrates a safety-critical trajectory generator with an advanced multi-view video generator. To tackle the challenges inherent in this integration, we first enhance scene understanding ability of the trajectory generator by incorporating visual context -- which is previously unavailable to such generator -- and leveraging a GRPO-finetuned vision-language model to achieve more realistic and context-aware trajectory generation. Second, recognizing that existing multi-view video generators struggle to render realistic collision events, we introduce a two-stage, controllable trajectory generation mechanism that produces collision-evasion trajectories, ensuring both video quality and safety-critical fidelity. Finally, we employ a diffusion-based multi-view video generator to synthesize high-quality safety-critical driving videos from the generated trajectories. Experiments conducted on an E2E AD planner demonstrate a significant increase in collision rate when tested with our generated data, validating the effectiveness of SafeMVDrive in stress-testing planning modules.
Due to file size limitations, videos are compressed and quality might be affected.
Adversarial vehicle suddenly cuts in; ego vehicle slightly steers right to avoid.
Rear adversarial vehicle suddenly accelerates; ego vehicle also speeds up to evade.
Rear adversarial vehicle suddenly accelerates; ego vehicle changes lane left to evade.
Front adversarial vehicle suddenly slows down; ego vehicle changes lane and decelerates to avoid.
At night, rear adversarial car suddenly accelerates; ego vehicle accelerates forward to evade and avoids the vehicle ahead.
Rear adversarial vehicle suddenly accelerates; ego vehicle first speeds up, then changes lane right to evade.
Front adversarial vehicle suddenly slows down; ego vehicle also decelerates to evade.
Rear adversarial vehicle suddenly accelerates; ego vehicle also speeds up to evade.
Rear adversarial vehicle suddenly accelerates; ego vehicle also speeds up to evade.