Steerable Adversarial Scenario Generation through Test-Time Preference Alignment

Adversarial scenario generation is a cost-effective approach for the safety assessment of autonomous driving systems. However, existing methods are either overly aggressive, resulting in low realism, or constrained to a fixed trade-off between multiple objectives, lacking the flexibility to adapt to diverse training and testing needs without costly retraining. Here, we reframe the task as a multi-objective preference alignment problem and introduce the first framework for Steerable Adversarial scenario GEneration (SAGE). SAGE enables fine-grained test-time control over the trade-off between adversariality and realism without any retraining. We first propose hierarchical group-based preference optimization, a data-efficient offline alignment method that learns to balance competing objectives by decoupling hard feasibility constraints from soft preferences. Instead of training a fixed model, SAGE fine-tunes two experts on opposing preferences and constructs a continuous spectrum of policies at inference time by linearly interpolating their weights. We provide theoretical justification for this scheme through the lens of linear mode connectivity. Extensive experiments demonstrate that SAGE not only generates scenarios with a superior balance of adversariality and realism but also enables more effective closed-loop training of driving policies.