Statistical Estimation of Adversarial Risk in Large Language Models under Best-of-N Sampling
IntermediateMingqian Feng, Xiaodong Liu et al.Jan 30arXiv
Real attackers can try many prompts in parallel until a model slips, so testing safety with only one try badly underestimates risk.
#Best-of-N sampling#Adversarial risk#Attack Success Rate (ASR)