Mohamed Elfeki, Tu Trinh, Kelvin Luu, Guangze Luo, Nathan Hunt, Ernesto Montoya, Nandan Marwaha, Yannis He, Charles Wang, Fernando Crabedo, Alessa Castilo, Bing Liu

Researchers introduced HiL-Bench, a benchmark that measures whether AI agents know when to ask for help rather than guess on incomplete tasks. Current benchmarks reward execution correctness on fully specified problems, masking a critical failure mode: agents that make lucky guesses score the same as those that would have escalated for clarification. The new benchmark surfaces human-validated blockers through progressive exploration and uses Ask-F1, a metric balancing question precision against blocker recall, to measure selective escalation skill. Testing across coding and SQL domains shows frontier models recover only a fraction of their full-information performance when deciding whether to ask, but reinforcement learning on shaped Ask-F1 rewards makes this judgment trainable.