Sihang Jiang, Lipeng Ma, Zhonghua Hong, Keyi Wang, Zhiyu Lu, Shisong Chen, Jinghao Zhang, Tianjun Pan, Weijia Zhou, Jiaqing Liang, Yanghua Xiao

Researchers introduce SEA-Eval, the first benchmark for evaluating self-evolving agents that can learn and improve across multiple tasks over time, rather than treating each task in isolation. Current LLM-based agents perform well on individual tasks but fail to accumulate experience or adapt their toolsets, creating a gap between episodic performance and genuine long-term learning. The benchmark measures both immediate execution reliability and evolutionary gains by analyzing success rates and token consumption across sequential task streams, revealing that identical success rates can mask up to 31.2 times differences in efficiency and divergent learning patterns.