The Resolution Hypothesis: Agent Evolution's Key Is Identifying Improvements
11 tiếng nói · chưa gắn nhãn trưởng lab · 1 nền tảng · 14 tương tác đã đo · Nguồn crawl
Chi tiết nguồn
Bài báo mô tả vòng lặp tiến hóa tác nhân: proposer tạo các ứng viên (prompts, skills, memories) và selection chọn những ứng viên có điểm cao hơn. Tuy vậy, họ nói rằng evolution có thể thất bại khi khởi động trên một số tác vụ vì đánh giá nhiễu không thể phân biệt đáng tin cậy giữa ứng viên tốt và xấu.
Agent evolution follows a simple loop: a proposer writes candidate artifacts (prompts, skills, memories) and selection accepts those that score higher. This loop has shown promise, yet practitioners face a puzzle: methods trade wins across papers, and evolution often fails to bootstrap on certain tasks. We trace this puzzle to the Resolution Hypothesis: agent evolution stalls not because methods fail to propose better artifacts, but because noisy evaluations cannot reliably identify them. Across seven evolution methods and six models (over 10M LLM calls), we find support for the hypothesis at both levels where evaluation scores are used. At the candidate level (choosing artifacts), helpful candidate artifacts exist, but selection is no better than random at telling them from harmful ones, so errors accumulate across rounds. At the method level (ranking methods), typical margins of a few percentage points are smaller than the difference between two independent runs of the same method, so published rankings often rest on luck. Together, these findings indicate that agent evolution is often bottlenecked by reliably identifying improvements. This calls for a shift in how we develop and evaluate self-improving agents, from designing ever more candidate improvement systems to making true improvements detectable and accumulable.
We trace this puzzle to the Resolution Hypothesis: agent evolution stalls not because methods fail to propose better artifacts, but because noisy evaluations cannot reliably identify them.
We trace this puzzle to the Resolution Hypothesis: agent evolution stalls not because methods fail to propose better artifacts, but because noisy evaluations cannot reliably identify them.
Across seven evolution methods and six models (over 10M LLM calls), we find support for the hypothesis at both levels where evaluation scores are used.
Together, these findings indicate that agent evolution is often bottlenecked by reliably identifying improvements. This calls for a shift in how we develop and evaluate self-improving agents, from designing ever more candidate improvement systems to making true improvements detectable and accumulable.
- The Resolution Hypothesis: Agent Evolution's Key Is Identifying Improvements Hugging Face Papers · 17 thg 11, 2026