IBM research proposes consistency guidelines, narrowing ReAct agents' Pass^5 gap from 24.4pp to 12.0pp
IBM researchers describe new consistency guidelines in altk-evolve, designed to address unstable results when an agent runs the same task repeatedly.
Why it matters: The paper quantifies the gap between Mean@k and Pass^k and shows that consistency guidelines cut it in half, offering a technique for agent-evaluation design.