IBM Reports ALTK-Evolve Guidelines Improve GPT-4.1 Customer Service Benchmark Results

IBM Research reports that ALTK-Evolve improved GPT-4.1’s reliability on customer service tasks by adding learned behavioral guidelines to its prompts, raising Pass^4 on held-out τ²-bench retail tasks from 57.5% to 67.5%. Its result on the full base split for retail also exceeded every entry in the benchmark’s October 12, 2026 leaderboard snapshot. ALTK-Evolve extracts guidelines from agents’ training attempts without updating model weights or using human annotation. The useful guideline set varied by task: airline reliability benefited most from a smaller set. Adding guidelines increased input token costs, while slightly reducing conversation turns when comparing successful runs on the same tasks.

Generated by AI from reports · Updated 3 hours ago

There is not enough continuous observation data to plot a trend yet.