Chain-of-Experience for Continual LLM Improvement

Summary

This paper introduces Chain-of-Experience (CoE), a paradigm enabling large language models (LLMs) to improve at test time through iterative interaction with self or environmental feedback. Across math, coding, and knowledge domains, using 8 LLMs including GPT-5, Gemini-2.5 Pro, and Claude-4.5 Sonnet, the authors show that leveraging iterative experience consistently outperforms feedback-free baselines, with self-feedback alone yielding substantial gains, a 5.6% overall improvement, and 19% lower API cost. Combining complementary feedback channels (e.g., model and correctness signals) brings additional gains, and CoE achieves higher accuracy per token than existing test-time strategies. The study finds a positive correlation between base ability and improvement capacity, and robustness under weak or spurious feedback, with most gains emerging early in iterations.