ARTEMIS platform automatically tunes LLM agent configurations, boosting performance by 10-37%

Large language model-based agents can automate intricate processes across domains like software development and customer service. Yet these systems frequently fall short of their potential because their internal settings—prompts, tool specifications, and parameters—remain poorly configured. Manually refining these components typically demands weeks of specialized effort.

A new optimization approach#

Researchers have introduced ARTEMIS, a platform that handles configuration tuning without requiring code expertise or architectural changes to agents. The system uses evolutionary algorithms informed by semantic understanding to jointly refine multiple components at once, capturing how different settings influence each other. Given just a benchmark test and a statement of goals in plain language, ARTEMIS discovers which parts of an agent can be adjusted, reads performance data from execution logs, and evolves better configurations automatically.

Real-world results#

The team tested ARTEMIS across four distinct agent implementations:

• ALE Agent (competitive programming via AtCoder Heuristic Contest): 13.6% improvement in acceptance rate
• Mini-SWE Agent (code optimization on SWE-Perf): 10.1% statistically significant performance gain
• CrewAI Agent (math reasoning and cost efficiency on Math Odyssey): 36.9% reduction in token usage
• MathTales-Teacher Agent (based on open-source Qwen2.5-7B model, GSM8K math problems): 22% accuracy improvement

Broader implications#

The results demonstrate that ARTEMIS works across both proprietary and open-source models. By treating agent components as interconnected rather than isolated, the system captures optimization opportunities that single-component approaches typically miss—addressing a key limitation of existing methods that either prove too specialized for widespread adoption or ignore these critical relationships.

Source: arXiv:2512.09108