AI
Multi-agent system harnesses LLMs to automate REST API testing at scale
Researchers introduce MASTEST, which pairs large language models with programmed agents to generate comprehensive API test suites from OpenAPI specifications.
Researchers have developed MASTEST, a multi-agent testing framework that leverages large language models alongside traditional programmed agents to streamline the creation and execution of test suites for RESTful APIs. The system processes OpenAPI Swagger specifications as input and orchestrates the full testing workflow—from generating test cases to validating API behaviour—while incorporating human review checkpoints to ensure quality.
How MASTEST works#
The system combines two types of agents. LLM-powered agents reason about API specifications and generate test artefacts, while programmed agents handle structured tasks and coordinate the overall testing pipeline. This hybrid approach covers the entire API testing lifecycle, with human testers embedded in the loop to review and refine LLM-generated outputs before execution.
Testing performance and results#
The research team evaluated MASTEST using two leading models—GPT-4o and DeepSeek V3.1 Reasoner—on five public APIs. Key findings include:
- Coverage metrics: Generated test cases achieved 94% (GPT-4o) and 98% (DeepSeek) unit test coverage, with 79% and 78% system test coverage respectively, when compared against human-authored baselines
- Code quality: Test scripts maintained 100% syntax correctness and required only minimal manual adjustments for semantic accuracy
- Bug detection: The system identified between 2.13 to 4.50 bugs per API operation, demonstrating strong fault-finding ability
- Assertion coverage: Generated tests included assertions on response status codes and message contents
Implications for QA workflows#
As cloud-native applications grow in complexity, REST API testing has become both more critical and more resource-intensive. MASTEST's automation of the test creation and execution pipeline addresses this bottleneck, reducing manual effort while maintaining high quality assurance standards. Both LLMs showed strong overall performance but demonstrated distinct strengths across different testing activities, suggesting complementary roles in automated testing strategies.
