Large Language Models Show Promise for Mobile App Testing Text Input Generation

Mobile app quality assurance depends heavily on automated GUI testing, but text-input components have long posed a challenge—many app screens require semantically meaningful text before users can proceed. Researchers have turned to large language models as a potential solution, yet critical questions remain about which UI representations work best, how feedback mechanisms affect outcomes, and where human input adds value.

A comprehensive empirical study published on arXiv examines these questions systematically. The work evaluates nine state-of-the-art LLMs tested across 115 real-world Android applications, comparing three distinct approaches to providing UI context:

• Extracted textual context: Plain-text descriptions of UI elements
• UI-hierarchy XML: Structured markup representing the application interface
• Vision-based input: Screenshots processed by multimodal models

Key Findings#

The research reveals that both extracted context and XML representations achieved comparable page-pass-through rates of around 71%, while vision-based input reached 65%—but at substantially higher token costs, making it less economical for large-scale testing.

When testing the models' ability to detect actual bugs through invalid text input, LLMs achieved roughly 51% detection rates. However, introducing a feedback-enhanced protocol—where execution outcomes inform subsequent attempts—boosted page-pass-through rates to 69–74% and raised bug-detection performance to 51–65%.

Human testers further refined the process, yielding additional improvements beyond what fully automated approaches achieved. The researchers integrated their findings into DroidBot, an existing open-source Android testing framework, to augment its exploration capabilities.

Implications for Practice#

The study distills actionable insights on selecting appropriate context representations, optimizing cost-effectiveness, designing feedback strategies, and structuring human-LLM collaboration. Rather than treating LLMs as standalone solutions, the most effective approach pairs them with iterative feedback loops and human oversight—a lesson that extends beyond mobile testing to broader software engineering practices.