New System Generates Design Specs from Screenshots While Keeping Evidence Auditable

Researchers have introduced Secret MCP, an open-source tool that transforms web screenshots into detailed design specifications while maintaining clear boundaries between what's measured, observed, inferred, and unknown.

The Problem with Screenshot-to-Code Systems#

Existing tools that convert screenshots into code prioritize getting a rendered result, but they skip over crucial details. They lose document structure, interaction rules, responsive design patterns, and the source evidence needed to verify where conclusions come from. When systems process multiple references together, they can accidentally mix evidence from one design with logic from another, creating contaminated outputs.

How Secret MCP Works#

Secret MCP addresses these issues through strict separation of concerns. The workflow breaks into distinct stages: retrieval, evidence preparation, model invocation, storage, and inspection.

Large screenshots are resized and divided into overlapping tiles. The system carefully records both the coordinates in the prepared image and the original space, and it measures color palettes directly. It then builds a comprehensive 19-section design contract that covers:

• Page inventories and navigation geometry
• Responsive design matrices
• Component descriptions
• Accessibility requirements
• Acceptance criteria
• Explicit labeling of measurements, observations, inferences, and unknowns

Each web reference is processed individually through a sampler interface. The system sends just one request per reference with context isolation enabled, ensuring that evidence from one design cannot leak into another.

Validation Results#

Testing of the system (commit c130c9c) demonstrated the approach works in practice:

• A live integration test processed two web references and generated two complete documents with zero cross-reference contamination
• An audit of three externally generated design specifications confirmed all 19 required sections were present in every document, with one unique identifier per design
• The audited documents contained 1,943 pixel-valued measurements and 319 color values, all traceable to source evidence