AI
New Framework Proposes Stability-Based Approach to Measuring AI General Intelligence
Researchers suggest AGI evaluation should measure how capabilities persist under pressure rather than relying on static test scores with equal domain weights.
Contemporary approaches to assessing artificial general intelligence tend to evaluate performance across multiple domains but treat each area as equally significant and rely on single snapshots in time. A new arXiv paper challenges this methodology, arguing that intelligence—both human and machine—functions more like a self-regulating system in which related capabilities support one another and remain robust under pressure.
The Case for Stability Over Static Scores#
The core argument rests on how natural intelligence actually works. Human cognitive abilities don't operate independently; rather, they form interconnected clusters where certain foundational skills help maintain others when conditions change. Current AGI testing misses this dynamic by assigning equal importance to every domain and measuring performance at a single point in time. Such approaches fail to distinguish between skills that remain solid under real-world stress and those that crumble when tested differently or with delay.
A Homeostatic View of Intelligence#
The paper frames general intelligence as a homeostatic property cluster—a collection of abilities paired with the mechanisms that preserve their co-presence even when circumstances shift. Under this model, certain domains carry more weight than others because they contribute disproportionately to the stability of the entire system. The research draws on human intelligence research, particularly the Cattell-Horn-Carroll (CHC) cognitive framework, to identify which capabilities matter most for keeping the whole system balanced.
Proposed Measurement Tools#
The authors offer two evaluation extensions designed to work with existing multi-domain testing batteries:
- Centrality-Prior Score: Incorporates CHC-informed weights that reflect the causal importance of each domain, with transparent sensitivity analysis showing how results shift as weightings change.
- Cluster Stability Index Family: A set of measures that separately assess profile persistence across test sessions, durable learning over time, and error correction abilities.
