Study reveals how AlphaZero internalizes search strategy into learned behavior

AlphaZero fuses a neural network with Monte Carlo tree search, but a fundamental question remains unanswered: when search provides useful tactical guidance during training, does the network absorb that knowledge into its weights, or does the agent continue to depend on real-time search at test time?

The research question#

A new paper on arXiv tackles this by introducing Cross-Phase Prior Intervention (CPI), a technique that independently switches tactical guidance on and off during training versus evaluation. This allows researchers to isolate what the network has genuinely learned from what it merely borrows from search during play.

The focus is deliberately specific: whether a network's policy network will enforce a forced defensive move when search-time hints are absent. Testing occurred on a sealed, single-shot evaluation in 9x9 Gomoku and 19x19 Go—games where the ability to recognize and execute basic tactical obligations is measurable.

What the experiment revealed#

The results show that networks do absorb some internalized capability, but the effect is partial and context-dependent:

• Gomoku: Removing the prior dropped defensive responses from 13.8% to 26.3%, a substantial residual.
• Go: The shift was more dramatic—from 0.6% to 33.8%—indicating stronger internalization in the larger game.
• Re-enabling the prior restored near-complete performance (around 100%), revealing dependency gaps of 73.7 and 65.8 percentage points between online and internalized competence.

Soft reweighting of action probabilities proved as effective as hard action restriction, suggesting the mechanism is not simple action pruning but rather a deeper shift in learned representations.

What the findings mean#

The research frames search as a behavioral curriculum during training. The lessons agents absorb are real—persistent in weights—but also partial and narrowly tied to the geometric patterns encountered. A network trained to defend specific positions generalizes weakly; the same defensive knowledge transferred poorly to structurally novel positions (+1.3 points versus +17.3 on in-distribution cases).

This distinction matters for understanding what AlphaZero agents actually know: online competence (performance with search) and internalized competence (performance relying on learned weights alone) are different quantities that standard ablation studies may conflate.