New benchmark tests whether neural language models need innate linguistic rules

A new study from researchers on arXiv evaluates a fundamental debate in linguistics: whether human brains must contain inborn knowledge about grammar to learn language. The work examines this question by measuring how well artificial neural networks—including Transformers and LSTMs—can pick up sophisticated grammatical patterns from surprisingly small training datasets.

The poverty of the stimulus debate#

Linguists have long argued that children acquire language despite receiving limited, imperfect input. This observation, called the Poverty of the Stimulus Hypothesis, has traditionally led researchers to conclude that the human brain must come pre-equipped with innate rules about how language works. Without such scaffolding, the reasoning goes, the available examples would be too sparse to explain how children learn grammar so quickly and correctly.

Recent work has tested this assumption by training neural networks on complex linguistic phenomena. Early results hinted that these models could generalize to new grammatical structures without explicit programming for language rules. Yet previous studies examined isolated cases using different testing methods, raising questions about whether findings held across different types of language patterns and learning scenarios.

A unified testing framework#

The new research introduces a comprehensive benchmark called PoSBench that evaluates performance across four canonical phenomena relevant to the poverty of the stimulus debate. The team trained standard neural network architectures—Transformers, LSTMs, and n-gram models—and measured their ability to make correct grammatical generalizations from limited input around 10 million words.

The key findings paint a nuanced picture. Neural models did achieve above-chance performance at generalizing from surprisingly constrained data, suggesting that acquired linguistic rules alone might be sufficient for certain grammatical competence without inborn linguistic constraints. However, the learning efficiency of these systems lagged far behind that of human children as the volume of training data increased.

The role of cognitive biases#

The researchers also tested whether adding cognitively motivated inductive biases—design choices inspired by how human brains might work—would improve performance. These biases did bolster overall syntactic abilities, but they did not reliably enhance the models' capacity to solve the specific language problems targeted by poverty of the stimulus arguments.