Researchers use machine learning to help screen readers understand webpage context

Screen reader users who navigate webpages through element lists often encounter vague labels—like "Read more"—that technically comply with web accessibility standards but make no sense without surrounding context. Researchers are investigating whether machine learning can automatically identify which nearby page elements should be linked to these orphaned labels.

The accessibility problem#

When users navigate by element list, they bypass the visual layout that sighted people use to understand relationships between page components. A link labeled "Read more" sitting alone in a list becomes meaningless; a human reader would look at nearby text or images to grasp what it references, but screen reader users cannot easily do this. Although the relevant context exists elsewhere on the page, no mechanism currently connects it to the target element.

The research approach#

A team has registered a study to test whether human-perceived contextual associations between webpage elements can be recovered from the accessibility tree using link prediction models. They plan to build a dataset by having three independent annotators mark contextual relationships on 35 websites, sampled across the Tranco top-million list. Each webpage will be represented as a graph derived from its accessibility tree, enhanced with spatial and semantic information pulled from the DOM and CSS.

Early findings#

The researchers have completed a five-site pilot study to validate their methodology. They compared four machine learning approaches—MLP, GCN, GAT, and SEAL—against two simpler heuristic baselines, measuring performance with metrics like Hit@10 (whether the correct association ranks in the top 10) and Mean Reciprocal Rank. Pilot results ranged from 0.16 to 0.85 across the learned models and 0.08 to 0.30 for heuristic baselines, suggesting machine learning offers promise over simple rules.

Broader implications#

The full study will evaluate whether associations learned from one set of websites generalize to others—a critical test for practical deployment. The research contributes a human-annotated dataset and an empirical benchmark for using graph-based link prediction to improve web accessibility.