New Japanese Dataset Shows Native Data Outperforms Translated Content for AI Vision Models

Researchers have created a large-scale Japanese image-text dataset that reveals an important lesson for multimodal AI: locally sourced training data can deliver stronger results for culture-specific tasks than relying solely on translated global datasets.

The team behind the effort, detailed in a paper submitted to arXiv, built WAON, described as the largest publicly available native Japanese image-text collection, extracted from authentic Japanese web content in Common Crawl. The dataset contains roughly 155 million image-caption pairs.

Why Native Data Matters#

While recent advances in contrastive vision-language models have shown that training on diverse global data—without English-only filters—improves cross-cultural performance, the researchers asked a more specific question: can models trained on natively sourced content outperform those fine-tuned on machine-translated versions of English data?

The answer, according to their experiments, is yes. Models trained on WAON consistently achieved better results on Japanese cultural benchmarks than counterparts trained on English-to-Japanese translated datasets, even when controlling for scale, filtering, and training resources.

Benchmark and Resources#

The team also introduced WAON-Bench, a manually curated evaluation set spanning 374 Japanese cultural classes, designed to measure performance on culture-specific understanding tasks. They conducted controlled experiments across two different model families to isolate the impact of native origin versus other variables.

The researchers have released the dataset, benchmark, trained models, and code to the research community, providing a foundation for further work on culturally adapted AI systems.