AI
New Dataset Brings Multimodal Peer Review Analysis Beyond Computer Science
Researchers introduce FMMD, a dataset that combines visual manuscript elements with open peer reviews across multiple scientific disciplines to improve AI systems for scholarly evaluation.
Automated systems for evaluating scholarly papers are becoming more common in the peer review process, spurring renewed interest in AI-assisted manuscript evaluation research. However, a fundamental gap has hindered progress: most datasets used to train and test these systems focus narrowly on text alone and concentrate on computer science research, missing the visual and structural complexity that reviewers encounter in practice.
The Dataset Gap#
Today's peer review datasets have significant blind spots. Reviewers routinely examine figures, tables, and complex visual layouts when assessing scientific claims—yet existing resources remain overwhelmingly text-focused. Additionally, the field's traditional emphasis on computer science publications limits the ability to develop review systems that generalize across diverse scientific disciplines. Equally important, most datasets fail to precisely link review comments to the specific manuscript versions being evaluated, obscuring how feedback drives iterative paper revisions.
Introducing FMMD#
To address these limitations, researchers have created FMMD (Foundational Multimodal Multidisciplinary dataset), a new peer review resource drawn from F1000Research. The dataset combines visual and structural data from manuscripts—including figures, tables, and layouts—alongside version-specific reviewer reports and editorial decisions. A key innovation is its explicit alignment of individual review comments with the exact manuscript version under examination, enabling detailed study of how peer review and manuscript development interact over time.
Scope and Research Applications#
FMMD extends beyond computer science to cover a wide range of scientific domains, reflecting the multidisciplinary nature of modern research. The dataset supports multiple research tasks, including assessing visual-semantic consistency in manuscripts, generating review comments triggered by figures, and predicting editorial decisions based on multimodal manuscript content. These capabilities provide a more comprehensive foundation for developing and validating multimodal automated peer review systems.
