Researchers tackle aspect-based summarization when key topics are missing or irrelevant

Aspect-based summarization—where systems create summaries focusing on specific user-requested topics—is gaining traction for producing more targeted outputs. However, a practical gap remains: real-world documents often contain content that doesn't align with predefined aspects users ask for, or the guidance may be incomplete or irrelevant.

Researchers have now formalized this challenge in a new task called Content-Aware Refinement of Provided Aspects for Summarization (CARPAS), exploring how large language models can intelligently adapt requested aspects to match what's actually in a document before generating summaries.

The core problem#

Most existing aspect-based summarization approaches assume users provide a clean, complete set of topics that align well with the source material. In reality, the mismatch between user expectations and document content creates friction. Rather than blindly following misaligned instructions, the researchers argue that systems should intelligently filter, adjust, or expand aspects based on what the document actually contains.

The research approach#

The team created three new benchmark datasets and tested four different prompting strategies with LLMs to understand how they currently handle aspect refinement. A key finding: models tend to over-generate aspects, producing unnecessarily long summaries that diverge from the user's intended focus.

To address this, the researchers introduced a two-stage framework. The first stage generates "scope guidance"—a lightweight assessment of how many and which aspects the document actually supports—before the refinement and summarization steps proceed. This upstream guidance reduces overproduction and improves alignment.

Key insights#

Experiments across all datasets showed significant performance gains with the proposed approach. Perhaps most revealing: the research exposed how LLMs comply differently when the number of aspects they naturally identify mismatches the number requested. This behavioral pattern is critical for understanding how these models will perform in real-world deployment where aspect guidance is often imperfect.

The work opens a pathway toward more robust summarization systems that can negotiate between user intent and document reality rather than rigidly following potentially flawed input.