Researchers develop framework to identify and manage accountability gaps in AI-powered software

Responsibility gaps—situations where accountability becomes murky or diffused across human and system actors—pose growing challenges for software built with generative AI. Researchers have proposed a structured methodology to identify these gaps early and embed oversight requirements directly into the development process.

The Challenge#

When AI systems make decisions or contribute to outcomes, it becomes harder to pin down who bears responsibility for failures or adverse outcomes. This ambiguity increases as more tasks shift between humans and machines, creating "responsibility gaps" that traditional requirements engineering approaches often fail to capture.

The Proposed Solution#

The new framework operates across three layers:

• Conceptualization: Defines responsibility along human and system dimensions, explaining how gaps emerge from their interaction
• Methodology: Provides a step-by-step process to spot responsibility gaps by analyzing these dimensions and deriving oversight requirements using established engineering practices
• Formalization: Packages findings into a reusable artifact called a Deductive Backbone Table, which documents the reasoning from gap identification through to oversight requirements

Validation Results#

Testing with users and expert reviewers found that the enhanced approach outperformed a conventional goal-oriented requirements engineering baseline on most quality metrics. Participants viewed the new method favorably, and experts awarded higher scores for consistency and traceability in the generated requirement artifacts—particularly valuable for teams trying to ensure accountability remains clear as systems become more complex.

The framework suggests that baking human oversight requirements into the design phase, rather than adding them afterward, can help prevent ambiguity from taking root in AI-enabled software development.