New Framework Harnesses AI to Score Student Assessments Across Multiple Subjects

Researchers have developed a human-centered approach that enables large language models to grade student responses more reliably across science, computing, and engineering courses. The system, called CoTAL (Chain-of-Thought Prompting + Active Learning), pairs AI grading with iterative refinement from teachers and students to improve scoring accuracy and explanation quality.

The Problem with Generic AI Grading#

Most existing methods for using LLMs in education rely on generic prompts without careful tailoring to specific learning objectives. This raises questions about whether such approaches can fairly and accurately assess student work across different subjects and pedagogical contexts. Prior research has not thoroughly tested how well prompt engineering techniques transfer beyond narrow domains.

How CoTAL Works#

The framework rests on three key pillars:

• Evidence-Centered Design: Aligns assessment tasks and grading rubrics directly to curriculum goals rather than treating them separately
• Iterative Refinement: Teachers and students provide feedback on both AI scores and explanations, which then guides improvements to the prompts and rubrics
• Chain-of-Thought Reasoning: Prompts LLMs to explain their reasoning step-by-step, making grading decisions more transparent and traceable

Rather than running the LLM once and accepting its output, the system continuously refines itself based on human input and evidence of where it succeeds or fails.

Results and Validation#

Testing the system on GPT-4 revealed substantial gains: the human-in-the-loop approach with chain-of-thought reasoning and labeled examples yielded scoring improvements up to 38.9% over a baseline model with no prompt engineering or feedback loop.

Both teachers and students rated CoTAL as effective at scoring responses and providing educational feedback. Their critiques identified gaps in explanation quality and grading accuracy, revealing insights that would be difficult to obtain through automated testing alone.

Why It Matters#

The work demonstrates that LLMs can become more reliable and fair assessment tools when paired with structured instructional design and human oversight—suggesting a pathway for bringing AI-assisted grading into real classrooms while maintaining educator control and transparency.