Automated Prolog Feedback System Boosts Student Grades and Satisfaction

Delivering timely, personalized feedback to hundreds of students in a single course remains one of higher education's most persistent challenges. A new study on an automated assessment system for Prolog programming reveals how intelligent feedback can meaningfully improve student outcomes and satisfaction at scale.

The ProHelp Platform#

Researchers evaluated ProHelp, an automated assessment tool built on the GitSeed framework and deployed in a 365-student logic programming course. The system automatically grades student submissions and generates targeted feedback without requiring instructor intervention—a critical advantage for large undergraduate classes where manual assessment becomes prohibitively time-consuming.

Seven Types of Feedback, Ranked by Usefulness#

The study surveyed 144 students and assessed how they valued seven distinct feedback mechanisms:

• Automatic testing — checking code against expected outputs
• Open choice point warnings — alerting students to potential logic errors
• Predicate scoring — evaluating individual function performance
• Syntax error highlighting — catching grammar mistakes in code
• Score rankings — showing student standing relative to peers
• Solution type validation — confirming logical correctness
• Unknown predicate name suggestions — recommending corrections for undefined functions

Students ranked automatic testing as by far the most useful, followed by choice point warnings and predicate scoring—differences that reached statistical significance.

Student Outcomes and Preferences#

Three-quarters of surveyed students felt the feedback directly helped raise their grades. The system earned a System Usability Scale score of 78.5, placing it in the "B+" range and indicating solid usability. Notably, neither student interest level, effort on optional exercises, nor reliance on large language models significantly shaped how students perceived the feedback's value.

When asked about future enhancements, students expressed strong interest in seeing side-by-side comparisons of the outputs their code actually generated versus what the tests expected—a feature that could further narrow the gap between writing code and understanding correctness.