Funded by the European Union. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or the European Education and Culture Executive Agency (EACEA). Neither the European Union nor EACEA can be held responsible for them. Project number: 2023-1-NL01-KA220-HED-000155675. Casus 5: Een AI-gebaseerd systeem voor formatieve en summatieve beoordeling in data science-cursussen General information Amelio, A., & De Medio, C. (n.d.), 22 December 2020. An AI-Based System for Formative and Summative Assessment in Data Science Courses, International Journal of Artificial Intelligence in Education (2021) 31:159–185 https://doi.org/10.1007/s40593-020-00230-2 The paper discusses an AI-based system designed for formative and summative assessments in data science courses. This system automates the grading process and provides feedback to both students and professors. This study's aim is to evaluate the system's effectiveness by comparing the time taken for grading, the accuracy of the grading, and the impact on student learning outcomes. Description of case The study evaluated time efficiency on grading manually versus grading with the AI tool, the grading accuracy by comparing the AI tool's accuracy to the manual grading's accuracy, the learning outcomes (the impact of automated feedback of student performance in final exams and the usability of the tool, which was based on the students' feedback on the system's usability. Lessons learned The system was expected to enhance student learning by offering timely and accurate feedback. The Model performance showed that only a slight improvement in performance when distance-based features were included along with sentence embeddings, which suggests that sentence embeddings alone were effective in representing the semantic content of the answers, especially when the answers and correct solutions had high lexical overlap. It was useful for both formative and summative assessments. In formative assessments, students used the tool for homework and received automated feedback, which was later compared to manual feedback. In summative assessments, exams were corrected either manually or through the AI system, allowing for a comparison of performance between human and AI grading. Implications for practice Efficiency in Grading, since the AI system reduces grading time, allowing instructors to focus on other educational tasks, and ensures consistent, unbiased evaluations, enhanced, Student Feedback, since it provides immediate, detailed feedback, helping students learn and improve continuously, Scalability, since it facilitates handling large classes, making it ideal for MOOCs and large enrolment courses, and Focus on Learning, since it frees up instructor time to offer personalised support and improve teaching strategies.
RkJQdWJsaXNoZXIy NzYwNDE=