| | |
Summary: A formative evaluation of AnimalWatch
Ivon Arroyo1
, Tom Murray1,3
, Joseph E. Beck2
, Beverly P. Woolf1
, Carole R. Beal1
1
Universityof Massachusetts, Amherst- 2
Carnegie Mellon University - 3
Hampshire College
Abstract. We present the results of a deep formative evaluation of AnimalWatch, an
intelligent tutor for arithmetic. Students learned with AnimalWatch, and had a positive
experience with it. Still, we detected AnimalWatch selected too easy problems. We
proposed different hypotheses for this behavior and analyzed the performance of each
ITS component, by observing the interactions of hundreds of students with the system.
We conclude with a report of lessons learned that would make ITS better.
Introduction
Evaluating Intelligent Tutoring Systems (ITS) is not an easy task due to their dynamic
nature. Most quantitative ITS evaluations in the past consisted of pre to post test
improvements (Koedinger, 97; Luckin, 99]. These evaluations are appropriate for ITS in
|
