14
Koedinger, K. R., Anderson, J. R., Hadley, W. H., & Mark, M. A. (1997). Intelligent tutoring goes to school in the big city. International Journal of Artificial Intelligence in Education (IJAIED), 8, 30-43.
Background
Practical Algebra Tutor (PAT), is an intelligent tutoring system within the Pittsburgh city school system. The authors evaluated the PAT in several algebra classrooms in the Pittsburgh school system. The evaluation compares mathematical achievement of students using the tutoring system with students of similar backgrounds not using the system. A small sample of higher achieving students is also part of the comparison. The comparison allows the authors to measure the effectiveness of PAT. The PAT system is a companion of the PUMP, Pittsburgh Urban Mathematics Project, algebra curriculum. The PUMP curriculum’s design engages students in analysis of real-world math situations. It also has them use computational tools to solve real-world equations. The authors emphasize the use of real-world equations to provide students with context (Koedinger et al, 1997, 33).
PAT uses the ACT cognitive theory where a “cognitive model is written as a system of if-then production rules that are capable of generating the multitude of solution steps and mis-steps typical of students”(Koedinger et al, 1997, 32). PAT presents students with equations provided by classroom teachers. The teachers use problem which students could encounter in the real-world The students use computational tools such as spreadsheets and graphs to work through the equations. The intelligent tutoring system tracks their progress, provides feedback and assists the student if the student requests. By comparing PAT and non-PAT using students performance, the authors note that the PAT users scored 100 percent higher on targeted curricula and 15 percent higher on standardized tests overall than non-PAT users. Evaluators include two standardized tests to measure student learning, the Iowa Algebra Aptitude test and a subset of the Math SAT which is appropriate for students in the 9th grade (Koedinger et al, 1997, 39).
Key Points:
- The PAT intelligent tutoring had success being implemented in an urban school setting
- Real-world application of the content made problems more relatable
- Based on the ACT theory
- Programmed as if-then statements able to generate possible student solution steps or mis-steps
- Model tracing and Knowledge Tracing
- Model tracing tracks students progress on a problem to a model of the problem solution so when mis-steps occur or hints are needed the instruction is personalized to the student
- Knowledge tracing tracks student learning from problem to problem to identify student strengths and weaknesses to better suggest targeted instruction
Design Principles
The design of the PAT tutoring system utilizes several concepts to help ensure student success. Timely feedback allows users to see errors when they occur. For common errors, PAT offers instruction on what type of error was made. Switching the x and y-axis while plotting graph points is a common error for which PAT will supply instruction. However, if the mis-step is not a common mistake, an instructional message is not displayed. Instead, the mistake is clearly visible by changing its color. The user fixes their error and keeps working through the problem. By not providing too much feedback on inconsistent mistakes, the system does not discourage students in their progress. When the timely feedback is paired with targeted instruction, the user is better able to adjust their misconception regarding solving the equation (Koedinger et al, 1997, 35).
The PAT system also employs model tracing and knowledge tracing which allows the system to provide instruction that is individualized for the user. Model tracing allows the tutoring system to compare the student’s progression through the system with a model of how the problem is solved, including several potential mis-steps that the user could make. Model tracing also indicates that the tutor is tracking the approach that the user is using to solve the problem. If the user has a problem either they do a step incorrectly or they ask for the instruction, PAT knows the approach they are using to solve the problem and is able to provide instruction within that approach(Koedinger et al, 1997, 32). The user has the option to ask for help when solving problems, and the more help students ask for the more detailed the help becomes (Koedinger et al. 1997, 35).
PAT uses knowledge tracing to track the progress a user has as they do multiple problems. The system is able to keep track of the problems the user consistently have trouble completing as well as the ones they complete without issue. This allows the system to keep challenging the user with problems where they need extra instruction and practice(Koedinger et al., 1997, 32). These design principle could account for PAT’s success within the Pittsburgh classrooms due to its ability to provide such personalized instruction.
Discussion Questions
- Would all subjects be able to implement the design principles that PAT employed? Would humanities be able to successfully create a model which could track student progress?
- This particular Intelligent Tutoring System did not employee self-pacing due to the evaluations limited context. Would self-pacing further improve students outcomes when using PAT? What role could self-pacing play in further iterations of other tutoring systems?
Additional Resources
Ritter, S., Anderson, J., Cytrynowicz, M., & Medvedeva, O. (1998). Authoring content in the PAT algebra tutor. Journal of Interactive Media in Education, 1998(2).
