24 AI-Speak: How Adaptive Systems Learn the Learner, part 1

When looking at an adaptive learning system, it is very hard to tell where it adapts¹. What technology is used and what it is used for also changes across systems.

However, all adaptive learning systems know who they teach (knowledge about the learner), what they teach (knowledge about the domain), and how to teach (knowledge about pedagogy)².

An ideal ALS adapts itself in multiple ways. In the outer loop, the sequence of learning activities is adapted – similar to Youtube adapting recommended list of videos. The outer loop might also personalise learning approaches and difficulty levels.

In the inner loop, within each activity, the ALS monitors step-by-step progress. It adapts feedback and hints to correct misconceptions, if any. It might also point to additional content if the student has a problem remembering a previously learnt concept. Some experts argue that the inner loop is best left to the instructor: not only is it costly and time-consuming to program all the rules for the specific subject and task, but the teacher’s knowledge and experience will always trump that of the machine³.

How adaptive systems learn the learner

Like all recommendation problems (See How Youtube Learns You Part 1), ALS splits the task into one or more surrogate questions that can be answered by the machine. Again, the choice of what to ask -and thus, what to predict, has a big impact on what recommendation is shown.

Marketing material often mention multiple goals – improved scores, employability and engagement. Given the proprietary nature of the systems, it is usually unclear what questions are coded into the systems, what goals are being optimised for, and how short-term goals are differentiated from long-term goals (example, mastery of a given content before progressing to the next level)⁴.

Where machine learning is used, regardless of the goals chosen, the prediction itself is based on other learners with similar skill levels and preferences. That is, learners whose models are similar.

The learner model

For creating a student model, developers ask what student characteristics are relevant to the learning process. Unlike teachers who can directly observe their students and adjust their approach, machines are limited to the data that can be collected and processed by them.

Typical characteristics considered in a student model:

• What does the student know – their knowledge level, skills and misconceptions^5,2,6. These are usually inferred through assessments, for example, the answer a student submits for a maths problem¹. This prior knowledge is then compared with what they will need to know at the end of the learning period.
• How does a student prefer to learn:  the learning process and preferences^5,6. For example, the number of times a student attempts a question before getting it correct, the types of resources consulted, the ratings they gave for an activity¹, or the material that engaged them most – images, audio or text². ALSs may also record when and how skills were learned and which pedagogies worked best⁶.
• Is the student feeling motivated:  feelings and emotions can be recorded directly by the student or extracted indirectly from speech, facial expressions, eye tracking, body language, physiological signals, or combinations thereof. This information can then be used to nudge the student out of negative states such as boredom or frustration that inhibit learning, into positive states such as engagement or enjoyment⁷.
• What about cognitive aspects like memory, attention, problem solving skills, decision-making capability, analysis of situations and critical thinking⁵?
• How do they communicate and collaborate⁵? For example, do they post comments on other students’ feed and how do they discuss with others to solve problems¹?
• What about meta-cognitive skills such as self-regulation, self-explanation, self-assessment and self-management⁵, help-seeking, being aware of and being able to control their own thinking? For example, how they select their learning goals, use prior knowledge or intentionally choose problem-solving strategies⁵.

While this data changes and has to be recorded and updated, models also contain static characteristics such as age, gender, mother tongue and email identity².

Most of the ALSs create learner models based on interactions with students. Some also glean info from other sites, especially social media. Once a model is available for each learner, the machine calculates which students are similar to one another and estimate the probability that a given student will benefit from an activity, example or question³.

The domain model

We can draw a loose parallel between learning objects in an ALS and videos on Youtube. A subject can be broken down into concepts and skills; these are called knowledge units (KUs) and are what the learner needs to know³. Each KU has a set of learning objects through which content can be learned, and a set of activities to assess learning. Some authors further break down learning objects into learning activities; we do not do this here.

Learning objects can be text to read, a video, a set of problems, interactive activities (from simple fill-in-the-blanks to scenario-based learning activities), interactive animation, etc.¹ The learning objects give what the learner needs to know, and the assessment activities indicate if the knowledge has been acquired³. The domain model contains all the features of the learning objects, including the associated KU and assessment.
What a learner learns next will also depend on the inter-relationships between the KUs.

These KUs need to go into the model too: learning objects A and B might both be pre-requisites for learning object D. Thus, A and B have to be mastered before D. There is order amongst some KUs that tell us how we learn³. Conversely, if the student solves correctly a problem that corresponds to D, it would be a good bet that he or she mastered A and B too.

Subject-matter experts can provide some of these relationships. The other inferences can be learnt by the machine, which can predict the probability that a KU has been mastered. This mastery involves the system knowing that the learner has mastered A and B, given that he answered questions under D. It can then use this information, along with other features of learner and domain models, to recommend learning pathways and learning objects.

Other features of learning objects could include the activity’s level of difficulty, its popularity and its ratings. The goal here, as in the case of Youtube recommendation, is to squeeze out as much information as possible from the data available available.

¹ EdSurge, Decoding Adaptive, Pearson, London, 2016.

² Alkhatlan, A., Kalita, J.K., Intelligent Tutoring Systems: A Comprehensive Historical Survey with Recent Developments, International Journal of Computer Applications 181(43):1-20, March 2019.

³ Essa, A., A possible future for next generation adaptive learning systems, Smart Learning Environments, 3, 16, 2016.

⁴ Bulger M., Personalised Learning: The Conversations We’re Not Having, Data & Society Working Paper, 2016.

⁵ Chrysafiadi, K., Virvou, M., Student modeling approaches: A literature review for the last decade, Expert Systems with Applications, Elsevier, 2013.

⁶ Groff, J., Personalized Learning: The state of the field and future directions, Center for curriculum redesign, 2017.

⁷ du Boulay, B., Poulovasillis, A., Holmes, W., Mavrikis, M., Artificial Intelligence And Big Data Technologies To Close The Achievement Gap, In: Luckin, Rose ed. Enhancing Learning and Teaching with Technology. London: UCL Institute of Education Press, pp. 256–28, 2018.