A smart toy system to educate children with autism: A comparison of tree-structured programming and machine learning models for an IoT device

Derksen, Gerry

Permalink

https://hdl.handle.net/2142/125580 Copy

Description

Title

A smart toy system to educate children with autism: A comparison of tree-structured programming and machine learning models for an IoT device

Author(s)

Derksen, Gerry

Issue Date

2024-07-11

Director of Research (if dissertation) or Advisor (if thesis)

Ruecker, Stan

Doctoral Committee Chair(s)

Ruecker, Stan

Committee Member(s)

• Twidale, Michael
• Lundgren, Robb
• Salamanca, Juan

Department of Study

Illinois Informatics Institute

Discipline

Informatics

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

Ph.D.

Degree Level

Dissertation

Keyword(s)

• Machine Learning
• Autism
• Toy
• Internet of Things
• Education
• Design

Abstract

In this study, we examine the question of individualized versus average learning using a toy that allows children with mild autism to play color-matching games and math games. The toy was first configured with a decision tree that was the same for the first group of 16 participants. The toy was then reconfigured to use a machine learning (ML) system that customized the experience for each member of the second group of 16 participants. The toy was equipped with a number of sensors that were used by both groups to support the decision as to whether the student should get the next question that was easier, the same level, or harder than the previous question. The sensors included force sensors on the buttons, an accelerometer in the circuit board, and an internal clock to track duration. The other features were captured using cloud service for the Internet of Things (IoT) broker MQTT, which captures performance, difficulty level, and “frustration score,” which is the basis for one of the software’s predictions. The broker also controls the LED’s used for the math and color sequence game. One purpose of the individualized levels of difficulty is to help maintain engagement. It is among the most critical aspects of understanding how people learn. Engaging in a subject is the first step toward learning because it can motivate, stimulate, and instigate a cycle of observing and applying the results of new knowledge and applying it again. It is the precursor to when learning is about to happen or currently taking place. If a student's level of engagement is lost, so is the possibility of learning. This is often due to the targeted level of the information not hitting the audience when they need it. It is hard to struggle through if the information is too dense or complex. Too easy of a task becomes mundane and uninteresting. Learning a subject is often abandoned unless there is something to regain their interest and willingness to return to the subject. These scenarios become even more frustrating if you have autism. Autism spectrum disorder is a mixed array of challenges that are not the same in every individual and to a greater or lesser degree between people. A teaching aid that adjusts to each child's learning level is at the center of this study, and it attempts to capture the information needed to make these adjustments and improve the learning experience for children with autism. This study aims to determine if an educational toy can predict when levels of engagement are such that learning is probable or if it is necessary to re-engage the learner by making the problem less complex, for example. Understanding engagement from a psychological perspective often takes the form of a mental model to construct what is available to a student through perception stimuli, internal motivation, and the ability to assess the problem and other cognitive triggers. Evaluation of the model's mix of complex cognitive triggers only provides the potential for successful learning but does not measure the outcome to see if learning occurred. In this study, an intelligent toy captures physical and cognitive manifestations and performance assessments to quantify learning outcomes. This data is collected to map those outcomes in an effort to train the toy on how to predict future outcomes and modify its behavior to accommodate the student's level of engagement by increasing, decreasing, or steadying the level of difficulty. Game design considerations trigger cognitive stimuli seeking to find physical and cognitive data points corresponding to success and failure responses. As it turns out, engagement was slightly but significantly increased in the ML version of the toy. On average, the ML game group played 29 minutes, while the tree-structured group played 22 minutes. The ML group also played 14 sequences on average, and the TS group played an average of 12.5. The next questions we asked concerned the students' performance (how many answers they got right) and the toy's performance (did it make useful predictions), which is to say, was the student appropriately challenged. Toy prediction results were analyzed using both MANOVA and Kruskal-Wallis. MANOVA is a stronger method but requires parametric data, while Kruskal-Wallis is intended for non-parametric data. In the case of this study, the data formed a bell curve, but the curve was cut off on the left. This means that the data was not parametric, but there have been arguments in the literature for the use of MANOVA in similar circumstances since real-world data is often messy. The results indicate a significant difference between the performance of the toy versions; however, the influence of more accurate predictions on a player's performance is in doubt. The toy's ML prediction rate is a moderate 68%, and although rates over 75% were targeted, the realities of real-world data collection have hindered its performance. Despite this, the MANOVA analysis suggests a significant difference of 2.43 out of 5 questions versus 2.27 out of 5 between the two software's impact on player performance or how often the player answered correctly. However, more study is needed to determine the validity of this finding since Kruskal-Wallis indicates no significant difference. Our final question related to the possibility that the sensors would form super-clusters in terms of their influence on the prediction. If they did, it would be easier to determine which combination of features is most influential. In the design of the toy, pressure values exerted on buttons and motion data collected from the accelerometer were built into the toy and clustered into distinct groups. Still, connections between pressure and motion and the other data features did not reinforce these super-clusters. For example, ranges of pressure data fell into four distinct touch levels 1200-2100, 2100-2700, 2500-3300, and 2600-3600 using peak range values; within the light-touch group, each player had a unique motion pattern. We would have expected that if a player generally had a light touch on the buttons, they might move the toy less or less often. However, this is not the case: each light touch player has a unique motion pattern. This was true of all the pressure data groups. Statistically, each feature did have a significant impact on the prediction of the toy. However, without these super-clustered groups influencing the prediction, it is difficult to determine which combination of features is most influential. The mix of results is supported by the literature on educational research and also underlines the complexity of learning further complicated by the variability of autism.

Graduation Semester

2024-08

Type of Resource

Thesis

Handle URL

https://hdl.handle.net/2142/125580

Copyright and License Information

Copyright 2024 Gerry Derksen

Owning Collections