Ranking features used in modeling student collaboration using multimodal learning analytics

Rajarathinam, Robin Jephthah

Permalink

https://hdl.handle.net/2142/130113 Copy

Description

Title

Ranking features used in modeling student collaboration using multimodal learning analytics

Author(s)

Rajarathinam, Robin Jephthah

Issue Date

2025-05-13

Director of Research (if dissertation) or Advisor (if thesis)

Mercier, Emma

Doctoral Committee Chair(s)

• Mercier, Emma
• Kang, Jina

Committee Member(s)

• D'Angelo, Cynthia
• Paquette, Luc

Department of Study

Curriculum and Instruction

Discipline

Curriculum and Instruction

Degree Granting Institution

University of Illinois Urbana-Champaign

Degree Name

Ph.D.

Degree Level

Dissertation

Keyword(s)

• Collaborative learning
• Multimodal learning analytics
• Collaborative learning analytics
• feature ranking
• machine learning
• Group behavior modeling
• Engineering education

Abstract

This dissertation tackles a persistent challenge in computer-supported collaborative learning (CSCL): how to identify the behavioral signals that matter most for detecting productive peer interaction in noisy, face-to-face classrooms. Drawing on the fourth design cycle of the NSF-funded CSTEPS project, the study asks (a) how individual data streams—video, audio, log traces, and speech transcripts—characterize specific collaborative behaviors, (b) whether fusing those streams improves machine classification, and (c) which fine-grained features contribute most to accurate prediction. By ranking features across modalities, the work aims to guide researchers and learning-technology designers in selecting sensors and analytics that yield the greatest instructional value. A quasi-experimental field study captured 31 undergraduate engineering groups over seven weeks (53 students wore head-mounted microphones, generating 34 multimodal sessions) during tablet-mediated problem-solving discussions. After synchronizing video, individual audio, shared-tablet logs, and automatic transcripts, hundreds of prosodic, linguistic, visual, and interaction features were extracted. Feature space was reduced through correlation filtering, PCA, and recursive feature elimination, then modeled with six classical classifiers (decision tree, random forest, SVM, AdaBoost, XGBoost, Naïve Bayes). A five-fold group-level cross-validation protocol, augmented by SMOTE to counter class imbalance, ensured generalization to unseen groups. Unimodal baselines were followed by early- and late-fusion multimodal models. Unimodal models confirmed complementary strengths: log interaction ratios best signaled On Task engagement (balanced accuracy = 0.84) while audio prosody captured OffTopicTalk and ToolTalk. Early fusion raised balanced accuracy for most codes (e.g., 0.84 for On Task), but late fusion delivered the most consistent gains, peaking at 0.90 balanced accuracy and 0.93 F1 for On Task, with moderate improvements for cognitively demanding behaviors such as MetaCognition and BuildingIdeas. Feature-importance analysis revealed that (i) loudness and pitch slopes were dominant audio cues for unstructured off-task dialogue, (ii) sentiment-laden textual n-grams differentiated idea-building talk, (iii) gaze minima and cheek-raiser action units flagged explanations, and (iv) scroll-balance ratios in the whiteboard log robustly indexed group engagement. Methodologically, the study offers a tested pipeline—group-wise cross-validation, targeted SMOTE, and decision-level fusion—for researchers modeling small-group collaboration at scale. Theoretically, it demonstrates that multimodal signals are synergistic rather than redundant: preserving modality-specific classifiers before aggregation yields the most balanced view of complex behaviors. Practically, the ranked feature set pinpoints a lean, interpretable subset (e.g., loudness variance, gaze convergence, scrolling equality) that can power real-time teacher dashboards without excessive sensor overhead. Future work should extend adaptive fusion strategies and leverage emerging 360° video and unobtrusive audio wearables to boost data fidelity and ecological validity.

Graduation Semester

2025-08

Type of Resource

Thesis

Handle URL

https://hdl.handle.net/2142/130113

Copyright and License Information

Copyright 2025 Robin Jephthah Rajarathinam

Owning Collections