Attitudes, behaviors, and learning outcomes from using classtranscribe, a UDL-featured video-based online learning platform with learnersourced text-searchable captions

Zhang, Zhilin

Permalink

https://hdl.handle.net/2142/113047 Copy

Description

Title

Attitudes, behaviors, and learning outcomes from using classtranscribe, a UDL-featured video-based online learning platform with learnersourced text-searchable captions

Author(s)

Zhang, Zhilin

Issue Date

2021-07-19

Director of Research (if dissertation) or Advisor (if thesis)

• Angrave, Lawrence
• Karahalios, Karrie

Department of Study

Computer Science

Discipline

Computer Science

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

M.S.

Degree Level

Thesis

Keyword(s)

• Human-Computer Interaction
• Learning Sciences and Technology
• Information Systems
• Video Search
• Crowdsourcing
• Learnersourcing
• Accessibility
• Online Education
• Universal Design for Learning
• MOOCs
• Captions

Abstract

This thesis consisted of a series of three studies on students' attitudes, behaviors, and learning outcomes from using ClassTranscribe, a Universal Design for Learning (UDL) featured video-based online learning platform. ClassTranscribe provided accurate accessible transcriptions and captioning plus a custom text-searchable interface to rapidly find relevant video moments from the entire course. Users could edit the machine-generated captions in a crowdsourcing way. The system logged student viewing, searching, and editing behaviors as fine-grained web browser interaction events including full-screen-switching, loss-of-focus, caption searching and editing events, and continued-video-watching events with the latter at 15-second granularity. In Study I, lecture material of a sophomore large-enrollment (N=271) system programming 15-week class in Spring 2019 was delivered solely online using a new video-based web platform - ClassTranscribe. Student learning behaviors and findings from four research questions were presented using individual-level performance and interaction data. Firstly, we reported on learning outcomes from alternative learning paths that arose from the course's application of Universal Design for Learning principles. Secondly, final exam performance was equal or better to prior semesters that utilized traditional in-person live lectures. Thirdly, learning outcomes of low and high performing students were analyzed independently by grouping students into four quartiles based on their non-final-exam course performance of programming assignments and quizzes. We introduced and justified an empirically-defined qualification threshold for sufficient video minutes viewed for each group. In all quartiles, students who watched an above-threshold of video minutes improved their in-group final exam performance (ranging from +6% to +14%) with the largest gain for the lowest-performing quartile. The improvement was similar in magnitude for all groups when expressed as a fraction of unrewarded final exam points. Finally, we found that using ClassTranscribe caption-based video search significantly predicted improvement in final exam scores. Overall, the study presented and evaluated how learner use of online video using ClassTranscribe predicted course performance and positive learning outcomes. In Study II, we further explored learner's searching behavior, which was shown to be correlated with improved final exam scores in the first study. From Fall 2019 to Summer 2020, engineering students used ClassTranscribe in engineering courses to view course videos and search for video content. The tool collected detailed timestamped student behavioral data from 1,894 students across 25 engineering courses that included what individual students searched for and when. As the first study showed that using ClassTranscribe caption search significantly predicted improvement in final exam scores in a computer science course, in this study, we presented how students used the search functionality based on a more detailed analysis of the log data. The search functionality of ClassTranscribe used the timestamped caption data to find specific video moments both within the current video or across the entire course. The number of search activities per person ranged from zero to 186 events. An in-depth analysis of the students (N=167) who performed 1,022 searches was conducted to gain insight into student search needs and behaviors. Based on the total number of searches performed, students were grouped into “Infrequent Searcher” (< 18 searches) and “Frequent Searcher” (18 to 110 searches) using clustering algorithms. The search queries used by each group were found to follow the Zipf’s Law and were categorized into STEM-related terms, course logistics and others. Our study reported on students’ search context, behaviors, strategies, and optimizations. Using Universal Design for Learning as a foundation, we discussed the implications for educators, designers, and developers who are interested in providing new learning pathways to support and enhance video-based learning environments. In Study III, we investigated students' attitudes towards learnersourced captioning for lecture videos. We deployed ClassTranscribe in a large (N=387) text retrieval and mining course where 58 learners participated in editing captions of 89 lecture videos, and each lecture video was edited by two editors sequentially. In the following semester, 18 editors participated in follow-up interviews to discuss their experience of using and editing captions in the class. Our study showed how students use captions to learn, and shed light on students' attitudes, motivations, and strategies in collaborating with other learners to fix captions in a learnersourced way.

Graduation Semester

2021-08

Type of Resource

Thesis

Permalink

http://hdl.handle.net/2142/113047

Copyright and License Information

Copyright 2021 Zhilin Zhang

Owning Collections