2026-09-19T11:17:05Z https://www.ideals.illinois.edu/oai-pmh

oai:www.ideals.illinois.edu:2142/108007 2023-07-11 col_2142_5131 col_2142_10761 com_2142_5130 com_2142_10755 com_2142_234

Parameswaran, Aditya Lee, Angela 2020-08-26T21:54:57Z 2020-08-26T21:54:57Z 2020-05-11 2020-05 It is well-known that the process of developing machine learning (ML) workflows is a dark-art; even experts struggle to find an optimal workflow leading to a high accuracy model. Users currently rely on empirical trial-and-error to obtain their own set of battle-tested guidelines to inform their modeling decisions. In this study, we aim to demystify this dark art by understanding how people iterate on ML workflows in practice. We analyze over 475k user-generated workflows on OpenML, an open-source platform for tracking and sharing ML workflows. We find that users often adopt a manual, automated, or mixed approach when iterating on their workflows. We observe that manual approaches result in fewer wasted iterations compared to automated approaches. Yet, automated approaches often involve more preprocessing and hyperparameter options explored, resulting in higher performance overall---suggesting potential benefits for a human-in-the-loop ML system that appropriately recommends a clever combination of the two strategies. Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2020-08-25 without embargo terms The student, Angela Lee, accepted the attached license on 2020-05-07 at 12:17. The student, Angela Lee, submitted this Thesis for approval on 2020-05-07 at 12:34. This Thesis was approved for publication on 2020-05-11 at 11:02. DSpace SAF Submission Ingestion Package generated from Vireo submission #15274 on 2020-08-25 at 17:13:04 Made available in DSpace on 2020-08-26T21:54:57Z (GMT). No. of bitstreams: 2 LEE-THESIS-2020.pdf: 1919422 bytes, checksum: dd5eec0227826dc90420503326ab18f4 (MD5) LICENSE.txt: 4207 bytes, checksum: 000ee453a82396072bc74eb2752b0dcf (MD5) Previous issue date: 2020-05-11 application/pdf http://hdl.handle.net/2142/108007 en Copyright 2020 Angela Lee machine learning data analysis empirical studies user behavior Demystifying a dark art: Understanding real-world machine learning model development text Thesis Computer Science Computer Science University of Illinois at Urbana-Champaign Thesis M.S.