2026-09-18T20:16:16Z https://www.ideals.illinois.edu/oai-pmh

oai:www.ideals.illinois.edu:2142/101246 2023-07-11 col_2142_5131 col_2142_10761 com_2142_5130 com_2142_10755 com_2142_234

Kumar, Ranjitha Liu, Thomas F 2018-09-04T20:42:02Z 2018-09-04T20:42:02Z 2020-09-05T09:15:23Z 2018-04-27 2018-05 Given the growing number of mobile apps and their increasing impact on modern life, researchers have developed black-box approaches to mine mobile app design and interaction data. Although the data captured during interaction mining is descriptive, it does not expose the design semantics of UIs: what elements on the screen mean and how they are used. This thesis introduces an automatic approach for semantically annotating the elements comprising a UI given the data captured during interaction mining. Through an iterative open coding of 73k UI elements and 720 screens, we first created a lexical database of 24 types of UI components, 197 text button concepts, and 135 icon classes shared across apps. Using the labeled data created during this process, we learned code-based patterns to detect components, and trained a convolutional neural network which distinguishes between 99 icon classes with 94% accuracy. With this automated approach, we computed semantic annotations for the 72k unique UIs comprising the Rico dataset, assigning labels for 78% of the total visible, non-redundant elements. Submission published under a 24 month embargo labeled 'U of I Access', the embargo will last until 2020-05-01 The student, Thomas Liu, accepted the attached license on 2018-04-27 at 16:08. The student, Thomas Liu, submitted this Thesis for approval on 2018-04-27 at 16:13. This Thesis was approved for publication on 2018-04-27 at 16:36. DSpace SAF Submission Ingestion Package generated from Vireo submission #12547 on 2018-08-31 at 17:21:52 Made available in DSpace on 2018-09-04T20:42:02Z (GMT). No. of bitstreams: 2 LIU-THESIS-2018.pdf: 14557319 bytes, checksum: 53aab5cbc6fed062e55ef34562a4fa27 (MD5) LICENSE.txt: 4207 bytes, checksum: f25209dcc54c5ef7100df8d0321824ec (MD5) Previous issue date: 2018-04-27 Embargo set by: Seth Robbins for item 107331 Lift date: 2020-09-04T20:42:08Z Reason: Author requested U of Illinois access only (OA after 2yrs) in Vireo ETD system U of I Only Restriction Lifted for Item 107331 on 2020-09-05T09:15:23Z. application/pdf http://hdl.handle.net/2142/101246 en Copyright 2018 Thomas Liu design, mobile Mobile design semantics text text Computer Science Computer Science University of Illinois at Urbana-Champaign Thesis M.S.