Files in this item



application/pdfLIU-THESIS-2018.pdf (15MB)
(no description provided)PDF


Title:Mobile design semantics
Author(s):Liu, Thomas F
Advisor(s):Kumar, Ranjitha
Department / Program:Computer Science
Discipline:Computer Science
Degree Granting Institution:University of Illinois at Urbana-Champaign
Subject(s):design, mobile
Abstract:Given the growing number of mobile apps and their increasing impact on modern life, researchers have developed black-box approaches to mine mobile app design and interaction data. Although the data captured during interaction mining is descriptive, it does not expose the design semantics of UIs: what elements on the screen mean and how they are used. This thesis introduces an automatic approach for semantically annotating the elements comprising a UI given the data captured during interaction mining. Through an iterative open coding of 73k UI elements and 720 screens, we first created a lexical database of 24 types of UI components, 197 text button concepts, and 135 icon classes shared across apps. Using the labeled data created during this process, we learned code-based patterns to detect components, and trained a convolutional neural network which distinguishes between 99 icon classes with 94% accuracy. With this automated approach, we computed semantic annotations for the 72k unique UIs comprising the Rico dataset, assigning labels for 78% of the total visible, non-redundant elements.
Issue Date:2018-04-27
Rights Information:Copyright 2018 Thomas Liu
Date Available in IDEALS:2018-09-04
Date Deposited:2018-05

This item appears in the following Collection(s)

Item Statistics