Files in this item



application/pdfYEH-THESIS-2018.pdf (24MB)
(no description provided)PDF


Title:Improvement and measurement of neural style transfer
Author(s):Yeh, Mao-Chuang
Advisor(s):Forsyth, David A.
Department / Program:Computer Science
Discipline:Computer Science
Degree Granting Institution:University of Illinois at Urbana-Champaign
Subject(s):style transfer
gram matrix
texture synthesis
Abstract:Style transfer methods produce a transferred image which is a rendering of a content image in the manner of a style image. There is a rich literature of variant methods. We seek to understand how to improve style transfer: in particular, there is some evidence that cross-layer losses are helpful, and some evidence that optimization problems might present difficulties. To do so requires quantitative evaluation procedures, but current evaluation is qualitative, mostly involving user studies. We describe a novel quantitative evaluation procedure. Our procedure relies on two statistics: the Effectiveness (E) statistic measures the extent that a given style has been transferred to the target, and the Coherence (C) statistic measures the extent to which the original image's content is preserved. Our statistics are calibrated to human preference: targets with larger values of E (resp C) will reliably be preferred by human subjects in comparisons of style (resp. content). We use these statistics to investigate relative performance of a number of recent style transfer methods, revealing a number of intriguing properties. {Our experiments pool multiple style transfers from many different styles to many different content images using many different style weights, allowing us to make general statements about what influences style transfer. }Admissible methods lie on a Pareto frontier (i.e. improving E reduces C, or vice versa). Three methods are admissible: Universal style transfer produces very good C but weak E; modifying the optimization used for Gatys' loss produces a method with strong E and strong C; and a modified cross-layer method has slightly better E at strong cost in C. While the histogram loss improves the E statistics of Gatys' method, it does not make the method admissible. Surprisingly, style weights have relatively little effect, and most variability in transfer is explained by the style itself (meaning experimenters can be misguided by selecting styles).
Issue Date:2018-12-14
Rights Information:Copyright 2018 Mao-Chuang Yeh
Date Available in IDEALS:2019-02-06
Date Deposited:2018-12

This item appears in the following Collection(s)

Item Statistics