Files in this item



application/pdfRABE-THESIS-2020.pdf (2MB)
(no description provided)PDF


Title:Hyperparameter tuning and its effects on deep learning performance and generalization
Author(s):Rabe, Benjamin
Advisor(s):Kindratenko, Volodymyr
Department / Program:Electrical & Computer Eng
Discipline:Electrical & Computer Engr
Degree Granting Institution:University of Illinois at Urbana-Champaign
Subject(s):machine learning
hyperparameter search
Abstract:Hyperparameter tuning is an integral part of deep learning research. Finding hyperparameter values that effectively leverage the strengths of network architectures and training procedures is crucial to maximizing performance. However, extensive hyperparameter searches raise concerns about overfitting to re-used evaluation datasets. In this thesis, we perform a case study of hyperparameter search methods on SqueezeNet v1.0 with refinements added to the training procedure. We show that random search allows for improvement over baseline performance in few trials, achieve around a 2% increase in SqueezeNet accuracy on ImageNet, and provide evidence that contrary to the common notion of adaptive overfitting, accuracy gains achieved on a validation set through hyperparameter tuning result in larger gains on a held-out test set.
Issue Date:2020-05-06
Rights Information:Copyright 2020 Benjamin Rabe
Date Available in IDEALS:2020-08-26
Date Deposited:2020-05

This item appears in the following Collection(s)

Item Statistics