Hyperparameter tuning and its effects on deep learning performance and generalization
Rabe, Benjamin
Loading…
Permalink
https://hdl.handle.net/2142/107979
Description
Title
Hyperparameter tuning and its effects on deep learning performance and generalization
Author(s)
Rabe, Benjamin
Issue Date
2020-05-06
Director of Research (if dissertation) or Advisor (if thesis)
Kindratenko, Volodymyr
Department of Study
Electrical & Computer Eng
Discipline
Electrical & Computer Engr
Degree Granting Institution
University of Illinois at Urbana-Champaign
Degree Name
M.S.
Degree Level
Thesis
Keyword(s)
machine learning
hyperparameter search
Abstract
Hyperparameter tuning is an integral part of deep learning research. Finding hyperparameter values that effectively leverage the strengths of network architectures and training procedures is crucial to maximizing performance. However, extensive hyperparameter searches raise concerns about overfitting to re-used evaluation datasets. In this thesis, we perform a case study of hyperparameter search methods on SqueezeNet v1.0 with refinements added to the training procedure. We show that random search allows for improvement over baseline performance in few trials, achieve around a 2% increase in SqueezeNet accuracy on ImageNet, and provide evidence that contrary to the common notion of adaptive overfitting, accuracy gains achieved on a validation set through hyperparameter tuning result in larger gains on a held-out test set.
Use this login method if you
don't
have an
@illinois.edu
email address.
(Oops, I do have one)
IDEALS migrated to a new platform on June 23, 2022. If you created
your account prior to this date, you will have to reset your password
using the forgot-password link below.
