2026-09-20T07:49:12Z https://www.ideals.illinois.edu/oai-pmh

oai:www.ideals.illinois.edu:2142/108189 2023-07-11 col_2142_5131 col_2142_10761 com_2142_5130 com_2142_10755 com_2142_234

Telgarsky, Matus J Lizama, Justin N 2020-08-26T23:58:48Z 2020-08-26T23:58:48Z 2022-08-26T23:58:55Z 2020-05-12 2020-05 A new loss function is proposed which learns the hinge loss function an infinite number of times pushing $f(x_i)y_i \to \infty$. It is proven that for a linear model on linearly separable data this modified hinge loss function converges in the direction of the $\ell_2$ max-margin separator at a rate of $\bigO\left( \sqrt{d/t} \right)$ where $d$ is the dimension of the data. Then, an explicit formula for the underlying dynamical system of the gradient descent iterates for two-layer linear networks on the inner product loss function is derived. Using the derived dynamical system, a precise explicit algorithm is developed which when implemented reproduces the gradient descent iterates of two-layer ReLU nets on the inner product exactly. This result is studied further to extrapolate conclusions for neural network optimization. Submission published under a 24 month embargo labeled 'U of I Access', the embargo will last until 2022-05-01 The student, Justin Lizama, accepted the attached license on 2020-05-12 at 12:38. The student, Justin Lizama, submitted this Thesis for approval on 2020-05-12 at 13:23. This Thesis was approved for publication on 2020-05-12 at 15:00. DSpace SAF Submission Ingestion Package generated from Vireo submission #15352 on 2020-08-25 at 17:31:13 Made available in DSpace on 2020-08-26T23:58:48Z (GMT). No. of bitstreams: 2 LIZAMA-THESIS-2020.pdf: 1342025 bytes, checksum: a56819a442a48f4562e0e97587b99fd7 (MD5) LICENSE.txt: 4210 bytes, checksum: 0761322496976fd61d67a6ec4cb5f931 (MD5) Previous issue date: 2020-05-12 Embargo set by: Seth Robbins for item 115802 Lift date: 2022-08-26T23:58:55Z Reason: Author requested U of Illinois access only (OA after 2yrs) in Vireo ETD system Author requested U of Illinois access only (OA after 2yrs) in Vireo ETD system U of I Only application/pdf http://hdl.handle.net/2142/108189 en Copyright 2020 Justin Lizama implicit regularization hinge loss Completion of hinge loss has an implicit bias text Thesis Computer Science Computer Science University of Illinois at Urbana-Champaign Thesis M.S.