ISE Seminar Series

Upcoming Events

ISE Seminar Series Fall 2021

Time: THIS SEMINAR HAS BEEN CANCELLED.

Location: Mohler Laboratory Room 453

Speaker: René Vidal, Johns Hopkins University

Title: "Learning Dynamic and Implicit Bias of Gradient Flow in Overparameterized Linear Models"

Zoom: https://lehigh.zoom.us/j/98233793950

Abstract:

Contrary to the common belief that overparameterization may hurt generalization and optimization, recent work suggests that overparameterization may bias the optimization algorithm towards solutions that generalize well — a phenomenon known as implicit regularization or implicit bias — and may also accelerate convergence — a phenomenon known as implicit acceleration. This talk will provide a detailed analysis of the dynamics of gradient flow in overparameterized two-layer linear models showing that convergence to equilibrium depends on the imbalance between input and output weights (which is fixed at initialization) and the margin of the initial solution. The talk will also provide an analysis of the implicit bias, showing that large hidden layer width, together with (properly scaled) random initialization, constrains the network parameters to converge to a solution which is close to the min-norm solution. 

Bio:

René Vidal is the Herschel Seder Professor of Biomedical Engineering and Director of the Mathematical Institute for Data Science at Johns Hopkins University. He is also an Amazon Scholar, Chief Scientist at NORCE, and Associate Editor in Chief of TPAMI. He also directs the NSF-Simons Collaboration on the Mathematical Foundations of Deep Learning and the TRIPODS Institute on the Foundations of Graph and Deep Learning. His current research focuses on the foundations of deep learning and its applications in computer vision and biomedical data science. He is an AIMBE Fellow, IEEE Fellow, IAPR Fellow and Sloan Fellow, and has received numerous awards for his work, including the D’Alembert Faculty Award, J.K. Aggarwal Prize, ONR Young Investigator Award, NSF CAREER Award as well as best paper awards in machine learning, computer vision, controls, and medical robotics.