Upcoming Events

 

ISE Seminar Series Fall 2021

Time: THIS SEMINAR HAS BEEN CANCELLED.

Location: Mohler Laboratory Room 453

Speaker: René Vidal, Johns Hopkins University

Title: "Learning Dynamic and Implicit Bias of Gradient Flow in Overparameterized Linear Models"

Zoom: https://lehigh.zoom.us/j/98233793950

 

Abstract:

Contrary to the common belief that overparameterization may hurt generalization and optimization, recent work suggests that overparameterization may bias the optimization algorithm towards solutions that generalize well — a phenomenon known as implicit regularization or implicit bias — and may also accelerate convergence — a phenomenon known as implicit acceleration. This talk will provide a detailed analysis of the dynamics of gradient flow in overparameterized two-layer linear models showing that convergence to equilibrium depends on the imbalance between input and output weights (which is fixed at initialization) and the margin of the initial solution. The talk will also provide an analysis of the implicit bias, showing that large hidden layer width, together with (properly scaled) random initialization, constrains the network parameters to converge to a solution which is close to the min-norm solution. 

 

Bio:

René Vidal is the Herschel Seder Professor of Biomedical Engineering and Director of the Mathematical Institute for Data Science at Johns Hopkins University. He is also an Amazon Scholar, Chief Scientist at NORCE, and Associate Editor in Chief of TPAMI. He also directs the NSF-Simons Collaboration on the Mathematical Foundations of Deep Learning and the TRIPODS Institute on the Foundations of Graph and Deep Learning. His current research focuses on the foundations of deep learning and its applications in computer vision and biomedical data science. He is an AIMBE Fellow, IEEE Fellow, IAPR Fellow and Sloan Fellow, and has received numerous awards for his work, including the D’Alembert Faculty Award, J.K. Aggarwal Prize, ONR Young Investigator Award, NSF CAREER Award as well as best paper awards in machine learning, computer vision, controls, and medical robotics. 

 

 

 

 


Past Events

Fall 2021
 
Tuesday, November 16, 2021, 11:00 am
Kimia Ghobadi, Johns Hopkins University
"COVID-19 Hospital Capacity Management using Math Modeling"
 
Tuesday, November 9, 2021, 11:00 am
INFORMS DISTINGUISHED SPEAKER: Dr. Jong-Shi Pang, University of Southern California
"Some Nonsmooth Function Classes and Their Optimization"
 
Tuesday, November 2, 2021, 11:00 am
Nathan Kallus - Cornell Tech
"Smooth Contextual Bandits"
 
Tuesday, October 19, 2021, 11:00 am
Georg Stadler - NYU Courant Institute of Mathematical Sciences
"Optimal control of PDEs under uncertainty with joint chance contraints"
 
Tuesday, October 12, 2021, 11:00 am
Emily Tucker - Clemson University
"Reducing Drug Shortages by Improving Supply Chain Resiliency"
 
Tuesday, October 5, 2021, 11:00 am
Gabor Pataki - UNC Chapel Hill
"How do exponential size solutions arise in Semidefinite Programming?"
 
Tuesday, September 28, 2021, 11:00 am
John Wright - Columbia University
"Deep Networks and the Multiple Manifold Problem"
 
Spring 2021
 
Tuesday, May 4, 2021, 11:00am
INFORMS Chapter Distinguished Speaker - Dr. Amir Beck, Tel-Aviv University
"Dual Randomized Coordinate Descent Method for Solving a Class of Nonconvex Problems"
 
Tuesday, April 27, 2021, 11:00am
ISE Seminar Series - Dr. Swati Gupta, Stewart School of Industrial & Systems Engineering, Georgia Tech
"Bridging Classical and Quantum Computing with Warm-starts and Coupling Graphs"
 
Tuesday, April 20, 2021, 11:00am
ISE Seminar Series - Andrés Medaglia, Department of Industrial Engineering, Universidad de los Andes, Columbia
"Exact Bidirectional (pulse) Algorithm for Constrained Shortest Path Extensions"
 
Monday, April 19, 2021, 12:15pm
ISE Seminar Series - Akwum Onwunta, Center for Mathematics and Artificial Intelligence, George Mason University
"Optimization Under Uncertainty: Current Approaches and Beyond"
 
Monday, April 12, 2021, 12:15pm
ISE Seminar Series - Wesley Marrero Colón, Harvard Medical School
"Flexible Treatment Plans and Genetic Testing Strategies for the Management of Cardiovascular Diseases"
 
Thursday, April 8, 2021, 12:15pm
ISE Seminar Series - Anirudh Subramanyam, Argonne National Laboratory
"Mitigating Extreme Failures in Complex Infrastructure Systems"
 
Tuesday, April 6, 2021, 11:00am
ISE Seminar Series -John F. Raffensperger, RAND Corp., Santa Monica, CA USA
"A price on warming with a supply chain directed market"
 
Tuesday, March 30, 2021 11:00am
ISE Seminar Series -Siqian Shen, Department of Industrial and Operations Engineering, University of Michigan
"Facilty Location Involving Competition or Decision-dependent Uncertainty: New Models, Solution Approaches and Insights"
 
Tuesday, March 23, 2021 11:00am
ISE Seminar Series - Meisam Razaviyayn, University of Southern California
"Non-convex min-max optimization: challenges and recent advancements"
 
Tuesday, March 9, 2021 11:00am
ISE Seminar Series - Guanghui (George) Lan, Georgia Institute of Technology
"Advancing Stochastic Optimization for Reinforcement Learning"
 
Tuesday, February 23, 2021 11:00am
ISE Seminar Series - Andrea Qualizza, Amazon
"Transportation and Fulfillment Optimization at Amazon"
 
Tuesday, February 9, 2021 11:00am
ISE Seminar Series - Samantha Keppler, University of Michigan Stephen M. Ross School of Business
"On the Use of Operations Management in Public Education Systems"
 
Fall 2020
 
Tuesday, December 1, 2020 11:00am
ISE Seminar Series - Ilya Safro, University of Delaware
"Multiscale Methods in Action: From Machine Learning and Graphs to Realistic Future for Quantum Computing"
 
Tuesday, November 17, 2020 11:00am
ISE Seminar Series - Raghu Bollapragada, University of Austin (UT)
"Adaptive Sampling Methods for Derivative-Free Stochastic Optimization" 
 
Tuesday, October 27, 2020 11:00am
ISE Seminar Series - Kim-Chuan Toh, National University of Singapore (NUS)
"Exploiting Second Order Sparsity in Big Data Optimization" 
 
Tuesday, October 20, 2020 11:00am
ISE Seminar Series - Albert Del Pia, University of Wisconsin-Madison
"Short Simplex Paths in Lattice Polytopes"
 
Tuesday, October 13, 2020 11:00am
ISE Seminar Series - Coralia Cartis, Oxford University
"Dimensionality Reduction Techniques for Large-Scale Optimization Problems"
 
Tuesday, October 6, 2020 11:00am
ISE Seminar Series - Miles Lubin, Google NYC
"Placement, Provisioning, Packaging"
 
Tuesday September 29, 2020 11:00am
ISE Seminar Series - Juan C. Vera, Tilburg University
"The Simplex Way to Obtain Non-Negative Certificates Over Compact Semialgebraic Sets"
 
Tuesday September 8, 2020 11:00am
ISE Seminar Series - Jeff Zhang, Carnegie Mellon University
"On Local Minima in Polynomial Optimization"
 
Spring 2020
 
Tuesday February 18, 2020 11:00am
ISE Seminar Series - Andrew Papanicolaou, New York University
"PCA for Implied Volatility Surfaces"
 
Tuesday January 30, 2020 4:30pm
ISE Seminar Series - Elaheh Fata, Massachusetts Institute of Technology
"Multi-stage and Multi-customer Assortment Optimization with Inventory Constraints"
 
Thursday January 16, 2020 10:45am
ISE Seminar Series - Aida Khajavirad, Rutgers School of Business
"Convexification and Global Optimization of Mixed Integer Nonlinear Optimization Problems"
Fall 2019
 
Wednesday December 11, 2019 4:30pm
ISE Seminar Series - Temitayo Ajayi , Rice University
"Objective Selection for Cancer Treatment: An Inverse Optimization Approach"
 
Monday December 9, 2019 10:45am
ISE Seminar Series - Karmel S. Shehadeh, Carnegie Mellon University
"A Distributionally Robust Optimization Approach for Outpatient Colonoscopy Scheduling"
 
Tuesday December 3, 2019 4:30pm
ISE Seminar Series - Lavanya Marla, University of Illinois at Urbana-Champaign
"Data-driven Greedy Policies and Information-Relaxation Bounds for Ambulance Location and Deployment"
 
Tuesday, November 12, 2019 - 4:30pm
ISE Seminar Series - Kayse Maass, Northeastern University
"A Systems Approach to Disrupting Human Trafficking"
 
Tuesday, November 5, 2019 - 4:30pm
INFORMS Chapter Distinguished Speaker Series - Robert Vanderbei, Princeton University
"The Parametric Self-Dual Simplex Method — A Modern Perspective"
 
Tuesday, October 8, 2019 - 4:30pm
ISE Seminar Series - Miju Ahn, Southern Methodist University
"Learning with Difference-of-Convex Sparsity Functions"
 
Tuesday, September 17, 2019 - 4:30pm
ISE Seminar Series - Ramteen Sioshansi, The Ohio State University
"Can We Get Market and Regulatory Designs ‘Right’ for Energy Storage?"
 
Tuesday, September 10, 2019 - 4:30pm
ISE Seminar Series - Adam Elmachtoub, Columbia University
"Smart "Predict, then Optimize""
 
Spring 2019
 
Tuesday, May 7, 2019 - 4:00pm
ISE Seminar Series - Alec Koppel, U.S. Army Research Laboratory
"Policy Search for Reinforcement Learning in Continuous Spaces: Improved Limits and Reduced Variance"
 
Tuesday, April 9, 2019 - 4:00pm
ISE Seminar Series - Aida Khajavirad (Rutgers & NYU)
"Novel Polyhedral Relaxations for Mixed-Integer Polynomial Optimization Problems"
 
Tuesday, April 2, 2018 - 4:00pm
INFORMS Chapter Distinguished Speaker Series - Stefan Wild, Argonne National Laboratory
"Exploiting Structure in Derivative-Free Nonlinear Optimization to Advance Science and Engineering"
 
Tuesday, March 26, 2019 - 4:00pm
ISE Seminar Series - Clément Royer, University of Wisconsin-Madison
"Nonconvex Optimization via Newton-CG Methods with Complexity Guarantees"
 
Tuesday, March 19, 2019 - 4:00pm
ISE Seminar Series - William K. Klimack, Chevron
"Decision Analysis at Chevron"
 
Tuesday, March 5, 2019 - 4:00pm
ISE Seminar Series - James Luedtke, University of Wisconsin-Madison
"Optimizing Truck Dispatching Decisions in Open-pit Mining using Integer Programming"
 
Tuesday, February 12, 2019 - 4:00pm
ISE Seminar Series - J. Cole Smith, Clemson University
"The Maximum Semicontinuous Flow Problem"
 
Tuesday, January 29, 2019 - 4:00pm
ISE Seminar Series - Giacomo Nannicini, IBM T.J. Watson Research Center
"Fully Polynomial-Time Approximation Schemes for Stochastic Dynamic Programs: Theory and Applications"
 
Fall 2018
 
Tuesday, December 4, 2018 - 4:00pm
ISE Seminar Series - Andreas Wächter, Northwestern University
"Nonlinear Programming Formulations of Chance-Constraints"
 
Tuesday, November 27, 2018 - 4:00pm
INFORMS Chapter Distinguished Speaker Series - Gerard Cornuejols, Carnegie Mellon University
"Packing and Covering"
 
Tuesday, November 13, 2018 - 4:00pm
ISE Seminar Series - Mert Gürbüzbalaban, Rutgers University
"Momentum Acceleration Under Random Gradient Noise: From Convex to Non-Convex Optimization"
 
Tuesday, October 30, 2018 - 4:00pm
ISE Seminar Series - Michael Katehakis, Rutgers University
"Reinforcement Learning: Connections Between MDPs and MAB Problems"
 
Tuesday, October 2, 2018 - 4:00pm
ISE Seminar Series - Michael L. Overton, Courant Institute of Mathematical Sciences, NYU
"Nonsmooth, Nonconvex Optimization: Algorithms and Examples"
 
Spring 2018
 
Tuesday, May 8, 2018 - 4:00pm
ISE Seminar Series - Krzysztof Choromanski, Google
"Structure is all that you need - Learning Compressed RL Policies Via Orthogonal Gradient Sensing"
 
Tuesday, May 1, 2018 - 4:00pm
ISE Seminar Series - Henry Lam, Columbia University
"Assessing Solution Quality in Stochastic Optimization with Limited Data"
 
Tuesday, April 24, 2018 - 4:00pm
INFORMS Chapter Distinguished Speaker Series - James Renegar, Cornell University
"A Simple Nearly-Optimal Restart Scheme for Speeding-Up First Order Methods"
 
Thursday, April 19, 2018 - 4:00pm
ISE Seminar Series - Matthias Walter, RWTH Aachen University
"Investigating Polyhedra by Oracles"
 
Tuesday, April 10, 2018 - 4:00pm
ISE Seminar Series - Stefano Coniglio, University of Southampton
"Network Routing Through the Internet as a Stackelberg Game"
 
Tuesday, March 6, 2018 - 4:00pm
ISE Seminar Series - Bo Zeng, University of Pittsburgh
"Robust and Chance Constrained Optimization in Power and Logistics Systems"
 
Fall 2017
 
Tuesday, December 5, 2017 - 4:00pm
ISE Seminar Series - Garud Iyengar, Columbia University
"Automatic Event Detection in Basketball"
 
Thursday, November 30, 2017 - 4:00pm
Spencer C. Schantz Technical Lecture - William J. Cook, University of Waterloo
"Guiding the Cutting-Plane Method"
 
Wednesday, November 29, 2017 - 4:00pm
Spencer C. Schantz Distinguished Lecture Series Presents William J. Cook
"In Pursuit of the Traveling Salesman: Mathematics at the Limits of Computation"
 
Tuesday, November 14, 2017 - 4:00pm
ISE Seminar Series - Jonathan Eckstein, Rutgers University
"Asynchronous Projective Splitting for Convex Optimization and Monotone Inclusion Problems"
 
Tuesday, November 7, 2017 - 4:00pm
ISE Seminar Series - Osman Y. Ozaltin, North Carolina State University
"A Branch-and-cut Algorithm for Discrete Bilevel Linear Programs"
 
Tuesday, October 31, 2017 - 4:00pm
ISE Seminar Series - Oleg Prokopyev, University of Pittsburgh
"Sequential Max-Min Bilevel Linear Programming with Incomplete Information and Learning"
 
Tuesday, October 10, 2017 - 4:00pm
ISE Seminar Series - Ion Necoara, University Politehnica of Bucharest
"Conditions for Linear Convergence of (Stochastic) First Order Methods"
 
Thursday, October 5, 2017 - 4:00pm
INFORMS Chapter Distinguished Speaker Series: Nick Trefethen, University of Oxford
 
Tuesday, September 26, 2017 - 4:00pm
ISE Seminar Series - Andrew Dorsett, Wolfram Research
"Mathematica 11 in Education and Research"
 
Tuesday, September 19, 2017 - 4:00pm
ISE Seminar Series - Alejandro Ribeiro, University of Pennsylvania
"High Order Methods in Empirical Risk Minimization"
 
Tuesday, August 22, 2017 - 4:00pm
ISE Seminar Series - Giorgio Fasano, Thales Alenia Space
"Solving Non-Standard Packing Problems by Global Optimization and Heuristics"