Upcoming Events

 

ISE Seminar Series Fall 2021

Time: Tuesday, September 28, 2021, 11:00 a.m.

Speaker: John Wright, Columbia University

Title: Deep Networks and the Multiple Manifold Problems

 

Abstract:

Data with low-dimensional nonlinear structure are ubiquitous in engineering and scientific problems. We study a model problem with such structure—a binary classification task that uses a deep fully-connected neural network to classify data drawn from two disjoint smooth curves on the unit sphere. Aside from mild regularity conditions, we place no restrictions on the configuration of the curves. We prove that when (i) the network depth is large relative to certain geometric properties that set the difficulty of the problem and (ii) the network width and number of samples is polynomial in the depth, randomly-initialized gradient descent quickly learns to correctly classify all points on the two curves with high probability. To our knowledge, this is the first generalization guarantee for deep networks with nonlinear data that depends only on intrinsic data properties. Our analysis draws on ideas from harmonic analysis and martingale concentration for handling statistical dependencies in the initial (random) network. We sketch applications to invariant vision, and to gravitational wave astronomy, where leveraging low-dimensional structure leads to statistically optimal tests for identifying signals in noise. 

Joint work with Sam Buchanan, Dar Gilboa, Tim Wang, Jingkai Yan

 

Bio:     

 

John Wright is an associate professor in Electrical Engineering at Columbia University. He is also affiliated with the Department of Applied Physics and Applied Mathematics and Columbia’s Data Science Institute. He received his PhD in Electrical Engineering from the University of Illinois at Urbana Champaign in 2009. Before joining Columbia he was with Microsoft Research Asia from 2009-2011. His research interests include sparse and low-dimensional models for high-dimensional data, optimization (convex and otherwise), and applications in imaging and vision. His work has received a number of awards and honors, including the 2012 COLT Best Paper Award and the 2015 PAMI TC Young Researcher Award. 

 

 

 

 

 

 


Past Events

Spring 2021
 
Tuesday, May 4, 2021, 11:00 a.m
INFORMS Chapter Distinguished Speaker - Dr. Amir Beck, Tel-Aviv University
Dual Randomized Coordinate Descent Method for Solving a Class of Nonconvex Problems
 
Tuesday, April 27, 2021, 11:00 a.m
ISE Seminar Series - Dr. Swati Gupta, Stewart School of Industrial & Systems Engineering, Georgia Tech
Bridging Classical and Quantum Computing with Warm-starts and Coupling Graphs
 
Tuesday, April 20, 2021, 11:00am
ISE Seminar Series - Andrés Medaglia, Department of Industrial Engineering, Universidad de los Andes, Columbia
Exact Bidirectional (pulse) Algorithm for Constrained Shortest Path Extensions
 
Monday, April 19, 2021, 12:15pm
ISE Seminar Series - Akwum Onwunta, Center for Mathematics and Artificial Intelligence, George Mason University
Optimization Under Uncertainty: Current Approaches and Beyond
 
Monday, April 12, 2021, 12:15pm
ISE Seminar Series - Wesley Marrero Colón, Harvard Medical School
Flexible Treatment Plans and Genetic Testing Strategies for the Management of Cardiovascular Diseases
 
Thursday, April 8, 2021, 12:15pm
ISE Seminar Series - Anirudh Subramanyam, Argonne National Laboratory
Mitigating Extreme Failures in Complex Infrastructure Systems
 
Tuesday, April 6, 2021, 11:00am
ISE Seminar Series -John F. Raffensperger, RAND Corp., Santa Monica, CA USA
"A price on warming with a supply chain directed market"
 
Tuesday, March 30, 2021 11:00am
ISE Seminar Series -Siqian Shen, Department of Industrial and Operations Engineering, University of Michigan
Facilty Location Involving Competition or Decision-dependent Uncertainty: New Models, Solution Approaches and Insights
 
Tuesday, March 23, 2021 11:00am
ISE Seminar Series - Meisam Razaviyayn, University of Southern California
Non-convex min-max optimization: challenges and recent advancements
 
Tuesday, March 9, 2021 11:00am
ISE Seminar Series - Guanghui (George) Lan, Georgia Institute of Technology
Advancing Stochastic Optimization for Reinforcement Learning
 
Tuesday, February 23, 2021 11:00am
ISE Seminar Series - Andrea Qualizza, Amazon
Transportation and Fulfillment Optimization at Amazon
 
Tuesday, February 9, 2021 11:00am
ISE Seminar Series - Samantha Keppler, University of Michigan Stephen M. Ross School of Business
On the Use of Operations Management in Public Education Systems
 
Fall 2020
 
Tuesday, December 1, 2020 11:00am
ISE Seminar Series - Ilya Safro, University of Delaware
Multiscale Methods in Action: From Machine Learning and Graphs to Realistic Future for Quantum Computing
 
Tuesday, November 17, 2020 11:00am
ISE Seminar Series - Raghu Bollapragada, University of Austin (UT)
Adaptive Sampling Methods for Derivative-Free Stochastic Optimization 
 
Tuesday, October 27, 2020 11:00am
ISE Seminar Series - Kim-Chuan Toh, National University of Singapore (NUS)
Exploiting Second Order Sparsity in Big Data Optimization 
 
Tuesday, October 20, 2020 11:00am
ISE Seminar Series - Albert Del Pia, University of Wisconsin-Madison
Short Simplex Paths in Lattice Polytopes
 
Tuesday, October 13, 2020 11:00am
ISE Seminar Series - Coralia Cartis, Oxford University
Dimensionality Reduction Techniques for Large-Scale Optimization Problems
 
Tuesday, October 6, 2020 11:00am
ISE Seminar Series - Miles Lubin, Google NYC
"Placement, Provisioning, Packaging"
 
Tuesday September 29, 2020 11:00am
ISE Seminar Series - Juan C. Vera, Tilburg University
"The Simplex Way to Obtain Non-Negative Certificates Over Compact Semialgebraic Sets"
 
Tuesday September 8, 2020 11:00am
ISE Seminar Series - Jeff Zhang, Carnegie Mellon University
"On Local Minima in Polynomial Optimization"
 
Spring 2020
 
Tuesday February 18, 2020 11:00am
ISE Seminar Series - Andrew Papanicolaou, New York University
"PCA for Implied Volatility Surfaces"
 
Tuesday January 30, 2020 4:30pm
ISE Seminar Series - Elaheh Fata, Massachusetts Institute of Technology
"Multi-stage and Multi-customer Assortment Optimization with Inventory Constraints"
 
Thursday January 16, 2020 10:45am
ISE Seminar Series - Aida Khajavirad, Rutgers School of Business
"Convexification and Global Optimization of Mixed Integer Nonlinear Optimization Problems"
Fall 2019
 
Wednesday December 11, 2019 4:30pm
ISE Seminar Series - Temitayo Ajayi , Rice University
"Objective Selection for Cancer Treatment: An Inverse Optimization Approach"
 
Monday December 9, 2019 10:45am
ISE Seminar Series - Karmel S. Shehadeh, Carnegie Mellon University
"A Distributionally Robust Optimization Approach for Outpatient Colonoscopy Scheduling"
 
Tuesday December 3, 2019 4:30pm
ISE Seminar Series - Lavanya Marla, University of Illinois at Urbana-Champaign
"Data-driven Greedy Policies and Information-Relaxation Bounds for Ambulance Location and Deployment"
 
Tuesday, November 12, 2019 - 4:30pm
ISE Seminar Series - Kayse Maass, Northeastern University
"A Systems Approach to Disrupting Human Trafficking"
 
Tuesday, November 5, 2019 - 4:30pm
INFORMS Chapter Distinguished Speaker Series - Robert Vanderbei, Princeton University
"The Parametric Self-Dual Simplex Method — A Modern Perspective"
 
Tuesday, October 8, 2019 - 4:30pm
ISE Seminar Series - Miju Ahn, Southern Methodist University
"Learning with Difference-of-Convex Sparsity Functions"
 
Tuesday, September 17, 2019 - 4:30pm
ISE Seminar Series - Ramteen Sioshansi, The Ohio State University
"Can We Get Market and Regulatory Designs ‘Right’ for Energy Storage?"
 
Tuesday, September 10, 2019 - 4:30pm
ISE Seminar Series - Adam Elmachtoub, Columbia University
"Smart "Predict, then Optimize""
 
Spring 2019
 
Tuesday, May 7, 2019 - 4:00pm
ISE Seminar Series - Alec Koppel, U.S. Army Research Laboratory
"Policy Search for Reinforcement Learning in Continuous Spaces: Improved Limits and Reduced Variance"
 
Tuesday, April 9, 2019 - 4:00pm
ISE Seminar Series - Aida Khajavirad (Rutgers & NYU)
"Novel Polyhedral Relaxations for Mixed-Integer Polynomial Optimization Problems"
 
Tuesday, April 2, 2018 - 4:00pm
INFORMS Chapter Distinguished Speaker Series - Stefan Wild, Argonne National Laboratory
"Exploiting Structure in Derivative-Free Nonlinear Optimization to Advance Science and Engineering"
 
Tuesday, March 26, 2019 - 4:00pm
ISE Seminar Series - Clément Royer, University of Wisconsin-Madison
"Nonconvex Optimization via Newton-CG Methods with Complexity Guarantees"
 
Tuesday, March 19, 2019 - 4:00pm
ISE Seminar Series - William K. Klimack, Chevron
"Decision Analysis at Chevron"
 
Tuesday, March 5, 2019 - 4:00pm
ISE Seminar Series - James Luedtke, University of Wisconsin-Madison
"Optimizing Truck Dispatching Decisions in Open-pit Mining using Integer Programming"
 
Tuesday, February 12, 2019 - 4:00pm
ISE Seminar Series - J. Cole Smith, Clemson University
"The Maximum Semicontinuous Flow Problem"
 
Tuesday, January 29, 2019 - 4:00pm
ISE Seminar Series - Giacomo Nannicini, IBM T.J. Watson Research Center
"Fully Polynomial-Time Approximation Schemes for Stochastic Dynamic Programs: Theory and Applications"
 
Fall 2018
 
Tuesday, December 4, 2018 - 4:00pm
ISE Seminar Series - Andreas Wächter, Northwestern University
"Nonlinear Programming Formulations of Chance-Constraints"
 
Tuesday, November 27, 2018 - 4:00pm
INFORMS Chapter Distinguished Speaker Series - Gerard Cornuejols, Carnegie Mellon University
"Packing and Covering"
 
Tuesday, November 13, 2018 - 4:00pm
ISE Seminar Series - Mert Gürbüzbalaban, Rutgers University
"Momentum Acceleration Under Random Gradient Noise: From Convex to Non-Convex Optimization"
 
Tuesday, October 30, 2018 - 4:00pm
ISE Seminar Series - Michael Katehakis, Rutgers University
"Reinforcement Learning: Connections Between MDPs and MAB Problems"
 
Tuesday, October 2, 2018 - 4:00pm
ISE Seminar Series - Michael L. Overton, Courant Institute of Mathematical Sciences, NYU
"Nonsmooth, Nonconvex Optimization: Algorithms and Examples"
 
Spring 2018
 
Tuesday, May 8, 2018 - 4:00pm
ISE Seminar Series - Krzysztof Choromanski, Google
"Structure is all that you need - Learning Compressed RL Policies Via Orthogonal Gradient Sensing"
 
Tuesday, May 1, 2018 - 4:00pm
ISE Seminar Series - Henry Lam, Columbia University
"Assessing Solution Quality in Stochastic Optimization with Limited Data"
 
Tuesday, April 24, 2018 - 4:00pm
INFORMS Chapter Distinguished Speaker Series - James Renegar, Cornell University
"A Simple Nearly-Optimal Restart Scheme for Speeding-Up First Order Methods"
 
Thursday, April 19, 2018 - 4:00pm
ISE Seminar Series - Matthias Walter, RWTH Aachen University
"Investigating Polyhedra by Oracles"
 
Tuesday, April 10, 2018 - 4:00pm
ISE Seminar Series - Stefano Coniglio, University of Southampton
"Network Routing Through the Internet as a Stackelberg Game"
 
Tuesday, March 6, 2018 - 4:00pm
ISE Seminar Series - Bo Zeng, University of Pittsburgh
"Robust and Chance Constrained Optimization in Power and Logistics Systems"
 
Fall 2017
 
Tuesday, December 5, 2017 - 4:00pm
ISE Seminar Series - Garud Iyengar, Columbia University
"Automatic Event Detection in Basketball"
 
Thursday, November 30, 2017 - 4:00pm
Spencer C. Schantz Technical Lecture - William J. Cook, University of Waterloo
"Guiding the Cutting-Plane Method"
 
Wednesday, November 29, 2017 - 4:00pm
Spencer C. Schantz Distinguished Lecture Series Presents William J. Cook
"In Pursuit of the Traveling Salesman: Mathematics at the Limits of Computation"
 
Tuesday, November 14, 2017 - 4:00pm
ISE Seminar Series - Jonathan Eckstein, Rutgers University
"Asynchronous Projective Splitting for Convex Optimization and Monotone Inclusion Problems"
 
Tuesday, November 7, 2017 - 4:00pm
ISE Seminar Series - Osman Y. Ozaltin, North Carolina State University
"A Branch-and-cut Algorithm for Discrete Bilevel Linear Programs"
 
Tuesday, October 31, 2017 - 4:00pm
ISE Seminar Series - Oleg Prokopyev, University of Pittsburgh
"Sequential Max-Min Bilevel Linear Programming with Incomplete Information and Learning"
 
Tuesday, October 10, 2017 - 4:00pm
ISE Seminar Series - Ion Necoara, University Politehnica of Bucharest
"Conditions for Linear Convergence of (Stochastic) First Order Methods"
 
Thursday, October 5, 2017 - 4:00pm
INFORMS Chapter Distinguished Speaker Series: Nick Trefethen, University of Oxford
 
Tuesday, September 26, 2017 - 4:00pm
ISE Seminar Series - Andrew Dorsett, Wolfram Research
"Mathematica 11 in Education and Research"
 
Tuesday, September 19, 2017 - 4:00pm
ISE Seminar Series - Alejandro Ribeiro, University of Pennsylvania
"High Order Methods in Empirical Risk Minimization"
 
Tuesday, August 22, 2017 - 4:00pm
ISE Seminar Series - Giorgio Fasano, Thales Alenia Space
"Solving Non-Standard Packing Problems by Global Optimization and Heuristics"