ECE 414 - Machine Learning and Statistical Decision Making
Electrical and Computer Engineering Dept., Lehigh University

Dr. Rick Blum, 512 Packard Lab
Email: rblum@lehigh.edu (E-mail is a good way to contact me)
Phone: (610) 758-3459
Office Hours: TBD

Important Information

  • The goal of this course is to teach the theory of Machine Learning and Statistical Decision Making. The focus will be on statistical categorization/bounds on performance.   To add to the material taken from several different textbooks, some results from our own research will be employed in some parts of the course. The hope is that students may be able to ultimately contribute to this important topic where many needed results are missing.  For an example of one desired result, we want to be able to explain mathematically when deep learning will and will not work well.  Such results are still unavailable and highly desired. 

Rough Outline of topics

1. Learning Theory: PAC, VC Dimension and other ideas (see Understanding Machine Learning: From Theory to Algorithms, 2014 by Shai Shalev-Shwartz and Shai Ben-David).

2. Statistical Theory to understand the topics under 1. and to generalize to problems with some prior knowledge or partial model.

3. Deep Learning, Support Vector Machines, Related Approaches and Special Topics as suggested.

The plan is to hand out notes and not use a textbook. Below I provide some references that might be helpful for extra material beyond the notes. Some are available online.

  • Understanding Machine Learning: From Theory to Algorithms, 2014 by Shai Shalev-Shwartz and Shai Ben-David
     
  • Statistical Theory Reference: H. V. Poor, Introduction to Signal Detection and Estimation, Springer-Verlag, New York, 1988 (latest edition).
     
  • Machine Learning Reference: Kevin Murphy, Machine learning: a probabilistic perspective
     
  • Deep Learning Reference: Michael Nielsen, Neural networks and Deep learning

Requirements

Students should have taken probability theory, have strong math skills (those needed for graduate engineering classes) and an interest in theory (statistical performance analysis). This is all I will require. I would like the class to be taken by both ECE and non-ECE students. I will let outstanding undergraduates take the course if there is space.

Warning

These are my best thoughts at this time. I am trying to focus the class on theory and I intend to do this if at all possible. Exact topics may change. Class size could change my plans. Feedback welcome.