Student: Ryan Forelli
Project: Real-Time Machine Learning in Scanning Probe
View: Research Poster (PDF)
Institution: Lehigh University
Major: Computer Engineering
Advisor: Joshua Agar
Abstract
Increased development and utilization of multimodal scanning probe microscopy (SPM) and spectroscopy techniques have led to an orders-of-magnitude increase in the volume, velocity, and variety of collected data. Scientists traditionally rely on empirical models to make predictions on complex datasets; however, this is not possible in SPM due to the low-veracity and high-velocity of the data collected. Recently, there has been an increase in the application of machine and deep learning techniques that use batching and stochastic methods to regularize statistical models in order to execute functions or aid scientific discovery and interpretation of this data.
One method to accelerate machine learning inference is bringing the computational resources as close to the data acquisition source as possible, thus minimizing latencies associated with I/O and scheduling. Neural processing units generally used for this application typically have less memory, computational blocks, and flexibility than CPUs and GPUs. Here, we develop an implementation that uses cantilever resonances acquired in BEPS[1] to predict the simple harmonic oscillator (SHO) fit in real-time. To do this, we leverage the National Instruments PXI-platform to create a high-speed, peer-to-peer communication channel two FPGAs. Using a high-level synthesis tool called HLS4ML[2], we then deploy a pruned neural network on one FPGA to conduct real-time prediction of the SHO fit. Our latest benchmark of this system indicates a prediction latency on the order of nanoseconds. This work provides a foundation for deploying on-sensor neural networks using specialty hardware for real-time analysis and control of materials imaging systems.
References:
Borodinov, N., Neumayer, S., Kalinin, S. V., Ovchinnikova, O. S., Vasudevan, R. K. & Jesse, S. Deep neural networks for understanding noisy data applied to physical property extraction in scanning probe microscopy. npj Computational Materials 5, 25 (2019). doi:10.1038/s41524-019-0148-5
Duarte, J., Han, S., Harris, P., Jindariani, S., Kreinar, E., Kreis, B., Ngadiuba, J., Pierini, M., Rivera, R., Tran, N. & Wu, Z. Fast inference of deep neural networks in FPGAs for particle physics. J. Instrum. 13, P07027 (2018). doi:10.1088/1748-0221/13/07/P07027
About Ryan Forelli
Ryan Forelli, a first-year student at Lehigh University, is majoring in Computer Engineering and minoring in Physics with interests in field-programmable gate array (FPGA) and embedded programming as well as machine learning. He is currently performing undergraduate research in the Multifunctional Materials and Machine Learning (M3 Learning) Group under Dr. Josh Agar. His current research focuses on the use of neural networks on FPGAs to accelerate materials science discovery by performing real-time inference on collected scanning probe microscopy data. In the past he has been an undergraduate researcher in the Data for Impact Summer Institute through the Creative Inquiry Mountaintop initiative, and is currently a member of the Lehigh Formula SAE electrical sub-team and the Lehigh University Student Scholars Institute. In his free time, he enjoys reading about new topics and technologies, playing racquetball, and model building. After graduation, he plans to pursue a PhD in Physics or Computer Engineering.