Student(s): William Tang, Kevin Zhu
Project: DA-SEGFORMER: Damage-Aware Semantic Segmentation for Fine-Grained Disaster Assessment
Advisor(s): Maryam Rahnemoonfar
Abstract
Rapid and accurate damage assessment following natural disasters is essential for effective emergency response. While Unmanned Aerial Vehicle (UAV) imagery provides the high resolution necessary for automated semantic segmentation, distinguishing fine-grained damage levels remains a significant challenge. This difficulty is primarily driven by the degradation of critical texture cues during standard image resizing and extreme class imbalance, where combined damage pixels constitute less than 8% of the dataset.
To address these challenges, this paper introduces DA-SegFormer, a damage-aware adaptation of the SegFormer architecture specifically optimized for high-resolution disaster imagery. The proposed methodology incorporates a Class-Aware Sampling strategy, ensuring exposure to rare damage features to counteract background dominance. Additionally, the architecture integrates Online Hard Example Mining (OHEM) combined with Dice Loss. This combination allows the model to dynamically focus gradient updates on difficult, underrepresented pixels while optimizing for region overlap on a per-class basis. Finally, the approach utilizes a resolution-preserving sliding window inference protocol that maintains native texture details without aggressive downsampling.
Evaluations conducted on the RescueNet UAV dataset demonstrate the efficacy of the proposed model. DA-SegFormer achieves a Mean Intersection over Union (mIoU) of 74.67%, outperforming the baseline SegFormer architecture by 4.37%. Most notably, the model yields double-digit performance gains in the most challenging and critical damage categories, improving Minor Damage identification by 13.9% and Major Damage by 12.5%.
About William Tang
Major: Computer Science and Engineering
William Tang is a third-year Computer Science student at Lehigh University, where he is part of the Computer Science and Business Honors Program. Originally from the New York Metro area, he is passionate about software engineering, artificial intelligence, and building impactful, user-focused technology. At Lehigh, William serves as a Computer Vision Research Lead at Bina Lab, where he co-authored an IEEE paper on UAV-based hurricane damage detection and developed large-scale machine learning pipelines to improve model performance. He has gained professional experience as a Full Stack UI/UX Developer Intern at RevSend and as an Accounting and Database Intern for the City of Norwalk, where he automated financial systems and improved data reliability. William is also working on a Merck-sponsored capstone project focused on modernizing legacy applications using AI. An Eagle Scout, he enjoys rowing, playing the cello, strength training, and outdoor recreation.
About Kevin Zhu
Major: Computer Science and Engineering
Kevin Zhu is a junior studying computer science with a minor in data science. He serves as the Research Lead at Bina Labs, focusing on optimizing the DA-SegFormer computer vision model for post-disaster damage assessment. Kevin is currently building Explainable AI (XAI) tools to make this model more transparent, and he recently authored an IEEE paper based on his findings with DA-SegFormer. Beyond his research, he was on the winning team at the JPMorgan Chase Data for Good Hackathon and previously interned at Incyte Corporation. This upcoming summer, Kevin will join Honeywell as a Software Engineer Intern. After graduation, he plans to continue creating innovative AI solutions. Originally from Media, Pennsylvania, Kevin enjoys playing poker and going to the gym outside of the lab.