One only needs to glance at the news, social media, or even just out the window to understand the devastation caused by flooding. Recent back-to-back major hurricanes have brought catastrophic rainfall that has devastated communities across the southeastern United States.
With climate change, experts predict these extreme weather events will increasingly become the norm. Among the many ways that researchers are devising strategies to protect and assist vulnerable areas, one such effort involves increasing the speed and accuracy of damage assessments.
“Research shows that if you can save a single day in the initial response phase, it could save thousands of days in the recovery phase because the effect is logarithmic,” says Lehigh University researcher Maryam Rahnemoonfar, an associate professor of computer science and engineering with a joint appointment in the Department of Civil and Environmental Engineering.
Rahnemoonfar is part of a team led by North Carolina Agricultural and Technical State University (NCAT) that recently received a nearly $1 million grant from the National Science Foundation to support their innovative approach to mapping and assessing flood damage. The award, of which Lehigh will receive $160,000, is part of NSF’s Historically Black Colleges and Universities – Excellence in Research program, which funds research at public and private HBCUs to strengthen research capacity and promote engagement with NSF.
This collaboration between Lehigh and NCAT grew out of efforts within Lehigh's Interdisciplinary Research Institutes (IRIs) to foster cross-institutional partnerships that address critical challenges.
A ‘more comprehensive picture’ of disaster zones
Current post-disaster damage assessments typically rely on data from time-consuming manual methods, like ground-based gauges, and remote sensing technologies, such as satellite imagery. Although useful, such tools are localized and provide only a fragmented view of affected areas. They also lack multi-resolution data.
“Optical imagery doesn’t allow you to see below the surface, so you can miss critical information about what’s under the water, like roads and bridges,” says Rahnemoonfar. “But with radar sensors, you can detect submerged objects and get a more comprehensive picture.”
The team will combine multi-modal data sources—including satellite, UAV (unmanned aerial vehicle), and various remote-sensing technologies, such as optical, SAR (Synthetic Aperture Radar), and LiDAR (Light Detection and Ranging)—into a novel framework that automatically extracts spatial and temporal features from these data streams. And they’ll combine all of that with an innovative vision language model that the Lehigh team led by Rahnemoonfar will create.
“Vision language models will be built into a flying or ground-based robot that a response team can communicate with,” she says. “First responders might ask questions like, ‘What roads are flooded?’ or ‘Which neighborhoods are under water?’ and based on the robot’s answer, the team could more efficiently allocate their resources. We’re combining the language and visual domains.”
Rahnmoonfar has developed such models in the past for RGB sensors, but the algorithms that powered them are fully supervised, meaning they require a huge amount of labeled training data in order to identify and learn patterns. Her team will develop a self-supervised algorithm that requires far less data and will use information from their multi-modal data sources and from previous floods.
“Ultimately, we want to develop a system that can assist the rapid response and recovery required after weather disasters,” she says.
Lehigh’s IRIs: Seeding collaborations
The partnership between Lehigh and NCAT was initially facilitated by a seed grant from the Institute for Data, Intelligent Systems, and Computation (I-DISC), one of Lehigh’s three Interdisciplinary Research Institutes (IRIs)
“The goal of all Lehigh’s IRIs is to bring faculty from different areas of expertise and different backgrounds together to address major societal problems,” says Himanshu Jain, T.L. Diamond Distinguished Chair in Engineering and Applied Science and a professor of materials science and engineering. Jain recently stepped down as inaugural director of Lehigh’s Institute for Functional Materials and Devices (I-FMD), a role he held since 2018.
In his capacity as a faculty member, Jain has worked with HBCUs for the past 20 years—including a teaching stint at Tuskegee University—and through his work with I-FMD, he connected with Lehigh alum Jagannathan Sankar '83 PhD, a distinguished professor in NCAT’s Department of Mechanical Engineering.
“NCAT’s researchers are highly regarded,” says Jain, “and through our connection with Professor Sankar, we started developing ideas around how we could bring our faculty together.”
The discussions eventually led to Jain organizing “seed interactions”—virtual meetings where faculty from both institutions could brainstorm possible collaborations. Jain also established seed funding within Lehigh’s IRIs to help potential ideas get off the ground.
As a participant in these meetings, Rahnemoonfar met a colleague at NCAT, Leila Hashemi Beni, an associate professor in the Department of Built Environment, whose research area and interests complemented her own. With the seed funding, Rahnemoonfar and Beni were able to refine the idea that eventually led to the funded NSF award.
“What we wanted to demonstrate with these seed interactions is that they’re effective and lead to fruitful collaboration,” says Jain. “It’s the beginning of what we want to establish as a track record of our institutions successfully working together. What we ultimately want from the IRIs is for them to help incubate focused research centers devoted to addressing big, consequential societal issues. With NCAT, we are not only technically interested in each other’s expertise, but we have a scientific synergy that can have a significant impact on some of the world’s biggest problems.”
