No matter what time of day you read this story, it’s a near certainty that you’ve come in contact with artificial intelligence at some point since rolling out of bed. AI is on our phones and in our cars. It’s sounding off in any number of rooms in our houses, where we work, and where we play. Such ubiquity is ripe for misappropriation.
“Over the past five years or so, AI technology has developed to the point where nearly all of us interact with it on a daily basis,” says Lichao Sun, an assistant professor of computer science and engineering. “But with that level of development, there comes significant issues around trustworthiness, privacy, and security. As a field, we’re way behind in addressing those concerns.”
Sun has devoted his career to tackling security and privacy challenges related to AI technology. One of his current projects confronts the persistent problem of protecting online audio content from being appropriated to train generative AI models, including large language models (LLMs) such as ChatGPT and DeepSeek, and diffusion models such as Dall-E and Sora.
“When these models collect audio that is not protected, it can create a host of problems for the creator of that data,” says Sun. “For example, it can infringe on their copyright protections and cause them to lose out on revenue, or the data can be used to create false or misleading information like when the speaker’s voice is appropriated to endorse something or someone they do not personally believe in, or to do something nefarious.”
Sun and his team are collaborating with researchers at the University of Tennessee to develop a framework that will make audio content—specifically music in this case—“unlearnable” by AI. Called HarmonyCloak, the approach essentially introduces white noise, or perturbations, that are imperceptible to the human ear. The additions don’t affect fidelity, but they make the music incomprehensible to AI models.
“It gives the content creator control over how their work is used,” he says. “If a tech company wants to use that music to train their model, the company has to pay for it. Once they’ve paid for it, the creator can strip out that protective layer.”
The team, which includes Syed Irfan Ali Meerza, a PhD student, and Jian Liu, an assistant professor of electrical engineering and computer science, both from UT Knoxville, has shown that HarmonyCloak can effectively shield music across a range of commonly used AI models. Since such technology is constantly evolving, the group is working on developing a universal perturbation that can work across any model.
The ultimate goal, says Sun, is to develop a plug-in of sorts that creators can deploy within their music before it’s published online.
Sun’s other projects include developing a benchmark called TrustLLM that can evaluate the trustworthiness of large language models around toxicity, bias, and value-alignment inherent in the LLM (value-alignment refers to matching, generally, human values).
The effort seeks to improve our understanding of how conversation generation models perform and help guide the technology to advance following a more ethical and socially responsible path. Essentially, it evaluates the safety, security, and privacy issues of generative AI, especially large language models.
He’s also working on an open-source, vision-language foundation model called BiomedGPT that can perform various medical tasks. This model may one day help doctors by interpreting complex medical images, assist researchers by analyzing scientific literature, or even aid in drug discovery by predicting how molecules behave.
“This work combines two types of AI into a decision-support tool for medical providers,” says Sun, who is the lead author of a paper recently published in the prestigious journal Nature Medicine detailing the approach. “One side of the system is trained to understand biomedical images, and one is trained to understand and assess biomedical text. The combination of these allows the model to tackle a wide range of biomedical challenges, using insight gleaned from databases of biomedical imagery and from the analysis and synthesis of scientific and medical research reports.”
All three projects involve Sun’s PhD students, and extensive collaboration with other universities and, in the case of BiomedGPT, other Lehigh faculty members and medical institutions like Massachusetts General Hospital, which validated the model’s effectiveness and applicability in real-world healthcare settings.
“I really enjoy these collaborations,” says Sun. “These problems are so complex and require expertise across a wide range of areas. Working together, we have a real opportunity to make meaningful contributions when it comes to privacy, security, trustworthiness, and the potential of AI.”
