A UMBC computer scientist is applying game theory to a question the AI industry hasn’t fully answered — how do you find a model’s failure points before someone else does?
Every large language model deployed today carries a hidden liability: nobody knows exactly where it will fail until someone — a curious user, a malicious actor, or a red team — finds the crack. Jailbreaks, prompt injections, and toxic outputs tend to surface only after a model is already in production, at which point the fix is a patch instead of a redesign. Dr. Samit Shivadekar, a full-time Assistant Teaching Professor in the Computer Science and Electrical Engineering (CSEE) Department at the University of Maryland, Baltimore County (UMBC), thinks the industry needs to find those cracks earlier — and more systematically.
His 2025 paper, “Red Teaming LLMs: A Stackelberg Game Approach to AI Safety,” presented at the IEEE International Conference on Innovative Mechanisms for Industry Application (ICIMIA), reframes AI red-teaming as a formal contest between two rational players. A Stackelberg game — a concept borrowed from economics, where one player moves first and the other reacts with full knowledge of that move — lets researchers model the interaction between an AI system’s defenders and the adversaries probing it for weaknesses. Instead of red-teaming as an ad hoc stress test, Shivadekar’s framework treats it as a repeatable, game-theoretic process for anticipating how an attacker would respond to a given set of defenses, then hardening the model against that response before deployment.
“The industry has largely treated red-teaming as something you do once, right before launch,” Shivadekar’s work suggests. “But if you model it as an ongoing strategic game instead of a one-time audit, you can anticipate how an adversary will adapt — and close the gap before it’s exploited.” *(Illustrative — to be confirmed with Dr. Shivadekar’s own words before publication.)*
Why This Matters Beyond the Lab
AI safety failures aren’t hypothetical anymore. Enterprises are wiring LLMs into customer service, coding pipelines, and decision support, and every one of those deployments inherits the model’s blind spots. A systematic, adversarially-modeled approach to red-teaming — rather than one-off penetration testing — offers a path toward the kind of pre-deployment assurance that regulators, enterprise buyers, and safety researchers are increasingly asking for. It’s a small but concrete contribution to a much larger industry problem: how to move AI safety testing from reactive to predictive.
A Career Built on Fusing Signals Others Miss
What makes Shivadekar’s perspective on AI safety distinctive is that it didn’t emerge from a pure NLP or security background — it emerged from years spent building AI systems that fuse messy, high-stakes, real-world data streams under NASA and NOAA funding at UMBC’s Center for Accelerated Real Time Analytics (CARTA), led by Professor Milton Halem.
Shivadekar has built deep learning systems to detect and track wildfires from satellite imagery, part of the broader “Dynamic Multiscale Wildfire Digital Twin” project spanning UMBC, NASA, and NOAA collaborators. He’s designed machine learning models that estimate Planetary Boundary Layer Height — the layer of atmosphere closest to Earth’s surface that determines how pollution disperses — by fusing ceilometer lidar readings with Weather Research and Forecasting (WRF) model output along the U.S. East Coast. And he’s applied tensor decomposition and compressive sensing — mathematical techniques for reconstructing complete signals from incomplete or noisy data — to merge satellite CO2 observations from NASA’s OCO-2 and Japan’s GOSAT missions, tracking regional carbon emissions across the U.S., Canada/Alaska, and the Amazon.
That pattern — pulling reliable signal out of noisy, high-dimensional, adversarial-feeling data — is the same instinct now driving his AI-safety work. Earlier in his career, he applied similar deep learning methods to health data, building CNN and ResNet-50 models to classify COVID-19 from lung radiography and contributing to open-source models for sub-categorizing adult heart failure using NHLBI-funded datasets. Across climate, health, and now AI safety, the throughline is the same: build systems robust enough to trust when the data — or the adversary — won’t cooperate.
Shivadekar’s path to this point wasn’t linear. Before earning his Ph.D. in Computer Science from UMBC, he spent years as a senior Java software developer at FedEx, Bank of America, and Wells Fargo — enterprise environments where systems have to work under real operational pressure, not just in a research sandbox. That grounding in production-scale engineering shows up in how his academic work is framed: not as theory for its own sake, but as infrastructure meant to hold up under stress.
Where It’s Headed
Shivadekar continues to mentor graduate and undergraduate students at UMBC on research methodology, while extending the Stackelberg framework toward more systematic, repeatable red-teaming pipelines for LLMs [VERIFY: specific next-phase research plans or funding not confirmed]. As AI safety shifts from a research niche to a deployment requirement — driven by enterprise risk teams, insurers, and regulators alike — game-theoretic frameworks like his offer one answer to an increasingly urgent question: not just whether a model can be broken, but whether its defenders can see the break coming first.
*Dr. Shivadekar holds a Ph.D. in Computer Science from the University of Maryland, Baltimore County, an M.S. in Computer Science from California State University, Fullerton, and a B.E. in Information Technology from Shivaji University, India. He has authored approximately 20 peer-reviewed publications in venues including IEEE, Springer, IGARSS, AMS, AGU, and SPIE, and won first place in the Great Learning AIML Hackathon in 2022.*
