For the first time in a decade, machines generate more web traffic than people do. Automated requests made up 51% of all web traffic in 2024, and the malicious portion, the bots that scrape pricing, hoard inventory, and probe for weak points, climbed to 37%. The instinctive response from most digital businesses is to block more traffic, more aggressively. But a blocking rule cannot tell a determined scraper apart from an impatient human with perfect accuracy, and when it guesses wrong, the loss does not show up in a security dashboard. It shows up in revenue.
Spandan Brahmbhatt, a Senior Data Scientist with more than a decade in cybersecurity and large-scale predictive modeling, has spent recent years working on the other side of that tradeoff. He leads detection for a real-time scraper protection platform that screens billions of requests a day across aviation, banking, and e-commerce clients, deciding within milliseconds whether each visitor is a person or a program. Beyond the platform, he serves as a technical program committee reviewer for the 8th International Conference on Blockchain Computing and Applications, evaluating peer research on security and decentralized systems. The question driving his work is the one the industry keeps answering badly: how do you stop the scrapers without turning away the customers?
When the Defense Becomes the Liability
The accuracy problem is larger than most teams admit. On the average e-commerce platform, between 2% and 10% of the orders rejected for suspected fraud are placed by real customers, people who wanted to buy and were turned away at the door. Each of those rejections is a sale lost at the worst possible moment, after the business has already paid to acquire the visitor and the visitor has signaled clear intent. Legacy detection tools, built to match known attack signatures, treat anything unfamiliar as suspect, which means they fail precisely against the traffic that matters most: the new, the rushed, the slightly unusual but entirely legitimate.
Spandan’s response has been to attack the false positive directly rather than treat it as acceptable collateral. He continuously audits legacy scraper-detection logic, isolating the rules that quietly block good traffic, and rebuilds them around data-driven feedback loops that retrain on real outcomes instead of static assumptions. Where older systems optimized only for catch rate, his models carry a strict threshold for human impact, tuned so that raising the wall against scrapers does not raise it against buyers. The work is unglamorous and constant, because every gain an attacker makes forces another pass through the same tradeoff.
“A false negative gets you a headline. A false positive gets you a churned customer who never tells you why they left,” notes Spandan. “The second one is harder to measure, which is exactly why the industry underinvests in it.”
Why a Single Percentage Point Decides the Quarter
The financial asymmetry is stark. Globally, false declines cost retailers roughly $443 billion a year, about 9 times the losses attributed to actual fraud. In other words, the machinery built to protect revenue destroys far more of it than the threat it was designed to stop. For high-stakes verticals, the margin for error is thinner still. A small misclassification rate on an airline booking engine or a ticketing platform translates into seats and tickets that real customers could not buy, and into a competitor or reseller who profited from the gap.
This is why Spandan treats precision as a financial discipline rather than a technical metric. On the platforms he protects, a single point of false positives can erase several points of monthly revenue, so detection has to hold its accuracy across hundreds of distinct customer architectures at once, from a niche retailer to a multinational carrier. He approaches each deployment as forensic analysis first, profiling what normal human behavior looks like for that specific site before deciding what counts as anomalous. The goal is not a universal rulebook but a tailored baseline, because the traffic of a banking portal shares little with that of a flash sale.
“Most teams ask how many bots they caught. I care more about how many people we let through who should never have been doubted,” Spandan explains. “Precision is the only number the business actually feels.”
Modeling the Human, Not the Attacker
Signature-based detection has a structural flaw it can never outgrow. It can only recognize attacks it has already seen, which leaves it a step behind every operator willing to change tactics, and modern operators change tactics constantly. A different approach inverts the problem. Instead of cataloguing every possible attack, it builds a detailed model of verified human behavior and flags whatever deviates from that baseline, an approach often called negative security. The advantage is durability: a defender who understands what humans do does not need to predict every move an adversary will invent.
Spandan has built much of his career inside this shift. His detection framework layers deterministic rules, unsupervised clustering that surfaces anomalies without being told what to look for, and models trained on human behavioral and biometric signals such as mouse dynamics and navigation rhythm, the micro-patterns that automation struggles to fake convincingly. That depth of method is why he is regularly asked to evaluate others’ work. He recently judged the Hack-Nation Global AI Hackathon, an MIT-rooted competition, assessing AI builds under real time pressure. Judging sharpens the same instinct his day job demands, the ability to separate something that genuinely works from something that merely looks impressive.
“You cannot win by memorizing the attacker. They iterate faster than any signature list,” says Spandan. “But human behavior is stable. If you model that well, the bot has to imitate a person perfectly to get through, and almost none of them can.”
The Adversary Has AI Now
The barrier to running a sophisticated attack has collapsed. What once required real engineering skill, reverse-engineering a site’s defenses and writing custom evasion code, can now be assembled from off-the-shelf tools and AI assistants that handle the hard parts. A commercial market of bot services has grown around this, letting operators with little technical depth launch high volumes of convincing traffic. The result is not only more attacks but a steady erosion of the old signals defenders relied on, since browser fingerprints, device identifiers, and network parameters can all be spoofed at scale.
Spandan spends a meaningful share of his time taking these tools apart. He reverse-engineers the commercial scraping frameworks circulating in underground forums and builds preemptive defenses before the tools reach wide use, working alongside threat intelligence teams to track how adversary techniques shift. When an operator does adapt and slips past current defenses, his systems trigger immediate re-analysis and retraining rather than waiting for a quarterly model refresh. The defense, in his framing, has to behave like a loop rather than a wall, because a wall only holds against the last attack it was built to stop.
“We used to assume that if traffic looked technically clean, it was probably human. That assumption is dead,” Spandan observes. “The interesting signal now is behavior over time, not any single fingerprint in a single moment.”
Precision as a Business Strategy
The reframing this demands is uncomfortable for a security industry that has long measured success by the number of threats it blocks. The more honest measure is harder: how many legitimate customers reach the checkout, and how few of them ever notice the security working at all. Detection, viewed this way, behaves less like a gate than a referee. The businesses that grasp the difference are beginning to fund accuracy with the same seriousness they once reserved for raw catch rates.
Spandan sees the next phase moving further toward behavior modeled across whole sessions rather than judged at single moments, and toward systems that adapt in real time as attackers do. Vector representations of how a user moves through a site, retrieved and compared at speed, can flag the repetitive workflows that betray a scraper even when each individual request looks ordinary. He is candid that none of this ends the contest. The attackers will keep adapting, the defenders will keep rebuilding, and the measure of success will stay the same: did real people get through.
“The win condition was never zero bots. That is not achievable, and chasing it is how you end up blocking your own customers,” Spandan reflects. “The win is a person who buys their ticket in 3 seconds and never knows a fight happened on their behalf. If we did our job, they felt nothing at all.”
