The two-stage architecture achieves a 0.981 area under the receiver operating characteristic curve (ROC-AUC) and 92.4% agentic recall on request sequences with at least five mouse events, reliably ranking autonomous agent traffic above human sessions.
Overall session recall sits at 52.6%, but jumps to 97.9% on evaluable sessions. Performance varies significantly by tool based on how sparse their telemetry is — OpenAI Atlas sessions were 89.9% evaluable, whereas Perplexity Comet sessions were only 48.8% evaluable.
A leave-one-customer-out cross-validation demonstrated strong transferability across websites, maintaining a 0.956 ROC-AUC, proving the model learns universal behavioral signatures rather than site-specific patterns.
Agents display sparse, direct mouse movements with 2,000 ms to 3,000 ms idle gaps while “thinking,” contrasting sharply with the continuous, curvilinear trajectories of human users.
Designed as an insight signal rather than an outright blocker, the model is most effective when paired with passive fingerprinting to confirm AI browser activity and feed risk-scoring dashboards.
In part one of this two-part series, we introduced a category of web traffic that existing bot detection models weren't designed to catch: AI agents operating in “control browser” mode.
Tools like Perplexity Comet and the Claude Chrome Extension navigate websites autonomously — clicking purposefully, moving directly, and generating behavioral telemetry that is too sparse for traditional behavioral models to evaluate.
We built a two-stage detection architecture to address this: a Masked Autoencoder Transformer trained on 3.5 million unlabeled mouse event sequences to produce compact behavioral embeddings, followed by a shallow multilayer perceptron (MLP) classifier trained on 2,190 labeled agentic sessions collected from real, BMP-instrumented customer websites.
In this blog post, we present the results of that model — how it performs at the request level and at the session level, how well the behavioral signatures generalize across customers, and what agentic sessions actually look like in the data. We also discuss what the results mean in the context of a real-world deployment.
On a held-out test set of labeled sequences (with at least five mouse events), the final two-stage model’s achievements are shown in Table 1.
| Metric | Value |
|---|---|
ROC-AUC | 0.981 |
Agentic recall (true positive rate) | 92.4% |
Human specificity (true negative rate) | 94.7% |
F2 score | 0.651 |
Table 1: MLP classifier performance metrics on held-out dataset of labelled telemetry sequences composed of at least five mouse events
Request-level recall is one thing. But in production, what matters is whether we can flag a session — and agents often produce many requests within a session, some with enough telemetry to score and some without.
We evaluated the model across all 816 labeled agentic sessions in our dataset using max-probability aggregation (a session is flagged if any single request scores above the threshold; Table 2).
| Metric | Value |
|---|---|
Overall session recall | 52.6% |
Recall on evaluable sessions (≥ 5 events in at least one request) | 97.9% |
Evaluable session fraction | 53.7% |
Table 2: MLP classifier performance metrics on sessions, where sessions are classified as agentic automation if the mouse telemetry sequence from at least one HTTP POST request was classified as agentic
The gap between 52.6% and 97.9% is almost entirely explained by the 46.3% of sessions in which every single request contains fewer than five mouse events. Once a session has at least one scoreable request, the model identifies it with near-perfect reliability.
Per-tool performance reflects behavioral differences between agents. Atlas (OpenAI) sessions are 89.9% evaluable — Atlas tends to generate more mouse activity than Comet. Comet (Perplexity) sessions, which make up 86% of our dataset, are only 48.8% evaluable; Comet's Control Browser mode produces very sparse telemetry (Figure 1).
To test whether the model was learning genuine behavioral patterns or overfitting to the specific customer websites in our training data, we ran a leave-one-customer-out cross-validation across the five customers with the most labeled agentic data.
All telemetry used in this analysis is fully anonymized so that no personally identifiable information is retained, and customer identities are not referenced in or recoverable from the dataset (Table 3).
| Metric | In-distribution | Cross-customer CV |
|---|---|---|
ROC-AUC | 0.983 | 0.956 ± 0.014 |
Precision (balanced) | 94.5% | 97.3% ± 2.2% |
Recall | ~92.1% | 56.3% ± 14.3% |
Table 3: K-fold cross-validation metrics comparing performance on in-distribution data (held-out test data from the set of customer websites included in the training set of a model’s fold) to performance on held-out data from customer websites entirely excluded from a model’s training fold
The 2.7 percentage point drop in ROC-AUC from in-distribution to cross-customer suggests the model is generalizing meaningfully — the behavioral signatures it learned transfer across websites and customer verticals.
The higher variance in recall (39.6%–72.7% across folds) reflects differences in how much telemetry each tool generates on different sites, not model failures.
K-fold cross-validation results, in which a model was trained using data from all but one customer’s data for each fold are shown in Figure 2. The held-out customer’s data was only used for evaluating model generalizability and determining if overfitting was occurring.
Precision, agentic recall, and ROC-AUC for models trained on each of the folds are shown in Figure 3.
The visualizations in Figure 4 compare mouse trajectory paths and interevent timing distributions from human and agentic sessions in our labeled dataset.
The trajectory visualizations in Figure 5 make the behavioral separation intuitive. A human session shows rich, curvilinear paths across the viewport — exploratory, wandering, occasionally backtracking.
An agent session, however, shows sparse, direct paths: appear at point A, move to point B, clicking, done. Interevent timing for agentic sessions often shows gaps of 2,000 ms to 3,000 ms between any mouse activity, reflecting the agent’s “thinking” between actions.
False positives (i.e., human sessions that the model flags as agentic) tend to be users who visited a page, clicked one link, and left. Their behavioral profile is sparse and direct, superficially resembling agent behavior.
These are genuine edge cases, not artifacts: These humans interacted efficiently, and the model's confusion is structurally reasonable, but can likely be mitigated with richer training performed in future research.
Updating transformer embeddings by concatenating interevent timing features (such as the mean and standard deviations shown in Figure 5) before they are passed to the MLP classifier is likely to yield improved model performance on the set of currently misclassified sequences.
Detection confidence improves substantially when behavioral signals are combined with passive fingerprinting signals. We already have such identification signals for agentic browsers that rely on fingerprint characteristics. The behavioral model is most powerful when applied conditionally on sessions where passive signals already suggest an AI browser, adding a layer of confirmation from behavior itself.
This research is a proof of concept (PoC), and several open questions will shape the path forward.
Prevalence estimation: How common is agentic traffic across all web traffic? Knowing true prevalence is essential to understanding real-world precision.
Hybrid sessions: Users sometimes cede control to an agent mid-session and resume manual browsing. How can we identify these mixed sessions?
Data and model scale: The current model was intentionally kept small as a PoC. Will larger labeled datasets and more expressive models yield more robust and generalizable detections?
Adversarial robustness: As agentic tools evolve, it will be important to ask: Will behavioral signatures remain stable or require ongoing model updates?
Agentic AI tools represent a genuinely new category of web traffic — not bots in disguise, not humans, but autonomous agents acting on human behalf. They interact with websites economically and purposefully, producing behavioral telemetry that is sparse enough to fall beneath the radar of models that were designed to detect traditional bots.
As a PoC, we've shown that a purpose-built approach — learning a compact behavioral representation from millions of unlabeled mouse event sequences, then training a lightweight supervised classifier on labeled agentic data — can identify automation-mode sessions with 92% request-level recall and 0.981 ROC-AUC, and generalizes meaningfully across customers in leave-one-out cross-validation.
These results validate the detection direction; translating this into a production-ready capability will require:
Scaling the dataset
Hardening the model against adversarial agents
Integrating the behavioral signal into a broader ensemble alongside passive fingerprinting detections
As agentic adoption grows, understanding what fraction of web traffic is driven by AI agents, and which agent tools are responsible, becomes an increasingly important capability for businesses and security teams alike. This research lays the groundwork for future capabilities in Akamai Bot & Agent Control to help better identify, understand, and manage AI agent traffic.
To learn more about how Akamai can help you gain visibility and control over your agentic traffic, contact an expert.
The research described in this post was conducted by the Akamai Threat Research and Data Science teams. The labeled agentic dataset was collected and anonymized following a standardized protocol using BMP-instrumented customer websites.
© 2026 Akamai Technologies
