The Telegraph Took Control Over AI Scrapers

Telegraph Media Group (TMG) has built its business on high-quality journalism delivered to a global audience. With a subscription-first model, TMG surpassed its ambitious goal of reaching 1 million subscriptions in 2023. Since then, the rise of AI-driven content scraping has created new challenges. Increasingly, AI bots have been harvesting The Telegraph’s journalism without permission. The Telegraph turned to its trusted partner Akamai to provide a solution. Through Akamai, The Telegraph is able to protect its content from AI bots and secure its APIs.

AI bots not only harvest content to train models, but also serve summaries to users — often without attribution or compensation. “People see a summary served up by AI and don’t click through to our site,” explained Ciro Rizzo, Director of Technology for TMG. “That’s lost traffic, lost engagement, and lost revenue.” 

Some outputs even misrepresent The Telegraph’s journalism, risking brand integrity. In addition, scraper traffic consumes costly infrastructure capacity without generating income. “When our content is licensed to partners, we’re compensated,” explained Rizzo. “But when it’s scraped and used without permission. It doesn’t align with our business model or strategy.”

The Telegraph needed to distinguish between “good” and “bad” bots, block unauthorized crawlers, and gather intelligence to pursue licensing agreements with those wanting access. Having worked with Akamai for years, the choice was straightforward. “No other provider can address the range of challenges we face, from ensuring fast content delivery to enabling sophisticated bot management,” Rizzo said.

To directly address the AI scraping problem, The Telegraph is running a proof of concept (PoC) with Content Protector. Early results have been revealing: “The first stage was an eye-opener — we finally understood the size of the AI crawler ecosystem and true scale of the challenge,” said Rizzo.

The PoC has already led to blocking large numbers of aggressive bots, many from obscure AI firms. This intelligence could support future negotiations with authorized crawlers, potentially creating new revenue streams while reducing brand and cost impacts.

At the heart of The Telegraph’s mobile-first strategy is its app ecosystem, powered by hundreds of APIs connecting its content management system (CMS), payment platform, and subscriber registration services.

By moving critical domains behind Akamai App & API Protector, The Telegraph now has consistent governance, visibility, and control across hundreds of APIs, which are especially important for high-value registration and payment services.

The Telegraph’s content library spans years of daily publishing — hundreds of articles per day — supported by a massive network of redirects for SEO and audience engagement. Previously, managing redirects was time-consuming and dependent on internal servers.

By moving to EdgeWorkers, The Telegraph transformed the process. What once took up to 15 minutes now happens in seconds. “When we want to drive traffic from one article to another, EdgeWorkers spins up redirects instantly across all locations — reliably, every time,” Rizzo said.

Understanding traffic patterns is key to operational efficiency and strategic planning. However, in the past, it was difficult for The Telegraph to aggregate and correlate all its logs. “That made it challenging to debug issues, especially for our CMS and website,” he explained.

TrafficPeak centralizes and correlates logs. For The Telegraph’s site reliability engineering team, that’s a game changer. “TrafficPeak shows us the whole picture, cutting troubleshooting from hours to minutes,” Rizzo said.

Moreover, Rizzo appreciates that TrafficPeak stores and manages large datasets for extended periods. “Access to that data means we can see correlations and patterns we could otherwise miss,” he continued.

Rizzo sees Akamai playing a role in the future evolution of The Telegraph’s app. “Akamai reliably delivers the industry-leading expertise, support, and solutions we need to both protect our journalism and grow our business,” he concluded.

Telegraph Media Group (TMG) is the publisher of The Daily Telegraph and The Sunday Telegraph, The Telegraph Magazine, Telegraph.co.uk, and the Telegraph app. TMG is a subscriber-first business, with quality journalism at our heart, and a goal to build deeper connections with our subscribers at scale.