Why Centralized Data Centers Fail Agentic AI

AI is changing the landscape. The traditional web front end, as we've known it for over 30 years, is changing. It's becoming agentic. And that means that the infrastructure that supports pretty much everything we do on the web is also having to change. So AI is fundamentally changing cloud computing.

What we're seeing now is a sea change in how cloud infrastructure itself is built. So from the highly CPU-focused cloud that we had, to now a cloud that comprises not just CPUs, but a large amount of specialized infrastructure, including GPUs and custom ASICs that allow people to be much more efficient in how they run their AI and how they price their AI. Because in the end, the price will in many ways determine adoption of such a transformative new technology.

When you look at some of the hyperscalers, they have built this massive, somewhat centralized data center. And there's a lot of power in that, right? Because the scale that they have been able to achieve and the elasticity is absolutely incredible. But in the world of AI, where you're going to have to bring these low latency experiences closer to the user, that paradigm begins to fall short a little bit. That is, I think, what the biggest part of the transformation is going to be, right?

The main limitation, I think, from the user's point of view is an awkward experience with high latency. And you've probably seen this, you've probably experienced it. If we're going to interact with agents and agents are going to help us get things done, the conversation has to be natural, it has to flow, it has to be spontaneous without having to sit there and wait for these long latency pauses.

From an enterprise perspective, AI traditionally or so far has been run in the big core data centers, the ones run by the hyperscalers and in some cases the neo clouds. Perhaps the biggest limitation in this new world of AI, and specifically AI inference, for some of the very well established hyperscalers and some of the well established clouds, is the same thing that made them incredibly powerful: the fact that they are centralized. As the world begins to shift towards AI, towards agentic workloads, it's inevitable that latency requirements and compliance requirements and security requirements are going to make it necessary to be decentralized, right? To be closer to the user, to be closer to the use cases. And then it becomes a matter of what clouds can quickly adapt to that paradigm shift.

So the one thing we have noticed in the 25 years that Akamai has been around is that user expectations only go up. We saw that with the web. We saw that with high definition streaming. We see that even with security applications. And there's no reason to think that AI will be any different. What you want in your web experience, in your internet experience, is going to continue to be extremely instant responsiveness, even when the enterprise that you're interacting with, let's say it's a shopping site, is trying to create a highly personalized experience for you. You're not going to wait 20 to 30 seconds for that highly personalized experience. You're going to want it within your attention span. And that essentially means that the infrastructure used for AI today needs to change. It needs to be able to bring that inference capability, that model capability close to you, close to whatever context you are in, and be able to respond within a couple of seconds.

Time keeps marching forward, and you can't pause and wait. A conversation requires an instant response. And it's surprising, I think, to many people how instant it has to be to have a natural conversation. If one side of the conversation is waiting a long time to answer, you can't get things done productively. Time keeps moving, so your applications have to keep moving.

If you think beyond humans, if you think about an agentic web, as agents talk to each other at massive scales, many orders of magnitude more than us humans talk to the web today, then response times become critical. And it's not just response time in isolation, it's response time when you're trying to deliver at massive scales. And essentially the entire architecture of the internet would need to change to deliver experiences like that.

Well, one principle to support the AI agents of the future is to support the full stack. All the different components, all the different microservices in the various clusters in which they're going to run. So that means not only the right hardware infrastructure, CPU, storage, and of course GPU, but also the right software infrastructure, which of course means enterprise Kubernetes.

You need to be able to put the intelligence and the data where enterprises are comfortable with them. From a latency perspective, you might want them to be closer. From a privacy perspective, you might want them to be within your control. And you will want very high guarantees that your data, the outputs of your model, or even the weights in your model are not being leaked to your competitors or to other parties outside. And these things will eventually dictate how that infrastructure has to evolve.

Securing the modern AI-agentic application poses a bunch of new challenges. AI applications inherently have a fairly large attack surface, and there are a lot of places that need to be addressed. You can't just focus all of your security in one component or in one location. The security stack has to be, in some sense, omnipresent to cover each of the components, each of the portions of the attack surface that a modern AI agent represents.

Openness and security. And I think they apply in many ways to different parts of the puzzle. You want to be open when it comes to the protocols, when it comes even to the software, to the models that you're building, to the ways in which you want your system to interact with other people. All of these things, I think when you build them in an open fashion, sometimes open source or sometimes completely accessible open APIs, you allow an ecosystem to actually grow. You allow people to innovate on top of what you have built and you innovate on top of what they build. But when it comes to customer data, to personally identifiable information, to the ability of somebody to hijack your infrastructure, I think you really have to think from the perspective of security. This is where you have to be able to build controls and guardrails that will keep customer data independent and safe from each other. You have to be able to ensure that not even your own employees are able to access data that they shouldn't, that they aren't able to make your models and AI systems do things that they shouldn't.

The AI world is very much still trying to find its footing in what it means to be fully trusted by users. And I think that's just part of incubation, right? That's just part of very, very rapidly innovating and then finding balance within security and performance and all these other things that users care about. I do think that regulation is going to end up driving a lot of this. And staying compliant is going to be one of the key driving factors for how customers think about the cloud that they need, right?

We have 20 years of experience at Akamai in delivering experiences globally that are performant, that are secure, and that ultimately satisfy what users are trying to get accomplished. Our core belief is that that experience carries over into the AI world. The paradigm is going to be the same that we saw 20 years ago when the internet itself was being transformed. We believe that that is exactly the same transformation that we're seeing right now with the shift to AI and the shift to agentic workloads.