Whether an application actively runs a model, connects to external AI services, feeds data to pipelines, or supplies core infrastructure, virtually every app is part of an AI system.
User requests now require complex, multistep pipelines that involve specialized compute (GPUs vs. CPUs), multi-tiered storage (vector databases and object storage), and low-latency networking to meet response targets of less than 500 milliseconds.
As inference expands to account for roughly two-thirds of all AI compute, real-time model execution — rather than one-off model training — is becoming the core driver of distributed system performance.
Because AI pipelines cross multiple locations and regulatory boundaries, data gravity forces models and infrastructure to move to where data legally and practically resides — not the other way around.
Modern AI apps rely on continuous data movement, rendering traditional, single-cloud pricing models obsolete. Ignoring data transfer fees early in the design review can lead to massive invoice shock later.
A few years ago, we talked about AI applications as if they were their own category of software: Traditional apps over here, AI apps over there. That line is disappearing for most applications.
While not every application executes a large language model (LLM), most applications now fit into one of four operational categories:
For example, a model or inference call runs inside the application's own code path, a support tool with an embedded chatbot, or a photo app with on-device object detection.
For example, the application calls an external AI service, even through an intermediate integration, or customer relationship management (CRM) software that routes tickets through a third-party summarization API counts, even if nobody on the team thinks of the CRM as “an AI product.”
For example, the application supplies data that a model somewhere else trains on, retrieves from, or gets fine-tuned against, or a 15-year-old enterprise resource planning (ERP) system that becomes part of this bucket the day someone points a retrieval-augmented generation (RAG) pipeline at its database, without a single line of the ERP itself changing.
For example, the application is infrastructure; that is, identity, storage, networking, observability, or orchestration that other AI systems depend on operationally, even if the application never processes a prompt itself.
Together, these relationships form what I think of as an AI ecosystem: applications, infrastructure, data, and services that collectively enable AI, whether or not any single component is “an AI application.”
Of course, you might read this and think your application is the exception. There will always be exceptions, but they're getting rarer every day. According to McKinsey's latest global AI survey, 88% of organizations now regularly use AI in at least one business function, up from 78% just a year earlier.
Not every application is calling an LLM every second of the day. Plenty of organizations still rely on applications that are decades old. Manufacturing systems, ERP platforms, healthcare software, government applications, and financial systems continue to run critical parts of the business. Some teams are maintaining code that predates the cloud entirely. But even those applications are getting pulled into AI workflows now, whether anyone planned for it or not.
Maybe a customer support copilot retrieves information from a legacy database. Maybe an internal search assistant is indexing 15 years of documents that nobody has touched. Or maybe operational data is quietly feeding a predictive maintenance model somewhere. The application itself hasn’t changed much, but everything around it has.
And this means your application lives in an AI ecosystem (whether you built it that way or not).
Application architecture used to follow a pretty familiar pattern: a user interacts with an application, the application queried a database, the database returned a result. You could draw that on a napkin … or ask AI to do it for you (Figure 1).
Today’s version is harder to draw on a napkin. It is increasingly common for a single user request to call multiple models, retrieve embeddings from a vector database, access object storage, invoke APIs, communicate with agent frameworks, and perform inference on GPU infrastructure before returning a response. That response may then be analyzed by observability platforms, logged for compliance and trigger another automated workflow (Figure 2).
Building applications today requires more than just choosing a model. Compute, storage, and networking have all become architectural decisions.
Not every step in that pipeline needs the same compute. An agent's reasoning runs on GPUs, but most of what it actually does — calling tools, reading files, querying databases, and hitting APIs — runs on CPUs. If you treat every step like it needs a GPU, you overprovision the parts that don't. If you treat every step like it can tolerate latency, you build something that can't ship.
AI workflows increasingly span vector databases, object storage, operational databases, and external APIs. Where that data lives affects latency, portability, and eventually egress architecture and cost. These are decisions that are not only deployment details, they're also architecture decisions.
Networking can introduce scale bottlenecks. A workflow with 50 sequential calls, which is not unusual for a multi-agent pipeline, can generate seconds of cumulative transport latency before reaching a user. According to our State of AI Inference 2026 report, that's a dealbreaker for the 82% of organizations that need end-to-end responses in less than 500 milliseconds, while a growing share (64%) are already targeting less than 250 milliseconds.
Applications are participants in distributed systems that increasingly revolve around AI. That's true even for companies that would tell you they aren't “building AI products.” For example:
None of those are AI applications themselves, but they are all essential parts of AI applications.
And GPU infrastructure now performs a more central role in distributed system performance. Deloitte projects inference will account for roughly two-thirds of all AI compute in 2026, up from one-third in 2023. The “perform inference” step is no longer a one-time event. It's becoming a recurring theme for modern application architecture.
Once you start looking at applications this way, a pattern shows up everywhere: It's all about moving data: every prompt, retrieval, embedding, model response, or API call.
None of this is new, exactly. Distributed systems have been data-intensive since long before anyone said “LLM” out loud. CDNs, big data pipelines, and mobile-first architectures all forced the same reckoning with data gravity a decade ago. What's different is the rate: AI is compounding the same problem faster than most architecture teams planned for.
Modern AI applications aren’t simply compute intensive (though they certainly are that), but they are also data intensive. The more capable an application gets, the more information it exchanges between services, regions, infrastructure, and users. This changes how cloud infrastructure was designed.
And while distributed architecture isn't new, what is new is that AI removes the last stable anchor point. A CDN still had an origin server. A multiregion database still had a primary. AI pipelines don't have an equivalent: the model, the data it needs, and the user asking the question can all be in three different places, and none of the three gets to claim “home.”
Models run in one location. Vector databases live somewhere else. Inference happens on specialized GPU infrastructure, and increasingly that infrastructure has to go wherever the data is allowed to live. Users still expect a fast response no matter where in the world they're sitting.
There's a name for this: data gravity. Digital Realty's Data Gravity Index describes it simply: “Data has a gravitational pull — as a data set grows, it attracts applications and services, creating a virtuous cycle of even more data creation.” That is, data has mass, and as its size and importance grow, it becomes harder to move or replicate, so it draws applications and infrastructure into its own orbit instead.
Some of that gravity is physics. Some of it is by design: Cloud providers charge no money to bring data in and meaningfully more money to move it out, which manufactures exactly the kind of gravity that keeps workloads anchored to wherever they first landed.
AI is accelerating both versions of that pull. Data gravity used to be a storage problem. Now it's an architecture problem, because the model has to go where the data can legally and practically sit, not the other way around.
More data movement doesn't automatically become a problem. It becomes a problem when infrastructure economics assume data should stay in one place. And that's exactly what most public cloud pricing models still reward.
Cloud pricing hasn't really kept up with how applications are built today. When most workloads ran in one place, charging for data that leaves the network was a reasonable model. AI applications don't work that way. They're distributed by design, and that means data is always on the move.
Every interaction is a series of distributed requests that move between services that often don't live in the same place. So, infrastructure costs are increasingly a reflection of how often applications move data rather than simply how much compute they consume.
Industry analysts estimate cloud data transfer fees to total between US$70 and US$80 billion annually, and that number continues to grow as organizations deploy more AI-powered applications. At the same time, egress pricing has become one of the biggest barriers to moving workloads between cloud providers.
Pricing has always shaped architecture, that's the entire premise of FinOps. What's changed is how early in the design process the pricing conversation has to happen. A team used to be able to design the ideal AI pipeline, then go find out what it costs. Increasingly, the egress bill now shows up in the design review before the architecture diagram is even final.
If every application contains AI, connects to AI, feeds data to AI, or supports something that feeds data to AI, then the practical question is not “Does my application use AI?” By our definition, it probably already does, either directly or by association. The more important question is whether your infrastructure was built assuming that data moves constantly, or whether that assumption got bolted on after the fact.
The answer to that question actually predicts what happens next. Platforms built for data in motion can absorb a new model, a new region, or a new compliance requirement without a re-architecture. Platforms designed on the assumption that data remains stationary incur unexpected operational costs, which often become visible only upon receipt of the cloud billing statements. Data gravity and egress pricing aren't edge cases anymore; they are standard architectural considerations.
You don't get to opt out of the ecosystem. You only get to decide whether you build for it on purpose.
Need help understanding your AI ecosystem? Meet with the Akamai team for your AI consult.
© 2026 Akamai Technologies
