Sooner or later, every successful AI project runs into a classic problem: Where do you put all the data?
One of our customers hit that point in a very specific way. Four production AI models were each writing 75 mebibyte (MiB) result files every few seconds, 24/7. Over time, their on‑prem platform started to creak: Arrays ran hot, backup windows slipped, and “Add more servers” stopped scaling.
The customer already used Akamai for content delivery, so centralizing AI outputs into Akamai Object Storage was an obvious next step. It kept traffic on a familiar network, minimized egress, and positioned those same files to be served downstream.
The open question was simple: Can Akamai Object Storage comfortably absorb this firehose today and as the models scale?
To answer that credibly, I built a small ingestion demo and measured it. In this blog post, I will describe the AI ingestion pattern, the test setup, and what the results mean for customers.
The AI ingestion pattern
The workload pattern looked like this:
Four independent AI models, each writing 75 MiB result files
Every six hours, the models collectively produce:
260 files from model 1
520 files from model 2
8,060 files from model 3 (effectively 31 “directories” of 260 files each)
328 files from model 4
That’s 9,168 files per batch, about 0.68 tebibyte (TiB) of data, landing in a single S3‑compatible bucket.
On‑prem, this stressed both storage controllers and the internal network. Each model had its own retry logic and assumptions about throughput, so capacity planning turned into guesswork.
The test setup
For the test setup, I used:
An 8 GB dedicated CPU instance in the Los Angeles (us‑lax‑4) region as a predictable client
An Object Storage bucket in the Chicago (us‑ord‑1) region as the S3‑compatible target
The platform’s documented rate limit and throughput guidance as a reference point
The demo: Four models in a box
Then, I built a lightweight tool that could behave like all four models at once, drivable from a browser by anyone on the team. The setup used a Node.js back end deployed on a us-lax-4 instance, paired with a single-page HTML/JS front end. The back end leveraged the AWS SDK for JavaScript v3 S3 client, pointed at Akamai’s S3-compatible endpoint.
The target was an Object Storage bucket in us-ord-1, accessible via the us-ord-1.linodeobjects.com endpoint.
The goal was not a fancy dashboard, but a transparent instrument that anyone on the team could use to see how the system behaves.
What the results tell us about Object Storage for AI
For this customer’s pattern (approximately 0.68 TiB per six‑hour batch), the bottleneck was the client instance and network, not Object Storage. After several runs, two other findings stood out:
- The storage tier had ample headroom. Even near 2 Gbps of sustained ingest, the job consumed only a small fraction of documented per‑bucket transactional capacity.
- Read‑after‑write behaved as advertised. Newly written objects appeared immediately in listings, which is exactly what near‑real‑time pipelines need.
In other words, the question shifted from “Can the platform keep up?” to “How fast do we want to go?”
Conclusion
For AI teams with multiple models emitting large artifacts, this pattern is both repeatable and observable: Point your outputs at Object Storage, run the demo (or a variant of it), and quickly see whether your current storage platform is your constraint. Your results may prove that it’s time to move to a more scalable ingest layer.
If your own models are already starting to outgrow their current storage platform, now is a good time to validate how Akamai Object Storage behaves with your specific workload. Recreate this simple test, plug in your file sizes, batch patterns, and regions, and use the results to inform your architecture decision-making.
Next steps
Get started with Object Storage by setting up a bucket using our documentation. From there, you can decide how fast you want to go — and how much future headroom you want to keep in reserve.
Tags
