[00:00–00:21] The hidden costs of data movement: Swapnil Bhartiya introduces the topic of egress costs surprising teams when moving AI into production. He questions how much of the centralized inference cost is attributed to data movement and how organizations can plan for these expenses.
[00:22–01:07] Modeling the complete AI bill: Ari Weil states that, over the next 12 to 36 months, successful companies will model their entire bill rather than relying on the line items touted by hyperscalers. He explains this requires understanding of scaling variables such as user acquisition, concurrency, and knowing the exact locations of both intellectual property and external learning data.
[01:08–01:40] Evaluating internal vs. cloud infrastructure: Weil suggests the first step for teams is to calculate the cost of managing data in their own environment, such as a leased space or private data center, instead of using cloud providers. This process helps teams identify data boundaries, availability zones, and the financial impact of distribution across different types of infrastructure.
[01:41–02:24] The financial barrier of cloud egress: Weil discusses the high expenses associated with exporting data or leaving a cloud partnership, noting that fees of 9 to 12 cents per gigabyte are explicitly designed to keep data locked within hyperscaler platforms. He adds that while some vendors provide a one-time data download exemption, the organization must genuinely intend to exit the provider’s cloud entirely.
[02:25–03:15] The shift to neutral architecture: The conversation shifts to neutral architectures, which allow companies to store data in one location, synchronize it across different areas, and serve it where it makes the most logical sense. Weil notes that a converged, core-to-edge architecture might initially appear expensive, but it becomes a cost benefit when eliminating the fees associated with moving data between microservices across geographic boundaries.
[03:16–03:48] Determining workload placement: To finalize an architecture strategy, Weil explains that organizations must determine exactly where inference runs, how frequently training and fine-tuning occur, and where the rest of the application activity takes place. By combining this operational knowledge with user growth modeling, teams can accurately decide which parts of their application should be centralized and which should be distributed.