Introduction
In the past decade, the narrative around artificial intelligence has often revolved around the staggering compute costs that accompany training and deploying large language and vision models. Early discussions framed the economics of AI as a barrier to entry, especially for mid‑sized enterprises that could not afford the continuous expense of cloud GPUs or the capital outlay for on‑prem clusters. Yet, as the technology matures and the competitive stakes rise, a new conversation has emerged: the true constraints are no longer the dollars spent per token but the speed of deployment, the flexibility to iterate, and the capacity to scale without interruption. This shift is evident in the operational philosophies of companies like Wonder, a cloud‑native food‑delivery platform, and Recursion, a biotech firm that leverages AI for drug discovery. Their experiences illustrate how the most advanced AI leaders are prioritizing rapid, sustainable delivery over cost containment, redefining what it means to run AI at scale.
The core of this transformation lies in a deeper understanding of the practical realities of AI workloads. Latency, for instance, can directly impact user experience in recommendation engines or real‑time logistics, while capacity constraints can halt growth when demand spikes. Moreover, the economics of model size and the trade‑offs between large, general models and small, hyper‑customized ones have become a central strategic decision. In the sections that follow, we will unpack how Wonder and Recursion navigate these challenges, the lessons they offer for other enterprises, and the broader implications for AI adoption across industries.
Main Content
Rethinking Capacity Constraints in a Cloud‑Native World
Wonder’s journey underscores a common misconception that cloud providers offer unlimited capacity for any workload. The company’s CTO, James Chen, explains that while AI adds only a few cents per order—a fraction of the overall cost—the real bottleneck emerged when the platform’s rapid growth exhausted CPU and storage quotas in a single region. The warning from cloud vendors about potential capacity limits arrived only six months after Wonder’s expansion, forcing a swift migration to a second region. This experience highlights the importance of designing for multi‑region resilience from the outset, even when the initial assumption is that the cloud will accommodate any surge.
Capacity is not merely a technical concern; it is a strategic one. A sudden need to scale can derail product launches, delay feature rollouts, and erode customer trust. Wonder’s approach—building a cloud‑native architecture that anticipates and mitigates such spikes—serves as a blueprint for other companies that rely on real‑time AI to power critical services.
The Economics of Model Size and the Quest for Micro‑Models
Another dimension of AI economics is the size of the models themselves. Wonder currently relies on large models to optimize restaurant recommendations, a scenario that demands high accuracy and rapid inference. However, the company envisions a future where each user is served by a micro‑model tailored to their preferences and clickstream. Chen notes that creating a unique model for every customer is “not economically feasible” at present because the cost of training, storing, and serving billions of tiny models would dwarf the marginal gains in conversion.
This tension between model size and cost is a microcosm of a broader industry debate. Large models offer versatility and can be fine‑tuned for multiple tasks, but they require significant compute and memory. Conversely, small models excel in speed and resource efficiency but may lack the nuance needed for complex decision‑making. Companies must therefore weigh the incremental value of a more personalized model against the operational overhead it introduces.
Budgeting for AI: An Artful Science
Budgeting for AI is notoriously difficult because the cost structure is dynamic and often opaque. Chen describes the process at Wonder as “art versus science,” where developers experiment freely but must remain vigilant about runaway compute bills. The challenge is compounded by the token‑based pricing models of many cloud providers, which can make it hard to predict monthly expenses until a model is fully deployed.
One strategy Wonder employs is the preservation of context across requests. By maintaining a corpus of relevant information that can be reused, the company reduces the number of tokens needed per inference. However, this approach also increases the payload size for each request, potentially inflating costs. Chen’s observation that “over 50%, up to 80% of your costs is just resending the same information back into the same engine again on every request” illustrates the delicate balance between efficiency and expense.
Hybrid Infrastructure: On‑Prem vs Cloud in Practice
Recursion’s experience offers a contrasting perspective on how to manage compute resources. The biotech firm began with on‑prem clusters built from Nvidia gaming GPUs—an unconventional choice that proved surprisingly durable. Over time, Recursion upgraded to enterprise‑grade GPUs such as the A100 and H100, integrating them into a Kubernetes‑managed cluster that can run both on‑prem and in the cloud.
The company’s hybrid strategy is driven by workload characteristics. Large, data‑intensive training jobs that require high‑bandwidth interconnects and parallel file systems are best handled on‑prem, where Recursion can maintain full control over data locality and network topology. Shorter inference tasks, on the other hand, are dispatched to the cloud, leveraging pre‑emptible GPUs and TPUs to keep costs low when latency is less critical.
From a cost perspective, Recursion reports that on‑prem training can be up to ten times cheaper than equivalent cloud workloads, translating to a 50% reduction in five‑year total cost of ownership. However, the trade‑off is a longer procurement cycle and the need for dedicated hardware maintenance teams. Recursion’s CTO, Ben Mabey, cautions that “cost‑effective solutions typically require multi‑year buy‑ins,” and that companies unwilling to commit to such infrastructure may find themselves perpetually paying on demand, which stifles innovation.
Strategic Takeaways for Enterprises
The narratives of Wonder and Recursion converge on a few key insights that can guide other organizations. First, capacity planning must be proactive; assuming unlimited cloud resources can lead to costly surprises. Second, the economics of model size are evolving, and while large models are currently the default for many applications, the future may see a shift toward smaller, more efficient models as hardware and software continue to improve. Third, budgeting for AI requires a blend of disciplined cost monitoring and an openness to experimentation; the line between innovation and overspending can be thin. Finally, a hybrid infrastructure model can offer the best of both worlds—on‑prem for high‑throughput, data‑heavy workloads and cloud for agility and burst capacity.
These lessons underscore a broader industry trend: the conversation is moving from “how much does AI cost?” to “how quickly can we deploy and sustain it?” The companies that succeed will be those that can balance speed, flexibility, and cost in a way that aligns with their strategic objectives.
Conclusion
The shift in focus from compute cost to deployment speed and capacity reflects a maturing AI ecosystem. Wonder’s experience with cloud capacity constraints and Recursion’s hybrid approach to training and inference illustrate that the real barriers to AI adoption are operational, not purely financial. By embracing proactive capacity planning, rethinking model economics, and adopting flexible infrastructure strategies, enterprises can unlock the full potential of AI without being hamstrung by cost concerns. As the technology continues to evolve, those who prioritize rapid, sustainable deployment will outpace competitors who cling to outdated cost‑centric mindsets.
Call to Action
If your organization is grappling with the same questions—how to scale AI workloads, how to balance cost and speed, or how to choose between on‑prem and cloud—you’re not alone. Start by mapping your most critical AI use cases and assessing their latency and capacity requirements. Engage with cloud providers early to understand their scaling limits and explore hybrid solutions that match your workload profile. Finally, invest in a culture of experimentation that is coupled with rigorous cost monitoring. By doing so, you’ll position your business to deploy AI faster, more efficiently, and with the confidence that you’re building a resilient foundation for the future.
