NVIDIA and Google Cloud Scale Agentic AI with 10x Cost Improvements
NVIDIA and Google Cloud announced expanded infrastructure for agentic and physical AI workloads at Google Cloud Next, introducing A5X bare-metal instances powered by NVIDIA Vera Rubin GPUs that deliver 10x lower inference cost per token and 10x higher token throughput per megawatt compared to prior generations. The partnership also previewed Google Gemini on Google Distributed Cloud running on NVIDIA Blackwell GPUs, added confidential computing capabilities for sensitive workloads, and integrated NVIDIA Nemotron models with the Gemini Enterprise Agent Platform. The infrastructure scales from single GPUs to 960,000 NVIDIA Rubin GPUs across multisite clusters, enabling enterprises to run frontier models, agentic systems, and physical AI applications in production.
TL;DR
- NVIDIA Vera Rubin-powered A5X instances deliver 10x lower inference cost per token and 10x higher token throughput per megawatt versus prior generation
- Google Distributed Cloud now supports Google Gemini on NVIDIA Blackwell and Blackwell Ultra GPUs in preview, enabling sovereign AI deployment
- Confidential computing with NVIDIA Blackwell protects prompts and fine-tuning data in encrypted environments for regulated workloads
- Infrastructure scales from fractional GPUs to 960,000 NVIDIA Rubin GPUs across multisite clusters, with leading labs like OpenAI and Thinking Machines already deploying
Why It Matters
The collaboration addresses a critical gap in production AI infrastructure: most agentic and physical AI systems remain in labs because they require massive scale, cost optimization, and security guarantees that few platforms provide. By combining Google's managed services with NVIDIA's hardware and software stack, this partnership makes it economically and operationally feasible for enterprises to deploy complex AI agents and robotics at scale. The 10x improvements in cost and throughput directly impact the viability of real-time inference for latency-sensitive applications like factory automation and autonomous systems.
Business Impact
For operators and founders building agentic or robotics products, this infrastructure reduces the capital and operational burden of scaling inference workloads. The ability to right-size GPU allocation from fractional GPUs to massive clusters means startups can begin on modest budgets while enterprises can run production systems without overprovisioning. Confidential computing and sovereign cloud options also unlock regulated industries like finance and healthcare, expanding the addressable market for AI applications.
Key Implications
- Cost-per-token improvements make real-time agentic AI economically viable for broader use cases beyond research and high-margin applications
- Confidential computing and distributed cloud options enable enterprises to deploy frontier models on sensitive data without moving data to public cloud, reducing compliance friction
- The infrastructure's flexibility across scale tiers (fractional to 960k GPUs) lowers barriers for startups while supporting hyperscale deployments, potentially consolidating market share around this stack
- Integration of NVIDIA Nemotron open models with Gemini Enterprise Agent Platform signals a shift toward hybrid proprietary-open model strategies for agentic systems
What to Watch
Monitor adoption rates among enterprises in regulated industries, particularly whether confidential computing removes barriers to AI deployment in finance, healthcare, and government. Track whether the cost improvements translate to new use cases in robotics and physical AI, or if they primarily benefit existing high-volume inference workloads. Watch for competitive responses from AWS and Azure, especially on sovereign cloud and confidential computing capabilities, as this partnership may set a new standard for enterprise AI infrastructure.
Subscribe to the newsletter
The latest stories and analysis, delivered to your inbox.
Free. No spam. Unsubscribe any time.
