AI inference moving to private clouds, Broadcom says

The majority of enterprises now either run or plan to run AI workloads in private clouds, according to a survey of 1,800 senior IT decision makers conducted by Radius Tech on behalf of Broadcom. Only 41% of enterprises are now using public clouds for inference workloads, down from 56% last year. Meanwhile, the use of private clouds for AI inference has risen slightly, from 55% to 56%. “The key takeaway this year is that we’ve seen an AI tipping point, driving towards private cloud as the preferred platform for running these workloads,” says Prashanth Shenoy, CMO and vice president of marketing for VMware Cloud Foundation division at Broadcom. Overall, 72% of enterprises intend to increase their private cloud spending over the next three years, up from just 51% in 2025’s survey. In addition, 50% of enterprises have already repatriated some workloads, up from 35% in 2025, and another 33% are considering doing so. Public clouds are also growing, the report shows, but at half the rate of private cloud investment. The increase in interest in private clouds is driven by a number of factors, including security and compliance, followed by cost predictability and performance. Agentic AI, in particular, can quickly cause cost overruns as the use of agents can increase large language model use exponentially. According to today’s survey, 62% of IT leaders are either “very” or “extremely” concerned about gen AI and agentic AI infrastructure costs. Enterprises are also concerned about data protection and privacy, followed closely by security and control, both of which are strengths of the private cloud deployment model. Last year, Shenoy says, there was huge growth in public cloud usage for AI pilots and for training. “Now that the majority of large-scale enterprise customers are done doing that, they want the models to be closer to where the data is and where the data is generated,” he says. “And that is in their own on-premise private cloud environment.” Public cloud is still the right answer for many workloads, says Mauricio Sanchez, analyst at Dell’Oro Group. “But the old assumption that every workload eventually moves to public cloud has broken down.” And it’s not just about AI. According to the survey, 97% of respondents say that some of their public cloud spending is wasted — and 52% say that the amount of waste is more than 25%. However, while costs are a concern, they’re not necessarily the biggest factor that determines where enterprises run their workloads. Security and compliance took the lead, with 32% selecting it as the most critical factor, followed by data sovereignty and control at 15%, performance and latency at 14%, and integration with existing systems also at 14%. Cost is tied with speed of deployment and scalability at 12%. Sanchez agrees that enterprises are concerned about data exposure, regulations, performance, and cost. “AI sharpens that trade-off,” he adds. “If a workload is highly variable or needs access to specialized cloud services, public cloud can be attractive. But if a company is running steady AI inference against sensitive data, wants more control over where data and models live, or needs predictable economics, a private cloud can look much better than it did a few years ago.” The difference between AI workloads and other types of applications is that AI pulls in large data sets and requires expensive accelerators. It also needs networking, security controls, and has unique governance requirements. For enterprises located outside the US, there are also sovereignty issues, adds Michela Menting, an analyst at ABI Research. “With the largest public cloud providers being US-based, there is concern in the rest of the world for data protection that meets local regulations,” she says. AI systems might use data, or process data, in a way that’s not compliant with regulations, she says. “Private cloud seems to offer more safeguards,” she says.