SA firms prepare to take AI private

Some of the next AI systems deployed by South African companies may stay out of the public cloud entirely. Within the next 12 months

more businesses are expected to run models and company data on infrastructure they control, marking a shift towards what is known as private AI.

The transition was a central theme at Nutanix Next on Tour Johannesburg, a regional enterprise technology event held earlier this month by the hybrid cloud software company.

According to Nutanix, the discussion has moved beyond choosing a virtualisation platform. Businesses increasingly need to decide where applications and data should reside, how virtual machines and containers fit together, and whether AI should run in a public cloud, private environment or a mixture of the two.

“From an AI perspective, I think the South African market is still in the experimentation stage,” Raif Abou Diab, Nutanix regional director for South Gulf, Oman and Sub-Saharan Africa, told Gadget during the event. “They haven’t moved into production yet.”

However, SA is not necessarily behind.

Sammy Zoghlami, Nutanix SVP for Europe, Middle East and Africa (EMEA), told Gadget: “I wouldn’t say SA is different. The Middle East is quite different. They run faster on AI, but the rest of the world is pretty much at the same stage.”

AI leaves the experiment

During a briefing, Zoghlami said South African customers he met during the week had already worked through the experimentation phase and identified potential AI applications.

The questions are now changing.

“They’ve done the experimentation, [and] they know the use case they want,” he said. “Now they’re [asking]: how do we ensure security and governance, manage costs, and do it privately?”

He expects that shift to become visible relatively quickly.

“If you ask me the same question in 12 months, there will be local production use cases. [Looking at] EMEA, private AI in production is in the tens of customers, not hundreds of thousands. That’s because of the maturity level. But in one year, because of the urgency around token costs, I can guarantee we’re going to be looking at least in the hundreds.”

AI services commonly measure usage in tokens, units of text processed and generated by a model, meaning greater use can increase costs.

The move towards private AI does not mean companies are abandoning public cloud services. Instead, Nutanix expects businesses to become more selective about what runs where.

Public cloud platforms made it relatively easy for companies to begin experimenting with AI, but production systems can introduce different considerations. These include the cost of repeatedly running AI models, restrictions around sensitive information and the need to govern who or what can access company systems.

Nutanix itself encountered the cost problem internally. Zoghlami told media that rising AI usage for coding and quality assurance drove token consumption sharply higher, prompting Nutanix to build its own AI infrastructure running open-weight models, whose underlying model parameters can be downloaded and run privately. Nutanix expects the $20-million investment to pay for itself in about a year.

The company says other organisations are likely to face similar calculations as AI moves from occasional experimentation to continuous use.

Miners explore AI agents and robots

One South African example shows how enterprise AI is set to move beyond experimentation.

Abou Diab said Nutanix had held discussions with an unnamed mining customer exploring how agentic AI and robotics could be used to reduce risks to human life while taking the surrounding community into account.

Agentic AI differs from a conventional chatbot because an AI agent can carry out tasks using connected applications, data and tools, rather than only generating a response to a prompt.

The mining discussion remains at an exploratory stage, but it illustrates why moving AI into production creates a different infrastructure problem. An AI assistant answering questions carries a different level of risk from an agent interacting with business systems or physical processes.

Nutanix has been expanding its own platform around that problem. Nutanix Enterprise AI 2.8, released in August, includes Nutanix Agent Gateway, which now has a Model Context Protocol gateway designed to govern how AI agents connect to applications and company data. An AI gateway acts as a control layer between AI systems and the applications and data they use.

Zoghlami said customers reached that governance question sooner than Nutanix had anticipated.

“When we brought the AI gateway to market, we felt we were well in advance of the problem,” he told Gadget. “We thought it was going to come in two years. Actually, like everything in AI, it comes much faster.”

He said customers he met in SA were already analysing gateways.

“That means they realise they already have an issue.”

Beyond the virtual machine

The AI push forms part of a wider infrastructure strategy Nutanix presented in Johannesburg.

The company says South African IT teams are increasingly having to support traditional virtualised applications alongside containers, modern databases and AI without creating a separate technology stack and specialist team for each one.

Containers package applications and the software they need to run without requiring a complete operating system for every workload. They have become important for cloud-native applications and increasingly for AI.

Nutanix is positioning its platform to run virtual machines and containers alongside one another rather than treating one as a replacement for the other.

“The answer to every new technology requirement cannot be another layer of infrastructure,” Zoghlami said in a release. “CIOs have to create the conditions for innovation while still managing the complexity already inside the enterprise. That means building a foundation that can support traditional applications, modern applications and AI in the environments that make sense for the business, rather than creating another silo every time the technology changes.”

Nutanix calls this a consistent platform across datacentres, public cloud and edge computing. The aim is to let organisations change where workloads run as requirements around cost, performance or regulation change.

The company is expanding support for existing hardware and storage rather than requiring every modernisation project to begin with a complete infrastructure replacement.

* Jason Bannier is a data analyst at World Wide Worx and deputy editor of Gadget.co.za. Follow him on Bluesky at @jas2bann.