Supermicro New Rack Is All CPU, and That Is the Point for Agentic AI
The Vera CPU rack puts 256 CPUs and 22,500 agent sandboxes behind liquid cooling. As agents move into production, the bottleneck is shifting from token generation to execution.
Supermicro has announced a rack with no GPUs in it at all. The NVIDIA Vera CPU rack holds 256 NVIDIA Vera CPUs, 22,528 cores and 45,056 threads in a single 48U cabinet, and it supports up to 22,500 concurrent agent sandboxes. It is the latest addition to Supermicro’s data centre building block portfolio, it sits alongside the Vera Rubin NVL72 GPU system that is now shipping, and it is available to quote now. The interesting part is not the specification. It is the premise: that the next bottleneck in AI infrastructure is not generating tokens but executing what agents do with them.
That premise has been building all year. Agentic workloads do not run as a single inference call. They run as loops of dependent steps: the model reasons, calls a tool, executes code, reads a result, and reasons again. GPUs generate the tokens, code and queries. CPUs run the sandboxes that execute the actions, evaluate the results and orchestrate the workflow. At factory scale, thousands or millions of those environments run in parallel, and each one wants its own core. Nvidia has said for months that agentic AI is bottlenecked by traditional CPUs, and it built Vera to answer that claim. Supermicro now sells the rack that puts the claim into production.
What Supermicro announced
The Vera CPU rack packs 32 liquid-cooled 1U compute trays, each with four nodes and eight Vera CPUs, for 256 CPUs per rack. Nvidia rates each chip at 88 custom Olympus cores with 176 threads, and the rack carries up to 400 terabytes of SOCAMM2 LPDDR5X memory with up to 300 terabytes per second of aggregate bandwidth. Networking runs through BlueField-4 data processing units at up to 200 gigabits per second per CPU. Supermicro puts operating power at 208 kilowatts and builds the rack on its DLC-2 direct liquid cooling stack, which it says makes a fully liquid-cooled design possible. The rack works alongside Vera Rubin NVL72 or on its own, and Supermicro will sell it with site survey, integration, deployment and support.
The rack is not only a companion to a GPU cluster. Supermicro also sells it as a standalone CPU platform, and it lists agent orchestration, sandboxing, database operations, cluster management, schedulers such as Slurm and Kubernetes, analytics and large software build pipelines as targets. That gives the product a life before the paired architecture is widely deployed: a customer can add CPU density now, for build and test farms or for a service that runs many small jobs, and add the GPU racks later. The 48U design carries 32 compute trays, each with four nodes and eight CPUs, which is the kind of packing that only liquid cooling makes possible at this density.
Why agents need CPUs
The CPU-heavy design follows from how agents actually run. A GPU cluster generates tokens quickly, but an agent that has decided to query a database, run a Python script and inspect the output spends its wall-clock time waiting for the CPU to do those things. Nvidia claims Vera runs those tasks up to 1.8 times faster than leading x86 processors, and CoreWeave, which tested it, reports sandbox startup more than three times faster than x86, with a single rack supporting more than 11,000 concurrent agent environments. Those are vendor and partner figures rather than independent benchmarks, and the honest reading is that the direction is credible while the multipliers need testing on a customer’s own workload.
What is not in doubt is the architectural point: if a factory runs 10,000 GPUs and each agent needs a sandbox, the CPU estate has to scale with the GPU estate, and it has not.
The power and cooling shift
The more consequential change may sit in the power path. The rack supports either a conventional AC feed or an 800-volt DC sidecar power rack, and the DC option lets a customer use battery backup units instead of a conventional uninterruptible power supply. That matters more than it sounds. Moving energy storage out of the compute rack and into a power rack frees rack space for compute, removes an AC-to-DC conversion stage, and cuts the copper needed to deliver a given amount of power, which one industry estimate puts at around 45 per cent. The Open Compute Project published a white paper on low-voltage DC distribution this month making the same case, and Nvidia, Google, Microsoft and Meta have converged on 800 volts DC as the direction of travel. Data centre power design stayed stable for two decades. It is not stable now.
Governance in silicon
Supermicro also leans on the security architecture that Nvidia published alongside Vera. The OpenShell reference architecture treats every agent as untrusted and grants access only as needed, through four components: a gateway that manages sandbox lifecycles, a supervisor that checks outbound requests and mediates credentials from outside the sandbox, the sandbox itself, and a policy prover that formally verifies that a policy’s permissions stay inside an organisation-defined ceiling. Where a customer wants more, Nvidia Sentry runs on the BlueField-4 DPUs in the I/O path, outside the agent’s reach, monitors behaviour without relying on the agent’s own reporting, and can quarantine an agent within milliseconds. That answers a question risk committees have asked all year: not whether an agent can be told what to do, but whether something outside the agent can stop it when it does something else.
The competitive race
Supermicro is not alone. Dell announced a PowerEdge server that packs 256 Vera CPUs into a 48U rack on the same day, along with an air-cooled two-socket system for standard racks and a modular data centre built on Nvidia’s DSX POD reference design. AMD used the Open Compute Project summit this month to launch Helion, a rack-scale reference design for agentic AI that supports up to 32,768 EPYC cores in a CPU-only configuration and mixes CPUs with GPUs in enterprise deployments. Microsoft has said its Cobalt 200 CPU targets agentic orchestration, retrieval and memory management. The consensus is forming quickly: the agentic data centre is a disaggregated one, with GPU racks for generation and CPU racks for execution, that fast fabric connects and that something outside the workload governs.
What buyers should ask
For a team planning agent capacity, the questions are concrete. How many concurrent agents do we actually need, and what per-agent core budget does that imply? What does a sandbox cost us today in startup time and memory, using our own workload rather than a benchmark? Does our facility support 800-volt DC, and if not, what would it take, because the power rack decision is harder to reverse than the server decision? Can we pin agent policy outside the workload, and can we prove to an auditor that something stopped an agent rather than merely logged it? And what is the plan for the GPU-to-CPU ratio as models get more agentic, because getting it wrong wastes capacity in one direction or starves it in the other?
What it signals
The Vera CPU rack is a hardware launch, and hardware launches are easy to over-read. But it marks something real: the industry has started building for agents as a distinct workload rather than assuming that more GPUs will cover it. That has three consequences. CPU capacity becomes a first-class purchase again, after years of being an afterthought next to accelerators. Power architecture becomes a competitive variable, because you cannot feed a megawatt rack through the electrical designs of 2015. And agent governance moves from a software feature into the silicon, which is where regulated buyers have wanted it. The companies that get the ratio and the power path right will run more agents per megawatt than those that do not, and in a business that counts tokens per dollar, that is the whole game. The timing reflects a broader change in how the industry talks about AI capacity.
Accelerators and Power
For two years the conversation was about accelerators and the power to feed them. Through 2026 it has shifted to what happens around the model: retrieval, orchestration, tool use and evaluation. Microsoft has said its Cobalt 200 CPU targets exactly those tasks, AMD built a rack-scale reference design for agentic workloads, and the Open Compute Project’s summit this month focused on coordination rather than raw scaling. A rack of CPUs is a mundane object in that story, and it is also the clearest sign that the workload has changed. None of this guarantees the rack sells.
Infrastructure buyers move slowly, and the 800-volt DC path asks them to change a facility design that has worked for years. But the direction is set, and the interesting question is no longer whether CPU capacity will scale alongside GPU capacity but who captures the value when it does. Nvidia has designed the chip and the reference architecture, and its manufacturing partners compete on how densely and how reliably they can package it.
For Supermicro, whose business has always been integration rather than silicon, that is a familiar position, and the rack is a reminder that in AI infrastructure the unglamorous layer often decides whether the expensive layer earns its keep. The companies that master it will rarely make headlines for it.