The Question Everyone Asked, and the Answer That Changed

For years, the question around GPU Confidential Computing was simple. How much slower is the GPU once encryption and isolation are switched on.

New research on NVIDIA Blackwell hardware, published by Hang Yin and Kevin Wang of Phala Network as The Serialized Bridge, gives a settled answer to that question.

The GPU itself is barely affected, but the path connecting the confidential virtual machine to the GPU becomes a serialized, high-cost channel that ordinary inference software was never built to survive.

The Compute Myth

Researchers Hang Yin and Kevin Wang measured BF16 matmul performance on the NVIDIA B300 and found it running at 0.998x of native speed under confidential computing. A CUDA graph chaining 96,000 matmuls together came in at 1.0012x, meaning compute was essentially unaffected.

Yet the same hardware lost 13 to 27 percent of throughput running dense LLM decode workloads, and mixture-of-experts serving lost up to 25 percent. KV-cache restore latency rose 131 percent, and loading a 120-billion-parameter model took 287 seconds using the default loader path.

This gap is the entire subject of the paper. If the GPU is fine, something else is absorbing the cost.

The Bridge, Not the Chip

The researchers trace the loss to what they call the CVM-GPU bridge, the runtime path connecting the confidential virtual machine’s protected memory to the GPU’s protected compute region. Because neither side can directly read the other’s protected memory, every transfer must cross through an encrypted staging buffer instead of moving as ordinary DMA traffic.

That bridge behaves nothing like the fast, concurrent memory transfers that modern inference engines expect. Three measured properties define its behavior, and each one directly undermines an assumption baked into today’s serving software.

First, transfers inside a single CUDA context do not run in parallel, no matter how many streams are issued against them. Sixteen streams issuing small device-to-host copies under confidential computing achieved close to zero parallel gain, compared with roughly 24 percent scaling when confidential computing was off.

Second, asynchronous copies are silently converted into blocking ones. A call marked non-blocking still holds the calling thread until the entire transfer finishes, because the driver cannot safely return control before the secure staging setup completes.

Third, every crossing pays a fixed setup cost regardless of size, measured at roughly 330 microseconds per call. For a small payload of only a few dozen bytes, that fixed cost is three orders of magnitude larger than the actual data being moved.

Why Faster Hardware Makes It Worse

One of the paper’s sharper findings is that the relative tax grows as hardware gets faster. The same serving stack that loses about 10 percent on an RTX Pro 6000 loses about 26 percent on the much faster B300, because every part of the pipeline that the GPU accelerates shrinks except the bridge.

A comparison against NVIDIA’s older Hopper generation, using an H200 GPU, confirms this is not a Blackwell-only quirk. The underlying bridge law already exists on Hopper, but Blackwell is fast enough that the fixed bridge cost becomes large enough to flip which scheduling strategy is actually optimal.

Policy Inversion in the Serving Stack

The clearest illustration comes from vLLM, a widely used open-source inference engine. Its default asynchronous scheduling mode normally overlaps a decode step’s output transfer with preparation for the next step, saving roughly three milliseconds per step when confidential computing is off.

Under confidential computing, that same optimization costs about four milliseconds per step instead of saving time. The overlapped copies serialize on the bridge anyway, so the runtime pays the overhead of attempting concurrency without ever receiving the benefit.

The researchers call this policy inversion, and they proved it through elimination rather than assumption. Patches targeting batching, pinned memory allocation, and CUDA graph capture modes all failed to close the gap, leaving scheduling policy as the only variable that mattered.

Recovering the Lost Performance

Once the mechanism was identified, the fixes followed directly from it. Disabling asynchronous scheduling with a single flag recovered 57 percent of the throughput gap on dense decode workloads and nearly all of the KV-cache restore penalty.

A deeper fix moved the blocking data drain onto a separate worker thread instead of the main serving engine thread. In a qualified high-concurrency test run, meaning the researchers’ hardware access window closed before they could independently reproduce it, that patch recovered up to 92 percent of the original performance gap, landing within roughly 8 percent of fully non-confidential throughput.

Model loading benefited from the same logic applied to a different layer. Pooling secure CUDA contexts and prewarming them before weight loading began cut load time for a 120-billion-parameter model from 287 seconds down to 8.4 seconds.

KV-cache offload policy needed a similar adjustment. Restricting offload to blocks that had actually been reused, rather than speculatively caching everything, cut spilled data volume dramatically and nearly tripled warm-start response time under confidential computing.

Blackwell’s Bigger Fabric Story

Blackwell also changes what a confidential compute tenant actually is. On the B300 HGX platform, a tenant is no longer a single GPU passed through to one workload, but a partition of a shared NVSwitch fabric spanning multiple GPUs.

The researchers qualified two-GPU confidential tenants sustaining 510 gigabytes per second of peer-to-peer bandwidth over NVLink, entirely inside the protected virtual machine. That number matters because it sits two orders of magnitude above what the CVM-GPU bridge itself can sustain, making the scale-up fabric the one data path confidential computing does not serialize.

That capability comes with an open trust question. A tenant can verify its own attestation evidence, GPU state, and visible fabric health, but it cannot yet verify the Fabric Manager software or NVSwitch routing tables controlling the shared fabric underneath it.

Confidential Tenancy Modes Compared

The paper documents three distinct confidential tenancy modes across GPU generations, summarized below. Each mode makes a different tradeoff between isolation, fabric bandwidth, and scheduling flexibility.

Mode	Typical Tenant Shape	Fabric Assumption	Scheduling Implication
Single-GPU GPU-CC	One GPU	No tenant-visible NVLink fabric	Scheduler assigns one confidential GPU
Protected PCIe (Hopper)	Whole attached GPU complex	Fabric private to the tenant by assignment	Scheduler sacrifices utilization to isolate a large unit
Blackwell multi-GPU GPU-CC	1, 2, 4, or 8-GPU NVSwitch partition	Shared fabric, host-managed, confidential partitions	Scheduler allocates a fabric-valid partition shape

Why Hardware-Level Visibility Matters

Findings like these make one thing clear about confidential computing on modern AI infrastructure. Software-level fixes such as scheduling flags and pooled contexts recover most of the lost performance, but none of them can confirm that the firmware, GPU components, and fabric control plane underneath that software are what they claim to be.

That distinction sits below the operating system, in a layer traditional endpoint detection and response tools and legacy vulnerability scanners were never designed to reach. Eclypsium addresses that gap by giving data center teams continuous visibility into GPU firmware, driver versions, and hardware integrity across AI infrastructure, independent of whichever confidential computing scheduling policy is running above it.

As the paper itself notes, the Fabric Manager binary, its configuration, and the NVSwitch routing state that programs a confidential partition all remain outside a tenant’s attestation evidence today.

Continuous firmware and hardware monitoring at the infrastructure layer is what closes that visibility gap while platform-level attestation standards continue to mature.

Frequently Asked Questions

What is GPU Confidential Computing on Blackwell hardware?

GPU Confidential Computing pairs a confidential virtual machine, protected by Intel TDX, with an NVIDIA GPU running in a protected compute region. Together they isolate AI workloads and data from the host system and other tenants sharing the same physical hardware.

Does confidential computing slow down the GPU itself?

No, GPU compute and GPU-local memory stay close to native performance under confidential computing on Blackwell. Measured BF16 matmul throughput on the B300 came in at 0.998x of non-confidential speed, with GPU-local HBM traffic only slightly reduced.

Where does the performance loss actually come from?

The loss comes from the bridge connecting the confidential virtual machine to the GPU, not from the GPU itself. That bridge forces host-to-device transfers through an encrypted staging path that serializes traffic and adds a fixed setup cost to every crossing.

Can the lost performance be recovered without new hardware?

Yes, the paper demonstrates a recovery hierarchy built entirely from software changes. A single scheduling flag recovered 57 percent of the throughput gap, and a worker-thread patch pushed recovery as high as 92 percent in a qualified high-concurrency test that the researchers were not able to independently reproduce before their hardware access ended.

What changes with Blackwell’s multi-GPU fabric?

On the B300 HGX platform, a confidential tenant can now span a partition of a shared NVSwitch fabric rather than a single passed-through GPU. Two-GPU confidential tenants sustained 510 gigabytes per second of NVLink bandwidth entirely inside the protected virtual machine.

Does confidential computing verify GPU firmware and hardware integrity?

Confidential computing attests the software and execution state inside the protected boundary, but it does not independently verify the firmware, drivers, or fabric control plane running underneath it. That verification gap is why continuous infrastructure-level monitoring remains necessary alongside confidential computing.

Is the performance tax the same across every workload?

No, the tax scales with how often a workload crosses the bridge in small pieces. Rate-limited serving barely notices the cost, dense decode pays a moderate tax, and bulk KV-cache restore pays the largest penalty of all.

Conclusion

GPU Confidential Computing on Blackwell keeps its promise on compute, running AI workloads at essentially native speed inside a protected boundary. The real work of recovering lost performance happens in the bridge connecting that boundary to the rest of the system, and in the firmware and fabric layers that neither scheduling flags nor worker threads were ever designed to secure.