Huawei OceanStor M900: 64 PB KV Cache Storage for AI Inference

By AiCybr Editorial · Published September 17, 2026

Huawei introduced OceanStor M900 Context Memory Storage on September 17, 2026, positioning the system as a shared KV-cache tier for hyperscale AI inference. Huawei specifies 64 PB of capacity per cluster, 40 TB/s of aggregate access bandwidth and 60 microseconds of access latency, using its UnifiedBus fabric to connect NPUs directly to SSD-backed storage.

The design extends inference context storage beyond accelerator memory and DRAM into a globally shared, multi-tier KV-cache pool. Huawei says available KV-cache capacity per NPU can move from gigabyte scale to terabyte scale, allowing long-context and multi-turn workloads to retain and reuse more previously computed key-value data.

OceanStor M900 is aimed at Huawei SuperPoD deployments and other large inference systems built around the company's UnifiedBus architecture. Its published performance figures are Huawei measurements and claims from the launch; independent production benchmarks for the M900 were not available at launch.

OceanStor M900 specifications

Huawei says the 60-microsecond access path represents a 90% latency reduction compared with the millisecond-scale path used as its reference. The company also claims 40 TB/s is 1.5 times the aggregate bandwidth of peer solutions. Huawei has not published a standardized third-party comparison behind that peer figure.

Why KV cache is moving into storage

Transformer inference generates key-value cache data for previously processed tokens. Reusing that data avoids recomputing attention state across an existing context, which is particularly valuable for long prompts, multi-turn conversations and agents that repeatedly operate over large working contexts.

Accelerator memory provides the fastest cache tier, but its capacity is constrained and expensive. DRAM adds capacity at a lower tier. OceanStor M900 introduces SSDs as another shared layer, trading some latency for a much larger cache pool while retaining direct integration with the compute fabric.

Huawei describes the resulting hierarchy as a global multi-tier KV cache spanning on-chip memory, DRAM and SSD storage. A larger shared cache can retain more context across an inference cluster and increase the probability that reusable KV data remains available when a subsequent request needs it.

This architecture is relevant as inference systems increasingly optimize around memory capacity, memory movement and cache reuse, rather than accelerator arithmetic alone. The value depends on the workload's cache reuse pattern: a workload that repeatedly references long or shared contexts has more opportunity to benefit than independent short prompts with little reusable state.

One-hop NPU-to-SSD access

Huawei says M900 integrates CPU, network-controller and NAND-controller functions into its storage architecture and provides native KV semantics. UnifiedBus then enables a one-hop path between a SuperPoD NPU and SSD storage.

The goal is to remove protocol-conversion and CPU-forwarding stages from the KV-cache data path. Huawei attributes the system's claimed 60 µs access latency to this architecture.

That latency remains substantially different from local accelerator memory, so M900 functions as a capacity-oriented cache tier within a hierarchy. Placement software determines which KV data should remain in faster memory and which data can move to the SSD-backed pool.

Huawei claims 2× token throughput in an AI coding workload

Huawei reports that OceanStor M900 doubled inference-cluster token throughput and cut time to first token by half in what it describes as a typical AI programming scenario.

Those results are vendor launch measurements. Huawei has not published enough workload, model, accelerator-count, concurrency and baseline configuration detail to reproduce the comparison independently. They are best read as an indication of the performance target for cache-heavy inference rather than a universal multiplier for every model or deployment.

The underlying mechanism is plausible at the systems level: when an inference workload repeatedly reuses large contexts, a higher KV-cache hit rate can reduce recomputation and keep accelerator resources available for new token generation. Actual gains will depend on model architecture, prompt lengths, request concurrency, cache locality and the relative performance of the faster cache tiers.

KV-aware SSD management targets endurance

Using SSDs as an active inference cache creates a write-endurance problem. KV data can have short and highly variable lifetimes, and a high-throughput cache can generate substantially more writes than conventional enterprise storage workloads.

Huawei says its KV-aware adaptive storage predicts cache-data lifetimes and distributes data across storage media according to value and expected retention. The company specifies support for up to 24 drive writes per day (DWPD) and claims the technique extends SSD endurance by 16 times, supporting three years of stable operation under its target workload.

The endurance claim is important to M900's economics because replacing SSDs frequently would erode the cost advantage of moving large KV-cache capacity out of DRAM. Production operators will need workload-specific endurance data to calculate media replacement rates and total cost of ownership.

M900's role in Huawei's SuperPoD architecture

Huawei introduced M900 alongside a broader expansion of its SuperPoD and UnifiedBus infrastructure. The company describes context memory storage as an L3.5 KV-cache layer within its agentic computing architecture.

Huawei's updated TaiShan 950 SuperPoD supports up to 4,096 nodes and a unified memory pool of up to 256 TB. Its larger agentic SuperCluster design combines Ascend SuperPoDs, Kunpeng general-purpose SuperPoDs and KV-cache clusters through UnifiedBus. Huawei says a two-tier, four-plane Clos topology can interconnect up to 512,000 NPUs, with a multi-rail topology extending the design target to one million NPUs.

M900 addresses a different scaling dimension from adding accelerators: retaining and moving inference state. At very large cluster sizes, the ability to reuse context without repeatedly rebuilding it can affect accelerator utilization as directly as raw compute capacity.

What operators should evaluate

For infrastructure teams, the most useful evaluation is workload-specific. Long-context coding agents, enterprise assistants operating over persistent sessions, multi-agent systems and repeated inference over shared prefixes are natural candidates for a large external KV-cache tier.

The key measurements are cache hit rate, TTFT, output-token throughput, SSD write amplification, media endurance, tail latency and the amount of accelerator memory freed for active inference. Those metrics should be measured together because maximizing cache capacity has limited value if the external tier adds excessive latency or storage churn.

Integration is another boundary. Huawei's launch material centers M900 on UnifiedBus and Huawei SuperPoD infrastructure, so it should be evaluated as part of that systems architecture rather than as a generic drop-in NVMe cache for arbitrary GPU clusters.

Sources

• Huawei: Context Memory Storage OceanStor M900 launch
• Huawei: HUAWEI CONNECT infrastructure keynote
• HPCwire and AIwire provided independent launch coverage on September 17, 2026.

Bottom line

OceanStor M900 turns KV cache into a dedicated infrastructure tier: Huawei is offering up to 64 PB of shared cache capacity per cluster and 40 TB/s of aggregate bandwidth, with a one-hop UnifiedBus path designed to keep SSD-backed context close to NPUs.

The release reflects a broader shift in large-scale inference engineering. As contexts, concurrency and agent workloads grow, accelerator utilization increasingly depends on where inference state is stored and how efficiently it can be reused. Huawei's launch figures are substantial; production benchmarks will determine how much of the claimed throughput and latency improvement carries across real models and request patterns.