Microsoft has outlined the Maia 300 inference system it plans to deploy in 2027: a rack with 96 accelerators inside a 167 kW power envelope. In its October 12 announcement, Microsoft says the earlier Maia 200 is already serving production workloads. Maia 300 is a forthcoming design, not a currently deployed upgrade.
Memory and data movement
Each Maia 300 accelerator is specified with eight HBM4 stacks providing 288 GB of capacity and 18 TB/s of peak bandwidth, with a 950 W thermal design power (TDP). Compute chiplets use TSMC's N2P process, while memory-controller and I/O functions sit on separate dies. In its architecture deep dive, Microsoft says a non-uniform memory layout and software-defined dataflow are meant to keep data nearer the compute using it. Software coordinates movement among on-die SRAM, HBM and the network.
That focus matters for long-context and decode-heavy inference, where feeding model weights and retained context to the compute can constrain useful throughput. Microsoft also describes narrower formats, including enhanced MXFP4 and sub-3-bit storage, to move fewer bytes. These are architectural mechanisms; the result for any given model will depend on its precision, memory footprint and software implementation.
A dense rack with two network tiers
Microsoft's rack design places the 96 accelerators in 24 trays of four. Each tray also contains its host processor, memory, storage and cooling. The 167 kW figure describes the rack power envelope, not a single chip. Microsoft says liquid cooling removes more than 80% of the rack's thermal load, a material facility requirement for operators.
The scale-up network uses electrical links within a rack and optical links between racks. Microsoft describes a superpod of up to 128 racks, or 12,288 accelerators, under one communication model. That does not make every path equally fast: directly connected accelerator pairs, other chips in the same rack and chips in other racks occupy different latency tiers. Placement still matters for traffic-sensitive workloads.
Claims that need real workloads
Microsoft expects 40% more tokens per megawatt than what it calls leading GPU-based systems and claims up to seven times the rack-scale token throughput of Maia 200 in a full 96-accelerator configuration. Those are Microsoft comparisons, not independently verified results in the two linked posts. The rack comparison also includes twice as many accelerators per rack, so it is not a sevenfold per-chip claim.
Neither post supplies a complete workload, model, precision, utilization and comparison-hardware breakdown for readers to reproduce those figures. Microsoft says Maia 300 production deployments will begin in 2027, but the posts do not specify general Azure customer availability or pricing. Actual deployed performance and operating cost remain to be assessed.
