Blog


ReplaySSM: Cache SSM Inputs, Not State

41 min read   ·   2026

SonicMoE: A Hardware-Efficient and Software-Extensible Blueprint for Fine-Grained MoEs

46 min read   ·   2026

Gram Newton-Schulz: A Fast, Hardware-Aware Newton-Schulz Algorithm for Muon

78 min read   ·   2026

Mamba-3 Part 1

15 min read   ·   2026

FlashAttention-4: Algorithm and Kernel Pipelining Co-Design for Asymmetric Hardware Scaling

17 min read   ·   2026