Speculative-Decoding on Musings of an AI Wrangler

Speculative-Decoding on Musings of an AI Wranglerhttps://mazurov.dev/tags/speculative-decoding/Recent content in Speculative-Decoding on Musings of an AI WranglerHugo -- 0.148.0en-usMon, 07 Sep 2026 15:33:53 -0600Speculative Decoding on GLM-5.3-Flash: DFlash vs Native MTPhttps://mazurov.dev/posts/glm-speculation-benchmark/Mon, 07 Sep 2026 11:00:00 -0700https://mazurov.dev/posts/glm-speculation-benchmark/A four-arm benchmark of speculative decoding on GLM-5.3-Flash (NVFP4, TP2 over two GB10 nodes): DFlash BF16, DFlash FP8 draft KV, native MTP, and no speculation. Native MTP's bundled draft head costs about 20% of short-prompt throughput and leaves an 89% larger KV pool - a good trade for long-context serving.