Speculative Decoding on GLM-5.3-Flash: DFlash vs Native MTP
Serving GLM-5.3-Flash on two GB10 Spark nodes means every speculation choice is a trade between single-stream speed and KV memory. A draft model accelerates decode, but the draft’s KV cache eats pool that the target could use for context. We benchmarked four serving configurations against the same workloads to find out which one to run. The Four Arms GLM-5.3-Flash ships with a native MTP head (NEXTN) bundled in the checkpoint, so serving has two ways to draft: an external drafter, or the model’s own prediction head. Every drafter has a price - DFlash2’s draft weights and draft KV eat pool the target could otherwise use for context, while the native head reuses the target’s own layers, so its draft KV is smaller and there’s no extra model to load. This benchmark measures what a drafter costs in speed and KV, and what it buys in capacity. ...