Liquid AI’s DSpark Draft Models Make LFM2.5 Up To 3.2x Faster Without Changing A Single Token

Posted on by
Contents show
A flat vector illustration of a large grey speedometer gauge with a thin teal needle near the top and a small amber draft page shape beside it, representing Liquid AI LFM2.5-DSpark speculative decoding draft models for faster decoding

The most useful kind of speed claim in AI is the one that costs nothing. Liquid AI released DSpark draft model checkpoints for its LFM2.5 family on 20 August 2026, adding speculative decoding to a model family already designed for on-device use, and the property that makes the release worth a builder's attention is not the headline speedup, up to 3.18x faster decoding on a GPU, but the guarantee behind it: speculative decoding is exact, the target model verifies every proposed token, and the emitted sequence is identical to what the model would have produced alone. No quality trade, no benchmark erosion, just the same outputs sooner.

The release covers three LFM2.5 models, 1.2B-Instruct, 2.6B and the 8B-A1B mixture-of-experts, each getting a draft model of roughly 300 million parameters, with day-one support in llama.cpp and SGLang. That last detail is the difference between a research artifact and an usable feature: the runtime support landed upstream at release, so adopting speculative decoding here is a flag, not a fork. This article works from Liquid AI's own technical blog, read in full, and covers how speculative decoding works, how DSpark's three components improve on the plain recipe, the real measured speedups per model and workload, and where the gains shrink, which the announcement unusually discloses in detail.

Why decoding is slow, and how speculation fixes it

LLM inference has two phases, and the slow one is not the one people expect. Decoding, generating tokens one at a time, is memory-bound: most of the latency comes from streaming the model's weights from DRAM into SRAM, not from computation. Each generated token pays the full cost of moving the weights, and the arithmetic itself is nearly free by comparison.

Speculative decoding attacks exactly that cost structure, and it does so without touching the model's weights or outputs. A lightweight draft model proposes several candidate tokens, and the target model verifies all of them in a single forward pass, sharing the cost of loading its weights across every token verified. If the draft is right often, one forward pass yields several accepted tokens, and throughput rises with the acceptance rate, which is why speculative decoding performance is reported alongside acceptance counts rather than speed alone. If the draft is wrong, the target model's own token takes its place, and the only cost is the wasted proposal.

The critical property of speculative decoding is exactness: under greedy decoding, a draft token is accepted only if it matches the target model's distribution. The output sequence is therefore identical to baseline by construction, which means benchmark accuracy is unchanged, not approximately unchanged. That is what separates speculative decoding from quantization or distillation, which trade quality for speed, and it is why the technique pairs so well with the local-model deployments this site covers: the speedup arrives without any re-evaluation burden.

The three DSpark components

DSpark builds on prior speculation methods, most prominently EAGLE-3 and DFlash, and combines three components that address the three limits of a naive draft model.

First, a DFlash-style parallel backbone conditioned on the target model's context features produces hidden states for all draft tokens in a single forward pass, rather than running the draft model sequentially per token, which is what keeps the draft's own cost low enough that speculative decoding remains a net win. Second, a lightweight sequential head, modeled as a Markov chain between neighboring tokens, adds inter-token dependency to the parallel proposals, raising the acceptance rate at later draft positions where a purely parallel draft would guess blind. Third, a confidence-scheduled verifier predicts each token's survival probability and prunes low-confidence suffixes when verification would cost more than it saves, which prevents the draft from over-proposing on hard passages where acceptance collapses.

The draft models themselves are attention-only, five layers with a block of nine, around 300 million parameters each, small enough that the draft's own inference cost stays negligible against the target's weight streaming, trained for 15 epochs on a data mix covering supervised fine-tuning, chat, code and function-calling data, with the epoch selected by highest acceptance rate rather than lowest loss. That last choice is the small tell of a team optimizing for the actual objective: a draft model's job is to be accepted, not to be a good language model.

The measured speedups, per model and per workload

Liquid AI publishes full tables rather than a single number, on an H100 with SGLang and on an M4 Max MacBook with llama.cpp and Metal, batch size 1, temperature 0, across five datasets, and the per-dataset acceptance rates are printed alongside every speedup, which is how speculative decoding results should always be presented. The per-model picture differs enough to matter for deployment decisions.

LFM2.5-2.6B averages 2.67x on the H100, from 323 to 864 tokens per second, and 2.27x on the MacBook, from 61 to 139, with speculative decoding acceptance between 4.3 and 5.4 of ten draft tokens across the five datasets. Acceptance rates hover between 4.3 and 5.4 of 10 draft tokens. The on-device result is the strategically important one: 139 tokens per second on a laptop pushes interactivity beyond what most proprietary cloud models deliver, and Liquid AI states it plainly. Function-calling latency across multi-tool scenarios drops 57 percent on average for this model, which is the number that matters for agent builders, since tool-calling round-trips are where agentic latency accumulates.

LFM2.5-1.2B-Instruct averages 2.10x on GPU and 2.54x on-device, with acceptance varying by as much as 52 percent depending on the text distribution, a variance the announcement discloses explicitly. The smaller the target model, the less a draft can help on hardware-bound server inference, but the on-device gain is larger because the MacBook's memory bandwidth is the tighter bottleneck.

LFM2.5-8B-A1B, the MoE model, shows the most instructive gap in the release, and it is the case study in why speculative decoding gains are stack-dependent: acceptance rates are the highest of the three, averaging 6.95 of 10, yet the on-device speedup averages only 1.18x. Liquid AI attributes this to the current MoE implementation in llama.cpp's Metal backend and to the fact that verifying multiple tokens activates more experts, and therefore more weight traffic, than a single decode step. The GPU result for the same model averages 2.54x, with 3.18x on MATH500, the release's headline. The lesson is that speculative decoding's gains are architecture-and-stack dependent, and a vendor publishing the case where its own technique underperforms is doing the ecosystem a service.

Choosing where speculative decoding pays

The release's own tables support a deployment rule of thumb that is worth stating explicitly, because speculative decoding is not uniformly beneficial and the conditions where it shines are identifiable in advance.

Speculative decoding pays most when a single stream of tokens is the bottleneck and the content is predictable. Code with common idioms, math with conventional notation, and chat with formulaic phrasing all show acceptance rates near or above five of ten, which translates to the two-to-three-x range. It pays least when content is high-entropy, the model is small enough that weight loading is already fast, or the hardware and runtime combination handles the draft poorly, which is exactly what the 8B MoE on-device results demonstrate. A team can measure the acceptance rate on its own traffic from the per-response timings that both integrations emit, and that measurement, rather than any published table, is the decision input. The workflow is unglamorous and fast: turn the draft on, run a sample of real prompts, read draft_n and draft_n_accepted from the timings, and compute the ratio. If the ratio is comfortably above half your block size, keep it; if not, the draft is costing more than it returns on that workload, and turning it off is a one-flag change.

The second-order effect is on batching. Speculative decoding primarily accelerates latency-bound single-stream generation, which is the local and edge case. Server deployments running high concurrency often find that batched decoding already amortizes weight loading across requests, shrinking the speculative advantage. That is why the H100 measurements here are batch size 1, and why a serving team should benchmark at its real concurrency before assuming the speedup transfers. The technique and batching are not opposed, but they are partially redundant, and which one wins depends on the traffic shape.

For local deployments the speculative decoding calculus is clean, because the traffic is single-stream by nature and the hardware is memory-bound by design, which is exactly the profile the technique was invented for. A MacBook user running LFM2.5-2.6B with a DSpark draft gets 139 tokens per second where 61 stood before, and function-calling agents get a 57 percent latency cut on the round-trips that dominate agent feel. On-device agentic inference has been the goal of the small-model movement all along, and this is the kind of release that moves it from a demos category to a shipping category.

How to run it

Both speculative decoding integrations are upstream, not forked: SGLang takes three speculative flags pointing at the draft model, and llama.cpp takes the draft as a sidecar with a spec-type flag, with block size read from the model metadata. The vLLM-style verification guarantee is stated in the llama.cpp section: speculative decoding is exact, the target verifies every proposed token, greedy output equals the target alone, and per-response timings report draft tokens proposed and accepted, which makes the acceptance rate observable per request in production.

The draft checkpoints ship in Safetensors and GGUF for all three models, so the same speculative decoding acceleration works from a datacenter GPU down to a laptop, and the acceptance telemetry ships with it.

Why this release matters for local inference economics

The broader context is that decoding speed is the binding constraint on local AI, and DSpark-style draft models attack it without the usual costs, which is what distinguishes this release from the quantization-and-prune school of local-model acceleration. A 2.6B model generating at 139 tokens per second on a MacBook crosses the threshold where local inference feels faster than a cloud round-trip, and the 57 percent function-calling latency reduction is the number that makes on-device agentic inference practical rather than demonstrative. An agent that calls three tools in a loop feels the compounding of every round-trip, and speculative decoding attacks exactly the generation half of that loop.

The release also fits the pattern this site has tracked across the autumn: efficiency is arriving through stacked, composable techniques rather than through bigger models. Nemotron ships MTP heads, Kyutai ships corrected RL losses, Liquid ships draft models, and each is a several-percent-to-several-x gain that composes with the others. For a builder running local models, the practical sequence is: pick the model whose quality fits the task, apply exact acceleration like DSpark where the runtime supports it, and only then consider quality-affecting trade-offs like quantization, which is the order that preserves the most quality per unit of speed gained. The economics thread runs through inference caching, model routing and the idle-capacity analysis, and speculative decoding is the local-inference entry in the same ledger.

Where the numbers stop

  • All throughput figures are Liquid AI's own measurements under specific conditions: batch size 1, temperature 0, FP16 GGUF on an M4 Max, BF16 on a single H100, and the named llama.cpp and SGLang builds. Different hardware, quantization, batch sizes or temperatures will produce different numbers, and batched serving in particular changes the calculus because speculative decoding primarily helps latency-bound single-stream generation.
  • The headline 3.18x is the best single cell in the tables, LFM2.5-8B-A1B on MATH500 on H100, not a mean. The honest means are 2.10x to 2.67x on GPU and 1.18x to 2.54x on-device depending on model.
  • Quality parity holds for greedy decoding by construction. Sampling with temperature above zero reintroduces distribution-matching considerations, and the published measurements are all at temperature 0, so parity under production sampling settings should be verified rather than assumed.
  • The 8B-A1B on-device gap is attributed by Liquid AI to llama.cpp's Metal MoE implementation and expert-activation traffic. That is the vendor's diagnosis of its own dependency stack, and it is plausible, but it is not independently profiled; the number will move as llama.cpp improves.
  • The 57 percent function-calling latency reduction is averaged across various multi-tool scenarios for LFM2.5-2.6B; the scenario list and per-scenario numbers are not fully itemized in the release.
  • I have not run the benchmarks. Every figure here is cited from Liquid AI's published tables.

FAQ

What are LFM2.5-DSpark draft models?

They are roughly 300-million-parameter draft models released by Liquid AI for three models in its LFM2.5 family, 1.2B-Instruct, 2.6B and 8B-A1B. Attached to the target model through llama.cpp or SGLang, they enable speculative decoding: the draft proposes candidate tokens, the target verifies them in one forward pass, and the output is identical to what the target alone would produce. The result is up to 3.18x faster decoding on a GPU and up to 2.9x on-device, with no change to output quality under greedy decoding.

Does speculative decoding affect output quality?

No, by construction, under greedy decoding. A draft token is accepted only if it matches the target model's distribution, and on rejection the target's own token takes its place, so the emitted sequence is identical to baseline. This is why the technique is preferable to quantization or distillation when speed is the goal: the speedup arrives without re-running any benchmark. One caveat: the published measurements are all at temperature 0, and parity under sampling settings needs verification in your own stack.

Should I use DSpark on the 8B MoE model locally?

The published numbers say be careful: the 8B-A1B model has the highest acceptance rate of the three, averaging 6.95 of 10 draft tokens accepted, yet only 1.18x average speedup on a MacBook, which Liquid AI attributes to the MoE implementation in llama.cpp's Metal backend and to extra expert activation during verification. On an H100 the same model averages 2.54x. So the draft models deliver clearly on GPU for all three targets, and on-device they deliver for the two dense models while the MoE case waits for runtime improvements.

The short version

Liquid AI's DSpark draft models add exact speculative decoding to three LFM2.5 models: roughly 300M-parameter drafts that make decoding up to 3.18x faster on GPU and 2.9x on-device while leaving the output bit-identical to the target model's own. The honest means are 2.1x to 2.7x, the function-calling latency cut of 57 percent is the number that matters for agents, and the release is unusually candid about where the technique underperforms, the MoE model on-device. For local inference, this is the rare free lunch: same tokens, sooner, with the draft checkpoints and upstream runtime support published for anyone to verify.

Sources
Subscription Form