Qwen3.8-27B on One RTX 3090
Four Qwen3.8-27B configurations measured on the same 24GB RTX 3090: mainline llama.cpp against ik_llama.cpp, unsloth UD-IQ4_XS and UD-Q4_K_M against ubergarm IQ4_KS. Throughput on the club-3090 harness, prompt processing at 10K and 90K, and retrieval through a 106,612-token working context.
The RTX 3090 remains a useful local AI card because 24 GB of VRAM is enough to run models that would otherwise require much newer hardware. But fitting the weights is only the first test. A practical configuration also has to generate quickly, ingest large prompts, leave usable memory headroom and keep working once the context fills.
That is what we tested with Qwen3.8-27B.
Qwen released the 27B model on 14 August 2026. Its official model card lists a native 262,144-token context and training with Multi-Token Prediction (MTP). For production or high-throughput serving, Qwen recommends dedicated engines such as SGLang, vLLM and TokenSpeed. Qwen3.8-27B model card · Qwen3.8 release repository
Two days earlier, club-3090 had retired the relevant single-card Qwen3.6-27B llama.cpp and ik_llama.cpp recommendation paths while consolidating that surface around vLLM. The old measurements still stood, but those paths were no longer maintained defaults. club-3090 retired Qwen3.6-27B single-card profile
So the question was straightforward: if we still want Qwen3.8-27B on one 24 GB RTX 3090 using GGUF, which configuration is worth running?
What we tested
All published results came from the same RTX 3090 with 24,576 MiB of VRAM. The server context was configured to 131,072 tokens, K/V cache was q8_0, and MTP used a maximum draft depth of four.
| Configuration | Engine | Weights |
|---|---|---|
| mainline + UD-IQ4_XS | llama.cpp | unsloth UD-IQ4_XS |
| ik + IQ4_KS | ik_llama.cpp | ubergarm MTP-IQ4_KS |
| ik + UD-IQ4_XS | ik_llama.cpp | unsloth UD-IQ4_XS |
| ik + UD-Q4_K_M | ik_llama.cpp | unsloth UD-Q4_K_M |
The duplicate UD-IQ4_XS row is deliberate. Running the same weights in both engines lets us distinguish an engine difference from a quantisation difference.
For the short-prompt benchmark we used club-3090's harness, which we did not write and did not modify: three warm-ups followed by five measured requests for both narrative and code workloads. Thinking was disabled and the harness used temperature 0.6. The 10K and 90K probes measure prompt processing rather than generation.
Short prompts: IQ4_KS wins decode, mainline wins prefill
| Configuration | Narrative | Code | Prefill 10K | Prefill 90K | Peak VRAM |
|---|---|---|---|---|---|
| mainline + UD-IQ4_XS | 44.73 tok/s | 61.96 tok/s | 876.38 tok/s | 636.30 tok/s | 19,944 MiB |
| ik + IQ4_KS | 45.58 tok/s | 63.97 tok/s | 730.97 tok/s | 507.56 tok/s | 23,052 MiB |
| ik + UD-IQ4_XS | 41.32 tok/s | 56.68 tok/s | 715.61 tok/s | 499.34 tok/s | 20,978 MiB |
| ik + UD-Q4_K_M | 34.08 tok/s | 49.04 tok/s | 724.61 tok/s | 493.58 tok/s | 22,922 MiB |
The short-prompt decode winner is ik + IQ4_KS, ubergarm's IQK-format quant, but the lead over mainline is small: about 1.9% on narrative and 3.2% on code.
Prefill is different. Mainline processes the 10K probe about 20% faster than IQ4_KS and the 90K probe about 25% faster. That matters for code assistants, retrieval systems and agents because their prompts grow as files, tool results and conversation history accumulate.
The memory difference is also substantial. Peak VRAM is 19,944 MiB for mainline against 23,052 MiB for IQ4_KS, a gap of 3,108 MiB, or about 3.0 GiB.
These are the final figures from the shipped configuration. Mainline and IQ4_KS are means of two canonical runs; the other two rows are single canonical runs. Across the measured runs, decode was noisier than prefill, so the small short-prompt gap should be read as small rather than as a universal ranking.
Engine versus quantisation
The matched UD-IQ4_XS rows give the cleanest engine comparison:
| Test | ik / mainline |
|---|---|
| Narrative decode | 0.92x |
| Code decode | 0.91x |
| 10K prefill | 0.82x |
| 90K prefill | 0.78x |
With the same weights and draft depth, mainline was faster in every measured workload.
Inside ik_llama.cpp, however, switching from UD-IQ4_XS to IQ4_KS raised decode by about 1.10x on narrative and 1.13x on code. That explains why IQ4_KS can still top the short-prompt table even though the matched-weight engine comparison favours mainline.
This also helps explain club-3090's earlier Qwen3.6-27B result, where its single-card IQ4_KS recipe was documented at roughly 18 to 20% faster decode than its mainline Q4_K_M path. Our Qwen3.8 test does not reproduce that comparison directly because we did not run Q4_K_M on mainline. What it does show is that the engine alone is not enough to explain the short-prompt IQ4_KS lead: the matched UD-IQ4_XS comparison favours mainline, while changing the quant inside ik materially changes decode speed. club-3090 single-card comparison
The bits-per-weight data points in the same direction. Repeating-layer BPW was 4.140 for UD-IQ4_XS, 4.461 for IQ4_KS and 4.745 for UD-Q4_K_M, yet the middle file was the fastest of the three on ik. Fewer bits therefore did not map directly to higher throughput in this test. We did not isolate which kernel or layout differences caused that result.
At 106K context, the ranking changes
A configured context size is not the same as a tested working context, so we also ran a retrieval test through a 106,612-token prompt.
The deterministic haystack contained six unique codes distributed from early context through the middle, deep context, near the end and at the end. Every run used the same prompt, a cold cache, temperature 0 and reasoning at medium. The generator used a fixed seed, and scoring was exact string retrieval.
| Configuration | Retrieval | Prefill | Decode | Peak VRAM | Wall time |
|---|---|---|---|---|---|
| mainline + UD-IQ4_XS | 6/6 | 613.81 tok/s | 49.06 tok/s | 19,940 MiB | 3m06s |
| ik + IQ4_KS | 6/6 | 503.31 tok/s | 38.26 tok/s | 23,150 MiB | 4m02s |
| ik + UD-IQ4_XS | 6/6 | 484.07 tok/s | 37.36 tok/s | 21,074 MiB | 4m12s |
| ik + UD-Q4_K_M | 6/6 | 482.68 tok/s | 34.16 tok/s | 23,020 MiB | 4m14s |
All four retrieved 6/6, so this test did not separate them on addressability. It did separate them on performance.
With the working context loaded, mainline decoded at 49.06 tok/s against 38.26 tok/s for IQ4_KS, about 28% faster, while also retaining the lower VRAM footprint. The complete request finished in 3m06s instead of 4m02s.
That reversal is the main reason our recommendation is not based on the highest short-prompt TPS number.
Why we stayed at 131K context
Qwen3.8-27B supports a native 262,144-token context, but our benchmark used 131,072 with q8_0 KV. That was a choice about serving quality, not a model limit.
club-3090 has separately demonstrated that the same single-card UD-IQ4_XS configuration can boot at the full 262,144 tokens by dropping K/V cache to q4_0. Its own profile labels that path experimental and below its serving KV floor; the q8_0 serving-grade override is 131,072 tokens. club-3090 q4_0 262K single-card profile
That is the trade-off in practical terms: q4_0 KV buys the full 262K window on one 3090; q8_0 KV keeps the higher-fidelity cache and tops out at 131K on this single-card setup. For this benchmark, and for our production default, we stay on q8_0. Our validated working-context result is therefore the narrower claim we actually exercised: 106,612 prompt tokens, 6/6 retrieval.
MTP check
MTP acceptance was 70.0% for IQ4_KS, 68.9% for UD-IQ4_XS and 68.6% for UD-Q4_K_M across the ik runs. Those values are too close to explain the larger throughput spread, so MTP acceptance is not the main differentiator here.
What should you run?
General-purpose single-card default: mainline llama.cpp + UD-IQ4_XS. It gives up only a few percent on short-prompt decode, while winning prompt processing, VRAM headroom and the 106K working-context performance test.
Short-context decode first: ik_llama.cpp + ubergarm IQ4_KS. It produced the highest narrative and code decode figures in the short-prompt benchmark, but with about 3.0 GiB more peak VRAM and slower large-context processing.
ik + UD-IQ4_XS: useful if you specifically need ik_llama.cpp features, but there is no throughput case for it in these measurements because the same file ran faster on mainline.
ik + UD-Q4_K_M: the slowest decode configuration we tested. Its prefill was close to the other ik rows, so this is not a claim that it lost every metric. It is also a throughput result, not a quality judgement.
What this benchmark does not answer
We did not measure quantisation fidelity. There is no perplexity, KL divergence or controlled behavioural evaluation here. The short benchmark follows the harness sampler used for these measured throughput runs; it is not a quality evaluation. The files span about 0.6 BPW across their repeating layers, so a slower file could still preserve model behaviour better. A quality comparison needs a separate experiment.
This is also not a vLLM versus llama.cpp benchmark. Qwen recommends vLLM among its production serving engines, and club-3090 discussion #1037 measured Qwen3.8-27B on two RTX 3090s at 84.56 tok/s narrative and 109.11 tok/s code with vLLM INT8, versus 44.22 and 52.91 tok/s for Q8 llama.cpp. That is useful context, but it uses different hardware and precision, so we do not project it onto this one-card GGUF test. club-3090 discussion #1037
Reproducibility and deployment notes
The final measurements used one RTX 3090 with 24,576 MiB VRAM, NVIDIA driver 610.57.04, 131,072-token context, q8_0 K/V cache and MTP draft depth four. The tested serving images were:
- mainline:
ghcr.io/mostlygeek/llama-swap:unified-cudaatsha256:5cb8cc8221b802a62dbb051d3534d4bd0e88f38fcef2c93d04e7733740fd978a - ik:
ghcr.io/ikawrakow/ik-llama-cpp:cu13-swap-4819atsha256:aab468c5e4976109966df4a4e4af1d5269688017459d1852db9f3b0c5f3bfc44
Two revisions of the harness were in use across the session, and the two differ in what they capture: the later one adds the cache-busted prefill probes, the PCIe and VRAM captures and the acceptance capture. Every figure published here came from the later revision. Both are hosted verbatim in the Configs library under club-3090's Apache 2.0 licence, with digests, so a given figure can be checked against the exact code that produced it. For the maintained harness, use club-3090 rather than our copies.
The weight files were SHA256-pinned in the test record, and all four configurations are published as launcher scripts in the Configs library so the measured setups can be reproduced without translating prose back into flags. For agentic use we used the corrected Qwen Fixed Chat Templates. A current llama.cpp build is recommended; release b10434 added reasoning_effort passthrough to chat templates. We do not claim an exact upstream llama.cpp revision for the benchmark container because its exposed version uses the distributor's build scheme.
Configured capacity and validated working context are separate claims: the server was configured for 131,072 tokens, while our long-context test actually exercised 106,612 prompt tokens and retrieved all six planted values.
Coming next: the largest quant that fits
A note on where this series goes, not a fifth result. Everyone wants to run the biggest file their card will take, and so do we. On 24GB you have to be realistic about what that means.
We have since run unsloth UD-Q4_K_XL on ik_llama.cpp through the same 106,612-token prompt at 131,072 configured context and q8_0 KV, on 2 September 2026. It completes: 6/6 codes, 486.28 tok/s prefill, 31.78 tok/s decode, cold cache confirmed. It also peaks at 24,068 MiB of the card's 24,576, which leaves 508 MiB free, and it is the slowest decode we have measured on this model. Roughly two percent of the card spare is a very different proposition from the headroom any configuration above leaves, and it is the sharpest illustration we have of what "it fits at 131K" is actually claiming: one measured peak, on one prompt.
That deserves its own article rather than a row in this one, because the question it raises is operational, not comparative: how you run a file that tight day to day, what it costs you, and what the alternatives are for going higher on quality still, including vLLM. That is the next piece in this series. The comparison above is unaffected by it and stands as measured.
Sources
- Qwen3.8-27B model card: architecture, native context, MTP, recommended serving engines and sampling parameters.
- Qwen3.8 official repository: 14 August 2026 release date for Qwen3.8-27B.
- club-3090: benchmark methodology and historical single-card configurations.
- club-3090 Qwen3.8 q4_0 262K single-card profile: full-context q4_0 fit, q8_0 131K serving-grade override and caveats.
- club-3090 Qwen3.6-27B retirement record: 12 August 2026 retirement and scope.
- club-3090 discussion #1037: dual-3090 Qwen3.8 vLLM versus llama.cpp comparison.
- unsloth Qwen3.8-27B GGUF and ubergarm Qwen3.8-27B GGUF: tested quantisation artefacts.
- froggeric Qwen Fixed Chat Templates: Qwen3.8 tool-call and reasoning template fixes.
- llama.cpp b10434:
reasoning_effortsupport in chat templates.
All benchmark figures, BPW values, VRAM measurements, image digests and file hashes in this article are from the 30 August 2026 test record included with the pack. The UD-Q4_K_XL figures in the closing note are from a separate run on 2 September 2026 and are not part of the comparison.
Something wrong here? Corrections are welcome at hello@devfactory.mu and will be noted on the article.