https://blog.fuzzing.uk/posts/ Recent content in Posts on fuzzing.uk Hugo en-us Sat, 29 Aug 2026 00:00:00 +0000 https://blog.fuzzing.uk/posts/run-20b-ternary-moe-in-0-63-gb/ Sat, 29 Aug 2026 00:00:00 +0000 https://blog.fuzzing.uk/posts/run-20b-ternary-moe-in-0-63-gb/ <p>Maple is a 20B-parameter A1B ternary MoE: 24 layers, 256 experts, top-8 routing. Serving it normally costs <strong>~5.9–6.5 GB</strong> of resident memory. With file-backed expert offload and KV-cache quantization, it runs in <strong>0.63 GB</strong> — bit-exact against the full model.</p> <p>Maple comes from the <strong>DeepGrove fork of <code>ml-explore/mlx-lm</code></strong>; the architecture and the 2-bit ternary weights are theirs (<code>deepgrove/maple-2bit-mlx</code> on Hugging Face). This post is about serving that exact model on less RAM — the offload adds no approximation.</p>