2.3% · 6.8%
more character error for the 1.7B model at 4-bit, on Prism and on Probe
50% · 57%
less on-disk size at 4-bit, 0.6B and 1.7B (decoder plus audio encoder)

What changes on disk

Only the decoder is quantized. The audio encoder stays in 16-bit in every run, for accuracy. The 16-bit format is BF16 and the 4-bit format is Q4_K_M, both from llama.cpp. You load two files: the decoder and the audio encoder.

Decoder 16-bitDecoder 4-bitAudio encoderTotal 16-bitTotal 4-bit
1.7B3,188 MB1,000 MB642 MB3,830 MB1,642 MB
0.6B1,068 MB342 MB379 MB1,447 MB721 MB
File sizes in MB (1 MB = 1,000,000 bytes). Total is decoder plus audio encoder.

Accuracy

Prism CERPrism WERPrism SERProbe CERProbe WERProbe SER
1.7B, 16-bit decoder6.0512.975.838.5216.644.65
1.7B, 4-bit decoder6.1913.496.139.1018.185.52
0.6B, 16-bit decoder7.9016.338.889.7919.166.75
0.6B, 4-bit decoder7.8716.239.059.7019.366.99
Percent, lower is better. CER and WER use the DarijaCS protocol of the parent page. SER is the meaning-error rate from one judge, GPT-5.6 Luna.

The 1.7B model loses a little at 4-bit. Its character error rises by 2.3% on Prism (6.05 to 6.19) and by 6.8% on Probe (8.52 to 9.10). Word error and meaning error move the same way on both sets.

The 0.6B model does not. Its character error is 7.90 at 16-bit and 7.87 at 4-bit on Prism, and 9.79 and 9.70 on Probe. Word error and meaning error differ by 0.24 points or less, in both directions.

How much of this is noise

Two things limit how far these differences can be read. Probe has only 109 clips, so one or two bad transcripts move its score. And the 16-bit runs were decoded on a CPU while the 4-bit runs were decoded on a GPU. The two do not compute exactly the same numbers: when we decoded the same 4-bit model on both, a few transcripts changed by a word or a punctuation mark. So the small differences of the 0.6B model should not be read as an effect of the precision. For the 1.7B model, every measure moves in the same direction, but we have not measured how much of the gap comes from the precision alone.

Speed

On an RTX 2060, a small desktop GPU, we transcribed the same 16 clips (214 seconds of audio) at both precisions. The 0.6B model runs at 40× real time at 4-bit against 32× at 16-bit. The 1.7B model runs at 27× against 19×.

Which one to use

For the 0.6B model, use 4-bit: on disk it is 721 MB instead of 1447 MB, it is faster, and we measured no clear loss. For the 1.7B model, 4-bit (1642 MB on disk against 3830 MB) is the practical choice on a small GPU or a laptop, and 16-bit is worth the space when accuracy matters most. The headline DarijaCS figures in the article use the 16-bit decoder; the 4-bit rows are in the table on the parent page.