1. Install llama.cpp
Use build 9199 or newer. Older builds load the files without any complaint and produce fluent text with wrong words. A CPU-only build is enough:
git clone https://github.com/ggml-org/llama.cpp cd llama.cpp git checkout 39cf5d619 # build 9199, or any newer commit cmake -B build -DCMAKE_BUILD_TYPE=Release cmake --build build -j --target llama-server llama-mtmd-cli ./build/bin/llama-server --version # expect 9199 or newer
2. Download the model
llama.cpp handles speech through its multimodal support, so you need two files: the decoder, fihria-asr-0.6b-Q4_K_M.gguf (4-bit, 342 MB), and the audio encoder, mmproj-fihria-asr-0.6b-BF16.gguf (16-bit, 379 MB). Together that is 721 MB on disk. Keep the audio encoder in 16-bit.
hf download <repo-id> \ fihria-asr-0.6b-Q4_K_M.gguf \ mmproj-fihria-asr-0.6b-BF16.gguf \ --local-dir ./fihria-asr
3. Transcribe a file
Any sample rate works: llama.cpp converts the audio to 16 kHz mono itself. Always pass the text prompt after the audio.
llama-mtmd-cli \ -m fihria-asr-0.6b-Q4_K_M.gguf \ --mmproj mmproj-fihria-asr-0.6b-BF16.gguf \ --audio sample.wav \ -p "Transcribe the speech in the original language." \ --jinja -c 8192 -n 512 --temp 0
To transcribe many files, start a server once, so the model is loaded only once, and send the files to it. The release includes transcribe.py, a client that needs only the Python standard library:
llama-server \ -m fihria-asr-0.6b-Q4_K_M.gguf \ --mmproj mmproj-fihria-asr-0.6b-BF16.gguf \ --no-mmproj-offload -ngl 0 -c 8192 \ --host 127.0.0.1 --port 8080 python3 transcribe.py --audio sample.wav python3 transcribe.py --input-jsonl clips.jsonl --output results.jsonl
4. Read the output
The model writes the detected language, then the transcript, with no space between them: language Darija followed directly by the Arabic text. Some builds insert an <asr_text> tag between the two. This pattern handles both forms:
import re
ASR_RE = re.compile(r"language\s+([A-Za-z]+)\s*(?:<asr_text>)?\s*(.*)", re.DOTALL)
def parse(raw: str) -> tuple[str | None, str]:
m = ASR_RE.search(raw.strip())
return (m.group(1), m.group(2).strip()) if m else (None, raw.strip())Match the language name with [A-Za-z]+ rather than \S+; otherwise the first Arabic word ends up in the language field.
Speed on a laptop
A laptop transcribes one clip at a time and spends most of its decoding time reading weights from memory, so on a laptop the size of the model and the precision of its weights matter as much as the tokenizer. We started from MoulSot as released and changed one thing at a time:
MoulSot · 1.7B · Qwen tokenizer · 16-bit
1.74× real time
Starting point · 1,771 tokens written · 4,059 MB on disk
Smaller model · 0.6B · Qwen tokenizer · 16-bit
×1.83 · 3.18× real time
1.7B → 0.6B · 1,861 tokens written · 1,559 MB on disk
Darija tokenizer · 0.6B · Darija tokenizer · 16-bit
×1.43 · 4.54× real time
Qwen → Darija vocabulary · 1,092 tokens written · 1,447 MB on disk
4-bit weights · 0.6B · Darija tokenizer · 4-bit
×1.32 · 5.98× real time
16-bit → 4-bit · 1,089 tokens written · 721 MB on disk
3.4×
faster than MoulSot in total (1.74× → 5.98× real time), from the smaller size (×1.83), the tokenizer (×1.43) and quantization (×1.32).
Intel Core Ultra 7 258V laptop CPU, 8 threads, llama.cpp, batch 1. Each bar is the total wall-clock time needed to transcribe the same 24 test clips (5 minutes of audio), one clip at a time. Each step changes one thing relative to the row above it. If MoulSot is also converted to 4-bit, it runs at 3.06× real time, and our 0.6B model is still 2.0 times faster.
Going from 1.7B to 0.6B parameters gives 1.83 times, the Darija tokenizer another 1.43 times, and 4-bit weights another 1.32 times—delivering a 3.4× end-to-end speedup on CPU (from 1.74× to 5.98× real time) in a 721 MB footprint.