1. Install llama.cpp

Use build 9199 or newer. Older builds load the files without any complaint and produce fluent text with wrong words. A CPU-only build is enough:

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
git checkout 39cf5d619   # build 9199, or any newer commit
cmake -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build -j --target llama-server llama-mtmd-cli
./build/bin/llama-server --version   # expect 9199 or newer

2. Download the model

llama.cpp handles speech through its multimodal support, so you need two files: the decoder, fihria-asr-0.6b-Q4_K_M.gguf (4-bit, 342 MB), and the audio encoder, mmproj-fihria-asr-0.6b-BF16.gguf (16-bit, 379 MB). Together that is 721 MB on disk. Keep the audio encoder in 16-bit.

hf download <repo-id> \
  fihria-asr-0.6b-Q4_K_M.gguf \
  mmproj-fihria-asr-0.6b-BF16.gguf \
  --local-dir ./fihria-asr

3. Transcribe a file

Any sample rate works: llama.cpp converts the audio to 16 kHz mono itself. Always pass the text prompt after the audio.

llama-mtmd-cli \
  -m fihria-asr-0.6b-Q4_K_M.gguf \
  --mmproj mmproj-fihria-asr-0.6b-BF16.gguf \
  --audio sample.wav \
  -p "Transcribe the speech in the original language." \
  --jinja -c 8192 -n 512 --temp 0

To transcribe many files, start a server once, so the model is loaded only once, and send the files to it. The release includes transcribe.py, a client that needs only the Python standard library:

llama-server \
  -m fihria-asr-0.6b-Q4_K_M.gguf \
  --mmproj mmproj-fihria-asr-0.6b-BF16.gguf \
  --no-mmproj-offload -ngl 0 -c 8192 \
  --host 127.0.0.1 --port 8080

python3 transcribe.py --audio sample.wav
python3 transcribe.py --input-jsonl clips.jsonl --output results.jsonl

4. Read the output

The model writes the detected language, then the transcript, with no space between them: language Darija followed directly by the Arabic text. Some builds insert an <asr_text> tag between the two. This pattern handles both forms:

import re
ASR_RE = re.compile(r"language\s+([A-Za-z]+)\s*(?:<asr_text>)?\s*(.*)", re.DOTALL)

def parse(raw: str) -> tuple[str | None, str]:
    m = ASR_RE.search(raw.strip())
    return (m.group(1), m.group(2).strip()) if m else (None, raw.strip())

Match the language name with [A-Za-z]+ rather than \S+; otherwise the first Arabic word ends up in the language field.

Speed on a laptop

A laptop transcribes one clip at a time and spends most of its decoding time reading weights from memory, so on a laptop the size of the model and the precision of its weights matter as much as the tokenizer. We started from MoulSot as released and changed one thing at a time:

Three independent speed-ups, measured on a laptop

MoulSot · 1.7B · Qwen tokenizer · 16-bit

1.74× real time

183.0 s

Starting point · 1,771 tokens written · 4,059 MB on disk

Smaller model · 0.6B · Qwen tokenizer · 16-bit

×1.83 · 3.18× real time

100.3 s

1.7B → 0.6B · 1,861 tokens written · 1,559 MB on disk

Darija tokenizer · 0.6B · Darija tokenizer · 16-bit

×1.43 · 4.54× real time

70.3 s

Qwen → Darija vocabulary · 1,092 tokens written · 1,447 MB on disk

4-bit weights · 0.6B · Darija tokenizer · 4-bit

×1.32 · 5.98× real time

53.3 s

16-bit → 4-bit · 1,089 tokens written · 721 MB on disk

3.4×

faster than MoulSot in total (1.74× → 5.98× real time), from the smaller size (×1.83), the tokenizer (×1.43) and quantization (×1.32).

Intel Core Ultra 7 258V laptop CPU, 8 threads, llama.cpp, batch 1. Each bar is the total wall-clock time needed to transcribe the same 24 test clips (5 minutes of audio), one clip at a time. Each step changes one thing relative to the row above it. If MoulSot is also converted to 4-bit, it runs at 3.06× real time, and our 0.6B model is still 2.0 times faster.

Going from 1.7B to 0.6B parameters gives 1.83 times, the Darija tokenizer another 1.43 times, and 4-bit weights another 1.32 times—delivering a 3.4× end-to-end speedup on CPU (from 1.74× to 5.98× real time) in a 721 MB footprint.