Skip to content

moonshine_asr.md

Latest commit

 

History

History
109 lines (86 loc) · 4.38 KB

moonshine_asr.md

File metadata and controls

109 lines (86 loc) · 4.38 KB
Raw

Moonshine Streaming ASR

Build with -DENGINE_ENABLE_NEON=ON (default OFF) to enable the native path. On ARM CPU, --session-option 'moonshine_asr.exec_plan={"decoder":{"impl":"neon"}}' selects the complete NEON decoder, including embedding lookup, cross-KV preparation and cached autoregressive steps. The encoder and sampling policy stay unchanged. Omission, an empty plan, or decoder.impl=default/ggml preserves the graph path. Explicit NEON requires an AArch64 CPU session. No artifacts or params are accepted.

Moonshine is an English streaming ASR family. audio.cpp supports the upstream tiny, small, and medium streaming checkpoints through the moonshine_asr family.

Install

The recommended packages are standalone Q8_0 GGUFs in the aggregate audio.cpp GGUF repo:

python3 tools/model_manager_v2.py install moonshine_streaming_tiny_q8_0
python3 tools/model_manager_v2.py install moonshine_streaming_small_q8_0
python3 tools/model_manager_v2.py install moonshine_streaming_medium_q8_0

The GGUFs embed the model spec, configuration, and tokenizer sidecars, so each file can be loaded directly.

Quick Start

Offline transcription:

audiocpp_cli --task asr \
  --family moonshine_asr \
  --model models/Moonshine-Streaming-GGUF/moonshine-streaming-tiny-q8_0.gguf \
  --backend cuda \
  --audio assets/resources/sample_16k.wav \
  --text-out transcript.txt \
  --log

Streaming session with file input:

audiocpp_cli --task asr --mode streaming \
  --family moonshine_asr \
  --model models/Moonshine-Streaming-GGUF/moonshine-streaming-tiny-q8_0.gguf \
  --backend cuda \
  --audio assets/resources/sample_16k.wav \
  --text-out transcript.txt \
  --log

Model

Field Value
Family moonshine_asr
Task asr
Modes offline, streaming
Language English
Recommended package moonshine_streaming_tiny_q8_0
Output Transcript text
Timestamps Not exposed

Common Options (use directly)

Option Values Default Meaning
--audio WAV path required Speech input.
--max-tokens integer audio-duration derived Maximum generated transcript tokens. Defaults to audio-duration derived limit. Nonpositive values select the automatic budget.
--request-option audio_chunk_mode=... auto, fixed, vad, none auto Long-form audio chunking policy.
--request-option audio_chunk_duration_sec=... seconds 60 Maximum chunk duration for auto, fixed, and vad.
--text-out TXT path not set Transcript output.

Session Options (use with --session-option)

Option Values Default Meaning
moonshine_asr.weight_type native, f32, f16 backend/config dependent Shared matmul storage override. When omitted, the CPU encoder with hidden size 620, 10 layers and decoder hidden size 512 uses f16; other encoders use native.
moonshine_asr.conv_weight_type native, f32, f16 native Frontend convolution weight storage override.
moonshine_asr.decoder_weight_type native, f32, f16 explicit shared setting, otherwise native Decoder matmul storage override.
moonshine_asr.encoder_gelu erf, exact, tanh, quick quick Encoder GELU lowering.
moonshine_asr.cpu_blas_scheduler bool true Use BLAS/Accelerate for supported CPU encoder matmuls.
moonshine_asr.weight_context_mb integer MiB >= 1 256 Weight context arena size.
moonshine_asr.graph_arena_mb integer MiB >= 1 512 Graph arena size.
moonshine_asr.vad_model_path path assets/framework/models/silero_vad Silero VAD model path used by audio_chunk_mode=vad.

Conversion

The helper converts the upstream safetensors checkpoints into standalone GGUFs:

python3 tools/community_models/convert_moonshine_asr.py \
  --checkpoint models/moonshine-streaming-tiny \
  --converter build/debug/bin/audiocpp_gguf \
  --output models/Moonshine-Streaming-GGUF/moonshine-streaming-tiny-q8_0.gguf \
  --type q8_0 \
  --overwrite

Use the same command shape for moonshine-streaming-small and moonshine-streaming-medium.

Sources