As checked on 2026-08-29. No general 20% beta approval; no all-language accuracy or deployed-performance parity claim.
The verified opt-in Rust worker is 230df53c852ef7104dfdce78d5877a1c13fd108c29114ca44f54eb4b0011076a. Its exact evaluated source is now integrated: 26 MMS recognition/alignment routes, including Haitian, acoustic word grouping for Khmer/Lao/Burmese, and guarded primary/secondary language detection. It preserves 1,158 retained native outputs, including Cap. Explicit language requests bypass detection. Serving defaults and deployment are unchanged; an explicit MMS asset directory must now contain all 26 pinned adapters.
98/99 base languages have some reference benchmark evidence, across different model versions and uneven samples. That is not 98 quality passes or a current-binary rerun of every language. Turkmen remains unmeasured. The earlier 60-clip Haitian Creole, Latin and Sanskrit baseline comparison completes 119/120 local versus 120/120 provider jobs, but exposes substantial remaining errors: Latin is numerically worse, and Sanskrit automatic WER is 116.83% versus 88.94%, with only 1/20 versus 7/20 correct language labels. Haitian has since improved through two-source development and a small unused validation cohort below. Faroese, Hindi and Nynorsk also retain measured gaps. The four regional aliases lack dedicated accent/dialect evidence.
The evaluated 997048bf… worker adds sixteen native recognizers with acoustic word grouping and guarded language detection. On 320 new development recordings, it completes 640/640 local API jobs across explicit and automatic modes. All 640 match verified native outputs and have one inference attempt plus one exact-duration usage event. Its explicit predecessor improved completion from 230/320 to 320/320; the baseline's 90 failed jobs remain recorded and unbilled.
Sandchest has lower primary text error than AssemblyAI in all 16 languages in both modes on this corpus. Automatic labels are correct on 312/320 versus 200/320. Lao remains 13/20 and Uzbek 19/20; these are not all-language quality passes. AssemblyAI used Universal-2 for all explicit responses, and 295 Universal-2 / 25 Universal-3.5 Pro for automatic responses.
Local/remote API p50 is 0.529s/3.189s explicit and 0.526s/3.575s automatic. Automatic p95 is 1.026s/6.115s. These compare local M4 Max with a remote service, exclude startup and real remoteDB/billing, and do not establish production load or tail performance.
The preceding sixteen-language B5 worker preserves 518 native regression checks, including 50 Cap recordings and an unrestricted-hint replay of a case that caused a broader policy to be rejected. Rust suites pass 122 base / 147 Whisper / 171 CTC / 228 MMS tests. Another 188 API controls and same-ID retries pass, including 50 exact Cap responses, 161 uniquely billed completions and 27 expected unbilled errors. Two real one-second deadlines recover exactly into automatic Pashto. The longer-audio run completes all 256 API jobs on 64 derived controls, with lower primary text error in all 16 languages in both modes. These reuse the 320 development recordings; they are not new independent samples. A coarse pause audit flags 12 local word intervals spanning an entire inserted 200ms silence: six MMS results on three recordings (explicit/auto duplicates) and six Whisper fallback intervals on two recordings. No local word lies entirely inside an inserted pause and no local interval has invalid bounds, but the crossings remain under investigation. Its exact evaluated source is now integrated as an experimental opt-in; no general beta is approved. Ten diagnostic replays reproduce the affected API outputs exactly; the MMS intervals already occur before forced alignment. A stricter VAD partition proposal does not resolve the issue and was not adopted. The unchanged candidate has now completed a fresh 320-recording FLEURS test-split validation: 638/640 local and 640/640 provider jobs, with lower primary error for all 16 languages in both modes. Auto labels are 309/320 versus 211/320. Two local automatic requests (Burmese and Uzbek) fail explicitly and remain in accuracy denominators, unbilled. Local completed-job p50 is 0.651s explicit / 0.525s auto, versus 3.264s / 3.665s remotely. No validation result was used to tune the candidate; absolute errors and the longer-audio timing warnings remain open.
Multilingual punctuation/number formatting, fillers, code switching and human-annotated timestamp accuracy remain gaps. Three new development cohorts are ready: 20 Latin clips across 20 publisher speaker IDs, 20 Haitian clips sharing one speaker ID, and 20 Sanskrit recitation clips with unknown speakers. Their 240-job paired explicit/automatic API comparison is complete, with 239 completions and one unbilled local Sanskrit error. This adds three measured languages, with the quality limits above. Fresh isolated checks pass 194 application tests with 7 conditional model skips, whole-app typecheck/lint, 56 collector tests and five detector-preparation tests. The evaluation suite includes 27 tests covering the reusable timing audit. Native source hashes now match the tested 230-MMS-test Haitian binary; the preceding sixteen-language build had 228 MMS tests. Turkmen reference access remains open; the official Common Voice download requires authenticated access not configured locally.
See the frozen validation table and new language baselines. Repeated native/API requests do not add independent recordings.
The initial numerically verified native MMS recognition screen reduced Haitian WER from 63.99% to 26.57%, versus AssemblyAI 65.03%, on the same 20 development clips from one publisher speaker ID. That initial text-only screen led to the integrated recognition/alignment/automatic route and API studies below; manual word-time accuracy is still unverified. The Latin alternative regresses (72.58% versus current 69.23% and provider 65.55%) and is rejected. A full-Whisper follow-up now completes 78/80 local API jobs on the same Latin/Sanskrit recordings. It matches provider explicit Latin WER (65.55%) and improves explicit Sanskrit CER, but automatic Sanskrit worsens, one additional job fails, and explicit Sanskrit p99 rises to 18.384s. The global replacement is rejected; Latin and Sanskrit serving paths stay unchanged. A separate Latin macron diagnostic preserves the original primary scores.
The new Haitian candidate completes 80/80 full API jobs on 40 development recordings from two sources. On the broader CMU sample (20 clips, 14 conservative filename groups), explicit/auto WER is 13.97%, versus prior B5 48.88% and AssemblyAI 53.87% / 76.31%. On the original one-speaker-ID sample it is 26.57% / 36.01%, versus provider 65.03% / 90.91%. The new route uses two-detector agreement and preserves caller language restrictions. These gains are measured development evidence, not general Haitian or all-language superiority.
Candidate API requests each complete once with one exact-duration usage event; no word/encoding structure defects occur. CMU median latency is 0.519s explicit / 0.774s auto on the local M4, versus remote 2.376s / 2.878s. The automatic p95 is 1.036s versus current B5 0.788s, so there is a tail-cost tradeoff. On 10 unused validation clips from five different filename groups, explicit/auto WER is 17.03%, versus provider 56.04% / 84.62%; auto labels are 10/10 versus 3/10. All 40 paired validation jobs complete. The exact candidate is now integrated after 230 MMS tests, 1,158 retained native comparisons, 198 logical option checks/replays and two real deadline recoveries. The option run retains one extra completed crop probe: 199 actual jobs, 171 exact usage events and 28 expected unbilled errors. No deployment or broad beta approval follows. See the two-source comparison for denominators and source limitations.
- A fixed 100-recording, 20-language development screen tested native MMS candidates against the current worker and cached AssemblyAI on identical audio. MMS lowered primary error in 19/20 groups versus the worker and 17/20 versus AssemblyAI. Most groups contain only two clips; these are candidate-selection results, not parity passes. All 21 numerical probes pass.
- On 60 Hindi development clips, raw MMS improves WER from 32.52% to 22.00%, still behind AssemblyAI's 19.42%. Protected holdout/validation data were not used. No new candidate is promoted.
- Faroese now has 20 clips across 13 conservative speaker groups. The real paired API run completes 39/40 local and 40/40 provider jobs. Local failures are unbilled, successful jobs have exactly one usage event, and both providers get 0/20 automatic language labels right. Current explicit WER is 96.35% versus 92.71% AssemblyAI.
- A dedicated Faroese MMS prototype improves raw explicit WER to 51.56%, but remains unintegrated, with high absolute error and no verified automatic route or word timestamps.
- The new Faroese collector has seven focused tests. Fresh isolated checks pass 189 app tests, typecheck, lint, 36 collector tests and five detector-preparation tests. Serving source and defaults remain unchanged.
See the full candidate table and Faroese API results. Word-timestamp accuracy, larger language samples, automatic routing and Cap validation remain separate gates; these results do not approve an all-language 20% beta.
Forty unused ML-SUPERB recordings were collected only after freezing the final candidate: 20 Bashkir and 20 Tatar, with source, normalized development-text, audio and canonical-PCM exclusions. They are short 4.1–9.8 second public read-speech clips, publisher dev data used as local validation. Speaker identities and model-training overlap are unknown; there is no manual word-time gold.
Lower WER is better; rates above 100% include insertions.
| Language / mode | Sandchest WER | AssemblyAI WER | Correct auto labels: local / provider |
|---|---|---|---|
| Bashkir explicit | 10.14% (14/138) | 98.55% (136/138) | — |
| Bashkir automatic | 32.61% (45/138) | 104.35% (144/138) | 14/20 · 0/20 |
| Tatar explicit | 24.03% (31/129) | 96.90% (125/129) | — |
| Tatar automatic | 46.51% (60/129) | 106.98% (138/129) | 9/20 · 0/20 |
Both providers complete 80/80 jobs. Each local job has one attempt and one exact-duration usage event. All 23 local auto results using MMS have the correct source language; the 17 remaining Whisper fallbacks have incorrect labels. Local word/encoding structure defects are zero, versus one provider defect. These are measured gains on this corpus, not general language superiority or an all-language beta pass.
Full API medians are 0.529–0.530s explicit / 0.773–0.774s automatic locally, versus 2.127–2.189s / 2.473–2.540s remotely. Timing includes upload, queue, recognition/alignment, persistence and polling; excludes startup; and compares Apple M4 Max hardware with remote AssemblyAI. It is not matched hardware, load, cloud capacity or production-tail evidence.
On the reused 20-clip development sets, Bashkir automatic WER improves from 90.35% to 43.86%, and Tatar from 106.72% to 55.22%. Their explicit recognition remains 18.42%/25.37%. A more aggressive detector policy produced better development text but four wrong language labels; the conservative policy was chosen before validation.
- The first integrated build passes 59 disabled-feature checks and 517 enabled native checks: 469 retained cases, including 50 Cap recordings and 80 Malagasy/Albanian variants, plus 48 target automatic cases.
- 192/192 development API jobs complete in one attempt with 192 unique duration-matched usage events. All 144 explicit or Breton/Nynorsk auto responses match the prior API, and 48 target auto responses match the verified native outputs.
- The final binary changes only timing diagnostics from that first build. It passes 132 API cases and 132 same-ID replays, including 50 exact Cap responses, hints, fallback, thresholds, crops, spelling, formatting and stereo. The logical cohort has 112 completions and 20 expected unbilled errors. One extra completed stereo probe from a stopped assertion is separately retained; the actual 133 jobs reconcile to 113 unique completed usage events and 20 unbilled errors.
- Two real one-second deadlines recover exactly into automatic Tatar. Combined timing includes both detectors' cost.
- Final native suites pass 122 base, 147 Whisper, 171 CTC and 215 MMS tests. All 29 offline collector tests pass. Fresh isolated app checks pass 189 tests with 7 conditional skips, plus typecheck, lint, 29 collector tests and 5 detector-preparation tests. The source snapshot matches the promoted code.
- Six CPU-reference/native detector probes agree on the top label; maximum logit difference is 0.00003815. Factoring the shared encoder reproduces all six earlier native probe tensors exactly. This validates implementation math, not independent language quality.
The reusable harness freezes data/code/options, separates development from validation/holdout, preserves failures and proposes unused development audio. The ML-SUPERB collector now supports explicit validation/holdout roles while retaining the pinned publisher dev split and duplicate exclusions. No model or policy was tuned using the new validation results.
Full WER/CER denominators, latency, two-build provenance and remaining gaps are in the latest multilingual report. Earlier Malagasy development/validation and other language studies retain their own provenance there. No shared database, Cap source/data, environment or deployment changed.
The earlier 4b863621… candidate completed 400 fresh paired API jobs on 100 previously unused FLEURS test recordings in Amharic, Assamese, Gujarati, Tamil and Telugu. Both providers completed 200/200 jobs; Sandchest auto labels were 97/100 versus 60/100. Explicit WER was 27.20–38.75%, lower on each group, with wrong-script provider output explaining much of the gap outside Tamil. These validation results keep their original provenance; the new worker preserves the five paths but this is not another validation rerun. See the retained validation table.
The original 82-language screen remains a historical baseline: 904 API tasks on 276 recordings, with 357/452 local versus 451/452 provider completions. It deliberately targets weaknesses and must not be presented as the current candidate's production failure rate. Earlier language/Cap/timestamp studies below also retain their original corpus and run provenance.
The earlier broad-screen baseline used Whisper large-v3-turbo F16, Parakeet v2 for detected English, English CTC alignment, and the retained Spanish MMS timing envelope. Its native binary SHA-256: 6b6a355d3614c393004bbe53d3c1f3ca122fb478e3004f95a2a74dc0802cf12a.
This is an opt-in local candidate, not a deployed service. The bare native engine default is still Parakeet; the multilingual path requires SANDCHEST_INFERENCE_ENGINE=whisper, English routing and alignment configuration. The 99-language inventory must not be attributed to the bare default.
All serving inference discussed here is Rust/native. Python handles corpus collection, orchestration, scoring and isolated numerical reference checks; it is not the serving runtime. Measurements are from an Apple M4 Max with 128 GiB RAM, compared where specified with remote AssemblyAI. There is no matched cloud-hardware/load comparison.
WER is word error rate: lower is better. Each corpus is scored against reference text, not against the other provider. These are the latest retained results for the current model paths, not a claim that every corpus was rerun on the latest binary. The initial detector preserved 286 native off/on outputs; the secondary-detector integration checks 469 retained outputs and 48 target cases, with final API validation described above; the quality scores below retain their original corpus and run provenance. Repeated requests do not increase the independent sample count.
| Language/domain | Sample | Sandchest WER | AssemblyAI WER | Evidence freshness |
|---|---|---|---|---|
| English — meetings | 64 AMI development clips | 11.48% | 18.16% | Retained Parakeet path; earlier full-native run |
| Spanish | 48 reused validation clips / 494 words | 2.83% | 1.82% | Retained MMS envelope; earlier16-clip replay and latest4 explicit/1 automatic regression |
| Hindi | 100 FLEURS clips: 60 development + 40 prior holdout | 32.52% | 20.73% | Current 60-clip development replay reproduces 32.52%; prior holdout not rerun |
| Indonesian | 20 FLEURS development clips | 5.41% | 5.41% | Earlier retained Turbo path; explicit and auto tested |
| Finnish | 20 FLEURS development clips | 8.98% | 12.07% | Earlier retained Turbo path; explicit and auto tested |
| French | One 10-second diagnostic / 45 reference words | 35.56% | 28.89% | Latest exact worker; far too small for general French quality |
Additional retained results:
- Spanish development16: 6/175 errors (3.43%) versus 5/175 (2.86%). All64 development+validation clips:20/669 (2.99%) versus14/669 (2.09%). Validation speakers differ from development, but this is a previously used corpus, not newly unseen data.
- Hindi latest development60:479/1,473 (32.52%) versus286/1,473 (19.42%). Earlier holdout40:310/953 (32.53%) versus217/953 (22.77%). The100-clip aggregate above is not a new100-clip replay.
- English LibriSpeech development64: current retained path22/1,276 (1.72%); AssemblyAI was not compared in that run.
- Older English LibriSpeech128 test: earlier Parakeet pipeline63/2,492 (2.53%) versus56/2,492 (2.25%), using English-normalized text. This is historical pipeline evidence, not a latest-worker replay or the same normalization as the lexical AMI score.
- Earlier notes record English AMI128 holdout7.19% versus12.18%, with126/128 native and128/128 provider completions. Its original RAM-backed runtime directory is currently unavailable, so it is not treated as a freshly reverified current score here.
No defensible overall99-language WER can be calculated from these six languages. Finnish/Indonesian have small samples; the French row is a diagnostic. English meeting gains do not establish superiority on every English domain. The separate50-recording Cap validation set remains uncollected.
| Reference set and common matched words | Sandchest | AssemblyAI |
|---|---|---|
| Spanish validation:481 common words — mean start error | 21.32 ms | 56.84 ms |
| Spanish validation — mean end error | 24.47 ms | 57.03 ms |
| Spanish validation — p95 start / end error | 57.94 / 76.12 ms | 128.30 / 134.97 ms |
| Spanish validation — both endpoints within80ms | 453/481 (94.18%) | 300/481 (62.37%) |
| Latest Spanish development:169 common words — pooled mean boundary error | 22.83 ms | 53.14 ms |
| Latest French diagnostic:26 common words — pooled mean boundary error | 67.58 ms | 71.97 ms |
Spanish is a measured timestamp improvement on this corpus, while recognition still has more word errors. The French sample is too small to close its timing gate. No general manually verified timestamp accuracy exists for the remaining languages. English AMI forced-alignment timestamps and provider timestamps are not human timing gold. Matching/coverage counts are retained; missing or wrongly recognized words are not silently treated as accurate timestamps.
The latest fresh paired provider comparison used50 new Cap recordings from48 owners, totaling2.0003hours, with exact shared MP3 bytes, the original Cap language hints, serial alternating provider order and250ms polling. Local timing included TCP upload, API, durable queue, native inference, PGlite persistence and polling; model startup and real remoteDB/billing roundtrips were excluded. This run predates the last multichannel/no-speech changes.
| Same48 mutually completed pairs | Local Sandchest | Remote AssemblyAI |
|---|---|---|
| p50 | 1.564 s | 6.614 s |
| p95 | 8.048 s | 15.192 s |
| p99 | 19.550 s | 22.391 s |
Current Malagasy-route local API replay, reusing the 50 recordings: 45 speech completions, p50 1.645s, p95 8.170s, p99 16.688s. The preceding three-language replay measured 1.679/8.256/17.884s; the detector-only replay measured 1.667/8.373/18.324s. All50 response contracts match the prior run, including five no-spoken-language errors. Two of those match the provider errors; three differ from the provider's empty completed results. The preceding local recheck measured1.552/18.273/38.890s; variation between runs is substantial. No fresh remote latency was measured in these local replays.
Neither the older8.048-second paired p95 nor the current 8.170-second replay establishes a stable production tail. A separate24-job old/new ABBA diagnostic triggered its regression screen, with a1.232 median per-case ratio on four selected long recordings. Changing background activity and large within-binary variation were observed, but the cause is unresolved. Do not claim stable cloud latency or throughput from these numbers.
Small groups are diagnostics, not per-language service-level estimates. Labels are not verified spoken-language ground truth; the Hindi-labelled cases are diagnosed Indonesian hint conflicts, and the Russian-labelled group includes Ukrainian.
| Provider label | Recordings | Mutually completed pairs | Local p50 | Remote p50 |
|---|---|---|---|---|
| de | 2 | 2 | 6.733 s | 7.730 s |
| en | 35 | 35 | 1.541 s | 6.115 s |
| es | 1 | 1 | 2.589 s | 6.193 s |
| fr | 3 | 3 | 5.986 s | 7.312 s |
| hi | 2 | 2 | 2.875 s | 18.791 s |
| pt | 1 | 1 | 1.313 s | 5.432 s |
| ru | 3 | 3 | 6.474 s | 8.996 s |
| unknown | 2 | 0 | — | — |
| zh | 1 | 1 | 0.533 s | 2.614 s |
Native-only retained medians: Indonesian421ms and Finnish481ms on20 clips each; Hindi786ms on100 clips including word timing. The corresponding Hindi provider median4.688s includes upload/network/queue/polling, so this is not a native-versus-native speed comparison.
The50-recording fresh sample has no independent human transcript references. These are edit differences against AssemblyAI, not Sandchest WER or proof that either provider is correct. The latest API replay retained all45 nonempty transcripts exactly.
| Provider label | Recordings | Lexical disagreement | Interpretation |
|---|---|---|---|
| en | 35 | 8.09% | Small sample; no human reference |
| de | 2 | 3.80% | Small sample; no human reference |
| fr | 3 | 23.54% | Small sample; no human reference |
| es | 1 | 22.58% | Small sample; no human reference |
| pt | 1 | 7.50% | Small sample; no human reference |
| ru | 3 | 62.64% | Includes a Ukrainian hint conflict; not a pure Russian group |
| hi | 2 | 102.78% | Two actual Indonesian cases under restrictive hints; not a Hindi accuracy score |
| zh | 1 | 68.75% characters | 22/32 normalized characters; word segmentation makes whitespace WER unsuitable |
Language labels agreed on42/45 nonempty original Cap requests. Opening the hint list or explicitly selecting the diagnosed language made both providers agree on Indonesian/Indonesian/Ukrainian for all six follow-up pairs. Their lexical disagreements were8.46%,10.16%,35.06%. Fallback selection is still not fully compatible.
Across13,845 words paired with AssemblyAI in the original50-clip sample, average start/end differences were118.7/115.1ms and p95 differences353/281ms. These are provider-boundary differences, not human timing error. English alone averaged59/91ms with p95 differences167/234ms.
- Retained Cap/control API regression:70 jobs,58 completions and12 expected no-spoken-language errors;58 unique usage events, no failed-job usage,70 exact idempotent replays, one inference attempt per job. This reused50 Cap recordings plus controls; it is not a production success-rate estimate.
- Latest Q8 comparison:206 native HTTP requests,198 successful responses and8 expected invalid-input rejections, all followed by exact recovery. Q8 was rejected rather than promoted. A separate9-request timing profile preserved exact words/timestamps/confidence/language/model.
- Current app checkpoint: 189 passing tests, 7 conditional skips, 2,354 assertions; typecheck and lint rerun successfully from a fresh isolated source snapshot. The current 28-test corpus-collector suite passes, including source/text/audio deduplication, protected-reference non-access, parent-path rejection, pinned asset URLs and transfer limits. Latest native suites: 122 base,147 Whisper,171 CTC and 211 MMS. Five additional offline model-preparation tests pass, and conversion reproduces the exact85,221,060-byte tested artifact. SDK build evidence remains the earlier checkpoint.
- Implemented/tested locally:upload, asynchronous transcription/polling, word metadata, explicit/automatic language options, crop offsets, custom spelling, punctuation, English filler control, multichannel, sentences/paragraphs, word search, SRT/VTT, listing/deletion, official SDK compatibility, durable webhooks, tenant isolation and idempotent billing.
- Still not1:1:diarization, prompting/keyterms/boosting, localization/code switching, redaction, some reflected/default/error semantics, multilingual filler behavior, request limits and unverified edge cases. Physical channel separation is not speaker diarization.
- Three Cap no-speech status mismatches remain. Some earlier datasets also contain real uncertainty failures; a green control suite does not erase them.
- No representative deployed concurrency/capacity, cold-start distribution, cloud peak-memory figures, compute cost per audio hour, sustainedp95/p99, or complete Cap fragmented-live workflow proof. These remain unmeasured/unverified rather than assumed passes.
- Qwen Hindi:15.50% reference WER versus20.73% AssemblyAI on the prior100-clip comparison. Promising recognition, but not the integrated serving route and not proof of complete timestamp/API/long-recording parity.
- Qwen Finnish:28.79% versus current Turbo8.98%; this rules out treating Qwen as a universal replacement.
- Q8 Turbo:31.64% Hindi development WER versus current32.52%, but1.256 times the native HTTP latency on the two long Whisper cases; not adopted.
- Encoder-only fused attention:built and185 native tests pass. No real-inference benchmark has run yet; the evaluation driver is prepared but has not run. It is not used in any current result above.
- Full Whisper large-v3, Cohere, Omni, CUPE and alternative alignment/selection pilots remain unpromoted; their best isolated scores are not Sandchest service performance.
AssemblyAI currently documents18 Universal-3.5 Pro languages and99 Universal-2 languages; the combined request can fall back across them. Sandchest's opt-in native parser includes99 base codes plus en_au/en_uk/en_us/de_ch regional aliases. This is inventory/contract support, not measured quality, accent or dialect parity. AssemblyAI supported languages.
“Reference evidence” below means reference text independent of the compared provider outputs, not proof of training-data or model-selection independence. It includes public development screens with two, four, 20 or 22 recordings per language, plus the separate five-language fresh validation; explicit and auto sample counts differ. Use the fresh candidate table separately from the serving baseline; use the linked language report for numerical errors and completion gates. Earlier six-language studies retain their original freshness caveats. “Unmeasured” means no reference benchmark is retained, not that the model necessarily cannot transcribe the language. Cap labels are provider comparisons, not verified language truth. Older59-clip counts stay separate from the50-clip rates.
| Language | Code | Reference accuracy evidence | 50-recording Cap label count | Earlier59 Cap label count | Manual timestamp evidence |
|---|---|---|---|---|---|
| Afrikaans | af |
New reference screen; n=2 (see errors/completions) | 0 | 0 | Not established |
| Albanian | sq |
20 Gheg clips, 2 speaker IDs; 65.92% explicit/auto WER; other varieties unproved | 0 | 0 | Not established |
| Amharic | am |
Current opt-in: 20 development + 20 fresh validation clips; historical screen separate | 0 | 0 | Not established |
| Arabic | ar |
New reference screen; n=2 (see errors/completions) | 0 | 0 | Not established |
| Armenian | hy |
New reference screen; n=2 (see errors/completions) | 0 | 0 | Not established |
| Assamese | as |
Current opt-in: 20 development + 20 fresh validation clips; historical screen separate | 0 | 0 | Not established |
| Azerbaijani | az |
New reference screen; n=2 (see errors/completions) | 0 | 0 | Not established |
| Bashkir | ba |
Integrated MMS on 20 reused clips; 18.42% explicit WER; auto quality open | 0 | 0 | Not established |
| Basque | eu |
New reference screen; n=4 (see errors/completions) | 0 | 0 | Not established |
| Belarusian | be |
New reference screen; n=2 (see errors/completions) | 0 | 0 | Not established |
| Bengali | bn |
Current explicit replay n=2; separate unpromoted MMS comparison in multilingual report | 0 | 0 | Not established |
| Bosnian | bs |
New reference screen; n=2 (see errors/completions) | 0 | 0 | Not established |
| Breton | br |
Integrated MMS on 20 reused clips; 58.75% explicit WER; auto quality open | 0 | 0 | Not established |
| Bulgarian | bg |
New reference screen; n=2 (see errors/completions) | 0 | 0 | Not established |
| Catalan | ca |
New reference screen; n=2 (see errors/completions) | 0 | 0 | Not established |
| Chinese | zh |
New reference screen; n=2 (see errors/completions) | 1 | 1 | Not established |
| Croatian | hr |
New reference screen; n=2 (see errors/completions) | 0 | 0 | Not established |
| Czech | cs |
New reference screen; n=2 (see errors/completions) | 0 | 0 | Not established |
| Danish | da |
New reference screen; n=2 (see errors/completions) | 0 | 0 | Not established |
| Dutch | nl |
New reference screen; n=2 (see errors/completions) | 0 | 0 | Not established |
| English | en |
AMI / LibriSpeech, earlier model-path runs | 35 | 42 | Not established |
| Estonian | et |
New reference screen; n=2 (see errors/completions) | 0 | 0 | Not established |
| Faroese | fo |
20 clips, 13 conservative speaker groups; current explicit WER 96.35%; unpromoted raw MMS 51.56% | 0 | 0 | Not established |
| Finnish | fi |
20 development clips | 0 | 1 | Not established |
| French | fr |
1 short diagnostic only | 3 | 0 | 1 small diagnostic |
| Galician | gl |
New reference screen; n=2 (see errors/completions) | 0 | 0 | Not established |
| Georgian | ka |
Current explicit replay n=2; separate unpromoted MMS comparison in multilingual report | 0 | 0 | Not established |
| German | de |
New reference screen; n=2 (see errors/completions) | 2 | 1 | Not established |
| Greek | el |
New reference screen; n=2 (see errors/completions) | 0 | 0 | Not established |
| Gujarati | gu |
Current opt-in: 20 development + 20 fresh validation clips; historical screen separate | 0 | 0 | Not established |
| Haitian Creole | ht |
40 development + 10 unused validation clips; integrated candidate validation WER 17.03% explicit/auto; source limitations apply | 0 | 0 | Not established |
| Hausa | ha |
Current explicit replay n=2; separate unpromoted MMS comparison in multilingual report | 0 | 0 | Not established |
| Hawaiian | haw |
Hawaiian4; four speakers; see errors/completions | 0 | 0 | Not established |
| Hebrew | he |
Current explicit replay n=2; separate unpromoted MMS comparison in multilingual report | 0 | 0 | Not established |
| Hindi | hi |
100 clips; latest exact-worker replay60 | 2 | 1 | Not established |
| Hungarian | hu |
New reference screen; n=2 (see errors/completions) | 0 | 0 | Not established |
| Icelandic | is |
New reference screen; n=2 (see errors/completions) | 0 | 0 | Not established |
| Indonesian | id |
20 development clips | 0 | 1 | Not established |
| Italian | it |
New reference screen; n=2 (see errors/completions) | 0 | 2 | Not established |
| Japanese | ja |
New reference screen; n=2 (see errors/completions) | 0 | 2 | Not established |
| Javanese | jw |
New reference screen; n=2 (see errors/completions) | 0 | 0 | Not established |
| Kannada | kn |
Current explicit replay n=2; separate unpromoted MMS comparison in multilingual report | 0 | 0 | Not established |
| Kazakh | kk |
New reference screen; n=2 (see errors/completions) | 0 | 0 | Not established |
| Khmer | km |
Current explicit replay n=2; separate unpromoted MMS comparison in multilingual report | 0 | 0 | Not established |
| Korean | ko |
New reference screen; n=2 (see errors/completions) | 0 | 0 | Not established |
| Lao | lo |
Current explicit replay n=2; separate unpromoted MMS comparison in multilingual report | 0 | 0 | Not established |
| Latin | la |
20 development clips / 20 publisher speaker IDs; current explicit WER 69.23% versus 65.55%; orthography caveat | 0 | 0 | Not established |
| Latvian | lv |
New reference screen; n=2 (see errors/completions) | 0 | 0 | Not established |
| Lingala | ln |
New reference screen; n=2 (see errors/completions) | 0 | 0 | Not established |
| Lithuanian | lt |
New reference screen; n=2 (see errors/completions) | 0 | 0 | Not established |
| Luxembourgish | lb |
New reference screen; n=2 (see errors/completions) | 0 | 0 | Not established |
| Macedonian | mk |
New reference screen; n=2 (see errors/completions) | 0 | 0 | Not established |
| Malagasy | mg |
20 development + 20 unused validation clips; 46.58% validation WER and 40/40 API completions; high error remains | 0 | 0 | Not established |
| Malay | ms |
New reference screen; n=2 (see errors/completions) | 0 | 0 | Not established |
| Malayalam | ml |
Current explicit replay n=2; separate unpromoted MMS comparison in multilingual report | 0 | 0 | Not established |
| Maltese | mt |
New reference screen; n=2 (see errors/completions) | 0 | 0 | Not established |
| Maori | mi |
New reference screen; n=2 (see errors/completions) | 0 | 0 | Not established |
| Marathi | mr |
Current explicit replay n=2; separate unpromoted MMS comparison in multilingual report | 0 | 0 | Not established |
| Mongolian | mn |
Current explicit replay n=2; separate unpromoted MMS comparison in multilingual report | 0 | 0 | Not established |
| Myanmar | my |
Current explicit replay n=2; separate unpromoted MMS comparison in multilingual report | 0 | 0 | Not established |
| Nepali | ne |
Current explicit replay n=2; separate unpromoted MMS comparison in multilingual report | 0 | 0 | Not established |
| Norwegian | no |
New reference screen; n=2 (see errors/completions) | 0 | 0 | Not established |
| Nynorsk | nn |
New 20-clip explicit/auto study; quality gate open | 0 | 0 | Not established |
| Occitan | oc |
New reference screen; n=2 (see errors/completions) | 0 | 0 | Not established |
| Pashto | ps |
Current explicit replay n=2; separate unpromoted MMS comparison in multilingual report | 0 | 0 | Not established |
| Persian | fa |
New reference screen; n=2 (see errors/completions) | 0 | 0 | Not established |
| Polish | pl |
New reference screen; n=2 (see errors/completions) | 0 | 1 | Not established |
| Portuguese | pt |
New reference screen; n=2 (see errors/completions) | 1 | 1 | Not established |
| Punjabi | pa |
New reference screen; n=2 (see errors/completions) | 0 | 0 | Not established |
| Romanian | ro |
New reference screen; n=2 (see errors/completions) | 0 | 0 | Not established |
| Russian | ru |
New reference screen; n=2 (see errors/completions) | 3 | 0 | Not established |
| Sanskrit | sa |
20 development recitation clips, unknown speakers; current auto WER 116.83% versus 88.94%; quality gap | 0 | 0 | Not established |
| Serbian | sr |
New reference screen; n=2 (see errors/completions) | 0 | 0 | Not established |
| Shona | sn |
New reference screen; n=2 (see errors/completions) | 0 | 0 | Not established |
| Sindhi | sd |
New reference screen; n=2 (see errors/completions) | 0 | 0 | Not established |
| Sinhala | si |
New reference screen; n=4 (see errors/completions) | 0 | 0 | Not established |
| Slovak | sk |
New reference screen; n=2 (see errors/completions) | 0 | 0 | Not established |
| Slovenian | sl |
New reference screen; n=2 (see errors/completions) | 0 | 0 | Not established |
| Somali | so |
Current explicit replay n=2; separate unpromoted MMS comparison in multilingual report | 0 | 0 | Not established |
| Spanish | es |
64 clips; latest exact-worker replay16 | 1 | 3 | 64 Spanish clips; small controlled corpus |
| Sundanese | su |
New reference screen; n=4 (see errors/completions) | 0 | 0 | Not established |
| Swahili | sw |
New reference screen; n=2 (see errors/completions) | 0 | 0 | Not established |
| Swedish | sv |
New reference screen; n=2 (see errors/completions) | 0 | 0 | Not established |
| Tagalog | tl |
New reference screen; n=2 (see errors/completions) | 0 | 0 | Not established |
| Tajik | tg |
Current explicit replay n=2; separate unpromoted MMS comparison in multilingual report | 0 | 0 | Not established |
| Tamil | ta |
Current opt-in: 20 development + 20 fresh validation clips; historical screen separate | 0 | 0 | Not established |
| Tatar | tt |
Integrated MMS on 20 reused clips; 25.37% explicit WER; auto quality open | 0 | 0 | Not established |
| Telugu | te |
Current opt-in: 20 development + 20 fresh validation clips; historical screen separate | 0 | 0 | Not established |
| Thai | th |
New reference screen; n=2 (see errors/completions) | 0 | 0 | Not established |
| Tibetan | bo |
Current explicit replay n=4; separate unpromoted MMS comparison in multilingual report | 0 | 0 | Not established |
| Turkish | tr |
New reference screen; n=2 (see errors/completions) | 0 | 1 | Not established |
| Turkmen | tk |
Unmeasured | 0 | 0 | Not established |
| Ukrainian | uk |
New reference screen; n=2 (see errors/completions) | 0 | 0 | Not established |
| Urdu | ur |
New reference screen; n=2 (see errors/completions) | 0 | 0 | Not established |
| Uzbek | uz |
Current explicit replay n=2; separate unpromoted MMS comparison in multilingual report | 0 | 0 | Not established |
| Vietnamese | vi |
New reference screen; n=2 (see errors/completions) | 0 | 0 | Not established |
| Welsh | cy |
Current explicit replay n=2; separate unpromoted MMS comparison in multilingual report | 0 | 0 | Not established |
| Yiddish | yi |
Eastern Yiddish4; four speakers; see errors/completions | 0 | 0 | Not established |
| Yoruba | yo |
Current explicit replay n=2; separate unpromoted MMS comparison in multilingual report | 0 | 0 | Not established |
Regional aliases:en_au/en_uk/en_us use the English acoustic model; de_ch uses German. No separate dialect benchmark is established. The99-code inventory does not mean99 measured languages.
Status derived from retained local reports and independently checked result hashes. No audio, private transcript text, customer identifier, signed URL or credential is included here. This file aggregates recorded results; the new multilingual screen performed the real provider/API requests stated above. No Cap mutation, shared database operation, deployment or production inference was performed. Licensing is outside the acceptance gates.
New multilingual source, run, candidate and verification receipts are retained in .sandchest/benchmarks/multilingual-20260828. Earlier source hashes and machine-readable metrics remain under the ignored whisper-large-20260827/durable1/benchmark-status1 experiment directory. Major evidence:cohere-current1; mms-envelope-serving-eval1; hindi-fresh-comparison2/recognition-audit; id-fi-comparison1; turbo-q8-eval1/audited-report and supplement; cap-expansion-verification1; silence-contract-api1; native-stage-profile2/eval/audited-report. Older English128 holdout claims are explicitly historical and are not substituted for current receipts.