Hugging Face adds Egyptian and Nigerian English to its ASR benchmark

Choosing a speech recognition model gets harder when the people using it sound different from the people in its benchmarks. Hugging Face’s latest Open ASR Leaderboard update gives developers more evidence to work with, particularly when comparing systems for English-speaking audiences across countries.

Announced on November 6, 2026, the update adds private Liva AI recordings of Egyptian-accented and Nigerian-accented English. The conversational splits contain 1.34 and 1.55 hours respectively, with spontaneous speech and transcripts that retain disfluencies. These are additions to English accent coverage, rather than two new languages.

What the accent comparisons show

The Private data tab offers US versus non-US results and a comparison between an Inner Circle group covering the US, UK, Australia and Canada and combined Outer & Expanding Circles covering India, Nigeria and Egypt.

For a team building a transcription service, those views provide another way to question a shortlist. A strong overall result should lead to closer inspection of relevant speech conditions, especially when the intended audience differs from the benchmark’s most familiar speakers.

The groups are broad. A pooled result cannot tell a product team how every customer, regional accent or recording setup will perform. Treat it as a reason to investigate, rather than a substitute for testing the people and conditions the service actually needs to support.

Read the average before reading the rank

Word error rate, or WER, counts substitutions, deletions and insertions against a reference transcript, relative to its word count. Lower scores generally mean fewer transcription errors. Also check response time and running costs for your workload. The benchmark reports processing speed through RTFx, but a voice interface still needs acceptable end-to-end latency under the conditions in which people will use it.

There is an important detail in the current leaderboard configuration. Private scripted and conversational results contribute to the default average. Older launch coverage described private scores as optional, but the project’s changelog records their inclusion in the default average on July 24.

The current scoring implementation also explains how the Private data tab works. Each group averages its datasets, then the displayed overall score averages the two groups selected for that view. The two groups therefore receive equal weight, even when they contain different numbers of datasets.

That matters when a model moves up or down after a different comparison is selected. The ordering can change because the calculation gives different emphasis to existing results. It does not, by itself, show that the model has improved.

When comparing screenshots or vendor claims, keep the selected grouping and leaderboard version alongside the score. Otherwise, two apparently conflicting rankings may simply answer different questions.

Why the recordings stay private

Hugging Face’s private-data methodology describes held-out recordings as a way to reduce benchmark-specific optimization. It also acknowledges a remaining limitation. Models trained on similar data from a provider could still benefit, even without access to the exact evaluation recordings.

For readers, the sensible distinction is between reducing an opportunity to optimize for a test and proving that every possible advantage has disappeared. Private evaluation adds evidence, but it cannot establish that a system will handle every production workload reliably.

How developers can use the update

Start with the Open ASR Leaderboard and inspect the Private data tab. For model submissions, the evaluation repository documents public testing and links to the pull-request checklist. Maintainers run the private evaluations. The withheld recordings are not a public dataset developers can simply download.

For an application shortlist, add a small evaluation using recordings collected with appropriate consent and permission. Include the devices, background noise and speaking styles the product expects. Review errors that change meaning, such as names, quantities or negations, rather than relying only on the position in a table.

The useful outcome is a better-supported selection decision. Broader benchmark coverage can help identify candidates worth testing, while application-specific evidence determines whether those candidates are ready for the people who will use them.