Human speech in modern business, tech, and cultural contexts is rarely strictly monolingual. In many Arabic-speaking regions, speakers slide seamlessly between Arabic and English, mixing vocabulary, grammar, and expressions—a phenomenon known as code-switching.
Accurately recognizing code-switched speech requires overcoming dynamic vocabulary boundaries, fluid phonetic shifts, and contextual ambiguity across distinct linguistic systems.
From Ragini & Akshara to the Arabic Frontier
At ConvoZen Research, our exploration of code-switching began with our foundational models for Indian languages—Ragini and Akshara. Designed to solve complex multilingual blending like Hinglish and Tamlish, Ragini and Akshara proved that capturing mixed-language speech requires modeling language transitions natively within a unified neural representation.
However, applying this code-switching paradigm to the Arabic-speaking world presented a fundamentally different and more complex obstacle: severe data fragmentation across geographic boundaries.
The Arabic Data Challenge: Regional Diversity vs. Industry Fragmentation
While speech data collection in regions like India has seen structured digital momentum, the Middle East & North Africa (MENA) landscape remains acutely fragmented. The Arab world spans over two dozen countries, each with distinct spoken dialects—ranging from Gulf (Khaleeji) and Levantine to Egyptian—that diverge significantly from Modern Standard Arabic (MSA) in phonology, vocabulary, and syntax.
Historically, there has been very little unified effort across industry or academia to aggregate and harmonize these diverse spoken sources. Most existing speech models are trained either on pristine news broadcasts (like MGB-2) or isolated monolingual corpora, failing completely when exposed to real-world dialectal variations mixed with English technical terms.
Alif v1: Unifying Dialects Across Disparate Sources
Alif v1 overcomes this fragmentation by establishing a multi-source acoustic and textual normalization pipeline. By sourcing and curating datasets across broadcast media (MGB-2), studio recordings (MASC Clean), noisy operational environments (MASC Noisy), Gulf colloquial speech (SADA), and North African dialects (Casablanca), Alif builds a single shared latent representation capable of handling both dialectal diversity and code-switching seamlessly.
By adopting a unified encoder-decoder architecture with a shared multilingual vocabulary tokenization system, Alif tracks language switches natively as joint acoustic-textual probabilities across seamless end-to-end decoding.
Click play to hear speech and view the live transcription stream.
Bilingual ASR Benchmarks
We evaluated Alif on custom code-switched speech test sets against public and commercial speech engines.
| Model | Type | Avg WER (%) | Casablanca | CommonVoice | MASC Clean | MASC Noisy | MGB-2 | SADA |
|---|---|---|---|---|---|---|---|---|
| Alif v1 | Local Unified | 23.88% | 56.37% | 13.63% | 12.23% | 24.43% | 12.52% | 27.89% |
| ElevenLabs Scribe v2 | Proprietary API | 24.59% | 54.05% | 10.90% | 20.25% | 28.25% | 10.22% | 33.74% |
| Deepgram Nova 3 | Proprietary API | 36.06% | 58.62% | 8.35% | 22.35% | 30.76% | 27.09% | 45.09% |
| Nvidia Conformer (with LM) | Open Source | 32.90% | 68.90% | 8.80% | 23.74% | 34.29% | 17.20% | 44.52% |
| Whisper Large v3 (HF) | Open Source | 36.86% | 71.81% | 17.83% | 24.66% | 34.63% | 16.26% | 55.96% |
| Seamless M4T v2 Large | Open Source | 38.16% | 66.25% | 21.70% | 25.04% | 33.24% | 20.23% | 62.52% |
Across the benchmark suite, Alif v1 achieves an overall average 23.88% Word Error Rate (WER)—outperforming Whisper Large v3 by 35.2% relative and demonstrating remarkable stability across regional dialects (such as 27.89% on Gulf SADA speech and 12.23% on studio MASC Clean). By unifying disparate spoken sources into a single model, Alif establishes a new state of the art for bilingual Arabic speech recognition.
Unified Architecture
Alif's end-to-end decoding projects multi-dialect Arabic and English phonology directly into a shared acoustic latent space, eliminating boundary errors altogether.
Fig 1 — Alif Unified End-to-End Architecture
Ultra-Low Latency for Conversational Voice Agents
Beyond high accuracy across complex dialects, real-world conversational AI demands instantaneous responsiveness. Alif v1 is engineered specifically for real-time streaming voice applications, achieving an ultra-low latency profile with sub-120ms time-to-first-token (TTFT).
By utilizing chunked streaming attention and dynamic prefix decoding, Alif processes live audio frames continuously without waiting for full silence boundaries. This makes it the ideal foundational engine for next-generation interactive voice agents, automated customer support bots, IVR telephony systems, and live bilingual meeting transcription where fluid, human-like turn-taking is critical.
Try out Alif v1 from the ConvoZen platform. For any inquiries or to request early access to our next-generation models, reach out to the research team at contact@convozen.ai.