今天,我们以 Apache 2.0 许可证开源了 Rumble-2-Flash。这是一个拥有 5 亿参数的语音识别模型,支持 40 多种语言,其性能可与体积为其 3 到 60 倍的模型相媲美。
- 随处运行:在 🤗 Transformers 和 vLLM 中提供原生支持,无需自定义代码
- 支持 40 多种语言:自动检测语言
- 输出控制:数字以阿拉伯数字或单词形式输出、填充词、逐字重复、强制指定语言
- 极速处理:约半秒即可处理一小时音频,全批量处理时 RTFx 超过 7,000
- 真实电话通话表现优异:在 302 小时的呼叫中心基准测试中,与 NVIDIA 的 Nemotron 3.5 Transcribe (30B) 相当
快速开始
使用 🤗 Transformers 进行转录
uv pip install torch transformers accelerate
from transformers import AutoProcessor, AutoModelForMultimodalLM
model_id = "Reson8/Rumble-2-Flash"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForMultimodalLM.from_pretrained(model_id, device_map="auto")
inputs = processor.apply_transcription_request(
audio="https://hf.co/datasets/hf-internal-testing/dummy-audio-samples/resolve/main/bcn_weather.mp3",
language="English", # 省略此参数可自动检测
).to(model.device, dtype=model.dtype)
output = model.generate(**inputs, max_new_tokens=512)
generated = output[:, inputs["input_ids"].shape[-1]:]
print(processor.decode(generated, return_format="transcription_only")[0]) # 如果检测语言,请使用 return_format="parsed"
audio 参数接受本地路径、URL、NumPy 数组或这些内容的列表。
构建原理
Rumble-2-Flash 采用 Qwen3-ASR 架构,以实现开箱即用的支持。如果您的技术栈运行的是 Transformers 或 vLLM,它就能运行 Rumble-2-Flash,无需分叉运行时环境或使用 trust_remote_code。
为了找出模型在精度下降前能缩小到什么程度,我们使用在线策略蒸馏(on-policy distillation)将我们的更大规模生产模型 Rumble-2 蒸馏到了该模型中。学生模型并不在教师模型编写的转录文本上进行训练。它自己转录音频,然后由教师模型对转录结果进行逐 token 评分:在每个位置,教师模型给出下一个 token 的概率,学生模型则学习匹配这种概率分布。由于学生模型是从自己的输出中学习,因此它能针对自己犯下的错误得到纠正,并且每个 token 都能获得反馈,而不是仅针对整个音频片段给出一个分数。
在线策略蒸馏通常要求教师和学生共享同一个分词器,而我们的模型并没有这样做:选择 Qwen3-ASR 意味着也采用了它的词汇表。GOLD 消除了这一限制。它将两个分词器对文本分割方式不同的 token 进行合并,从而使两个模型能够在相同的文本基础上进行比较。存在于两个词汇表中的 token 直接进行比较,其余 token 则按其排序后的概率进行比较。

使用 GOLD 进行在线策略蒸馏
结果
与其规模相当的精度
在 Open ASR Leaderboard 多语言赛道上,Rumble-2-Flash 在中文、德语、法语、意大利语、西班牙语、葡萄牙语、印地语、荷兰语和英语上的平均错误率为 6.96%。这优于 Meta 的 omniASR-LLM 7B(一个体积为其 15 倍的模型),并领先于 Whisper large-v3。在 英语赛道上,其平均词错误率(WER)为 6.6%。

同规模下的最佳成绩
速度
Rumble-2-Flash 的速度也非常快。其吞吐量取决于批处理的填充程度:在 HF 的 Open ASR Leaderboard 中较大的数据集上,当批处理保持满负荷时,其吞吐量可达超过 7,000 RTFx。这意味着大约半秒即可处理一小时音频。
真实电话通话
干净音频的处理相对容易。电话通话经过压缩且伴有噪音,说话者往往带有非常不同的口音。在 AppTek 的呼叫中心基准测试(302 小时热线通话,14 种英语口音,使用 AppTek 自有脚本评分)中,Rumble-2-Flash 达到7.86% WER,与 NVIDIA 的 Nemotron 3.5 Transcribe (30B) 的 7.78% 相当。这明显优于来自同一模型家族的 Qwen3-ASR 0.6B(8.7%),并接近 Qwen3-ASR 1.7B(7.4%),后者是此次比较中唯一表现更好的模型。

真实电话通话,14 种英语口音
试用
模型权重托管在 Hugging Face Hub 上的 Reson8/Rumble-2-Flash。您也可以通过我们的 API 使用它。
我们非常好奇您将用它构建什么。请在 Discussions 标签页告诉我们您的作品、喜欢之处以及希望在下一版本中看到哪些更改。
查看原文
Today we're open sourcing Rumble-2-Flash under the Apache 2.0 license. It's a 0.5B-parameter speech recognition model for 40+ languages that competes with models 3-60x its size.
- Runs anywhere: native support in 🤗 Transformers and vLLM, no custom code
- 40+ languages: automatic language detection
- Output control: numbers as digits or words, fillers, verbatim repeats, forced language
- Extremely fast: an hour of audio in about half a second, with over 7,000 RTFx on full batches
- Holds up on real phone calls: on par with NVIDIA's Nemotron 3.5 Transcribe (30B) on a 302-hour call-centre benchmark
Quickstart
Transcribe with 🤗 Transformers
uv pip install torch transformers accelerate
from transformers import AutoProcessor, AutoModelForMultimodalLM
model_id = "Reson8/Rumble-2-Flash"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForMultimodalLM.from_pretrained(model_id, device_map="auto")
inputs = processor.apply_transcription_request(
audio="https://hf.co/datasets/hf-internal-testing/dummy-audio-samples/resolve/main/bcn_weather.mp3",
language="English", # omit to detect automatically
).to(model.device, dtype=model.dtype)
output = model.generate(**inputs, max_new_tokens=512)
generated = output[:, inputs["input_ids"].shape[-1]:]
print(processor.decode(generated, return_format="transcription_only")[0]) # return_format="parsed" if detecting language
audio accepts a local path, a URL, a NumPy array, or a list of these.
How it's built
Rumble-2-Flash uses the Qwen3-ASR architecture for day-0 support. If your stack runs Transformers or vLLM, it runs Rumble-2-Flash, with no forked runtime or trust_remote_code.
To find out how small a model could get before accuracy dropped off, we distilled Rumble-2, our larger production model, into it with on-policy distillation. The student doesn't train on transcripts the teacher wrote. It transcribes the audio itself, and the teacher grades that transcript token by token: at every position, it gives its own probability for what should come next, and the student learns to match it. Because the student learns from its own outputs, it gets corrected on exactly the mistakes it makes, with feedback on every token instead of one score per clip.
On-policy distillation normally requires teacher and student to share a tokenizer, and ours don't: choosing Qwen3-ASR meant taking its vocabulary too. GOLD removes that restriction. It merges tokens where the two tokenizers split the text differently, so both models are compared over the same text. Tokens that exist in both vocabularies are compared directly, and the rest by their sorted probabilities.

On-policy distillation with GOLD
Results
Accuracy for its size
On the Open ASR Leaderboard multilingual track, Rumble-2-Flash averages a 6.96% error rate across Chinese, German, French, Italian, Spanish, Portuguese, Hindi, Dutch and English. That's better than Meta's omniASR-LLM 7B, a model 15x larger, and ahead of Whisper large-v3. On the English track it averages 6.6% WER.

The best score for its size
Speed
Rumble-2-Flash is also fast. Its throughput depends on how full the batches are: on the larger datasets in HF's Open ASR Leaderboard, where batches stay full, it reaches over 7,000 RTFx. That's an hour of audio in about half a second.
Real phone calls
Clean audio is easy. Phone calls are compressed and noisy, and often have speakers with very different accents. On AppTek's call-centre benchmark (302 hours of helpline calls, 14 English accents, scored with AppTek's own script), Rumble-2-Flash reaches 7.86% WER, on par with NVIDIA's Nemotron 3.5 Transcribe (30B) at 7.78%. That's well ahead of Qwen3-ASR 0.6B (8.7%), from the same model family, and close to Qwen3-ASR 1.7B (7.4%), the only model in the comparison that does better.

Real phone calls, 14 English accents
Try it
The weights are on the Hub at Reson8/Rumble-2-Flash. It's also available through our API.
We're very curious what you'll build with it. Tell us in the Discussions tab what you've made, what you like, and what you'd like to see changed in a next version.
原文版权归 Hugging Face Blog 及原作者所有,本站转载仅供阅读。如需删除,请联系 support@aisa.one。