30B Total, 3B Active, ~3s StreamLAAL: Tencent's Hy-SimulMT Dragged Simultaneous Interpretation from Human Profession into Open Source
30B Total, 3B Active, ~3s StreamLAAL: Tencent’s Hy-SimulMT Dragged Simultaneous Interpretation from Human Profession into Open Source
Part One: The Real Difficulty of Simultaneous Interpretation
The history of SI goes back to the Nuremberg Trials after WWII, where engineers and linguists built the first electronic SI system so participants in English, French, German, and Russian could follow testimony simultaneously. Eighty-odd years later, the hardware has gone from vacuum tubes to smartphones, but the core physical constraint of SI hasn’t changed: the human ear has a hard tolerance cap on latency. Beyond 4–5 seconds, listener attention starts to drift; beyond 8 seconds, listeners start suspecting “did it break?” That’s why “3 seconds” carries a special weight in the SI world — it lands right on the edge between “acceptable” and “feels fluent.”
Isn’t Translation Quality, It’s “Jumping In Early”
On October 26, 2026, Tencent Hunyuan open-sourced Hy-SimulMT, a simultaneous interpretation large model that supports bidirectional Chinese-English speech-to-speech real-time translation, emitting translated text and voice in parallel while supporting real-time voice cloning — listeners hear the translation in a voice close to the speaker’s own. The model is first shipped free on Tencent’s Hy Translate app.
Simultaneous interpretation (SI) is not the same problem as text translation. Text can wait until a full paragraph ends; speech must be continuously chunked, with pauses / fillers / emotion / interruptions handled on the fly. The output side is harder still: it must not only be correct, it must be spoken in tone and timbre close to the original speaker, otherwise the listener hears “a machine speaking for someone” rather than “the other party speaking in a foreign language.” So the true difficulty of this field has never been translation quality — it is latency. Every extra second of delay adds to the listener’s sense of brokenness, making them notice “jumping in” or “stuttering.” Only once latency is pushed into seconds does SI become a wearable service rather than a helper tool.
Hunyuan’s headline number: StreamLAAL ~3 seconds — professional human interpreters average 4–5 seconds, so it’s a hair faster; translation quality on the RealSI long-form benchmark beats every open-source model compared and 4 out of 5 commercial APIs.
The "jump-in-early" problem of SI: delay vs. information completeness
+---------------------------------------------------+
| High information completeness (wait for the full |
| sentence or paragraph) |
| ^ |
| | |
| | Traditional translation: wait for end |
| | high delay, stable quality |
| | |
| | |
| | Hy-SimulMT: translate as you listen + |
| | post-correction |
| | ~3s delay, quality near human SI |
| | |
| v |
| Low information completeness (speak on half a |
| sentence) |
| speak early -> low latency ("jump in") |
| later context may overturn earlier judgment |
+---------------------------------------------------+
Part Two: 30B MoE, 3B Active — why “sparsity” pushes latency to seconds
Hy-SimulMT is not a “fat” 30B-dense model. It is a 30B-total / ~3B-active-per-token Mixture-of-Experts architecture. The core idea of MoE: inference routes a token through only a small number of relevant “experts,” so a 30B-class model can carry its compute cost close to that of a dense model several billions smaller.
# Intuitive demo of MoE activation sparsity
class MoEDemo:
"""30B total / 3B active MoE; each token activates ~10% of experts."""
def __init__(self, total_params_b=30.0, active_per_token_b=3.0):
self.total = total_params_b
self.active = active_per_token_b
self.sparsity = 1 - active_per_token_b / total_params_b
def compute_per_token(self):
# simplified: per-token compute ∝ active params
return self.active
def memory_footprint(self):
# VRAM / on-device memory must hold all weights;搬运量 is set by active
return self.total
def vs_dense(self, dense_b):
ratio = self.active / dense_b
return {"active_ratio_to_dense": f"{ratio:.2f}x",
"total_capacity": f"{self.total}B vs dense {dense_b}B"}
m = MoEDemo()
print(f"Sparsity : {m.sparsity:5.1%}")
print(f"Per-token compute : {m.compute_per_token()}B")
print(f"Memory footprint (all) : {m.memory_footprint()}B")
print(f"vs dense 3B : {m.vs_dense(3.0)}")
print(f"vs dense 30B : {m.vs_dense(30.0)}")
The intuition is direct: SI latency is forced into shape by compute. A fully-dense model has to move a lot of weights every inference pass and is hard to push to seconds; sparse activation only uses a handful of experts, compute cost drops, and only then can you “speak while listening.” This is not chasing the new; it is a sequence of compromises forced by the latency constraint. 30B total capacity guarantees enough knowledge coverage (especially domain terminology); 3B active compresses per-token compute — together they simultaneously hold “no quality drop” and “seconds-of-latency.”
Part Three: Key metrics — StreamLAAL, GEMBA, and the four short-audio benchmarks
StreamLAAL (Streaming Length-Adapted Audio Latency) is the SI-specific latency metric: delay in seconds between translation output and input audio; lower is better. Note it is not the same as time to first spoken output — the former measures whole-stream synchrony, the latter is just first-token latency.
RealSI long-form audio (mix of speeches, interviews, meetings, etc.):
- Hy-SimulMT: StreamLAAL ≈ 3s (lowest), GEMBA score beats every compared open model and 4/5 commercial APIs;
- Qwen3.8-LiveTranslate: slightly higher latency, slightly lower quality;
- Gemini 3.5-LiveTranslate: higher latency;
Short-audio four benchmarks (Tencent self-eval):
Model FLEURS CoVoST2 ACL6060 RealSI-Sent
Hy-SimulMT 90.3 75.5 81.2 84.7 <- 1st on RealSI-Sent
Qwen3.8-LiveTranslate 88.4 82.1 80.5 84.4 <- 1st on CoVoST2
Gemini3.5-LiveTranslate 92.4 74.4 82.6 82.2 <- 1st on FLEURS/ACL6060
Turning the relative rank on each leaderboard into a “domain dominance” index shows what Hy-SimulMT’s real position is — not a sweep of every board, but a balanced player that is lowest-latency + best-among-open-overall:
# "Domain dominance" index: score by rank in each leaderboard, sum for an overall
leaderboards = {
"RealSI-long": {"Hy": "1st", "Qwen": "3rd", "Gemini": "4th"},
"FLEURS": {"Hy": "2nd", "Qwen": "3rd", "Gemini": "1st"},
"CoVoST2": {"Hy": "2nd", "Qwen": "1st", "Gemini": "3rd"},
"ACL6060": {"Hy": "2nd", "Qwen": "3rd", "Gemini": "1st"},
"RealSI-Sentence": {"Hy": "1st", "Qwen": "2nd", "Gemini": "3rd"},
}
score = {"1st": 3, "2nd": 2, "3rd": 1, "4th": 0}
totals = {"Hy": 0, "Qwen": 0, "Gemini": 0}
for board, ranks in leaderboards.items():
for m, r in ranks.items():
totals[m] += score[r]
for m, t in sorted(totals.items(), key=lambda kv: kv[1], reverse=True):
print(f"{m:8s} aggregate = {t}")
The conclusion: Hy-SimulMT holds two 1sts (RealSI long, RealSI-Sentence), and 2nd on the other three — it is the strongest open-source model on the “lowest latency + overall quality” line, but on pure-quality boards (FLEURS, ACL6060) it still trails Gemini 3.5 and, on CoVoST2, Qwen 3.8. This “doesn’t chase individual 1sts, chases overall balance” positioning aligns with the SI business itself: users don’t need one sentence perfectly accurate; they need “the whole stream doesn’t stutter, nothing breaks immersion.”
Part Four: Head-to-head with humans
This duel matters because it breaks the old cognitive inertia that “SI = a human profession.” In the past few years, machine translation has caught up with — and in some benchmarks beaten — humans on text, but SI has long been regarded as “the last bastion machines couldn’t take” because it requires the model to chunk, understand, translate, and speak all at the same time without pauses longer than a few seconds. With ~3s StreamLAAL, Hy-SimulMT moves “machine SI” from concept to engineering-deployable.
But we also have to see this duel was set up in the model’s favor: the audio is pre-recorded, with no echo, no multi-speaker interruptions, no heavy accents. Real venues are far harsher. So “reaches human-expert level” should more precisely be read as “reaches human-expert level on clean audio, clear scenarios,” not “can replace human SI in any setting.”
From an engineering angle, the model has one advantage humans don’t: replicability. A trained model can run simultaneously across countless venues, with consistent quality, no fatigue, no breaks. A top human SI works at most a few hours a day and their state fluctuates. This means the supply curve of SI shifts from “scarce human labor” to “plentiful compute,” and the corresponding business model shifts from “per-hour high-premium service” to “per-session / per-user standardized service” — though the premium for high-end human SI will of course still exist.
— 6 CATTI Level-1/2 interpreters vs. the model
The most telling comparison is the “human-machine duel” Tencent deliberately set up. The team collected new Chinese and English audio from the internet across 10 domains (AI software, medicine, manufacturing/energy, natural science, agriculture/food, business management, law, education, culture/tourism, sports/lifestyle), covering speeches, interviews, meetings.
Participating human interpreters numbered 6 (3 with CATTI Level 1, 3 with Level 2), all with many years of SI practice; each recording was ~6 minutes, 18 in total. The evaluation took the best score among the three at each level. Tencent’s conclusion: Hy-SimulMT reached human-expert level in quality (higher score) and was faster (lower latency).
A “quality-latency quadrant” code demo places the model’s true position clearly:
# Quality-latency quadrant (values are demo; true numbers live only in official figures)
def quadrant(quality, latency_sec, name):
# quality > human avg and latency < human avg -> "ahead"; else "catching up"
human_quality_baseline = 80 # human expert quality baseline (demo)
human_latency_baseline = 4.5 # human expert avg latency (sec)
q = "ahead" if quality >= human_quality_baseline and latency_sec < human_latency_baseline \
else "catching"
return {"name": name, "quadrant": q,
"quality_delta": round(quality - human_quality_baseline, 1),
"latency_delta": round(human_latency_baseline - latency_sec, 1)}
print(quadrant(82.0, 3.0, "Hy-SimulMT")) # quality ~+, latency -1.5s -> ahead
print(quadrant(80.5, 4.5, "Human SI experts")) # baseline
print(quadrant(78.0, 5.5, "Traditional SI")) # quality -, latency high -> catching
Wang Weiwei, vice dean of the School of Interpreting and Translation Studies at Guangdong University of Foreign Studies, gave an insightful observation: the system behaves like a human SI in “speaking first and self-correcting afterward.” That is, it doesn’t wait until all information is in before translating; like a human it emits the most likely translation in the low-latency window and then corrects with later context — an engineering trade of “locally correctable” for “globally low-latency,” which is also the core skill of SI as a profession.
Part Five: Six variants, SimulInfer, SimulTrain — what exactly is open-sourced
Six variants on Hugging Face, arranged along two orthogonal axes:
- Text output mode: ① Base, only translation; ② Interleave, original transcript + translation alternating (friendlier for “source-aligned” review);
- Weight precision: BF16 full (for vLLM high-accuracy deployment), FP8 (single-GPU memory-friendly), GGUF (llama.cpp quantized — official support not yet merged, must follow the integration procedure).
Bundled tools:
- SimulInfer: vLLM-based inference service + Web UI + WebSocket API + optional TTS; requires vLLM ≥ 0.31, Linux, Python ≥ 3.12, NVIDIA GPU + CUDA;
- SimulTrain: scripts for continued training from existing checkpoints on domain data;
- License: Apache 2.0 (no commercial restriction; the TTS service itself is separately licensed).
# Simplified deployment decision tree
def pick_variant(use_case, target):
"""Pick variant by use case and target hardware."""
if use_case == "demo/research":
fmt = "bf16"
elif use_case == "single-GPU prod":
fmt = "fp8" # memory-friendly, fast enough
elif use_case == "edge":
fmt = "gguf" # llama.cpp, note: official not merged
else:
fmt = "fp8"
if use_case == "source-aligned":
mode = "interleave"
else:
mode = "base"
return f"Hy-SimulMT-30B-A3B-{mode}-{fmt}"
print("research/demo :", pick_variant("demo/research", "multi-GPU"))
print("single-GPU :", pick_variant("single-GPU prod", "A100"))
print("edge :", pick_variant("edge", "edge device"))
print("source-aligned:", pick_variant("source-aligned", "prod"))
Deployment gotchas worth flagging: ① Currently Chinese and English only — other languages need further training; ② Speech output and voice cloning depend on an external TTS service — Hy-SimulMT itself produces text only, the voice side is a separate service chain; ③ The benchmark numbers are Tencent self-eval, no independent third-party validation yet; ④ Real-venue echo, cross-talk, and accents are the “real exam” leaderboards don’t cover.
Part Six: From Text to Speech — Hunyuan’s “real-time” turning point
Placing Hy-SimulMT in Hunyuan’s open-source timeline makes it more interesting. Before this, Hunyuan mostly open-sourced text models and toolchains; this time it jumps straight into real-time speech, picking the SI scenario that is the most latency-sensitive and hardest to productize, backed by two judgments.
First, the translation battlefield is moving from text to real-time speech. Text translation has already been solved to “good enough” by all LLMs; the real incremental space is “meeting / speech / conversation” scenarios that simultaneously eat latency, quality, and timbre — miss any one and you can’t close a business loop. Tencent’s free rollout on Hy Translate app is a “cold start” strategy: feed it real audio, trade scenarios for data, data for iteration — more profitable than squeezing a few more points on a leaderboard.
Second, MoE + low activation is the most reliable way to drive latency down. 30B capacity for knowledge, 3B active for speed, FP8 for single-GPU, GGUF for edge — this combo stretches SI from “cloud only” to “privatizable and edgeable,” a genuinely usable option for enterprise customers.
# Three-condition check for SI commercialization
def production_readiness(quality_vs_human, latency_sec, edge_ready, voice_clone):
checks = {
"quality reaches human": quality_vs_human >= 1.0, # ≥1.0 is the bar
"latency ≤ 4s": latency_sec <= 4.0, # hard SI commercial threshold
"edge-deployable": edge_ready,
"voice-cloning ready": voice_clone,
}
ready = all(checks.values())
return checks, ready
c, r = production_readiness(1.05, 3.0, edge_ready=True, voice_clone=True)
print("entry conditions:", c)
print("commercial ready? :", r)
Hy-SimulMT checks all four: human-level quality, 3-second latency, edge-deployable (GGUF), voice-clone ready. It is the first open-source SI player to satisfy all four — which means the SI track is formally beginning to shift from “professional human occupation” to “scale-ready software service.”
Part Seven: Limits and outlook — don’t rush it into critical settings
Finally, the limits.
First, Chinese and English only. Multilingual SI requires retraining; short-term coverage is limited and not directly applicable to Europe/Southeast-Asia multilingual markets.
Second, self-eval numbers need independent validation. GEMBA, StreamLAAL, and the four short-audio scores are all Tencent self-eval, and the GEMBA judge model is GPT-6.1-Sol — using a rival as your own judge at least needs the “referee relationship” flagged.
Third, the evaluation doesn’t cover the hardest scenarios. Real venues have echo, cross-talk, multi-speaker interruptions, heavy accents, and dense jargon bursts. Leaderboards prove direction, not that performance holds in noisy environments.
Fourth, voice cloning is a separate pipeline. Hy-SimulMT itself only produces translated text; speech generation and voice cloning rely on external TTS services, and the permissions, privacy, and compliance of that layer need separate assessment — especially in legal, medical, and other high-sensitivity settings.
Fifth, “speak first, correct later” introduces visual/audible artifacts. In scenarios where “every sentence is final” is a hard requirement — formal diplomacy, legal testimony, etc. — this engineering tactic is not suitable.
For developers the practical advice: start SI in low-risk, latency-sensitive scenarios first — live streams, e-commerce, internal training, narration for cross-border meetings. For high-sensitivity scenarios (legal, medical, diplomatic) still keep a human fallback, treating Hy-SimulMT as a first-layer accelerator rather than the sole source. Its real value is not in being number one on some leaderboard, but in having simultaneously delivered four things that used to be mutually contradictory — “low latency + human-near + open-source self-deployable + voice-clone” — which starts to turn SI from a profession into an infrastructure.
Ninth, the real test for the open-source community. Apache 2.0 is the most permissive license, but “open-sourced” and “actually used” are two different things. The SI domain has seen almost no prior open-source success story, because the scenario needs far more than model weights — it needs TTS integration, real-time audio piping, noise reduction, meeting-system integration. Hy-SimulMT has made a start, but whether it can form a full open-source ecosystem (the way Whisper did for ASR) will depend on community activity, integration docs, and whether any enterprise puts it in production and actually feeds back real-world performance over the next year or two.
Tenth, the career transition for SI interpreters. This model won’t eliminate interpreters, but it will reshape the profession. Future SI practitioners are more like “SI system operators” or “quality fallback staff,” handling the high-sensitivity edge cases, the model’s blind spots, and stepping in when the model fails. Their revenue model may shift from “per-hour billing” to “service-tier billing” — basic SI done by the model, high-sensitivity or high-spec occasions layered by humans, and the premium is where the human value truly lives.
Eleventh, integration with Tencent Cloud. Hy-SimulMT will very likely land inside Tencent Meeting, Tencent SI, and Tencent Cloud’s audio-video product lines. For enterprise customers, “out-of-the-box Chinese-English SI” is a strong selling point; for developers, the open-source version plus the SimulInfer/SimulTrain combo is a sufficiently complete starting point. Tencent’s depth in “real-time audio-video” may be the differential versus OpenAI and Google’s SI approaches — the latter two are stronger in cloud infrastructure, but not necessarily in “edge integration + localized meeting scenarios.”
Sources: Tencent Hunyuan official release (2026-10-26); IT Home / Sina Tech / Jiemian / CodedTrip / Pandaily / AIWire reports; Hugging Face model card (tencent/Hy-SimulMT-30B-A3B, Apache 2.0); GitHub Tencent-Hunyuan/Hy-SimulMT. StreamLAAL, GEMBA, and the four short-audio scores and the human-SI comparison are Tencent self-eval; Python code is architectural/deployment illustration, not official implementation.