Blaze Labs · Engineering

Vietnamese voice cloning: a technical benchmark

A reproducible head-to-head across quality (speaker similarity, intelligibility, naturalness) and performance (first-byte latency, real-time factor), with the exact API calls so you can reproduce it. Latest quality run: 30 real Vietnamese speakers from the ViMD multi-dialect corpus, balanced 10 North / 10 Central / 10 South, zero-shot cloning, identical scoring.

Base URL https://api.blaze.vn · auth Authorization: Bearer $BLAZE_API_TOKEN · all audio Vietnamese (vi) · updated Jun 2026 · 22/06/2026

1. Benchmark design

• Data (current): 30 reference clips (4–8 s) from the ViMD multi-dialect corpus (real broadcast speech, EMNLP 2024), one per distinct speaker across 13 provinces, balanced 10 North / 10 Central / 10 South. Each paired with a separate target sentence — cross-sentence, zero-shot cloning.
• Generation: each system clones the reference and reads the target. Batch models via /v1/voices/test; realtime models via the realtime WebSocket from a per-item registered voice.
• Identical scoring applied to every system’s output.

Metrics

Note: SIM and WER use Blaze’s own embedding/ASR — applied identically to all systems, so the comparison is internally consistent (a fully neutral SIM would use a third-party verifier; the direction of results is robust either way).

2. Quality results — ViMD references (n=30)

Blaze, all 6 configs, on the clean region-balanced ViMD references.

By dialect region (SIM / WER% / UTMOS, n=10 each — indicative)

Reading the matrix

• 2.0-realtime has the highest speaker similarity of all (SIM 0.732); v2.0_pro has the lowest WER (4.6%). flash ≈ pro on similarity within a generation.
• v2.0 clearly wins naturalness (UTMOS ~2.62 vs v1.5 ~2.2). On these clean refs the v1.5 WER gap narrows a lot (6–8% vs v2.0’s 4.6%), but v1.5 still sounds more synthetic.
• By region: SIM is stable ~0.70–0.74 across North/Central/South (v1.5/1.5-realtime dip to ~0.67 on Northern dialect). Northern gets the lowest WER for v2.0 (0.7–2.2%); Central is the hardest for intelligibility (WER up to ~10%) yet scores the highest UTMOS; Southern sits in between. 2.0-realtime and v2.0_pro are the most consistent across regions.
• n=10/region → treat region deltas as directional, not definitive.

3. Latency & throughput

Realtime tier (the comparison that matters for live use). Blaze over the realtime WebSocket (warm), ElevenLabs over HTTP /stream. n=15 sentences, sequential, warmed up.

4. Model selection guide

5. Audio samples

Blaze vs ElevenLabs on representative Vietnamese clips — listen for the speaker-similarity and pronunciation differences.

6. API reference (the exact calls used)

6.1 Zero-shot clone + generate (one shot)

Clone from a reference clip and synthesize in a single call. Returns a presigned audio URL. This is what the batch quality numbers use.

curl -X POST https://api.blaze.vn/v1/voices/test \
  -H "Authorization: Bearer $BLAZE_API_TOKEN" \
  -F "text=Xin chào, đây là giọng nói được nhân bản." \
  -F "language=vi" -F "audio_speed=1.0" \
  -F "speaker_type=Narration" \        # Narration|Commentary|Podcast|Storytelling
  -F "speaker_province=Hà Nội" \       # required for vi
  -F "ref_text=<transcript of reference audio>" \
  -F "model=v2.0_pro" \                # v2.0_pro|v2.0_flash|v1.5_pro|v1.5_flash
  -F "file=@reference.wav;type=audio/wav"
# -> {"audio_url": "https://.../xxx.mp3?..."}

6.2 Register a reusable voice, then batch TTS

# Register (multipart) -> persistent speaker_id
curl -X POST https://api.blaze.vn/v1/voices \
  -H "Authorization: Bearer $BLAZE_API_TOKEN" \
  -F "voice_id=my_voice_01" -F "name=My Voice" \
  -F "type=Narration" -F "gender=Male" -F "language=vi" \
  -F "audio_speed=1.0" -F "ref_text=<transcript>" \
  -F "province=Hà Nội" -F "file=@reference.wav;type=audio/wav"

# Synthesize (async job)
curl -X POST https://api.blaze.vn/v1/tts \
  -H "Authorization: Bearer $BLAZE_API_TOKEN" -H "Content-Type: application/json" \
  -d '{"query":"Nội dung cần đọc.","language":"vi","speaker_id":"my_voice_01","model":"v2.0_pro","audio_format":"wav"}'
# -> 202 + job; fetch result via GET /v1/tts/{id}/download

6.3 Realtime streaming (WebSocket) — recommended for live use

Protocol: connect → authenticate → speech-start → query → speech-end → JSON status frames + binary PCM (int16, 24 kHz, mono). Realtime model ids: 2.0-realtime, 1.5-realtime.

import asyncio, json, websockets, numpy as np

async def synth(token, speaker_id, text):
    url = "wss://api.blaze.vn/v1/tts/realtime"
    async with websockets.connect(url, max_size=None) as ws:
        await ws.recv()                                   # {"type":"successful-connection"}
        await ws.send(json.dumps({"token": token, "strategy": "livekit"}))
        await ws.recv()                                   # {"type":"successful-authentication"}
        await ws.send(json.dumps({"event":"speech-start","language":"vi",
            "audio_format":"pcm","speaker_id":speaker_id,"normalization":"no",
            "model":"2.0-realtime","sample_rate":24000}))
        await ws.send(json.dumps({"query": text}))
        await ws.send(json.dumps({"event":"speech-end"}))
        pcm = bytearray()
        while True:
            m = await ws.recv()
            if isinstance(m, (bytes, bytearray)): pcm += m          # raw int16 @ 24k
            elif json.loads(m).get("type") == "finished-byte-stream": break
        return np.frombuffer(bytes(pcm), dtype=np.int16)            # TTFB = send(query) -> first frame

6.4 HTTP streaming of batch models

Two steps; the stream is framed as repeated [4-byte big-endian length][int16 PCM @ 24k].

# 1) create job (form). NOTE: text field is `text`, model is `model_id`.
curl -X POST https://api.blaze.vn/v1/tts/stream \
  -H "Authorization: Bearer $BLAZE_API_TOKEN" \
  -F "text=Nội dung." -F "speaker_id=my_voice_01" \
  -F "model_id=v2.0_pro" -F "language=vi" -F "audio_format=wav"
# -> {"audio_url":"/v1/tts/stream-audio/<job_id>"}
# 2) stream the framed PCM
curl https://api.blaze.vn/v1/tts/stream-audio/<job_id> -H "Authorization: Bearer $BLAZE_API_TOKEN"

6.5 Scoring endpoints (to reproduce SIM / WER)

# Speaker embedding (256-d) — max 10 s of audio
curl -X POST https://api.blaze.vn/v1/stt/speaker-embedding \
  -H "Authorization: Bearer $BLAZE_API_TOKEN" \
  -F "audio_file=@clip.wav;type=audio/wav"
# -> {"embedding":[...256...], "dimension":256}   (SIM = cosine of two embeddings)

# Speech-to-text for WER
curl -X POST "https://api.blaze.vn/v1/stt/execute?language=vi&model=stt-async-1.5" \
  -H "Authorization: Bearer $BLAZE_API_TOKEN" \
  -F "audio_file=@gen.wav;type=audio/wav"
# -> {"result":{"data":{"transcription":"..."}}}

7. Caveats & reproducibility

• Self-referential scoring: SIM uses Blaze’s embedding and WER uses Blaze STT. They’re applied identically to all systems (fair, internally consistent), but absolute SIM may flatter Blaze; a neutral verifier (e.g. WavLM/ECAPA) would remove that. The large, consistent direction holds regardless.
• UTMOS is English-trained → a relative naturalness proxy for Vietnamese, not an absolute MOS.
• Latency is network-dependent and measured from a single client location; treat cross-vendor TTFB as indicative, RTFx as more robust.
• ElevenLabs models: eleven_multilingual_v2 was excluded (renders Vietnamese with an English accent). eleven_v3 and eleven_flash_v2_5 handle Vietnamese.
• Reference set: quality numbers use ViMD (real broadcast, region-labelled). Sample size is 30 (10/region), so region splits are directional. Latency (§3) is reference-independent.