gRPC Integration Guide

Audience: developers writing a client (e.g. hibiki-stage) that consumes hibiki-translate over gRPC.

This is the contract document. For the underlying proto definition see api-reference.md. For an executable walk-through, see tests/spike_grpc_streaming_client.py — every code snippet here corresponds to a section of that file.

TL;DR

from protos.generated import translation_pb2, translation_pb2_grpc
import grpc

channel = grpc.aio.insecure_channel("translate.hibiki.local:50053")
stub = translation_pb2_grpc.TranslationServiceStub(channel)

async def request_iter():
    yield translation_pb2.TranslateStreamRequest(
        config=translation_pb2.StreamConfig(src_lang="en", tgt_lang="ja", session_id="...")
    )
    # ... yield AudioChunk messages with float32 16kHz PCM ...
    yield translation_pb2.TranslateStreamRequest(control=translation_pb2.StreamControl(end=True))

async for response in stub.TranslateStream(request_iter()):
    if response.WhichOneof("payload") == "sentence":
        print(response.sentence.translated_text)

That's the shape. Everything below explains the details.

Endpoint

The hibiki-translate gRPC server listens on port 50053. The address depends on the deployment mode:

In the split deploy, the service is internal-only — accessible from any task in the same VPC, not from the public internet. There is no authentication on the gRPC layer; the trust boundary is the network (VPC + security groups). Don't expose port 50053 publicly.

Use grpc.aio.insecure_channel(...) (no TLS for the internal hop). The shared infra layer terminates TLS at the per-app CloudFront edge for HTTP traffic; gRPC-internal stays plaintext.

RPC methods

The service definition (protos/translation.proto) exposes three RPCs:

99% of the integration work is TranslateStream. The rest of this doc focuses on it.

TranslateStream — the streaming contract

The message ordering is strict

Every TranslateStream call must follow this order:

1. First message: StreamConfig. If audio arrives before config, the server returns FAILED_PRECONDITION and closes the stream.
2. Then: zero or more AudioChunk messages (or the deprecated audio field — see below).
3. Optionally: StreamControl messages to signal flush or end.

yield TranslateStreamRequest(config=StreamConfig(...))
yield TranslateStreamRequest(audio_chunk=AudioChunk(pcm=..., pts_start_ms=..., pts_end_ms=...))
yield TranslateStreamRequest(audio_chunk=AudioChunk(...))
# ...
yield TranslateStreamRequest(control=StreamControl(flush=True))   # end-of-utterance
yield TranslateStreamRequest(control=StreamControl(end=True))     # end-of-session

StreamConfig fields

Per-call profile config (pre-buffer thresholds, smart-chunking, prompts) is not in StreamConfig. The server resolves a profile from (src_lang, tgt_lang) via the language-pair profile registry. To customise per-pair tuning, use POST /translate/update-profile (HTTP; persisted to DynamoDB) before opening the gRPC stream — or accept the defaults.

AudioChunk — the audio format

message AudioChunk {
    bytes pcm = 1;          // float32 PCM samples, mono, 16 kHz
    int64 pts_start_ms = 2; // PTS of the first sample (source-stream clock)
    int64 pts_end_ms = 3;   // PTS of the last sample
}

Required format:

• Sample rate: 16 kHz (the STT layer is hardwired to this; resample at the client)
• Channels: mono (1 channel)
• Format: float32, little-endian, in [-1.0, 1.0]
• Bytes per sample: 4
• Recommended chunk size: 100ms = 1600 samples = 6400 bytes. Smaller chunks add latency and overhead; larger chunks delay the pre-buffer's first fire.

PTS values: - Express timestamps in milliseconds. - The clock source is your stream, not the server's wall clock. Whatever monotonic timeline your audio comes from (a media file's timestamp, a live capture's elapsed time, etc.) is what should be in pts_start_ms / pts_end_ms. - These are echoed back in TranslatedSentence.src_pts_start_ms / src_pts_end_ms, which is how downstream consumers (e.g. a video dub mixer) correlate translation output with the source stream timing. - Setting both to 0 is acceptable but loses timing fidelity. Always populate them if you have them.

Deprecated: the audio field

TranslateStreamRequest.audio (a raw bytes field) is the legacy path. It still works for backward compatibility with HibikiVoice. New clients should always use audio_chunk so PTS information flows through.

StreamControl — flush and end

message StreamControl {
    bool flush = 1;   // end of utterance, drain buffers
    bool end = 2;     // end of session, close the stream
}

• flush=True: tells the server "I've finished an utterance; drain the pre-buffer and emit any pending translation now." Useful when the speaker pauses noticeably or you have a VAD-detected speech-end event.
• end=True: tells the server "I'm done sending audio; close gracefully." After end=True, no more audio chunks will be accepted; the server emits any final translations and closes the response stream.

Always send end=True before closing the channel — without it, the server may drop the last partial translation.

Receiving translations

The response stream emits TranslateStreamResponse messages with a oneof payload:

message TranslateStreamResponse {
    oneof payload {
        TranslatedSentence sentence = 1;
        PartialTranscript partial = 2;
        LiveTranscriptUpdate live = 3;
    }
}

TranslatedSentence — the main event

Emitted once per committed sentence (or chunk if skip_post_stitcher=True). All fields:

Translation chain status

The translator chain runs a hedged race: primary tier (default Sonnet) at t=0; secondary (Nova) joins at hedge_delay_s; tertiary (AWS Translate) at 2 × hedge_delay_s. The first non-empty result wins; the rest are cancelled. translation_status tells you which won:

PartialTranscript — the in-progress preview

message PartialTranscript {
    string text = 1;
    int32 speaker_id = 2;
}

Partial transcripts are emitted while a translation is being assembled — for example, the streaming Nova on_partial callback fires per token. They're useful for typewriter-style UI but should not be persisted as final output.

In current code, partials are emitted from the WLK STT path consistently and from Transcribe inconsistently. Don't rely on them for your client logic; treat them as a UX nicety.

LiveTranscriptUpdate — operator-UI display state

Emitted on every WLK tick (~5Hz). Carries the full snapshot of what the operator's screen should show — committed rows plus an in-flight row for content currently being recognised / translated. Consumer apps that just want committed translations can ignore this; it exists to drive the operator's live-display UX without requiring the client to reconstruct row state from sentence + partial events.

message LiveTranscriptUpdate {
    repeated TranscriptLine lines = 1;
    string buffer_transcription = 2;
    string buffer_diarization = 3;
    string buffer_translation = 4;
    string status = 5;
    bool speculative = 6;
    float remaining_time_transcription = 7;
    float remaining_time_diarization = 8;
}

message TranscriptLine {
    int32 speaker = 1;
    string text = 2;
    string translation = 3;
    string speaker_gender = 4;
    float start = 5;
    float end = 6;
    string detected_language = 7;
}

Field meanings:

The trailing-row contract was tightened in the 2026-06 line of work to fix a chunk-by-chunk flicker. The engine maintains _uncommitted_english — a running buffer of WLK-confirmed text since the last post-stitcher commit — which becomes the trailing row's text. It grows monotonically as the speaker continues; commits drain it by removing the matching prefix. Clients that mirror this behavior will get smooth blue-text rendering for free.

Error handling and reconnect

gRPC errors come through as grpc.RpcError. Common codes:

No session resumption. The server keeps no state between streams. If your stream drops mid-utterance, the partial audio is lost. Reconnects start from scratch with a new session_id.

For a UX where dropping audio mid-utterance is unacceptable, mitigate at the client: buffer the source audio for a short window (1–2 seconds), reconnect on UNAVAILABLE, and replay buffered audio with the same PTS values. The server will dedupe nothing, but you'll get a clean restart at sentence boundaries.

Cancellation

Two ways:

1. Stop the request iterator. async for ... exits, the stream closes from the client side, the server sees the half-close and stops emitting.
2. Close the channel. await channel.close() aborts immediately. The server raises CANCELLED on its side.

Always send StreamControl(end=True) before close if you want a graceful end. Skipping it is fine for emergencies but you lose any in-flight translation.

Worked example

See tests/spike_grpc_streaming_client.py — a complete working client. Run it against a local server:

# Start hibiki-translate locally (one terminal)
AUTH_MODE=dev STT_ENGINE=transcribe python web_server.py

# In another terminal — health check first
python tests/spike_grpc_streaming_client.py health

# Synthetic audio (silence + sine tone, ~5 seconds)
python tests/spike_grpc_streaming_client.py synthetic en ja

# Real WAV file (must be 16kHz mono int16; resample with ffmpeg if not)
python tests/spike_grpc_streaming_client.py stream sample.wav en ja

To point at a deployed server:

HIBIKI_TRANSLATE_GRPC=translate.hibiki.local:50053 \
    python tests/spike_grpc_streaming_client.py synthetic en ja

The script demonstrates: - Config-first message ordering - Real-time pacing of audio chunks (sleep between sends to match audio duration) - PTS tracking (so TranslatedSentence.src_pts_* is meaningful) - Both TranslatedSentence and PartialTranscript handling - Graceful Ctrl-C cancellation - gRPC error classification and exit codes

Common gotchas

• Sample rate: the server expects 16 kHz, period. Send other rates and STT will fail or produce garbage. Resample at the client (librosa, ffmpeg, soxr).
• float32, not int16: PCM bytes must be float32 little-endian. Many capture libraries default to int16; convert before sending: int16_value / 32768.0 → float32.
• Don't forget session_id: while optional, an empty session_id makes server logs harder to correlate. Always send a UUID.
• PTS units are milliseconds: not microseconds, not samples. pts_start_ms = sample_index * 1000 // 16000.
• Pace your sends: don't dump 60 seconds of audio in one second of network. The pre-buffer's timing logic depends on chunks arriving at roughly real-time pace. The spike script demonstrates this.
• enable_diarization requires WLK: with the AWS Transcribe STT engine, the flag is silently ignored. If diarization is critical, deploy in ecs-gpu or split mode with WLK enabled.
• whisper_model only affects WLK: ignored under Transcribe.

Next steps

• Generate gRPC stubs in your client language. Python: python -m grpc_tools.protoc -I=protos --python_out=. --grpc_python_out=. protos/translation.proto.
• For TypeScript/Node clients, consider @grpc/grpc-js plus the proto file directly, or pre-generate stubs with protoc-gen-ts.
• For TTS, see hibiki-tts's gRPC contract (separate microservice; consume TranslatedSentence.translated_text + emotion_instruction and pipe to hibiki-tts.SynthesizeStream).
• See api-reference.md for the raw proto field-by-field.
• See architecture.md for where this fits in the broader system.