Operator Guide

Audience: people configuring a hibiki-translate deployment — choosing STT engines, tuning translation profiles, managing GPU lifecycle, customising prompts.

This is about running the engine, not consuming it. If you're writing a gRPC client, see grpc-integration.md. If you're deploying it, see deployment.md.

What you can configure at runtime

The settings page at /translate/settings is the main operator UI. It surfaces these knobs:

All settings persist to DynamoDB (the HibikiTranslateProfiles table) where applicable. Restarting the server preserves them.

Settings that are NOT in the UI: - Region (deploy-time only; AWS_REGION env var) - Cognito pool / client ID (deploy-time) - Default target language at server start (TGT_LANG env var) - gRPC port (hardwired to 50053)

STT engine selection

The big choice. Two backends, very different tradeoffs.

WhisperLiveKit (WLK)

Runs on GPU. Uses faster-whisper + Sortformer for transcription with built-in speaker diarization.

Pros: - Multilingual: handles all 10 supported source languages - Speaker diarization (speaker_id per word, gender estimate via F0) - Large-v3 model is the most accurate option - Streaming partial transcripts work cleanly

Cons: - Requires a GPU instance (g6e.xlarge / L40S recommended) - ~60s cold start to load the model - More expensive — GPU is ~$1–4/hour vs. Transcribe's per-second-of-audio billing - Single language per WLK process; switching WLK_LANGUAGE requires restarting the WLK service

When to use: events with multiple speakers where diarization matters; when accuracy is the priority; when you have a sustained workload that justifies keeping the GPU warm.

AWS Transcribe

Cloud API. No GPU required.

Pros: - No GPU instance to manage - Instant start (no cold-start delay) - Pay per second of audio actually transcribed (no idle cost) - Multilingual within the supported set

Cons: - No speaker diarization — speaker_id is always 0 - No gender estimate - Network latency adds ~50–100ms per chunk - AWS-region-specific (use the same region as your deploy)

When to use: single-speaker scenarios; cost-sensitive deployments; bursty workloads where keeping a GPU warm doesn't pay off; development and testing.

Switching engines

In the operator UI: Settings → STT Engine → choose → Apply. The server immediately switches, and:

• If switching to WLK: the GPU lifecycle is automatically started (ASG desired capacity → 1). Wait ~60s for the model to load.
• If switching away from WLK: the GPU stays running unless you also manually stop it. (We removed automatic shutdown to prevent mid-event terminations.)

The change persists. Subsequent server restarts use the saved engine choice.

What if I need to change WLK_LANGUAGE mid-session?

Today: not possible without restarting the WLK service (the underlying Whisper model loads with a fixed language). The /translate/restart-wlk endpoint exists to do this gracefully.

This is a fundamental limitation of the WLK model loading, not an architectural choice. To work around it, deploy multiple WLK services with different language configs and route to the right one — but that's hibiki-stage's problem, not the engine's.

Language pair profiles

Translation behavior depends on the source-target pair. Different language pairs have different optimal pre-buffer thresholds, smart-chunking rules, and post-stitcher behavior.

Categories

Per-profile knobs (settings UI: "Pipeline tuning")

Defaults are tuned per category. The settings UI lets you override per language pair.

Recommended starting points

These are the post-revert defaults from June 2026. The event-translator branch had briefly used aggressive settings for SVO→SOV (timeout=2.5s, min_chars=20, min_words=3) — those produced fragmenty Japanese and were reverted. If you need to test the aggressive settings again (e.g. for very short utterances like tour guides), the override is preserved in README.md.

Smart-chunking word lists

Each source language has a list of "function words" — if the buffer's last word is on the list AND smart chunking is on, the buffer waits for more text before firing.

Examples: - English: the, and, or, but, of, to, in, ... - Japanese: は, が, を, に, で, と, も, の, ... (case particles) - French: le, la, les, et, ou, mais, ...

Edit via Settings → Smart Chunking Words. The list is per source language, not per pair.

GPU lifecycle

When STT_ENGINE=whisperlivekit, the engine needs a GPU. Lifecycle is manual.

Operator-driven

The GPU keeps running indefinitely once started. There's no idle auto-stop and no max-runtime cap. Both used to exist; we removed them after they triggered mid-event terminations.

If you want time-based auto-stop (e.g. for cost control on dev environments), implement it externally — a scheduled Lambda calling /translate/gpu/stop outside business hours, for example. The engine deliberately doesn't make this decision.

Cold-start behavior

When the GPU goes from stopped to running: 1. ASG sets desired capacity → 1 (~30s for instance launch) 2. ECS schedules the WLK task on the new instance (~30s) 3. Whisper model loads (~30–60s for large-v3) 4. WLK service registers in Cloud Map; gRPC server can now reach it

Total: ~1.5–2 minutes of "GPU is starting" before translations work. Plan accordingly for events.

Dual control planes (caveat)

There are two ways to control the GPU instance: - gpu_manager.start/stop (ASG-based, current default in the UI) - wlk_service.start/stop (ECS service-based, legacy)

Both manipulate the same physical instance via different AWS APIs. Stopping one doesn't necessarily stop the other. If they get out of sync, the operator UI may show contradictory state. To resolve: stop both, then start the one you want.

This dual-control-plane situation is a known issue. Captured in CHANNEL_FEATURE_REVIEW.md for resolution as part of the consumer-app extraction.

Translation model and prompts

Current model

Amazon Nova Lite via Bedrock Converse Stream API. Region-specific model IDs:

Streaming output (token-by-token) is enabled, which lets the operator UI show typewriter-style live translation.

System prompts

Two prompts, one with conversation context, one without. Both are heavily templated — they include:

• Source and target language names (filled in per stream)
• Foreign-word-handling rules (transliterate proper nouns to katakana for JA target, etc.)
• Disfluency-handling rules: --, ..., …, repeated commas in the input are treated as speech-recognition hesitation artifacts, not punctuation, and are NOT preserved in the output (Japanese specifically: never use —— or …… to mirror them)
• False-start rules: when the speaker abandons a phrase and restarts ("you-- you can extract"), output ONLY the recovered version, never both
• The [SKIP] rule for fillers, garbled input, and refusals
• Emotion tagging instructions (EXCITED|, HAPPY|, ..., NEUTRAL|)
• Output format constraints

Edit prompts in Settings → Translation Model. Both prompts have a "Reset to default" button. Custom prompts persist to DynamoDB.

Be careful editing prompts. The post-processing logic depends on the prompt producing output in the expected format (EMOTION|text or [SKIP]). Breaking that contract leads to malformed translations. If you need radically different output, the right path is the tool-use refactor in TOOLUSE_REFACTOR.md, not prompt acrobatics.

The defaults are tuned for live speech where the speaker hesitates and corrects themselves frequently. If you customise the prompt, keep the disfluency / false-start sections — without them the translator faithfully renders WLK's -- artifacts as Japanese em-dashes, which reads as broken speech.

When prompts produce wrong output

Common failure modes:

Day-to-day workflow

Starting a session

1. (If using WLK) Start the GPU via Settings → GPU → Start. Wait ~2 minutes.
2. Verify the green status badge for STT.
3. Open the operator UI (/translate).
4. Confirm the language pair displayed in the header.
5. Click the mic button. Speak. Translations appear.

During a session

• Switch language: Settings → Language Pair. Takes effect on next utterance for the gRPC consumer; the WS path may need a reconnect.
• Tune chunking: Settings → Pipeline Tuning. Test the change with a short utterance before assuming it works for your event.
• Pause translation: There's a pause/resume button on the operator UI that stops feeding the pre-buffer. Useful for off-the-record discussion.

Ending a session

• Click stop on the operator UI (sends end=True to the WS / gRPC).
• (Optionally) stop the GPU if you don't need it for the next session: Settings → GPU → Stop.

The session record (segments, timing, language, etc.) persists in memory until the server restarts. There's no automatic cleanup; this is captured as a known issue and will move to hibiki-stage with proper persistence.

Troubleshooting

"Translations are slow / not appearing"

1. Check the STT engine is up. Settings → STT Status. If GPU is starting, wait. If "stopped", start it.
2. Check Nova credentials. The startup log (/health or container logs) will show "Nova translator: credentials validated" or an error.
3. Check pre-buffer thresholds. If you've set min_chars=200, the buffer is waiting for a long input before firing. Try reset-to-defaults.
4. Check enable_emotion and enable_diarization aren't set on a non-supporting backend. Diarization on Transcribe is silently no-op; emotion on a stripped prompt produces empty strings.

"Translations are wrong language"

1. Check tgt_lang in the request. The settings UI shows the current default but per-stream gRPC config overrides it.
2. Check the prompt hasn't been edited. The "output entirely in {tgt_lang}" rule is critical.

"GPU won't start"

1. Check GPU_ASG_NAME is set in env. Without it, the GPU manager is disabled and shows "unmanaged".
2. Check ASG capacity limits in AWS console. If max=0, the ASG won't launch instances.
3. Check service quota. New AWS accounts have low GPU instance quotas.

"Operator UI shows wrong STT engine"

This was a known bug fixed in June 2026: /translate/config was reading the env var instead of the runtime state. If you see this on a deploy older than that, redeploy.

"I changed a profile setting and nothing happened"

Profile changes apply on the next stream open. Existing streams continue with the old config. Reconnect to pick up changes.

Reference

• grpc-integration.md — for clients consuming the gRPC API
• api-reference.md — full proto and HTTP endpoint reference
• deployment.md — CDK deploy modes, infrastructure
• architecture.md — how the engine relates to hibiki-stage and hibiki-tts
• ../HIBIKI_STAGE_PLAN.md — the broader plan to extract the operator/audience UX into its own app
• ../CHANNEL_FEATURE_REVIEW.md — known issues being deferred to that extraction