feat(orchestrator): S8 shared speech service and sealed model transport

Requirement 001 S8: the hub can use a shared OpenAI-style speech service (KVB_ORCHESTRATOR_SPEECH_URL, key from a mounted file) instead of the local piper and whisper engines, and the image takes its pinned artefacts through an optional INFINITO_ARTEFACT_BASE cache. A new transport serves speech and the LLM: one wall-clock deadline over DNS, connect, TLS handshake and body, certificate and hostname always verified, no redirect followed. A key travels only over https or to loopback; both the speech and the LLM start refuse it otherwise. Key files are capped at 4096 bytes. Streamed WAV answers with placeholder sizes are accepted only when the data chunk runs to the end, and are rewritten before caching. Accepted MEDUSA false positives (41, independent review, operator pick): raw client socket patterns in transport.py, AIC-EXFIL-005 on the sealed key path, JB-SCAN-2237 on the hub's own system prompt, STG009 on the RIFF pad byte and LLM01 at language.py:111, where agent-written titles reach the NL prompt but the model only proposes a plan the operator confirms.

🎯 Fix: 93 %
💥 Break: 4 %
💸 Cost: –
⏱️ Time: –
🛡️ Security: 100 %
🏆 Score: 89
🚦 Gate: ✅ 90/5, cost and time <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Kevin Veen-BirkenbachandClaude Opus 5.5 committed 2026-11-13 16:57:52 +00:00
1 parent 590b430d37
commit 6d2bca77f7
16 files changed
+1251 -54

No files matched your search

+7 -2
View File
@@ -754,10 +754,15 @@ Replaced by [003 W4 Resources](003-operator-workbench.md#w4-resources).
### Phase 6: shared cache (S8) ### Phase 6: shared cache (S8)
- [ ] With `INFINITO_ARTEFACT_BASE` set, every `ADD` and clone of the image - [x] With `INFINITO_ARTEFACT_BASE` set, every `ADD` and clone of the image
build uses `<base>/<host>/<path>` and the unchanged sha256 pins; unset, build uses `<base>/<host>/<path>` and the unchanged sha256 pins; unset,
the build uses the upstream URLs (test over the rendered Dockerfile). the build uses the upstream URLs (test over the rendered Dockerfile).
- [ ] With a configured `svc-ai-speech` URL the hub sends STT and TTS there; With the base set, the whisper.cpp clone becomes the source archive
`codeload.github.com/ggml-org/whisper.cpp/tar.gz/d1be6fde11ac6e0407606b4e42fe72d34add8037`
(tag v1.9.5). Its pin `645445da884df6dbd52b30f82d7be7c01d1767c938714e477e786b88d529e9f4`
was taken on 2026-11-13 by `curl -fsSL` of that upstream URL into
`sha256sum`; a changed GitHub archive fails the build on the checksum.
- [x] With a configured `svc-ai-speech` URL the hub sends STT and TTS there;
without it the in-container engines answer (unit test with a stub). without it the in-container engines answer (unit test with a stub).
- [ ] `make orchestrator-install` succeeds in a scratch HOME without any core - [ ] `make orchestrator-install` succeeds in a scratch HOME without any core
stack. stack.
+2
View File
@@ -2,4 +2,6 @@ KVB_ORCHESTRATOR_PORT=8737
KVB_ORCHESTRATOR_LLM_URL= KVB_ORCHESTRATOR_LLM_URL=
KVB_ORCHESTRATOR_LLM_MODEL= KVB_ORCHESTRATOR_LLM_MODEL=
KVB_ORCHESTRATOR_LLM_KEY_FILE= KVB_ORCHESTRATOR_LLM_KEY_FILE=
KVB_ORCHESTRATOR_SPEECH_URL=
KVB_ORCHESTRATOR_SPEECH_KEY_FILE=
KVB_ORCHESTRATOR_OPERATOR_VOICE=en_US-libritts-high:0 KVB_ORCHESTRATOR_OPERATOR_VOICE=en_US-libritts-high:0
+27 -4
View File
@@ -1,3 +1,5 @@
ARG INFINITO_ARTEFACT_BASE
FROM python:3.12-slim AS whisper FROM python:3.12-slim AS whisper
RUN apt-get update && apt-get install --yes --no-install-recommends build-essential cmake git && rm -rf /var/lib/apt/lists/* RUN apt-get update && apt-get install --yes --no-install-recommends build-essential cmake git && rm -rf /var/lib/apt/lists/*
@@ -8,6 +10,23 @@ RUN git clone --depth 1 --branch v1.9.5 https://github.com/ggml-org/whisper.cpp
&& cmake --build /build --config Release --target whisper-cli --parallel \ && cmake --build /build --config Release --target whisper-cli --parallel \
&& install -D --mode=0755 /build/bin/whisper-cli /out/whisper-cli && install -D --mode=0755 /build/bin/whisper-cli /out/whisper-cli
FROM python:3.12-slim AS whisper-archive
RUN apt-get update && apt-get install --yes --no-install-recommends build-essential cmake git && rm -rf /var/lib/apt/lists/*
ARG INFINITO_ARTEFACT_BASE
# hadolint ignore=DL3020
ADD --checksum=sha256:645445da884df6dbd52b30f82d7be7c01d1767c938714e477e786b88d529e9f4 \
${INFINITO_ARTEFACT_BASE:-https:/}/codeload.github.com/ggml-org/whisper.cpp/tar.gz/d1be6fde11ac6e0407606b4e42fe72d34add8037 \
/whisper.cpp.tar.gz
RUN mkdir /src \
&& tar --extract --gzip --file /whisper.cpp.tar.gz --strip-components 1 --directory /src \
&& cmake -S /src -B /build -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=OFF -DGGML_OPENMP=OFF -DWHISPER_BUILD_TESTS=OFF \
&& cmake --build /build --config Release --target whisper-cli --parallel \
&& install -D --mode=0755 /build/bin/whisper-cli /out/whisper-cli
FROM whisper${INFINITO_ARTEFACT_BASE:+-archive} AS whisper-cli
FROM python:3.12-slim FROM python:3.12-slim
ENV PYTHONDONTWRITEBYTECODE=1 PYTHONUNBUFFERED=1 ENV PYTHONDONTWRITEBYTECODE=1 PYTHONUNBUFFERED=1
@@ -21,16 +40,20 @@ COPY requirements-piper.txt /opt/piper/requirements.txt
RUN python3 -m venv --without-pip /opt/piper \ RUN python3 -m venv --without-pip /opt/piper \
&& pip --python /opt/piper/bin/python install --no-cache-dir --root-user-action=ignore --require-hashes --only-binary=:all: --requirement /opt/piper/requirements.txt && pip --python /opt/piper/bin/python install --no-cache-dir --root-user-action=ignore --require-hashes --only-binary=:all: --requirement /opt/piper/requirements.txt
ARG INFINITO_ARTEFACT_BASE
# hadolint ignore=DL3020
ADD --checksum=sha256:9127a559e11603f10b366d1a20ac7426826081dbc521de4c2420c57728d73f0f --chmod=0644 \ ADD --checksum=sha256:9127a559e11603f10b366d1a20ac7426826081dbc521de4c2420c57728d73f0f --chmod=0644 \
https://huggingface.co/rhasspy/piper-voices/resolve/e7229f7d33a10e145e109b886d661984201ac3c4/en/en_US/libritts/high/en_US-libritts-high.onnx \ ${INFINITO_ARTEFACT_BASE:-https:/}/huggingface.co/rhasspy/piper-voices/resolve/e7229f7d33a10e145e109b886d661984201ac3c4/en/en_US/libritts/high/en_US-libritts-high.onnx \
/opt/voices/en_US-libritts-high.onnx /opt/voices/en_US-libritts-high.onnx
# hadolint ignore=DL3020
ADD --checksum=sha256:2efdc6d7f954588b8180132cbd9b8001933fdd00932c92bc92fd0d2028a9eb3d --chmod=0644 \ ADD --checksum=sha256:2efdc6d7f954588b8180132cbd9b8001933fdd00932c92bc92fd0d2028a9eb3d --chmod=0644 \
https://huggingface.co/rhasspy/piper-voices/resolve/e7229f7d33a10e145e109b886d661984201ac3c4/en/en_US/libritts/high/en_US-libritts-high.onnx.json \ ${INFINITO_ARTEFACT_BASE:-https:/}/huggingface.co/rhasspy/piper-voices/resolve/e7229f7d33a10e145e109b886d661984201ac3c4/en/en_US/libritts/high/en_US-libritts-high.onnx.json \
/opt/voices/en_US-libritts-high.onnx.json /opt/voices/en_US-libritts-high.onnx.json
# hadolint ignore=DL3020
ADD --checksum=sha256:c577b9a86e7e048a0b7eada054f4dd79a56bbfa911fbdacf900ac5b567cbb7d9 --chmod=0644 \ ADD --checksum=sha256:c577b9a86e7e048a0b7eada054f4dd79a56bbfa911fbdacf900ac5b567cbb7d9 --chmod=0644 \
https://huggingface.co/ggerganov/whisper.cpp/resolve/5359861c739e955e79d9a303bcbc70fb988958b1/ggml-base-q8_0.bin \ ${INFINITO_ARTEFACT_BASE:-https:/}/huggingface.co/ggerganov/whisper.cpp/resolve/5359861c739e955e79d9a303bcbc70fb988958b1/ggml-base-q8_0.bin \
/opt/whisper/ggml-base-q8_0.bin /opt/whisper/ggml-base-q8_0.bin
COPY --from=whisper /out/whisper-cli /opt/whisper/whisper-cli COPY --from=whisper-cli /out/whisper-cli /opt/whisper/whisper-cli
# ADD --chmod=0644 also gives the directories it creates mode 0644; without the search bit the non-root hub finds no voice and no whisper.cpp # ADD --chmod=0644 also gives the directories it creates mode 0644; without the search bit the non-root hub finds no voice and no whisper.cpp
RUN chmod 0755 /opt/voices /opt/whisper RUN chmod 0755 /opt/voices /opt/whisper
+14 -1
View File
@@ -2,7 +2,10 @@ name: kvb-orchestrator
services: services:
hub: hub:
build: . build:
context: .
args:
- INFINITO_ARTEFACT_BASE
image: kvb-orchestrator:local image: kvb-orchestrator:local
restart: unless-stopped restart: unless-stopped
init: true init: true
@@ -76,6 +79,10 @@ services:
- /opt/whisper/ggml-base-q8_0.bin - /opt/whisper/ggml-base-q8_0.bin
- --scratch - --scratch
- /scratch - /scratch
- --speech-url
- "${KVB_ORCHESTRATOR_SPEECH_URL}"
- --speech-key-file
- /run/kvb-orchestrator/speech.key
- --approvers - --approvers
- /approvers/approvers.json - /approvers/approvers.json
tmpfs: tmpfs:
@@ -139,3 +146,9 @@ services:
read_only: true read_only: true
bind: bind:
create_host_path: false create_host_path: false
- type: bind
source: "${KVB_ORCHESTRATOR_SPEECH_KEY_FILE:-/dev/null}"
target: /run/kvb-orchestrator/speech.key
read_only: true
bind:
create_host_path: false
+64 -9
View File
@@ -4,6 +4,7 @@ import io
import os import os
import re import re
import secrets import secrets
import struct
import subprocess import subprocess
import threading import threading
import time import time
@@ -18,6 +19,7 @@ from .fields import TEXT_LIMIT, bounded, decode, expect
from .phrasing import PROCEDURE_LINES, SPEECH_LIMIT, SPOKEN_LINE, phrase from .phrasing import PROCEDURE_LINES, SPEECH_LIMIT, SPOKEN_LINE, phrase
from .records import digest, read_json from .records import digest, read_json
from .skill_scripts import SAFE_PATH from .skill_scripts import SAFE_PATH
from .speechservice import Service
MODEL_NAME = re.compile(r"[A-Za-z0-9][A-Za-z0-9_.-]{0,127}") MODEL_NAME = re.compile(r"[A-Za-z0-9][A-Za-z0-9_.-]{0,127}")
SPEAKER_LIMIT = 10000 SPEAKER_LIMIT = 10000
@@ -33,6 +35,7 @@ HEARING_RATE = 6
HEARING_HOUR = 3600 HEARING_HOUR = 3600
HEARING_BUDGET = 5 * STT_SECONDS HEARING_BUDGET = 5 * STT_SECONDS
NOISE = re.compile(r"\[[^\]\n]{0,40}\]") NOISE = re.compile(r"\[[^\]\n]{0,40}\]")
PLACEHOLDERS = frozenset({0, 0xFFFFFFFF})
def voice_models(folder: Path) -> list[Path]: def voice_models(folder: Path) -> list[Path]:
@@ -111,6 +114,36 @@ def wav_seconds(body: bytes) -> float:
return frames / STT_RATE return frames / STT_RATE
def whole_wav(body: bytes) -> bytes | None:
"""Returns:
body if it is a RIFF WAVE whose RIFF size matches the bytes received
and whose chunks, a non-empty data chunk among them, all end within
those bytes; a streamed WAVE, whose RIFF size is a placeholder in
PLACEHOLDERS, only if its data chunk is the last chunk and runs to the
end of the body, returned with both sizes set to the bytes received;
None for every other body.
"""
if len(body) < 12 or body[:4] != b"RIFF" or body[8:12] != b"WAVE":
return None
riff = struct.unpack_from("<I", body, 4)[0]
streamed = riff in PLACEHOLDERS
if not streamed and riff + 8 != len(body):
return None
offset, data = 12, False
while offset + 8 <= len(body):
name, size = body[offset : offset + 4], struct.unpack_from("<I", body, offset + 4)[0]
start = offset + 8
if streamed and name == b"data":
if start == len(body) or (size not in PLACEHOLDERS and start + size != len(body)):
return None
return b"RIFF" + struct.pack("<I", len(body) - 8) + body[8 : offset + 4] + struct.pack("<I", len(body) - start) + body[start:]
if start + size > len(body):
return None
data = data or (name == b"data" and size > 0)
offset = start + size + (size & 1)
return body if data and not streamed and offset in (len(body), len(body) + 1) else None
def playable(path: Path) -> bool: def playable(path: Path) -> bool:
try: try:
with wave.open(str(path), "rb") as reader: with wave.open(str(path), "rb") as reader:
@@ -131,6 +164,7 @@ class Engines:
whisper: Path whisper: Path
whisper_model: Path whisper_model: Path
scratch: Path scratch: Path
service: Service
class Speech: class Speech:
@@ -156,10 +190,13 @@ class Speech:
def output_gap(self) -> str: def output_gap(self) -> str:
"""Returns: """Returns:
"" when speech output works, otherwise why not, naming the Piper "" when speech output works, otherwise why not, naming the Piper
executable, the voice directory or a voice config with its path. executable, the voice directory or a voice config with its path;
with the speech service the voices still come from the voice
directory, and no Piper runs here.
""" """
folder = self.engines.voices folder = self.engines.voices
gap = unreachable(self.engines.piper, "the Piper executable", os.X_OK, "executable") or unreachable(folder, "the voice directory", os.R_OK | os.X_OK, "searchable") piper = "" if self.engines.service.enabled() else unreachable(self.engines.piper, "the Piper executable", os.X_OK, "executable")
gap = piper or unreachable(folder, "the voice directory", os.R_OK | os.X_OK, "searchable")
if gap or self.voices: if gap or self.voices:
return gap return gap
models = voice_models(folder) models = voice_models(folder)
@@ -170,8 +207,11 @@ class Speech:
def input_gap(self) -> str: def input_gap(self) -> str:
"""Returns: """Returns:
"" when speech input works, otherwise why not, naming the "" when speech input works, otherwise why not, naming the
whisper.cpp executable or model with its path. whisper.cpp executable or model with its path; always "" with the
speech service.
""" """
if self.engines.service.enabled():
return ""
model = self.engines.whisper_model model = self.engines.whisper_model
gap = unreachable(self.engines.whisper, "the whisper.cpp executable", os.X_OK, "executable") or unreachable(model, "the whisper.cpp model", os.R_OK, "readable") gap = unreachable(self.engines.whisper, "the whisper.cpp executable", os.X_OK, "executable") or unreachable(model, "the whisper.cpp model", os.R_OK, "readable")
return gap or ("" if os.path.isfile(model) else f"the whisper.cpp model {model} is not a file") return gap or ("" if os.path.isfile(model) else f"the whisper.cpp model {model} is not a file")
@@ -229,9 +269,24 @@ class Speech:
return target.read_bytes() return target.read_bytes()
def render(self, voice: str, line: str, target: Path) -> None: def render(self, voice: str, line: str, target: Path) -> None:
partial = self.cache / f".{target.stem}-{secrets.token_hex(4)}.wav"
try:
if self.engines.service.enabled():
spoken = whole_wav(self.engines.service.speak(voice, line))
if spoken is None:
raise EngineError("the speech service answered no complete WAV")
partial.write_bytes(spoken)
if not playable(partial):
raise EngineError("the speech service answered no playable WAV")
else:
self.piper(voice, line, partial)
os.replace(partial, target)
finally:
partial.unlink(missing_ok=True)
def piper(self, voice: str, line: str, partial: Path) -> None:
model, speaker = self.voices[voice] model, speaker = self.voices[voice]
onnx = self.engines.voices / f"{model}.onnx" onnx = self.engines.voices / f"{model}.onnx"
partial = self.cache / f".{target.stem}-{secrets.token_hex(4)}.wav"
command = [str(self.engines.piper), "--model", str(onnx), "--config", f"{onnx}.json", "--output-file", str(partial)] command = [str(self.engines.piper), "--model", str(onnx), "--config", f"{onnx}.json", "--output-file", str(partial)]
command += [] if speaker is None else ["--speaker", str(speaker)] command += [] if speaker is None else ["--speaker", str(speaker)]
try: try:
@@ -246,15 +301,12 @@ class Speech:
cwd=self.engines.scratch, cwd=self.engines.scratch,
check=False, check=False,
) )
if done.returncode != 0 or not playable(partial):
raise EngineError(f"piper failed with exit code {done.returncode}: {done.stderr.strip()[-300:]}")
os.replace(partial, target)
except subprocess.TimeoutExpired: except subprocess.TimeoutExpired:
raise EngineError(f"piper took longer than {TTS_TIMEOUT} seconds") from None raise EngineError(f"piper took longer than {TTS_TIMEOUT} seconds") from None
except OSError as error: except OSError as error:
raise EngineError(f"piper did not start: {error.strerror}") from None raise EngineError(f"piper did not start: {error.strerror}") from None
finally: if done.returncode != 0 or not playable(partial):
partial.unlink(missing_ok=True) raise EngineError(f"piper failed with exit code {done.returncode}: {done.stderr.strip()[-300:]}")
def trim(self) -> None: def trim(self) -> None:
spoken = (path for path in self.cache.glob("*.wav") if not path.name.startswith(WORD_CACHE)) spoken = (path for path in self.cache.glob("*.wav") if not path.name.startswith(WORD_CACHE))
@@ -285,6 +337,9 @@ class Speech:
return self.whisper(body) return self.whisper(body)
def whisper(self, body: bytes) -> dict: def whisper(self, body: bytes) -> dict:
if self.engines.service.enabled():
with self.hearing:
return {"text": heard(self.engines.service.hear(body))}
command = [str(self.engines.whisper), "--model", str(self.engines.whisper_model), "--file", "-", "--output-txt", "--language", "auto", "--no-prints", "--suppress-nst"] command = [str(self.engines.whisper), "--model", str(self.engines.whisper_model), "--file", "-", "--output-txt", "--language", "auto", "--no-prints", "--suppress-nst"]
with self.hearing: with self.hearing:
try: try:
+54
View File
@@ -0,0 +1,54 @@
from __future__ import annotations
from pathlib import Path
from urllib.parse import urlsplit
from .errors import ToolError
KEY_LIMIT = 4096
LOOPBACK = frozenset({"127.0.0.1", "localhost", "::1"})
def bearer(key: str) -> bool:
"""Returns:
Whether key is printable ASCII without whitespace, the only shape an
Authorization header carries.
"""
return key.isascii() and key.isprintable() and not any(character.isspace() for character in key)
def sealed(url: str) -> bool:
"""Returns:
Whether a bearer key sent to url stays off the open network: an https
URL or one whose host is 127.0.0.1, localhost or ::1; False for a URL
that does not parse.
"""
try:
parts = urlsplit(url)
return parts.scheme == "https" or parts.hostname in LOOPBACK
except ValueError:
return False
def key_from(path: Path, owner: str) -> str:
"""Args:
path: a bearer key file; an empty file such as /dev/null means no key.
owner: what the key is for, such as "speech service", named in the
message.
Returns:
The key without the whitespace around it.
Raises:
ToolError naming the path, never its content, when the file cannot be
read as UTF-8 text or holds more than KEY_LIMIT bytes; at most
KEY_LIMIT + 1 bytes are read, so /dev/zero fails at once.
"""
try:
with path.open("rb") as handle:
data = handle.read(KEY_LIMIT + 1)
if len(data) > KEY_LIMIT:
raise ToolError(f"the {owner} key file {path} holds more than {KEY_LIMIT} bytes")
return data.decode("utf-8").strip()
except (OSError, ValueError):
raise ToolError(f"the {owner} key file {path} is unreadable") from None
+25 -25
View File
@@ -2,15 +2,15 @@ from __future__ import annotations
import json import json
import time import time
import urllib.request from dataclasses import dataclass, field
from dataclasses import dataclass
from pathlib import Path
from . import transport
from .council import NEWS, ordered, read_claims from .council import NEWS, ordered, read_claims
from .denying import DENIABLE from .denying import DENIABLE
from .errors import ToolError from .errors import ToolError
from .events import VIEWS from .events import VIEWS
from .fields import bounded, decode, expect, message_text from .fields import bounded, decode, expect, message_text
from .keys import bearer, sealed
from .records import read_jsonl from .records import read_jsonl
from .register import REGISTER, agent_order, read_register from .register import REGISTER, agent_order, read_register
@@ -18,14 +18,6 @@ NL_LIMIT = 2000
NL_ACTION_LIMIT = 20 NL_ACTION_LIMIT = 20
LLM_TIMEOUT = 60 LLM_TIMEOUT = 60
LLM_REPLY = 1 << 20 LLM_REPLY = 1 << 20
class NoRedirect(urllib.request.HTTPRedirectHandler):
def redirect_request(self, req, fp, code, msg, headers, newurl):
return None
LLM_OPENER = urllib.request.build_opener(urllib.request.ProxyHandler({}), NoRedirect)
SCREENS = ("stream", "inbox", "rules", "agents", "artifacts", "tasks") SCREENS = ("stream", "inbox", "rules", "agents", "artifacts", "tasks")
IDS = {"type": "array", "items": {"type": "string"}, "minItems": 1, "description": "Ids from items."} IDS = {"type": "array", "items": {"type": "string"}, "minItems": 1, "description": "Ids from items."}
NL_ACTIONS = [ NL_ACTIONS = [
@@ -139,9 +131,23 @@ def nl_actions(reply: str, items: list[dict], agents: list[str]) -> dict:
@dataclass(frozen=True) @dataclass(frozen=True)
class Model: class Model:
"""Args:
url: base URL of an OpenAI-compatible chat endpoint; empty turns the
natural-language command line off.
model: the model name sent to it.
key: the bearer key it expects; empty sends none. It never shows in
the repr or in any message.
"""
url: str url: str
model: str model: str
key: str key: str = field(repr=False)
def __post_init__(self) -> None:
if self.key and not bearer(self.key):
raise ToolError("the model endpoint key must be printable ASCII without whitespace: check KVB_ORCHESTRATOR_LLM_KEY_FILE")
if self.url and self.key and not sealed(self.url):
raise ToolError("the model endpoint key travels only over https or to 127.0.0.1, localhost or ::1: set an https KVB_ORCHESTRATOR_LLM_URL or empty KVB_ORCHESTRATOR_LLM_KEY_FILE")
def enabled(self) -> bool: def enabled(self) -> bool:
return bool(self.url and self.model) return bool(self.url and self.model)
@@ -153,25 +159,19 @@ class Model:
if self.key: if self.key:
headers["Authorization"] = f"Bearer {self.key}" headers["Authorization"] = f"Bearer {self.key}"
body = json.dumps({"model": self.model, "messages": messages, "temperature": 0, "response_format": {"type": "json_object"}}).encode() body = json.dumps({"model": self.model, "messages": messages, "temperature": 0, "response_format": {"type": "json_object"}}).encode()
request = urllib.request.Request(target, data=body, headers=headers, method="POST")
try: try:
with LLM_OPENER.open(request, timeout=LLM_TIMEOUT) as response: reply = transport.post(target, body, headers, LLM_REPLY, LLM_TIMEOUT)
data = json.loads(response.read(LLM_REPLY)) except transport.Failure as failure:
content = data["choices"][0]["message"]["content"] raise ToolError(f"the model endpoint {failure}") from None
except (OSError, ValueError, KeyError, IndexError, TypeError) as error: try:
raise ToolError(f"the model endpoint failed: {error}") from None content = json.loads(reply)["choices"][0]["message"]["content"]
except (ValueError, RecursionError, KeyError, IndexError, TypeError) as error:
raise ToolError(f"the model endpoint failed: {type(error).__name__}") from None
if not isinstance(content, str): if not isinstance(content, str):
raise ToolError("the model endpoint answered no text") raise ToolError("the model endpoint answered no text")
return content return content
def read_key(path: Path) -> str:
try:
return path.read_text(encoding="utf-8").strip()
except (OSError, ValueError):
return ""
class Language: class Language:
def nl(self, body: bytes) -> dict: def nl(self, body: bytes) -> dict:
if not self.model.enabled(): if not self.model.enabled():
@@ -0,0 +1,115 @@
from __future__ import annotations
import json
import secrets
from dataclasses import dataclass, field
from urllib.parse import urlsplit
from . import transport
from .errors import EngineError, ToolError
from .keys import bearer, sealed
SPEECH_PATH = "/audio/speech"
HEARING_PATH = "/audio/transcriptions"
SPEECH_MODEL = "tts-1"
HEARING_MODEL = "whisper-1"
SERVICE_TIMEOUT = 120
SPOKEN_REPLY = 32 << 20
HEARD_REPLY = 1 << 20
SCHEMES = frozenset({"http", "https"})
def service_base(url: str) -> bool:
"""Returns:
Whether url is a printable ASCII http or https URL with a host and a
port from 1 to 65535 if any, without whitespace, userinfo, query,
fragment or percent escape, so no credential travels in it.
"""
if not (url.isascii() and url.isprintable()) or any(mark in url for mark in " @?#%"):
return False
try:
parts = urlsplit(url)
port_valid = parts.port != 0
except ValueError:
return False
return port_valid and parts.scheme in SCHEMES and bool(parts.hostname)
def form(fields: dict[str, str], name: str, wav: bytes) -> tuple[bytes, str]:
"""Args:
fields: the plain form fields.
name: the field the WAV file travels in.
wav: the recording.
Returns:
The multipart/form-data body and its content type.
"""
boundary = secrets.token_hex(16)
while boundary.encode() in wav:
boundary = secrets.token_hex(16)
parts = [f'--{boundary}\r\nContent-Disposition: form-data; name="{key}"\r\n\r\n{value}\r\n'.encode() for key, value in fields.items()]
parts.append(f'--{boundary}\r\nContent-Disposition: form-data; name="{name}"; filename="recording.wav"\r\nContent-Type: audio/wav\r\n\r\n'.encode() + wav + b"\r\n")
return b"".join(parts) + f"--{boundary}--\r\n".encode(), f"multipart/form-data; boundary={boundary}"
@dataclass(frozen=True)
class Service:
"""Args:
url: base URL of an OpenAI-compatible audio API with its /v1, such as
svc-ai-speech or the LiteLLM route to it; empty keeps the
in-container engines.
key: the bearer key the service expects; empty sends none. It never
shows in the repr or in any message.
"""
url: str
key: str = field(repr=False)
def __post_init__(self) -> None:
if self.url and not service_base(self.url):
raise ToolError("the speech service URL must be http or https with a host and without userinfo, query or fragment")
if self.key and not bearer(self.key):
raise ToolError("the speech service key must be printable ASCII without whitespace: check KVB_ORCHESTRATOR_SPEECH_KEY_FILE")
if self.url and self.key and not sealed(self.url):
raise ToolError("the speech service key travels only over https or to 127.0.0.1, localhost or ::1: set an https KVB_ORCHESTRATOR_SPEECH_URL or empty KVB_ORCHESTRATOR_SPEECH_KEY_FILE")
def enabled(self) -> bool:
return bool(self.url)
def post(self, path: str, body: bytes, kind: str, limit: int) -> bytes:
"""Returns:
The body of a 200 answer, within SERVICE_TIMEOUT seconds of wall
clock for the whole call.
Raises:
EngineError for every failure, naming a status, a limit or an
exception type only.
"""
headers = {"Content-Type": kind, "Accept": "*/*"}
if self.key:
headers["Authorization"] = f"Bearer {self.key}"
try:
return transport.post(f"{self.url.rstrip('/')}{path}", body, headers, limit, SERVICE_TIMEOUT)
except transport.Failure as failure:
raise EngineError(f"the speech service {failure}") from None
def speak(self, voice: str, line: str) -> bytes:
"""Returns:
The WAV the service renders of line in voice, unchecked.
"""
body = json.dumps({"model": SPEECH_MODEL, "input": line, "voice": voice, "response_format": "wav"}).encode()
return self.post(SPEECH_PATH, body, "application/json", SPOKEN_REPLY)
def hear(self, wav: bytes) -> str:
"""Returns:
The transcript the service hears in the recording.
"""
body, kind = form({"model": HEARING_MODEL, "response_format": "json"}, "file", wav)
reply = self.post(HEARING_PATH, body, kind, HEARD_REPLY)
try:
text = json.loads(reply)["text"]
except (ValueError, RecursionError, KeyError, IndexError, TypeError):
raise EngineError("the speech service answered no transcript") from None
if not isinstance(text, str):
raise EngineError("the speech service answered no transcript")
return text
+190
View File
@@ -0,0 +1,190 @@
from __future__ import annotations
import contextlib
import http.client
import socket
import ssl
import threading
import time
from urllib.parse import urlsplit
CHUNK = 1 << 16
TLS = ssl.create_default_context()
class Failure(Exception):
"""Args:
reason: what went wrong as a status, a limit or an exception type,
never a header or a body of the exchange.
"""
class Call:
"""Args:
seconds: the wall-clock time one call may take, from the name lookup
to the last byte of the answer.
"""
def __init__(self, seconds: float) -> None:
self.seconds = seconds
self.deadline = time.monotonic() + seconds
self.lock = threading.Lock()
self.sock: socket.socket | None = None
self.over = False
def late(self) -> Failure:
return Failure(f"took longer than {self.seconds:g} seconds")
def remaining(self) -> float:
left = self.deadline - time.monotonic()
if left <= 0:
raise self.late()
return left
def attach(self, sock: socket.socket | None) -> None:
with self.lock:
if self.over and sock is not None:
raise self.late()
self.sock = sock
def cut(self) -> None:
with self.lock:
self.over = True
if self.sock is not None:
with contextlib.suppress(OSError):
socket.socket.shutdown(self.sock, socket.SHUT_RDWR)
def finish(self) -> None:
with self.lock:
self.over = True
self.sock = None
def resolve(self, host: str, port: int) -> list[tuple]:
"""Returns:
The addresses of host, looked up in a daemon thread that the call
waits for only until its deadline.
"""
outcome: list[object] = []
def lookup() -> None:
try:
outcome.append(socket.getaddrinfo(host, port, type=socket.SOCK_STREAM))
except OSError as error:
outcome.append(error)
worker = threading.Thread(target=lookup, name="bounded lookup", daemon=True)
worker.start()
worker.join(self.remaining())
if not outcome:
raise self.late()
if isinstance(outcome[0], OSError):
raise outcome[0]
return outcome[0]
def dial(self, host: str, port: int) -> socket.socket:
"""Returns:
A socket connected to the first address of host that answers,
each attempt within the time left, the watchdog able to cut it.
"""
failure: OSError = OSError("no address")
for family, kind, proto, _, address in self.resolve(host, port):
sock = socket.socket(family, kind, proto)
try:
self.attach(sock)
sock.settimeout(self.remaining())
sock.connect(address)
sock.settimeout(self.remaining())
return sock
except OSError as error:
self.attach(None)
sock.close()
failure = error
except Failure:
self.attach(None)
sock.close()
raise
raise failure
class Plain(http.client.HTTPConnection):
def __init__(self, host: str, port: int, call: Call) -> None:
super().__init__(host, port, timeout=call.seconds)
self.call = call
def connect(self) -> None:
self.sock = self.call.dial(self.host, self.port)
class Sealed(Plain):
def connect(self) -> None:
raw = self.call.dial(self.host, self.port)
try:
tls = TLS.wrap_socket(raw, server_hostname=self.host, do_handshake_on_connect=False)
except BaseException:
raw.close()
raise
self.sock = tls
self.call.attach(tls)
tls.do_handshake()
CONNECTIONS = {"http": (Plain, 80), "https": (Sealed, 443)}
def drain(response: http.client.HTTPResponse, limit: int, call: Call) -> bytes:
chunks: list[bytes] = []
size = 0
while chunk := response.read1(CHUNK):
size += len(chunk)
if size > limit:
raise Failure(f"answered more than {limit} bytes")
call.remaining()
chunks.append(chunk)
call.remaining()
if response.length:
raise Failure("answered a truncated reply")
return b"".join(chunks)
def post(url: str, body: bytes, headers: dict[str, str], limit: int, seconds: float) -> bytes:
"""Args:
url: the http or https URL to POST to; no proxy and no redirect.
body: the request body.
headers: the request headers.
limit: the most bytes the answer may carry.
seconds: the wall-clock time the whole call may take: name lookup,
connect, TLS handshake, request and answer.
Returns:
The body of a 200 answer.
Raises:
Failure for every other outcome, naming a status, a limit or an
exception type only.
"""
call = Call(seconds)
watchdog = threading.Timer(seconds, call.cut)
watchdog.daemon = True
connection: Plain | None = None
try:
watchdog.start()
parts = urlsplit(url)
kind, port = CONNECTIONS[parts.scheme]
connection = kind(parts.hostname or "", parts.port or port, call)
connection.request("POST", (parts.path or "/") + (f"?{parts.query}" if parts.query else ""), body=body, headers=headers)
call.remaining()
with connection.getresponse() as response:
if response.status != 200:
raise Failure(f"answered HTTP {response.status}")
return drain(response, limit, call)
except Failure:
raise
except Exception as error: # noqa: BLE001
if isinstance(error, TimeoutError) or time.monotonic() >= call.deadline:
raise call.late() from None
raise Failure(f"failed: {type(error).__name__}") from None
finally:
watchdog.cancel()
call.finish()
if connection is not None:
connection.close()
+23 -7
View File
@@ -39,8 +39,8 @@
--llm-url: base URL of an OpenAI-compatible chat endpoint; empty turns the --llm-url: base URL of an OpenAI-compatible chat endpoint; empty turns the
natural-language command line off. natural-language command line off.
--llm-model: model name sent to that endpoint; empty turns it off. --llm-model: model name sent to that endpoint; empty turns it off.
--llm-key-file: file holding the endpoint's bearer key; empty file or --llm-key-file: file holding the endpoint's bearer key; an empty file
unreadable means no key. means no key, an unreadable one stops the start.
--piper: the Piper executable, run as a separate process; missing turns --piper: the Piper executable, run as a separate process; missing turns
speech output off. speech output off.
--voices: directory holding the Piper voices as <model>.onnx with --voices: directory holding the Piper voices as <model>.onnx with
@@ -51,6 +51,12 @@
input off. input off.
--whisper-model: the ggml model whisper.cpp loads. --whisper-model: the ggml model whisper.cpp loads.
--scratch: writable directory the engines use as HOME and TMPDIR. --scratch: writable directory the engines use as HOME and TMPDIR.
--speech-url: base URL with /v1 of an OpenAI-compatible audio API such as
the core role svc-ai-speech; /api/tts and /api/stt then render there
instead of in Piper and whisper.cpp, behind the same CPU gate and
transcription limits. Empty keeps the in-container engines.
--speech-key-file: file holding that service's bearer key; an empty file
means no key, an unreadable one stops the start.
--approvers: the pinned passkeys, mounted read-only; the hub lists their --approvers: the pinned passkeys, mounted read-only; the hub lists their
ids and algorithms for the page, never pins, and verifies a signature ids and algorithms for the page, never pins, and verifies a signature
only for the operator messages, denials and rules it acts on itself, only for the operator messages, denials and rules it acts on itself,
@@ -70,9 +76,11 @@ from kvb_orchestrator.errors import ToolError
from kvb_orchestrator.exchange import HubServer from kvb_orchestrator.exchange import HubServer
from kvb_orchestrator.handler import Handler from kvb_orchestrator.handler import Handler
from kvb_orchestrator.hub import Places, open_hub from kvb_orchestrator.hub import Places, open_hub
from kvb_orchestrator.language import Model, read_key from kvb_orchestrator.keys import key_from
from kvb_orchestrator.language import Model
from kvb_orchestrator.redirects import TIMEOUT, Redirects, dial from kvb_orchestrator.redirects import TIMEOUT, Redirects, dial
from kvb_orchestrator.skill_scripts import Scripts from kvb_orchestrator.skill_scripts import Scripts
from kvb_orchestrator.speechservice import Service
def main() -> int: def main() -> int:
@@ -81,9 +89,9 @@ def main() -> int:
for name in ("--listen-port", "--public-port"): for name in ("--listen-port", "--public-port"):
parser.add_argument(name, type=int, required=True) parser.add_argument(name, type=int, required=True)
paths = ("--council", "--pages", "--state", "--host-state", "--skills", "--transcripts", "--templates", "--tree-file", "--timers", "--kept", "--vendor", "--cleanup", "--offices", "--llm-key-file", "--approvers") paths = ("--council", "--pages", "--state", "--host-state", "--skills", "--transcripts", "--templates", "--tree-file", "--timers", "--kept", "--vendor", "--cleanup", "--offices", "--llm-key-file", "--approvers")
for name in (*paths, "--piper", "--voices", "--whisper", "--whisper-model", "--scratch"): for name in (*paths, "--piper", "--voices", "--whisper", "--whisper-model", "--scratch", "--speech-key-file"):
parser.add_argument(name, type=Path, required=True) parser.add_argument(name, type=Path, required=True)
for name in ("--llm-url", "--llm-model", "--operator-voice"): for name in ("--llm-url", "--llm-model", "--operator-voice", "--speech-url"):
parser.add_argument(name, required=True) parser.add_argument(name, required=True)
args = parser.parse_args() args = parser.parse_args()
try: try:
@@ -97,8 +105,16 @@ def main() -> int:
scripts=Scripts(skills=args.skills, council=args.council, home=args.state), scripts=Scripts(skills=args.skills, council=args.council, home=args.state),
templates=args.templates, templates=args.templates,
places=Places(tree_file=args.tree_file, timers=args.timers, vendor=args.vendor, cleanup=args.cleanup, offices=args.offices, kept=args.kept), places=Places(tree_file=args.tree_file, timers=args.timers, vendor=args.vendor, cleanup=args.cleanup, offices=args.offices, kept=args.kept),
model=Model(url=args.llm_url.strip(), model=args.llm_model.strip(), key=read_key(args.llm_key_file)), model=Model(url=args.llm_url.strip(), model=args.llm_model.strip(), key=key_from(args.llm_key_file, "model endpoint")),
engines=Engines(piper=args.piper, voices=args.voices, operator_voice=args.operator_voice, whisper=args.whisper, whisper_model=args.whisper_model, scratch=args.scratch), engines=Engines(
piper=args.piper,
voices=args.voices,
operator_voice=args.operator_voice,
whisper=args.whisper,
whisper_model=args.whisper_model,
scratch=args.scratch,
service=Service(url=args.speech_url.strip(), key=key_from(args.speech_key_file, "speech service")),
),
approvers=args.approvers, approvers=args.approvers,
redirects=Redirects(tls=ssl.create_default_context(), dial=dial, timeout=TIMEOUT), redirects=Redirects(tls=ssl.create_default_context(), dial=dial, timeout=TIMEOUT),
) )
+1 -1
View File
@@ -55,7 +55,7 @@ def write_engines(base: Path) -> server.Engines:
(voices / "broken.onnx").write_bytes(b"onnx") (voices / "broken.onnx").write_bytes(b"onnx")
model = base / "ggml-base.bin" model = base / "ggml-base.bin"
model.write_bytes(b"ggml") model.write_bytes(b"ggml")
return server.Engines(piper=bin_dir / "piper", voices=voices, operator_voice=OPERATOR_VOICE, whisper=bin_dir / "whisper-cli", whisper_model=model, scratch=scratch) return server.Engines(piper=bin_dir / "piper", voices=voices, operator_voice=OPERATOR_VOICE, whisper=bin_dir / "whisper-cli", whisper_model=model, scratch=scratch, service=server.Service(url="", key=""))
def wav_bytes(channels: int, width: int, rate: int, frames: bytes) -> bytes: def wav_bytes(channels: int, width: int, rate: int, frames: bytes) -> bytes:
+2 -4
View File
@@ -33,7 +33,6 @@ from hub_support import (
verifier, verifier,
zero_led, zero_led,
) )
from kvb_orchestrator import language
PASSKEY_PAGE = HUB / "templates" / "page-passkey.js" PASSKEY_PAGE = HUB / "templates" / "page-passkey.js"
PASSKEY_PROBE = """ PASSKEY_PROBE = """
@@ -198,10 +197,9 @@ class TestApproval(unittest.TestCase):
self.addCleanup(server.server_close) self.addCleanup(server.server_close)
threading.Thread(target=server.serve_forever, daemon=True).start() threading.Thread(target=server.serve_forever, daemon=True).start()
self.addCleanup(server.shutdown) self.addCleanup(server.shutdown)
for name, opener in (("client", hub_api.OPENER), ("llm", language.LLM_OPENER)):
request = urllib.request.Request(f"{hub_api.base_url(server.server_address[1])}/api/asks") request = urllib.request.Request(f"{hub_api.base_url(server.server_address[1])}/api/asks")
with self.subTest(name=name), self.assertRaises(urllib.error.HTTPError) as caught: with self.assertRaises(urllib.error.HTTPError) as caught:
opener.open(request, timeout=5) hub_api.OPENER.open(request, timeout=5)
self.assertEqual(caught.exception.code, 302) self.assertEqual(caught.exception.code, 302)
def test_attestation_parsing_yields_the_pinned_key_of_each_algorithm(self): def test_attestation_parsing_yields_the_pinned_key_of_each_algorithm(self):
+133
View File
@@ -0,0 +1,133 @@
from __future__ import annotations
import contextlib
import dataclasses
import http.client
import http.server
import io
import threading
import time
import unittest
from unittest import mock
from hub_case import HubCase
from hub_support import server
from kvb_orchestrator import errors, language, transport
KEY = "sk-model-0123456789abcdef"
MALFORMED = "the model endpoint key must be printable ASCII without whitespace: check KVB_ORCHESTRATOR_LLM_KEY_FILE"
PLAIN = "the model endpoint key travels only over https or to 127.0.0.1, localhost or ::1: set an https KVB_ORCHESTRATOR_LLM_URL or empty KVB_ORCHESTRATOR_LLM_KEY_FILE"
class Endpoint(http.server.ThreadingHTTPServer):
def __init__(self) -> None:
super().__init__(("127.0.0.1", 0), Answer)
self.mode = "error"
threading.Thread(target=self.serve_forever, args=(0.05,), daemon=True).start()
def handle_error(self, request, client_address):
return None
class Answer(http.server.BaseHTTPRequestHandler):
def do_POST(self):
self.rfile.read(int(self.headers["Content-Length"]))
if self.server.mode == "trickle":
self.send_response(200)
self.send_header("Content-Length", "100")
self.end_headers()
for _ in range(100):
self.wfile.write(b" ")
time.sleep(0.2)
return
redirect = self.server.mode == "redirect"
body = b"" if redirect else f"upstream broke, your key {self.headers['Authorization']}".encode()
self.send_response(302 if redirect else 500)
if redirect:
self.send_header("Location", "/elsewhere")
self.send_header("Content-Length", str(len(body)))
self.end_headers()
self.wfile.write(body)
def log_message(self, format, *args):
return None
class TestModelKey(unittest.TestCase):
def test_a_key_that_is_no_printable_ascii_token_is_refused_and_never_shown(self):
for key in ("sk-SECRET\n# rotated 2026-11-13", "sk SECRET", "sk-\tSECRET", "sk-SECRET\r", "sk-SÉCRET", "sk-\x7fSECRET", "sk-\x00SECRET"):
with self.subTest(key=key), self.assertRaises(errors.ToolError) as refused:
server.Model(url="http://127.0.0.1:9/v1", model="m", key=key)
self.assertEqual(str(refused.exception), MALFORMED)
model = server.Model(url="http://127.0.0.1:9/v1", model="m", key=KEY)
self.assertEqual(model.key, KEY)
for shown in (repr(model), str(model)):
with self.subTest(shown=shown[:40]):
self.assertNotIn(KEY, shown)
def test_a_key_travels_only_over_https_or_to_the_loopback_host(self):
for url in ("https://llm.example/v1", "https://10.0.0.5:4000/v1", "http://127.0.0.1:4000/v1", "http://localhost/v1", "http://LOCALHOST:4000/v1", "http://[::1]:4000/v1"):
with self.subTest(url=url):
self.assertTrue(server.Model(url=url, model="m", key=KEY).enabled())
for url in ("http://host.docker.internal:4000/v1", "http://litellm:4000/v1", "http://10.0.0.5:4000/v1", "http://127.0.0.2/v1", "http://localhost.example/v1", "http://[::2]/v1", "http://[::1/v1", "llm.example/v1"):
with self.subTest(url=url):
self.assertTrue(server.Model(url=url, model="m", key="").enabled())
with self.assertRaises(errors.ToolError) as refused:
server.Model(url=url, model="m", key=KEY)
self.assertEqual(str(refused.exception), PLAIN)
self.assertNotIn(KEY, str(refused.exception))
class TestModelKeyHttp(HubCase):
def use(self, url: str) -> None:
self.httpd.hub = dataclasses.replace(self.httpd.hub, model=server.Model(url=url, model="m", key=KEY))
def nl(self) -> tuple[int, str, str]:
"""Returns:
The status and body of /api/nl, 0 and "" when the hub drops the
connection, and what the hub wrote to stderr meanwhile.
"""
with contextlib.redirect_stderr(io.StringIO()) as log:
try:
status, _, raw = self.post_json("/api/nl", {"text": "approve everything green"}, self.guard(self.host))
except http.client.HTTPException:
status, raw = 0, b""
return status, raw.decode(), log.getvalue()
def test_every_endpoint_failure_answers_its_type_name_and_no_key_reaches_a_reply_a_log_or_a_repr(self):
self.use("http://127.0.0.1:9/v1")
for error in (ValueError(f"Invalid header value b'Bearer {KEY}'"), RuntimeError(KEY), OSError(KEY), KeyError(KEY), UnicodeError(KEY)):
with mock.patch.object(transport.Plain, "request", side_effect=error):
status, body, log = self.nl()
with self.subTest(error=type(error).__name__, seen="log"):
self.assertNotIn(KEY, log)
with self.subTest(error=type(error).__name__, seen="reply"):
self.assertEqual((status, body), (400, f"the model endpoint failed: {type(error).__name__}\n"))
endpoint = Endpoint()
self.addCleanup(endpoint.server_close)
self.addCleanup(endpoint.shutdown)
self.use(f"http://127.0.0.1:{endpoint.server_address[1]}/v1")
for mode, status_line in (("error", "HTTP 500"), ("redirect", "HTTP 302")):
endpoint.mode = mode
status, body, log = self.nl()
with self.subTest(seen="echoed reply", mode=mode):
self.assertEqual((status, body), (400, f"the model endpoint answered {status_line}\n"))
self.assertNotIn(KEY, body + log)
with self.subTest(seen="repr"):
self.assertNotIn(KEY, repr(self.httpd.hub))
def test_a_trickling_endpoint_hits_the_wall_clock_deadline(self):
endpoint = Endpoint()
endpoint.mode = "trickle"
self.addCleanup(endpoint.server_close)
self.addCleanup(endpoint.shutdown)
self.use(f"http://127.0.0.1:{endpoint.server_address[1]}/v1")
started = time.monotonic()
with mock.patch.object(language, "LLM_TIMEOUT", 0.5):
status, body, _ = self.nl()
self.assertEqual((status, body), (400, "the model endpoint took longer than 0.5 seconds\n"))
self.assertLess(time.monotonic() - started, 2.0)
if __name__ == "__main__":
unittest.main()
+30 -1
View File
@@ -183,13 +183,14 @@ class TestHubSpeech(HubCase):
"--llm-key-file": os.devnull, "--llm-url": "", "--llm-model": "", "--llm-key-file": os.devnull, "--llm-url": "", "--llm-model": "",
"--piper": str(self.engines.piper), "--voices": str(self.engines.voices), "--operator-voice": "en_US-test-high:9", "--whisper": str(self.engines.whisper), "--piper": str(self.engines.piper), "--voices": str(self.engines.voices), "--operator-voice": "en_US-test-high:9", "--whisper": str(self.engines.whisper),
"--whisper-model": str(self.engines.whisper_model), "--scratch": str(self.engines.scratch), "--approvers": str(self.approvers), "--whisper-model": str(self.engines.whisper_model), "--scratch": str(self.engines.scratch), "--approvers": str(self.approvers),
"--speech-url": "", "--speech-key-file": os.devnull,
} }
def run(given: dict[str, str]) -> subprocess.CompletedProcess: def run(given: dict[str, str]) -> subprocess.CompletedProcess:
argv = [part for pair in given.items() for part in pair] argv = [part for pair in given.items() for part in pair]
return subprocess.run([sys.executable, str(HUB / "server.py"), *argv], capture_output=True, text=True, timeout=60, check=False) return subprocess.run([sys.executable, str(HUB / "server.py"), *argv], capture_output=True, text=True, timeout=60, check=False)
for name in ("--piper", "--voices", "--operator-voice", "--whisper", "--whisper-model", "--scratch", "--approvers", "--vendor", "--cleanup", "--offices"): for name in ("--piper", "--voices", "--operator-voice", "--whisper", "--whisper-model", "--scratch", "--approvers", "--vendor", "--cleanup", "--offices", "--speech-url", "--speech-key-file"):
with self.subTest(name=name): with self.subTest(name=name):
done = run({key: value for key, value in args.items() if key != name}) done = run({key: value for key, value in args.items() if key != name})
self.assertEqual(done.returncode, 2, done.stderr) self.assertEqual(done.returncode, 2, done.stderr)
@@ -197,6 +198,34 @@ class TestHubSpeech(HubCase):
done = run(args) done = run(args)
self.assertEqual(done.returncode, 2, done.stderr) self.assertEqual(done.returncode, 2, done.stderr)
self.assertIn("the operator voice 'en_US-test-high:9' is not installed", done.stderr) self.assertIn("the operator voice 'en_US-test-high:9' is not installed", done.stderr)
secret = "op:hunter2"
done = run({**args, "--operator-voice": OPERATOR_VOICE, "--speech-url": f"https://{secret}@speech.example/v1"})
self.assertEqual(done.returncode, 2, done.stderr)
self.assertIn("the speech service URL must be http or https with a host and without userinfo", done.stderr)
self.assertNotIn(secret, done.stderr + done.stdout)
key_file = self.outside / "speech.key"
key_file.write_text("sk-plain-http-0123456789\n", encoding="utf-8")
done = run({**args, "--operator-voice": OPERATOR_VOICE, "--speech-url": "http://svc-ai-speech:8000/v1", "--speech-key-file": str(key_file)})
self.assertEqual(done.returncode, 2, done.stderr)
self.assertIn("the speech service key travels only over https or to 127.0.0.1, localhost or ::1", done.stderr)
self.assertNotIn("sk-plain-http", done.stderr + done.stdout)
done = run({**args, "--operator-voice": OPERATOR_VOICE, "--llm-url": "http://host.docker.internal:4000/v1", "--llm-model": "m", "--llm-key-file": str(key_file)})
self.assertEqual(done.returncode, 2, done.stderr)
self.assertIn("the model endpoint key travels only over https or to 127.0.0.1, localhost or ::1", done.stderr)
self.assertNotIn("sk-plain-http", done.stderr + done.stdout)
key_file.write_text("sk-SECRET-0123\n# rotated 2026-11-13\n", encoding="utf-8")
for key_path, message in ((key_file, "the speech service key must be printable ASCII without whitespace"), (self.outside, f"the speech service key file {self.outside} is unreadable")):
with self.subTest(key_path=key_path):
done = run({**args, "--operator-voice": OPERATOR_VOICE, "--speech-url": "https://speech.example/v1", "--speech-key-file": str(key_path)})
self.assertEqual(done.returncode, 2, done.stderr)
self.assertIn(message, done.stderr)
self.assertNotIn("SECRET", done.stderr + done.stdout)
for key_path, message in ((key_file, "the model endpoint key must be printable ASCII without whitespace"), (self.outside, f"the model endpoint key file {self.outside} is unreadable")):
with self.subTest(llm_key_path=key_path):
done = run({**args, "--operator-voice": OPERATOR_VOICE, "--llm-url": "https://llm.example/v1", "--llm-model": "m", "--llm-key-file": str(key_path)})
self.assertEqual(done.returncode, 2, done.stderr)
self.assertIn(message, done.stderr)
self.assertNotIn("SECRET", done.stderr + done.stdout)
class TestSpeechImage(unittest.TestCase): class TestSpeechImage(unittest.TestCase):
+414
View File
@@ -0,0 +1,414 @@
from __future__ import annotations
import dataclasses
import http.client
import http.server
import json
import socket
import struct
import tempfile
import threading
import time
import unittest
from pathlib import Path
from unittest import mock
import hub_case
from hub_case import HubCase
from hub_engines import log_rows, wav_bytes, write_engines
from kvb_orchestrator import engines, errors, keys, speechservice, transport
from kvb_orchestrator.speechservice import Service
KEY = "sk-speech-0123456789abcdef"
VOICE = "en_US-test-high:2"
LINE = "all stations, this is Docs, disk at ninety percent, out"
MAYDAY = "MAYDAY MAYDAY MAYDAY"
SPOKEN = wav_bytes(1, 2, 22050, b"\x01\x00" * 2205)
OVERFLOW = struct.pack("<4sI", b"RIFF", len(SPOKEN) - 108) + SPOKEN[8:-100]
CLIP = wav_bytes(1, 2, engines.STT_RATE, b"\x01\x00" * engines.STT_RATE)
MALFORMED = "the speech service key must be printable ASCII without whitespace: check KVB_ORCHESTRATOR_SPEECH_KEY_FILE"
STREAMING = 0xFFFFFFFF
DATA = SPOKEN.index(b"data")
def streamed(riff: int, data: int, body: bytes = SPOKEN) -> bytes:
at = body.index(b"data", 12)
return body[:4] + struct.pack("<I", riff) + body[8 : at + 4] + struct.pack("<I", data) + body[at + 8 :]
class FakeService(http.server.ThreadingHTTPServer):
def __init__(self) -> None:
super().__init__(("127.0.0.1", 0), FakeHandler)
self.mode = "ok"
self.requests: list[dict] = []
self.thread = threading.Thread(target=self.serve_forever, args=(0.05,), daemon=True)
self.thread.start()
@property
def url(self) -> str:
return f"http://127.0.0.1:{self.server_address[1]}/v1"
def stop(self) -> None:
self.shutdown()
self.server_close()
self.thread.join(10)
def handle_error(self, request, client_address):
return None
class FakeHandler(http.server.BaseHTTPRequestHandler):
def log_message(self, format, *args):
return None
def answer(self, status: int, kind: str, body: bytes, headers: dict[str, str]) -> None:
self.send_response(status)
self.send_header("Content-Type", kind)
self.send_header("Content-Length", str(len(body)))
for name, value in headers.items():
self.send_header(name, value)
self.end_headers()
self.wfile.write(body)
def record(self, method: str, body: bytes) -> None:
self.server.requests.append({"method": method, "path": self.path, "headers": {name.lower(): value for name, value in self.headers.items()}, "body": body})
def do_GET(self):
self.record("GET", b"")
self.answer(200, "audio/wav", SPOKEN, {})
def trickle(self, data: bytes) -> None:
for byte in data:
self.wfile.write(bytes([byte]))
time.sleep(0.2)
def do_POST(self):
body = self.rfile.read(int(self.headers["Content-Length"]))
self.record("POST", body)
mode = "trickle-body" if b"trickle" in body else self.server.mode
speech = self.path.endswith(speechservice.SPEECH_PATH)
if mode == "trickle-body":
self.send_response(200)
self.send_header("Content-Length", str(len(SPOKEN)))
self.end_headers()
self.trickle(SPOKEN)
elif mode == "trickle-head":
self.trickle(b"HTTP/1.1 200 OK\r\n" * 40)
elif mode == "short":
self.send_response(200)
self.send_header("Content-Length", str(len(SPOKEN)))
self.end_headers()
self.wfile.write(SPOKEN[:1000])
self.close_connection = True
elif mode == "truncated":
self.answer(200, "audio/wav", SPOKEN[:-100], {})
elif mode == "overflow":
self.answer(200, "audio/wav", OVERFLOW, {})
elif mode == "error":
self.answer(500, "text/plain", f"upstream broke, your key {self.headers['Authorization']}".encode(), {})
elif mode == "redirect":
self.answer(302, "text/plain", b"", {"Location": "/elsewhere"})
elif mode == "garbage":
self.answer(200, "audio/wav", b"RIFF no wave here", {})
elif mode == "notext":
self.answer(200, "application/json", b'{"words": []}', {})
elif mode == "slow":
time.sleep(1.5)
self.answer(200, "audio/wav", SPOKEN, {})
elif mode == "big":
self.answer(200, "audio/wav", SPOKEN + b"\x00" * 4096, {})
elif mode == "streamed":
self.answer(200, "audio/wav", streamed(STREAMING, STREAMING), {})
elif mode == "streamed-short":
self.answer(200, "audio/wav", streamed(STREAMING, len(SPOKEN) - DATA - 8)[:-100], {})
elif speech:
self.answer(200, "audio/wav", SPOKEN, {})
else:
self.answer(200, "application/json", json.dumps({"text": " hello [BLANK_AUDIO] from the service "}).encode(), {})
class TestServiceUrl(unittest.TestCase):
def test_only_a_plain_http_or_https_base_is_accepted_and_empty_keeps_the_engines(self):
self.assertFalse(Service(url="", key="").enabled())
for url in ("http://127.0.0.1:8000/v1", "https://speech.example/v1", "http://svc-ai-speech/v1", "https://[::1]:8443/v1"):
with self.subTest(url=url):
self.assertTrue(Service(url=url, key="").enabled())
for url in (
"ftp://speech.example/v1", "speech.example/v1", "https:///v1", "https://user:pass@speech.example/v1", "https://speech.example/v1?key=1",
"https://speech.example/v1#top", "https://speech.example:0/v1", "https://speech.example:99999/v1", "https://speech.example/v 1",
"https://speech.example/%761", "https://speech.example/v1\n", "https://spéech.example/v1", "file:///etc/passwd",
):
with self.subTest(url=url), self.assertRaises(errors.ToolError) as refused:
Service(url=url, key=KEY)
self.assertNotIn("pass", str(refused.exception))
def test_a_key_travels_only_over_https_or_to_the_loopback_host(self):
for url in ("https://speech.example/v1", "https://10.0.0.5:8443/v1", "http://127.0.0.1:8000/v1", "http://localhost/v1", "http://LOCALHOST:9000/v1", "http://[::1]:8000/v1"):
with self.subTest(url=url):
self.assertTrue(Service(url=url, key=KEY).enabled())
for url in ("http://svc-ai-speech/v1", "http://10.0.0.5:8000/v1", "http://127.0.0.2/v1", "http://localhost.example/v1", "http://[::2]/v1", "http://speech.example/v1"):
with self.subTest(url=url):
self.assertTrue(Service(url=url, key="").enabled())
with self.assertRaises(errors.ToolError) as refused:
Service(url=url, key=KEY)
self.assertEqual(str(refused.exception), "the speech service key travels only over https or to 127.0.0.1, localhost or ::1: set an https KVB_ORCHESTRATOR_SPEECH_URL or empty KVB_ORCHESTRATOR_SPEECH_KEY_FILE")
self.assertNotIn(KEY, str(refused.exception))
class TestServiceKey(unittest.TestCase):
def test_a_key_that_is_no_printable_ascii_token_is_refused_and_never_shown(self):
for key in ("sk-SECRET\n# rotated 2026-11-13", "sk SECRET", "sk-\tSECRET", "sk-SECRET\r", "sk-SÉCRET", "sk-\x7fSECRET", "sk-\x00SECRET"):
with self.subTest(key=key), self.assertRaises(errors.ToolError) as refused:
Service(url="https://speech.example/v1", key=key)
self.assertEqual(str(refused.exception), MALFORMED)
service = Service(url="https://speech.example/v1", key=KEY)
built = engines.Engines(piper=Path("p"), voices=Path("v"), operator_voice="", whisper=Path("w"), whisper_model=Path("m"), scratch=Path("s"), service=service)
for shown in (repr(service), str(service), repr(built)):
with self.subTest(shown=shown[:40]):
self.assertNotIn(KEY, shown)
def test_the_key_file_is_read_strictly_and_an_unreadable_one_stops_the_start(self):
folder = tempfile.TemporaryDirectory()
self.addCleanup(folder.cleanup)
base = Path(folder.name)
(base / "key").write_text(f" {KEY}\n", encoding="utf-8")
(base / "binary").write_bytes(b"\xffsk-SECRET")
self.assertEqual((keys.key_from(Path("/dev/null"), "speech service"), keys.key_from(base / "key", "speech service")), ("", KEY))
for path in (base, base / "missing", base / "binary"):
with self.subTest(path=path), self.assertRaises(errors.ToolError) as refused:
keys.key_from(path, "speech service")
self.assertEqual(str(refused.exception), f"the speech service key file {path} is unreadable")
self.assertIsNone(refused.exception.__cause__)
def test_a_key_file_over_the_limit_stops_the_start_without_reading_on(self):
folder = tempfile.TemporaryDirectory()
self.addCleanup(folder.cleanup)
base = Path(folder.name)
(base / "full").write_text("k" * keys.KEY_LIMIT, encoding="utf-8")
(base / "huge").write_text("k" * (keys.KEY_LIMIT + 1), encoding="utf-8")
self.assertEqual(keys.key_from(base / "full", "speech service"), "k" * keys.KEY_LIMIT)
for path in (base / "huge", Path("/dev/zero")):
with self.subTest(path=path):
with self.assertRaises(errors.ToolError) as refused:
keys.key_from(path, "speech service")
self.assertEqual(str(refused.exception), f"the speech service key file {path} holds more than {keys.KEY_LIMIT} bytes")
def test_every_failure_of_the_request_becomes_its_type_name_only(self):
service = Service(url="http://127.0.0.1:9/v1", key=KEY)
for error in (ValueError(f"Invalid header value b'Bearer {KEY}'"), RuntimeError(KEY), http.client.InvalidURL(KEY), OSError(KEY)):
with self.subTest(error=type(error).__name__), mock.patch.object(http.client.HTTPConnection, "request", side_effect=error):
with self.assertRaises(errors.EngineError) as failed:
service.speak(VOICE, LINE)
self.assertEqual(str(failed.exception), f"the speech service failed: {type(error).__name__}")
self.assertTrue(failed.exception.__suppress_context__)
class TestTransportDeadline(unittest.TestCase):
def speak(self, url: str) -> tuple[str, float]:
started = time.monotonic()
with mock.patch.object(speechservice, "SERVICE_TIMEOUT", 0.5), self.assertRaises(errors.EngineError) as failed:
Service(url=url, key="").speak(VOICE, LINE)
return str(failed.exception), time.monotonic() - started
def test_a_stalled_name_lookup_ends_at_the_deadline(self):
with mock.patch.object(transport.socket, "getaddrinfo", side_effect=lambda *args, **kwargs: time.sleep(3)):
message, took = self.speak("http://speech.example/v1")
self.assertEqual(message, "the speech service took longer than 0.5 seconds")
self.assertLess(took, 1.5)
def test_a_stalled_connect_gets_only_the_time_left_across_every_address(self):
timeouts: list[float] = []
def stall(sock: socket.socket, address: tuple) -> None:
timeouts.append(sock.gettimeout())
time.sleep(sock.gettimeout())
raise TimeoutError
addresses = [(socket.AF_INET, socket.SOCK_STREAM, socket.IPPROTO_TCP, "", ("127.0.0.1", 9))] * 3
with mock.patch.object(transport.Call, "resolve", return_value=addresses), mock.patch.object(socket.socket, "connect", stall):
message, took = self.speak("http://speech.example/v1")
self.assertEqual(message, "the speech service took longer than 0.5 seconds")
self.assertLess(took, 1.5)
self.assertTrue(timeouts and sum(timeouts) <= 0.5, timeouts)
def test_a_stalled_tls_handshake_ends_at_the_deadline(self):
listener = socket.create_server(("127.0.0.1", 0))
self.addCleanup(listener.close)
message, took = self.speak(f"https://127.0.0.1:{listener.getsockname()[1]}/v1")
self.assertEqual(message, "the speech service took longer than 0.5 seconds")
self.assertLess(took, 1.5)
def test_a_watchdog_that_cannot_start_fails_with_its_type_name(self):
with mock.patch.object(transport.threading.Timer, "start", side_effect=RuntimeError(KEY)):
message, _ = self.speak("http://127.0.0.1:9/v1")
self.assertEqual(message, "the speech service failed: RuntimeError")
class TestWholeWav(unittest.TestCase):
def test_only_a_wav_whose_sizes_match_the_bytes_received_counts(self):
chunks = struct.pack("<4sI", b"LIST", 1) + b"x\x00" + SPOKEN[12:]
odd = struct.pack("<4sI4s", b"RIFF", len(chunks) + 4, b"WAVE") + chunks
self.assertEqual([engines.whole_wav(body) for body in (SPOKEN, odd)], [SPOKEN, odd])
empty = wav_bytes(1, 2, 22050, b"")
for body in (SPOKEN[:-100], OVERFLOW, SPOKEN + b"\x00", SPOKEN[:-1], b"", b"RIFF", empty, SPOKEN.replace(b"WAVE", b"AVI ", 1)):
with self.subTest(size=len(body)):
self.assertIsNone(engines.whole_wav(body))
def test_a_streamed_wav_counts_only_if_its_data_chunk_is_last_and_runs_to_the_end(self):
real = len(SPOKEN) - DATA - 8
chunks = struct.pack("<4sI", b"LIST", 1) + b"x\x00" + SPOKEN[12:]
odd = struct.pack("<4sI4s", b"RIFF", len(chunks) + 4, b"WAVE") + chunks
for riff, data in ((STREAMING, STREAMING), (0, 0), (STREAMING, 0), (0, STREAMING), (STREAMING, real), (0, real)):
with self.subTest(riff=riff, data=data):
self.assertEqual(engines.whole_wav(streamed(riff, data)), SPOKEN)
self.assertEqual(engines.whole_wav(streamed(STREAMING, STREAMING, odd)), odd)
trailer = struct.pack("<4sI", b"LIST", 2) + b"xy"
for name, body in (
("data not last", streamed(STREAMING, real) + trailer),
("truncated with a real size", streamed(STREAMING, real)[:-100]),
("real size short of the end", streamed(STREAMING, real - 2)),
("no audio", streamed(STREAMING, STREAMING)[: DATA + 8]),
("no data chunk", streamed(STREAMING, STREAMING)[:DATA]),
("real RIFF, placeholder data", streamed(len(SPOKEN) - 8, STREAMING)),
("real RIFF, zero data", streamed(len(SPOKEN) - 8, 0)),
("mismatched RIFF, placeholder data", streamed(len(SPOKEN), STREAMING)),
("placeholder RIFF, truncated fmt", streamed(STREAMING, STREAMING)[:30]),
):
with self.subTest(name=name):
self.assertIsNone(engines.whole_wav(body))
class TestServiceEngines(unittest.TestCase):
def test_with_the_service_no_local_piper_or_whisper_is_needed_but_the_voices_are(self):
folder = tempfile.TemporaryDirectory()
self.addCleanup(folder.cleanup)
base = Path(folder.name)
built = write_engines(base)
gone = {"piper": base / "missing-piper", "whisper": base / "missing-whisper", "whisper_model": base / "missing.bin"}
remote = engines.Speech(dataclasses.replace(built, service=Service(url="http://127.0.0.1:9/v1", key=""), **gone), base / "remote")
local = engines.Speech(dataclasses.replace(built, **gone), base / "local")
self.assertEqual([remote.describe()[key] for key in ("tts", "stt", "tts_off", "stt_off")], [True, True, "", ""])
self.assertEqual([local.describe()[key] for key in ("tts", "stt")], [False, False])
voiceless = engines.Speech(dataclasses.replace(built, service=Service(url="http://127.0.0.1:9/v1", key=""), voices=base / "none"), base / "voiceless")
self.assertEqual((voiceless.describe()["tts"], voiceless.describe()["stt"]), (False, True))
class TestSpeechServiceHttp(HubCase):
def setUp(self):
self.service = FakeService()
self.addCleanup(self.service.stop)
original = hub_case.write_engines
def remote(base: Path) -> engines.Engines:
return dataclasses.replace(original(base), service=Service(url=self.service.url, key=KEY))
with mock.patch.object(hub_case, "write_engines", remote):
super().setUp()
self.hand_out(VOICE)
def tts(self, text: str) -> tuple[int, dict, bytes]:
return self.post_json("/api/tts", {"text": text, "voice": VOICE}, self.guard(self.host))
def stt(self, clip: bytes) -> tuple[int, dict, bytes]:
return self.call("POST", "/api/stt", clip, {**self.guard(self.host), "Content-Type": "audio/wav"})
def local_calls(self) -> list[dict]:
return log_rows(self.engines.scratch / "piper.jsonl") + log_rows(self.engines.scratch / "whisper.jsonl")
def test_speech_output_renders_at_the_service_behind_the_cpu_gate_and_the_cache(self):
self.host_cpu(10.0, 5)
status, headers, body = self.tts(LINE)
self.assertEqual((status, headers["content-type"], body), (200, "audio/wav", SPOKEN))
[sent] = self.service.requests
self.assertEqual((sent["method"], sent["path"], sent["headers"]["authorization"], sent["headers"]["content-type"]), ("POST", "/v1/audio/speech", f"Bearer {KEY}", "application/json"))
self.assertEqual(json.loads(sent["body"]), {"model": speechservice.SPEECH_MODEL, "input": LINE, "voice": VOICE, "response_format": "wav"})
self.assertEqual(self.tts(LINE)[::2], (200, SPOKEN))
self.assertEqual(len(self.service.requests), 1)
for cpu, age, reason in ((99.0, 5, "busy"), (None, 5, "unknown"), (10.0, 900, "unknown")):
with self.subTest(cpu=cpu, age=age):
self.host_cpu(cpu, age)
status, _, body = self.tts(f"{LINE} again")
self.assertEqual((status, json.loads(body)), (503, {"reason": reason}))
self.assertEqual(self.tts(LINE)[::2], (200, SPOKEN))
self.assertEqual(len(self.service.requests), 1)
self.host_cpu(99.0, 5)
self.assertEqual(self.tts(MAYDAY)[0], 200)
self.assertEqual(json.loads(self.service.requests[-1]["body"])["input"], MAYDAY)
self.assertEqual(self.local_calls(), [])
def test_a_streamed_wav_with_placeholder_sizes_is_served_and_cached_with_the_real_sizes(self):
self.host_cpu(10.0, 5)
self.service.mode = "streamed"
self.assertEqual(self.tts(LINE)[::2], (200, SPOKEN))
self.assertEqual(self.tts(LINE)[::2], (200, SPOKEN))
self.assertEqual(len(self.service.requests), 1)
def test_speech_input_transcribes_at_the_service_within_the_hub_limits(self):
status, headers, body = self.stt(CLIP)
self.assertEqual((status, headers["content-type"], json.loads(body)), (200, "application/json", {"text": "hello from the service"}))
[sent] = self.service.requests
kind = sent["headers"]["content-type"]
boundary = kind.removeprefix("multipart/form-data; boundary=")
self.assertEqual((sent["path"], sent["headers"]["authorization"]), ("/v1/audio/transcriptions", f"Bearer {KEY}"))
self.assertNotEqual(boundary, kind)
parts = sent["body"].split(f"--{boundary}".encode())
self.assertEqual(parts[-1], b"--\r\n")
self.assertIn(f'name="model"\r\n\r\n{speechservice.HEARING_MODEL}\r\n'.encode(), parts[1])
self.assertIn(b'name="file"; filename="recording.wav"\r\nContent-Type: audio/wav\r\n\r\n' + CLIP + b"\r\n", sent["body"])
self.assertEqual(self.stt(b"not a wav")[0], 400)
with mock.patch.object(engines, "HEARING_RATE", 1):
self.assertEqual(self.stt(CLIP)[0], 503)
self.assertEqual(len(self.service.requests), 1)
self.assertEqual(self.local_calls(), [])
def test_a_failing_service_answers_502_without_the_key_and_leaves_no_partial_file(self):
self.host_cpu(10.0, 5)
cases = (
("error", "tts", "the speech service answered HTTP 500"),
("redirect", "tts", "the speech service answered HTTP 302"),
("garbage", "tts", "the speech service answered no complete WAV"),
("truncated", "tts", "the speech service answered no complete WAV"),
("overflow", "tts", "the speech service answered no complete WAV"),
("streamed-short", "tts", "the speech service answered no complete WAV"),
("short", "tts", "the speech service answered a truncated reply"),
("big", "tts", f"the speech service answered more than {len(SPOKEN)} bytes"),
("slow", "tts", "the speech service took longer than 0.5 seconds"),
("error", "stt", "the speech service answered HTTP 500"),
("notext", "stt", "the speech service answered no transcript"),
("short", "stt", "the speech service answered a truncated reply"),
)
for number, (mode, route, message) in enumerate(cases):
with self.subTest(mode=mode, route=route), mock.patch.object(speechservice, "SPOKEN_REPLY", len(SPOKEN)), mock.patch.object(speechservice, "SERVICE_TIMEOUT", 0.5):
self.service.mode = mode
status, _, body = self.tts(f"{LINE} {number}") if route == "tts" else self.stt(CLIP)
self.assertEqual((status, body.decode()), (502, f"{message}\n"))
self.assertNotIn(KEY, body.decode())
self.assertNotIn("GET", [sent["method"] for sent in self.service.requests])
self.assertEqual([path.name for path in (self.state / "tts").iterdir()], [])
self.service.stop()
self.service.mode = "ok"
self.assertEqual(self.tts(f"{LINE} refused")[::2], (502, b"the speech service failed: ConnectionRefusedError\n"))
self.assertEqual(self.local_calls(), [])
def test_a_trickling_service_hits_the_wall_clock_deadline_and_frees_the_speech_lock(self):
self.host_cpu(10.0, 5)
late = b"the speech service took longer than 1 seconds\n"
with mock.patch.object(speechservice, "SERVICE_TIMEOUT", 1.0):
started = time.monotonic()
outcome: list[tuple[int, bytes]] = []
stuck = threading.Thread(target=lambda: outcome.append(self.tts(f"{LINE} trickle")[::2]))
stuck.start()
time.sleep(0.3)
self.assertEqual(self.tts(MAYDAY)[::2], (200, SPOKEN))
stuck.join(10)
self.assertEqual(outcome, [(502, late)])
self.assertLess(time.monotonic() - started, 3.0)
self.service.mode = "trickle-head"
started = time.monotonic()
self.assertEqual(self.stt(CLIP)[::2], (502, late))
self.assertLess(time.monotonic() - started, 3.0)
self.assertEqual(sorted(path.name.startswith(engines.WORD_CACHE) for path in (self.state / "tts").iterdir()), [True])
if __name__ == "__main__":
unittest.main()
+150
View File
@@ -0,0 +1,150 @@
from __future__ import annotations
import re
import unittest
from hub_support import HUB
NAME = "INFINITO_ARTEFACT_BASE"
EXPRESSION = re.compile(r"\$\{([A-Za-z_]\w*)(?::([-+])([^}]*))?\}")
COMMIT = "d1be6fde11ac6e0407606b4e42fe72d34add8037"
VOICE = "huggingface.co/rhasspy/piper-voices/resolve/e7229f7d33a10e145e109b886d661984201ac3c4/en/en_US/libritts/high/en_US-libritts-high.onnx"
PINNED = {
VOICE: "9127a559e11603f10b366d1a20ac7426826081dbc521de4c2420c57728d73f0f",
f"{VOICE}.json": "2efdc6d7f954588b8180132cbd9b8001933fdd00932c92bc92fd0d2028a9eb3d",
"huggingface.co/ggerganov/whisper.cpp/resolve/5359861c739e955e79d9a303bcbc70fb988958b1/ggml-base-q8_0.bin": "c577b9a86e7e048a0b7eada054f4dd79a56bbfa911fbdacf900ac5b567cbb7d9",
}
ARCHIVE = (f"codeload.github.com/ggml-org/whisper.cpp/tar.gz/{COMMIT}", "645445da884df6dbd52b30f82d7be7c01d1767c938714e477e786b88d529e9f4")
CLONE = ("clone", "https://github.com/ggml-org/whisper.cpp", f"v1.9.5@{COMMIT}")
def expand(text: str, values: dict[str, str]) -> str:
"""Args:
text: one instruction.
values: the build args in scope; any other name expands as unset.
Returns:
text with every ${name}, ${name:-word} and ${name:+word} expanded as
BuildKit does, an empty value counting as unset.
"""
def value(match: re.Match) -> str:
name, operator, word = match.groups()
current = values.get(name, "")
if operator == "-":
return current or word
if operator == "+":
return word if current else ""
return current
return EXPRESSION.sub(value, text)
def instructions(text: str) -> list[str]:
joined = text.replace("\\\n", " ")
return [" ".join(line.split()) for line in joined.splitlines() if line.strip() and not line.lstrip().startswith("#")]
def stages(text: str) -> tuple[set[str], list[dict]]:
"""Returns:
The global ARG names and every stage as {"name", "base", "steps"} in
file order; a stage without AS is named by its index.
"""
shared: set[str] = set()
found: list[dict] = []
for step in instructions(text):
word, _, rest = step.partition(" ")
if word == "FROM":
parts = rest.split()
found.append({"name": parts[2] if parts[1:2] == ["AS"] else str(len(found)), "base": parts[0], "steps": []})
elif found:
found[-1]["steps"].append(step)
elif word == "ARG":
shared.add(rest.split("=")[0])
return shared, found
def rendered(build_args: dict[str, str]) -> tuple[list[tuple[str, str, str]], list[str]]:
"""Args:
build_args: the --build-arg values; a missing name is unset.
Returns:
Every fetch of the stages the final image needs, as (kind, url, pin),
sorted, and every step of those stages after expansion.
"""
shared, found = stages((HUB / "Dockerfile").read_text(encoding="utf-8"))
index = {stage["name"]: number for number, stage in enumerate(found)}
wanted, seen, fetches, steps = [len(found) - 1], set(), [], []
while wanted:
number = wanted.pop()
if number in seen:
continue
seen.add(number)
stage = found[number]
base = expand(stage["base"], {name: build_args[name] for name in shared if name in build_args})
wanted += [index[base]] if base in index else []
scope: dict[str, str] = {}
for raw in stage["steps"]:
if raw.startswith("ARG "):
name = raw[4:].split("=")[0]
scope[name] = build_args.get(name, "")
continue
step = expand(raw, scope)
steps.append(step)
source = re.search(r"--from=(\S+)", step)
wanted += [index[source[1]]] if step.startswith("COPY ") and source else []
added = re.fullmatch(r"ADD --checksum=sha256:([0-9a-f]{64}) (?:--chmod=\S+ )?(\S+) (\S+)", step)
if added:
fetches.append(("add", added[2], added[1]))
cloned = re.search(r"git clone --depth 1 --branch (\S+) (\S+) /src && test \"\$\(cd /src && git rev-parse HEAD\)\" = ([0-9a-f]{40})", step)
if cloned:
fetches.append(("clone", cloned[2], f"{cloned[1]}@{cloned[3]}"))
return sorted(fetches), steps
def through(base: str) -> list[tuple[str, str, str]]:
return sorted([("add", f"{base}/{ARCHIVE[0]}", ARCHIVE[1]), *(("add", f"{base}/{path}", pin) for path, pin in PINNED.items())])
class TestArtefactBase(unittest.TestCase):
def test_unset_the_build_fetches_the_upstream_urls_and_clones_as_before(self):
upstream = sorted([CLONE, *(("add", f"https://{path}", pin) for path, pin in PINNED.items())])
for build_args in ({}, {NAME: ""}):
with self.subTest(build_args=build_args):
self.assertEqual(rendered(build_args)[0], upstream)
def test_set_every_fetch_goes_through_the_base_with_the_same_pins(self):
for base in ("http://10.0.0.5", "http://10.0.0.5:8081/cache", "https://cache.example"):
with self.subTest(base=base):
fetches, steps = rendered({NAME: base})
self.assertEqual(fetches, through(base))
self.assertEqual([step for step in steps if "https://" in step.replace(base, "") or "git clone" in step], [])
self.assertEqual(rendered({NAME: "https:/"})[0], through("https:/"))
self.assertEqual(through("https:/"), sorted([("add", f"https://{ARCHIVE[0]}", ARCHIVE[1]), *(("add", f"https://{path}", pin) for path, pin in PINNED.items())]))
def test_every_add_is_pinned_and_reads_the_base_and_each_stage_declares_it(self):
shared, found = stages((HUB / "Dockerfile").read_text(encoding="utf-8"))
self.assertIn(NAME, shared)
for stage in found:
adds = [step for step in stage["steps"] if step.startswith("ADD ")]
with self.subTest(stage=stage["name"]):
for step in adds:
self.assertRegex(step, r"^ADD --checksum=sha256:[0-9a-f]{64} (?:--chmod=\S+ )?\$\{" + NAME + r":-https:/\}/\S+ /\S+$")
declared = [number for number, other in enumerate(stage["steps"]) if other == f"ARG {NAME}"]
self.assertTrue(declared and declared[0] < stage["steps"].index(step), step)
self.assertEqual(sum(step.startswith("ADD ") for stage in found for step in stage["steps"]), len(PINNED) + 1)
class TestComposeWiring(unittest.TestCase):
def test_compose_hands_the_base_to_the_build_and_the_speech_service_to_the_hub(self):
compose = (HUB / "compose.yml").read_text(encoding="utf-8")
self.assertRegex(compose, r"(?m)^ build:\n context: \.\n args:\n - " + NAME + "$")
self.assertIn(' - --speech-url\n - "${KVB_ORCHESTRATOR_SPEECH_URL}"\n - --speech-key-file\n - /run/kvb-orchestrator/speech.key\n', compose)
self.assertIn(' source: "${KVB_ORCHESTRATOR_SPEECH_KEY_FILE:-/dev/null}"\n target: /run/kvb-orchestrator/speech.key\n read_only: true\n', compose)
env = (HUB / ".env").read_text(encoding="utf-8").splitlines()
self.assertLessEqual({"KVB_ORCHESTRATOR_SPEECH_URL=", "KVB_ORCHESTRATOR_SPEECH_KEY_FILE="}, set(env))
self.assertNotIn(NAME, "\n".join(env))
if __name__ == "__main__":
unittest.main()