feat(orchestrator): S8 shared speech service and sealed model transport
Requirement 001 S8: the hub can use a shared OpenAI-style speech service (KVB_ORCHESTRATOR_SPEECH_URL, key from a mounted file) instead of the local piper and whisper engines, and the image takes its pinned artefacts through an optional INFINITO_ARTEFACT_BASE cache. A new transport serves speech and the LLM: one wall-clock deadline over DNS, connect, TLS handshake and body, certificate and hostname always verified, no redirect followed. A key travels only over https or to loopback; both the speech and the LLM start refuse it otherwise. Key files are capped at 4096 bytes. Streamed WAV answers with placeholder sizes are accepted only when the data chunk runs to the end, and are rewritten before caching. Accepted MEDUSA false positives (41, independent review, operator pick): raw client socket patterns in transport.py, AIC-EXFIL-005 on the sealed key path, JB-SCAN-2237 on the hub's own system prompt, STG009 on the RIFF pad byte and LLM01 at language.py:111, where agent-written titles reach the NL prompt but the model only proposes a plan the operator confirms. 🎯 Fix: 93 % 💥 Break: 4 % 💸 Cost: – ⏱️ Time: – 🛡️ Security: 100 % 🏆 Score: 89 🚦 Gate: ✅ 90/5, cost and time <= 100, security 100 % Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
590b430d37
commit
6d2bca77f7
16 files changed
+1251
-54
No files matched your search
@@ -754,10 +754,15 @@ Replaced by [003 W4 Resources](003-operator-workbench.md#w4-resources).
|
|||||||
|
|
||||||
### Phase 6: shared cache (S8)
|
### Phase 6: shared cache (S8)
|
||||||
|
|
||||||
- [ ] With `INFINITO_ARTEFACT_BASE` set, every `ADD` and clone of the image
|
- [x] With `INFINITO_ARTEFACT_BASE` set, every `ADD` and clone of the image
|
||||||
build uses `<base>/<host>/<path>` and the unchanged sha256 pins; unset,
|
build uses `<base>/<host>/<path>` and the unchanged sha256 pins; unset,
|
||||||
the build uses the upstream URLs (test over the rendered Dockerfile).
|
the build uses the upstream URLs (test over the rendered Dockerfile).
|
||||||
- [ ] With a configured `svc-ai-speech` URL the hub sends STT and TTS there;
|
With the base set, the whisper.cpp clone becomes the source archive
|
||||||
|
`codeload.github.com/ggml-org/whisper.cpp/tar.gz/d1be6fde11ac6e0407606b4e42fe72d34add8037`
|
||||||
|
(tag v1.9.5). Its pin `645445da884df6dbd52b30f82d7be7c01d1767c938714e477e786b88d529e9f4`
|
||||||
|
was taken on 2026-11-13 by `curl -fsSL` of that upstream URL into
|
||||||
|
`sha256sum`; a changed GitHub archive fails the build on the checksum.
|
||||||
|
- [x] With a configured `svc-ai-speech` URL the hub sends STT and TTS there;
|
||||||
without it the in-container engines answer (unit test with a stub).
|
without it the in-container engines answer (unit test with a stub).
|
||||||
- [ ] `make orchestrator-install` succeeds in a scratch HOME without any core
|
- [ ] `make orchestrator-install` succeeds in a scratch HOME without any core
|
||||||
stack.
|
stack.
|
||||||
|
|||||||
@@ -2,4 +2,6 @@ KVB_ORCHESTRATOR_PORT=8737
|
|||||||
KVB_ORCHESTRATOR_LLM_URL=
|
KVB_ORCHESTRATOR_LLM_URL=
|
||||||
KVB_ORCHESTRATOR_LLM_MODEL=
|
KVB_ORCHESTRATOR_LLM_MODEL=
|
||||||
KVB_ORCHESTRATOR_LLM_KEY_FILE=
|
KVB_ORCHESTRATOR_LLM_KEY_FILE=
|
||||||
|
KVB_ORCHESTRATOR_SPEECH_URL=
|
||||||
|
KVB_ORCHESTRATOR_SPEECH_KEY_FILE=
|
||||||
KVB_ORCHESTRATOR_OPERATOR_VOICE=en_US-libritts-high:0
|
KVB_ORCHESTRATOR_OPERATOR_VOICE=en_US-libritts-high:0
|
||||||
+27
-4
@@ -1,3 +1,5 @@
|
|||||||
|
ARG INFINITO_ARTEFACT_BASE
|
||||||
|
|
||||||
FROM python:3.12-slim AS whisper
|
FROM python:3.12-slim AS whisper
|
||||||
|
|
||||||
RUN apt-get update && apt-get install --yes --no-install-recommends build-essential cmake git && rm -rf /var/lib/apt/lists/*
|
RUN apt-get update && apt-get install --yes --no-install-recommends build-essential cmake git && rm -rf /var/lib/apt/lists/*
|
||||||
@@ -8,6 +10,23 @@ RUN git clone --depth 1 --branch v1.9.5 https://github.com/ggml-org/whisper.cpp
|
|||||||
&& cmake --build /build --config Release --target whisper-cli --parallel \
|
&& cmake --build /build --config Release --target whisper-cli --parallel \
|
||||||
&& install -D --mode=0755 /build/bin/whisper-cli /out/whisper-cli
|
&& install -D --mode=0755 /build/bin/whisper-cli /out/whisper-cli
|
||||||
|
|
||||||
|
FROM python:3.12-slim AS whisper-archive
|
||||||
|
|
||||||
|
RUN apt-get update && apt-get install --yes --no-install-recommends build-essential cmake git && rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
ARG INFINITO_ARTEFACT_BASE
|
||||||
|
# hadolint ignore=DL3020
|
||||||
|
ADD --checksum=sha256:645445da884df6dbd52b30f82d7be7c01d1767c938714e477e786b88d529e9f4 \
|
||||||
|
${INFINITO_ARTEFACT_BASE:-https:/}/codeload.github.com/ggml-org/whisper.cpp/tar.gz/d1be6fde11ac6e0407606b4e42fe72d34add8037 \
|
||||||
|
/whisper.cpp.tar.gz
|
||||||
|
RUN mkdir /src \
|
||||||
|
&& tar --extract --gzip --file /whisper.cpp.tar.gz --strip-components 1 --directory /src \
|
||||||
|
&& cmake -S /src -B /build -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=OFF -DGGML_OPENMP=OFF -DWHISPER_BUILD_TESTS=OFF \
|
||||||
|
&& cmake --build /build --config Release --target whisper-cli --parallel \
|
||||||
|
&& install -D --mode=0755 /build/bin/whisper-cli /out/whisper-cli
|
||||||
|
|
||||||
|
FROM whisper${INFINITO_ARTEFACT_BASE:+-archive} AS whisper-cli
|
||||||
|
|
||||||
FROM python:3.12-slim
|
FROM python:3.12-slim
|
||||||
|
|
||||||
ENV PYTHONDONTWRITEBYTECODE=1 PYTHONUNBUFFERED=1
|
ENV PYTHONDONTWRITEBYTECODE=1 PYTHONUNBUFFERED=1
|
||||||
@@ -21,16 +40,20 @@ COPY requirements-piper.txt /opt/piper/requirements.txt
|
|||||||
RUN python3 -m venv --without-pip /opt/piper \
|
RUN python3 -m venv --without-pip /opt/piper \
|
||||||
&& pip --python /opt/piper/bin/python install --no-cache-dir --root-user-action=ignore --require-hashes --only-binary=:all: --requirement /opt/piper/requirements.txt
|
&& pip --python /opt/piper/bin/python install --no-cache-dir --root-user-action=ignore --require-hashes --only-binary=:all: --requirement /opt/piper/requirements.txt
|
||||||
|
|
||||||
|
ARG INFINITO_ARTEFACT_BASE
|
||||||
|
# hadolint ignore=DL3020
|
||||||
ADD --checksum=sha256:9127a559e11603f10b366d1a20ac7426826081dbc521de4c2420c57728d73f0f --chmod=0644 \
|
ADD --checksum=sha256:9127a559e11603f10b366d1a20ac7426826081dbc521de4c2420c57728d73f0f --chmod=0644 \
|
||||||
https://huggingface.co/rhasspy/piper-voices/resolve/e7229f7d33a10e145e109b886d661984201ac3c4/en/en_US/libritts/high/en_US-libritts-high.onnx \
|
${INFINITO_ARTEFACT_BASE:-https:/}/huggingface.co/rhasspy/piper-voices/resolve/e7229f7d33a10e145e109b886d661984201ac3c4/en/en_US/libritts/high/en_US-libritts-high.onnx \
|
||||||
/opt/voices/en_US-libritts-high.onnx
|
/opt/voices/en_US-libritts-high.onnx
|
||||||
|
# hadolint ignore=DL3020
|
||||||
ADD --checksum=sha256:2efdc6d7f954588b8180132cbd9b8001933fdd00932c92bc92fd0d2028a9eb3d --chmod=0644 \
|
ADD --checksum=sha256:2efdc6d7f954588b8180132cbd9b8001933fdd00932c92bc92fd0d2028a9eb3d --chmod=0644 \
|
||||||
https://huggingface.co/rhasspy/piper-voices/resolve/e7229f7d33a10e145e109b886d661984201ac3c4/en/en_US/libritts/high/en_US-libritts-high.onnx.json \
|
${INFINITO_ARTEFACT_BASE:-https:/}/huggingface.co/rhasspy/piper-voices/resolve/e7229f7d33a10e145e109b886d661984201ac3c4/en/en_US/libritts/high/en_US-libritts-high.onnx.json \
|
||||||
/opt/voices/en_US-libritts-high.onnx.json
|
/opt/voices/en_US-libritts-high.onnx.json
|
||||||
|
# hadolint ignore=DL3020
|
||||||
ADD --checksum=sha256:c577b9a86e7e048a0b7eada054f4dd79a56bbfa911fbdacf900ac5b567cbb7d9 --chmod=0644 \
|
ADD --checksum=sha256:c577b9a86e7e048a0b7eada054f4dd79a56bbfa911fbdacf900ac5b567cbb7d9 --chmod=0644 \
|
||||||
https://huggingface.co/ggerganov/whisper.cpp/resolve/5359861c739e955e79d9a303bcbc70fb988958b1/ggml-base-q8_0.bin \
|
${INFINITO_ARTEFACT_BASE:-https:/}/huggingface.co/ggerganov/whisper.cpp/resolve/5359861c739e955e79d9a303bcbc70fb988958b1/ggml-base-q8_0.bin \
|
||||||
/opt/whisper/ggml-base-q8_0.bin
|
/opt/whisper/ggml-base-q8_0.bin
|
||||||
COPY --from=whisper /out/whisper-cli /opt/whisper/whisper-cli
|
COPY --from=whisper-cli /out/whisper-cli /opt/whisper/whisper-cli
|
||||||
# ADD --chmod=0644 also gives the directories it creates mode 0644; without the search bit the non-root hub finds no voice and no whisper.cpp
|
# ADD --chmod=0644 also gives the directories it creates mode 0644; without the search bit the non-root hub finds no voice and no whisper.cpp
|
||||||
RUN chmod 0755 /opt/voices /opt/whisper
|
RUN chmod 0755 /opt/voices /opt/whisper
|
||||||
|
|
||||||
|
|||||||
@@ -2,7 +2,10 @@ name: kvb-orchestrator
|
|||||||
|
|
||||||
services:
|
services:
|
||||||
hub:
|
hub:
|
||||||
build: .
|
build:
|
||||||
|
context: .
|
||||||
|
args:
|
||||||
|
- INFINITO_ARTEFACT_BASE
|
||||||
image: kvb-orchestrator:local
|
image: kvb-orchestrator:local
|
||||||
restart: unless-stopped
|
restart: unless-stopped
|
||||||
init: true
|
init: true
|
||||||
@@ -76,6 +79,10 @@ services:
|
|||||||
- /opt/whisper/ggml-base-q8_0.bin
|
- /opt/whisper/ggml-base-q8_0.bin
|
||||||
- --scratch
|
- --scratch
|
||||||
- /scratch
|
- /scratch
|
||||||
|
- --speech-url
|
||||||
|
- "${KVB_ORCHESTRATOR_SPEECH_URL}"
|
||||||
|
- --speech-key-file
|
||||||
|
- /run/kvb-orchestrator/speech.key
|
||||||
- --approvers
|
- --approvers
|
||||||
- /approvers/approvers.json
|
- /approvers/approvers.json
|
||||||
tmpfs:
|
tmpfs:
|
||||||
@@ -139,3 +146,9 @@ services:
|
|||||||
read_only: true
|
read_only: true
|
||||||
bind:
|
bind:
|
||||||
create_host_path: false
|
create_host_path: false
|
||||||
|
- type: bind
|
||||||
|
source: "${KVB_ORCHESTRATOR_SPEECH_KEY_FILE:-/dev/null}"
|
||||||
|
target: /run/kvb-orchestrator/speech.key
|
||||||
|
read_only: true
|
||||||
|
bind:
|
||||||
|
create_host_path: false
|
||||||
@@ -4,6 +4,7 @@ import io
|
|||||||
import os
|
import os
|
||||||
import re
|
import re
|
||||||
import secrets
|
import secrets
|
||||||
|
import struct
|
||||||
import subprocess
|
import subprocess
|
||||||
import threading
|
import threading
|
||||||
import time
|
import time
|
||||||
@@ -18,6 +19,7 @@ from .fields import TEXT_LIMIT, bounded, decode, expect
|
|||||||
from .phrasing import PROCEDURE_LINES, SPEECH_LIMIT, SPOKEN_LINE, phrase
|
from .phrasing import PROCEDURE_LINES, SPEECH_LIMIT, SPOKEN_LINE, phrase
|
||||||
from .records import digest, read_json
|
from .records import digest, read_json
|
||||||
from .skill_scripts import SAFE_PATH
|
from .skill_scripts import SAFE_PATH
|
||||||
|
from .speechservice import Service
|
||||||
|
|
||||||
MODEL_NAME = re.compile(r"[A-Za-z0-9][A-Za-z0-9_.-]{0,127}")
|
MODEL_NAME = re.compile(r"[A-Za-z0-9][A-Za-z0-9_.-]{0,127}")
|
||||||
SPEAKER_LIMIT = 10000
|
SPEAKER_LIMIT = 10000
|
||||||
@@ -33,6 +35,7 @@ HEARING_RATE = 6
|
|||||||
HEARING_HOUR = 3600
|
HEARING_HOUR = 3600
|
||||||
HEARING_BUDGET = 5 * STT_SECONDS
|
HEARING_BUDGET = 5 * STT_SECONDS
|
||||||
NOISE = re.compile(r"\[[^\]\n]{0,40}\]")
|
NOISE = re.compile(r"\[[^\]\n]{0,40}\]")
|
||||||
|
PLACEHOLDERS = frozenset({0, 0xFFFFFFFF})
|
||||||
|
|
||||||
|
|
||||||
def voice_models(folder: Path) -> list[Path]:
|
def voice_models(folder: Path) -> list[Path]:
|
||||||
@@ -111,6 +114,36 @@ def wav_seconds(body: bytes) -> float:
|
|||||||
return frames / STT_RATE
|
return frames / STT_RATE
|
||||||
|
|
||||||
|
|
||||||
|
def whole_wav(body: bytes) -> bytes | None:
|
||||||
|
"""Returns:
|
||||||
|
body if it is a RIFF WAVE whose RIFF size matches the bytes received
|
||||||
|
and whose chunks, a non-empty data chunk among them, all end within
|
||||||
|
those bytes; a streamed WAVE, whose RIFF size is a placeholder in
|
||||||
|
PLACEHOLDERS, only if its data chunk is the last chunk and runs to the
|
||||||
|
end of the body, returned with both sizes set to the bytes received;
|
||||||
|
None for every other body.
|
||||||
|
"""
|
||||||
|
if len(body) < 12 or body[:4] != b"RIFF" or body[8:12] != b"WAVE":
|
||||||
|
return None
|
||||||
|
riff = struct.unpack_from("<I", body, 4)[0]
|
||||||
|
streamed = riff in PLACEHOLDERS
|
||||||
|
if not streamed and riff + 8 != len(body):
|
||||||
|
return None
|
||||||
|
offset, data = 12, False
|
||||||
|
while offset + 8 <= len(body):
|
||||||
|
name, size = body[offset : offset + 4], struct.unpack_from("<I", body, offset + 4)[0]
|
||||||
|
start = offset + 8
|
||||||
|
if streamed and name == b"data":
|
||||||
|
if start == len(body) or (size not in PLACEHOLDERS and start + size != len(body)):
|
||||||
|
return None
|
||||||
|
return b"RIFF" + struct.pack("<I", len(body) - 8) + body[8 : offset + 4] + struct.pack("<I", len(body) - start) + body[start:]
|
||||||
|
if start + size > len(body):
|
||||||
|
return None
|
||||||
|
data = data or (name == b"data" and size > 0)
|
||||||
|
offset = start + size + (size & 1)
|
||||||
|
return body if data and not streamed and offset in (len(body), len(body) + 1) else None
|
||||||
|
|
||||||
|
|
||||||
def playable(path: Path) -> bool:
|
def playable(path: Path) -> bool:
|
||||||
try:
|
try:
|
||||||
with wave.open(str(path), "rb") as reader:
|
with wave.open(str(path), "rb") as reader:
|
||||||
@@ -131,6 +164,7 @@ class Engines:
|
|||||||
whisper: Path
|
whisper: Path
|
||||||
whisper_model: Path
|
whisper_model: Path
|
||||||
scratch: Path
|
scratch: Path
|
||||||
|
service: Service
|
||||||
|
|
||||||
|
|
||||||
class Speech:
|
class Speech:
|
||||||
@@ -156,10 +190,13 @@ class Speech:
|
|||||||
def output_gap(self) -> str:
|
def output_gap(self) -> str:
|
||||||
"""Returns:
|
"""Returns:
|
||||||
"" when speech output works, otherwise why not, naming the Piper
|
"" when speech output works, otherwise why not, naming the Piper
|
||||||
executable, the voice directory or a voice config with its path.
|
executable, the voice directory or a voice config with its path;
|
||||||
|
with the speech service the voices still come from the voice
|
||||||
|
directory, and no Piper runs here.
|
||||||
"""
|
"""
|
||||||
folder = self.engines.voices
|
folder = self.engines.voices
|
||||||
gap = unreachable(self.engines.piper, "the Piper executable", os.X_OK, "executable") or unreachable(folder, "the voice directory", os.R_OK | os.X_OK, "searchable")
|
piper = "" if self.engines.service.enabled() else unreachable(self.engines.piper, "the Piper executable", os.X_OK, "executable")
|
||||||
|
gap = piper or unreachable(folder, "the voice directory", os.R_OK | os.X_OK, "searchable")
|
||||||
if gap or self.voices:
|
if gap or self.voices:
|
||||||
return gap
|
return gap
|
||||||
models = voice_models(folder)
|
models = voice_models(folder)
|
||||||
@@ -170,8 +207,11 @@ class Speech:
|
|||||||
def input_gap(self) -> str:
|
def input_gap(self) -> str:
|
||||||
"""Returns:
|
"""Returns:
|
||||||
"" when speech input works, otherwise why not, naming the
|
"" when speech input works, otherwise why not, naming the
|
||||||
whisper.cpp executable or model with its path.
|
whisper.cpp executable or model with its path; always "" with the
|
||||||
|
speech service.
|
||||||
"""
|
"""
|
||||||
|
if self.engines.service.enabled():
|
||||||
|
return ""
|
||||||
model = self.engines.whisper_model
|
model = self.engines.whisper_model
|
||||||
gap = unreachable(self.engines.whisper, "the whisper.cpp executable", os.X_OK, "executable") or unreachable(model, "the whisper.cpp model", os.R_OK, "readable")
|
gap = unreachable(self.engines.whisper, "the whisper.cpp executable", os.X_OK, "executable") or unreachable(model, "the whisper.cpp model", os.R_OK, "readable")
|
||||||
return gap or ("" if os.path.isfile(model) else f"the whisper.cpp model {model} is not a file")
|
return gap or ("" if os.path.isfile(model) else f"the whisper.cpp model {model} is not a file")
|
||||||
@@ -229,9 +269,24 @@ class Speech:
|
|||||||
return target.read_bytes()
|
return target.read_bytes()
|
||||||
|
|
||||||
def render(self, voice: str, line: str, target: Path) -> None:
|
def render(self, voice: str, line: str, target: Path) -> None:
|
||||||
|
partial = self.cache / f".{target.stem}-{secrets.token_hex(4)}.wav"
|
||||||
|
try:
|
||||||
|
if self.engines.service.enabled():
|
||||||
|
spoken = whole_wav(self.engines.service.speak(voice, line))
|
||||||
|
if spoken is None:
|
||||||
|
raise EngineError("the speech service answered no complete WAV")
|
||||||
|
partial.write_bytes(spoken)
|
||||||
|
if not playable(partial):
|
||||||
|
raise EngineError("the speech service answered no playable WAV")
|
||||||
|
else:
|
||||||
|
self.piper(voice, line, partial)
|
||||||
|
os.replace(partial, target)
|
||||||
|
finally:
|
||||||
|
partial.unlink(missing_ok=True)
|
||||||
|
|
||||||
|
def piper(self, voice: str, line: str, partial: Path) -> None:
|
||||||
model, speaker = self.voices[voice]
|
model, speaker = self.voices[voice]
|
||||||
onnx = self.engines.voices / f"{model}.onnx"
|
onnx = self.engines.voices / f"{model}.onnx"
|
||||||
partial = self.cache / f".{target.stem}-{secrets.token_hex(4)}.wav"
|
|
||||||
command = [str(self.engines.piper), "--model", str(onnx), "--config", f"{onnx}.json", "--output-file", str(partial)]
|
command = [str(self.engines.piper), "--model", str(onnx), "--config", f"{onnx}.json", "--output-file", str(partial)]
|
||||||
command += [] if speaker is None else ["--speaker", str(speaker)]
|
command += [] if speaker is None else ["--speaker", str(speaker)]
|
||||||
try:
|
try:
|
||||||
@@ -246,15 +301,12 @@ class Speech:
|
|||||||
cwd=self.engines.scratch,
|
cwd=self.engines.scratch,
|
||||||
check=False,
|
check=False,
|
||||||
)
|
)
|
||||||
if done.returncode != 0 or not playable(partial):
|
|
||||||
raise EngineError(f"piper failed with exit code {done.returncode}: {done.stderr.strip()[-300:]}")
|
|
||||||
os.replace(partial, target)
|
|
||||||
except subprocess.TimeoutExpired:
|
except subprocess.TimeoutExpired:
|
||||||
raise EngineError(f"piper took longer than {TTS_TIMEOUT} seconds") from None
|
raise EngineError(f"piper took longer than {TTS_TIMEOUT} seconds") from None
|
||||||
except OSError as error:
|
except OSError as error:
|
||||||
raise EngineError(f"piper did not start: {error.strerror}") from None
|
raise EngineError(f"piper did not start: {error.strerror}") from None
|
||||||
finally:
|
if done.returncode != 0 or not playable(partial):
|
||||||
partial.unlink(missing_ok=True)
|
raise EngineError(f"piper failed with exit code {done.returncode}: {done.stderr.strip()[-300:]}")
|
||||||
|
|
||||||
def trim(self) -> None:
|
def trim(self) -> None:
|
||||||
spoken = (path for path in self.cache.glob("*.wav") if not path.name.startswith(WORD_CACHE))
|
spoken = (path for path in self.cache.glob("*.wav") if not path.name.startswith(WORD_CACHE))
|
||||||
@@ -285,6 +337,9 @@ class Speech:
|
|||||||
return self.whisper(body)
|
return self.whisper(body)
|
||||||
|
|
||||||
def whisper(self, body: bytes) -> dict:
|
def whisper(self, body: bytes) -> dict:
|
||||||
|
if self.engines.service.enabled():
|
||||||
|
with self.hearing:
|
||||||
|
return {"text": heard(self.engines.service.hear(body))}
|
||||||
command = [str(self.engines.whisper), "--model", str(self.engines.whisper_model), "--file", "-", "--output-txt", "--language", "auto", "--no-prints", "--suppress-nst"]
|
command = [str(self.engines.whisper), "--model", str(self.engines.whisper_model), "--file", "-", "--output-txt", "--language", "auto", "--no-prints", "--suppress-nst"]
|
||||||
with self.hearing:
|
with self.hearing:
|
||||||
try:
|
try:
|
||||||
|
|||||||
@@ -0,0 +1,54 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from pathlib import Path
|
||||||
|
from urllib.parse import urlsplit
|
||||||
|
|
||||||
|
from .errors import ToolError
|
||||||
|
|
||||||
|
KEY_LIMIT = 4096
|
||||||
|
LOOPBACK = frozenset({"127.0.0.1", "localhost", "::1"})
|
||||||
|
|
||||||
|
|
||||||
|
def bearer(key: str) -> bool:
|
||||||
|
"""Returns:
|
||||||
|
Whether key is printable ASCII without whitespace, the only shape an
|
||||||
|
Authorization header carries.
|
||||||
|
"""
|
||||||
|
return key.isascii() and key.isprintable() and not any(character.isspace() for character in key)
|
||||||
|
|
||||||
|
|
||||||
|
def sealed(url: str) -> bool:
|
||||||
|
"""Returns:
|
||||||
|
Whether a bearer key sent to url stays off the open network: an https
|
||||||
|
URL or one whose host is 127.0.0.1, localhost or ::1; False for a URL
|
||||||
|
that does not parse.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
parts = urlsplit(url)
|
||||||
|
return parts.scheme == "https" or parts.hostname in LOOPBACK
|
||||||
|
except ValueError:
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def key_from(path: Path, owner: str) -> str:
|
||||||
|
"""Args:
|
||||||
|
path: a bearer key file; an empty file such as /dev/null means no key.
|
||||||
|
owner: what the key is for, such as "speech service", named in the
|
||||||
|
message.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
The key without the whitespace around it.
|
||||||
|
|
||||||
|
Raises:
|
||||||
|
ToolError naming the path, never its content, when the file cannot be
|
||||||
|
read as UTF-8 text or holds more than KEY_LIMIT bytes; at most
|
||||||
|
KEY_LIMIT + 1 bytes are read, so /dev/zero fails at once.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
with path.open("rb") as handle:
|
||||||
|
data = handle.read(KEY_LIMIT + 1)
|
||||||
|
if len(data) > KEY_LIMIT:
|
||||||
|
raise ToolError(f"the {owner} key file {path} holds more than {KEY_LIMIT} bytes")
|
||||||
|
return data.decode("utf-8").strip()
|
||||||
|
except (OSError, ValueError):
|
||||||
|
raise ToolError(f"the {owner} key file {path} is unreadable") from None
|
||||||
@@ -2,15 +2,15 @@ from __future__ import annotations
|
|||||||
|
|
||||||
import json
|
import json
|
||||||
import time
|
import time
|
||||||
import urllib.request
|
from dataclasses import dataclass, field
|
||||||
from dataclasses import dataclass
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
|
from . import transport
|
||||||
from .council import NEWS, ordered, read_claims
|
from .council import NEWS, ordered, read_claims
|
||||||
from .denying import DENIABLE
|
from .denying import DENIABLE
|
||||||
from .errors import ToolError
|
from .errors import ToolError
|
||||||
from .events import VIEWS
|
from .events import VIEWS
|
||||||
from .fields import bounded, decode, expect, message_text
|
from .fields import bounded, decode, expect, message_text
|
||||||
|
from .keys import bearer, sealed
|
||||||
from .records import read_jsonl
|
from .records import read_jsonl
|
||||||
from .register import REGISTER, agent_order, read_register
|
from .register import REGISTER, agent_order, read_register
|
||||||
|
|
||||||
@@ -18,14 +18,6 @@ NL_LIMIT = 2000
|
|||||||
NL_ACTION_LIMIT = 20
|
NL_ACTION_LIMIT = 20
|
||||||
LLM_TIMEOUT = 60
|
LLM_TIMEOUT = 60
|
||||||
LLM_REPLY = 1 << 20
|
LLM_REPLY = 1 << 20
|
||||||
|
|
||||||
|
|
||||||
class NoRedirect(urllib.request.HTTPRedirectHandler):
|
|
||||||
def redirect_request(self, req, fp, code, msg, headers, newurl):
|
|
||||||
return None
|
|
||||||
|
|
||||||
|
|
||||||
LLM_OPENER = urllib.request.build_opener(urllib.request.ProxyHandler({}), NoRedirect)
|
|
||||||
SCREENS = ("stream", "inbox", "rules", "agents", "artifacts", "tasks")
|
SCREENS = ("stream", "inbox", "rules", "agents", "artifacts", "tasks")
|
||||||
IDS = {"type": "array", "items": {"type": "string"}, "minItems": 1, "description": "Ids from items."}
|
IDS = {"type": "array", "items": {"type": "string"}, "minItems": 1, "description": "Ids from items."}
|
||||||
NL_ACTIONS = [
|
NL_ACTIONS = [
|
||||||
@@ -139,9 +131,23 @@ def nl_actions(reply: str, items: list[dict], agents: list[str]) -> dict:
|
|||||||
|
|
||||||
@dataclass(frozen=True)
|
@dataclass(frozen=True)
|
||||||
class Model:
|
class Model:
|
||||||
|
"""Args:
|
||||||
|
url: base URL of an OpenAI-compatible chat endpoint; empty turns the
|
||||||
|
natural-language command line off.
|
||||||
|
model: the model name sent to it.
|
||||||
|
key: the bearer key it expects; empty sends none. It never shows in
|
||||||
|
the repr or in any message.
|
||||||
|
"""
|
||||||
|
|
||||||
url: str
|
url: str
|
||||||
model: str
|
model: str
|
||||||
key: str
|
key: str = field(repr=False)
|
||||||
|
|
||||||
|
def __post_init__(self) -> None:
|
||||||
|
if self.key and not bearer(self.key):
|
||||||
|
raise ToolError("the model endpoint key must be printable ASCII without whitespace: check KVB_ORCHESTRATOR_LLM_KEY_FILE")
|
||||||
|
if self.url and self.key and not sealed(self.url):
|
||||||
|
raise ToolError("the model endpoint key travels only over https or to 127.0.0.1, localhost or ::1: set an https KVB_ORCHESTRATOR_LLM_URL or empty KVB_ORCHESTRATOR_LLM_KEY_FILE")
|
||||||
|
|
||||||
def enabled(self) -> bool:
|
def enabled(self) -> bool:
|
||||||
return bool(self.url and self.model)
|
return bool(self.url and self.model)
|
||||||
@@ -153,25 +159,19 @@ class Model:
|
|||||||
if self.key:
|
if self.key:
|
||||||
headers["Authorization"] = f"Bearer {self.key}"
|
headers["Authorization"] = f"Bearer {self.key}"
|
||||||
body = json.dumps({"model": self.model, "messages": messages, "temperature": 0, "response_format": {"type": "json_object"}}).encode()
|
body = json.dumps({"model": self.model, "messages": messages, "temperature": 0, "response_format": {"type": "json_object"}}).encode()
|
||||||
request = urllib.request.Request(target, data=body, headers=headers, method="POST")
|
|
||||||
try:
|
try:
|
||||||
with LLM_OPENER.open(request, timeout=LLM_TIMEOUT) as response:
|
reply = transport.post(target, body, headers, LLM_REPLY, LLM_TIMEOUT)
|
||||||
data = json.loads(response.read(LLM_REPLY))
|
except transport.Failure as failure:
|
||||||
content = data["choices"][0]["message"]["content"]
|
raise ToolError(f"the model endpoint {failure}") from None
|
||||||
except (OSError, ValueError, KeyError, IndexError, TypeError) as error:
|
try:
|
||||||
raise ToolError(f"the model endpoint failed: {error}") from None
|
content = json.loads(reply)["choices"][0]["message"]["content"]
|
||||||
|
except (ValueError, RecursionError, KeyError, IndexError, TypeError) as error:
|
||||||
|
raise ToolError(f"the model endpoint failed: {type(error).__name__}") from None
|
||||||
if not isinstance(content, str):
|
if not isinstance(content, str):
|
||||||
raise ToolError("the model endpoint answered no text")
|
raise ToolError("the model endpoint answered no text")
|
||||||
return content
|
return content
|
||||||
|
|
||||||
|
|
||||||
def read_key(path: Path) -> str:
|
|
||||||
try:
|
|
||||||
return path.read_text(encoding="utf-8").strip()
|
|
||||||
except (OSError, ValueError):
|
|
||||||
return ""
|
|
||||||
|
|
||||||
|
|
||||||
class Language:
|
class Language:
|
||||||
def nl(self, body: bytes) -> dict:
|
def nl(self, body: bytes) -> dict:
|
||||||
if not self.model.enabled():
|
if not self.model.enabled():
|
||||||
|
|||||||
@@ -0,0 +1,115 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import secrets
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from urllib.parse import urlsplit
|
||||||
|
|
||||||
|
from . import transport
|
||||||
|
from .errors import EngineError, ToolError
|
||||||
|
from .keys import bearer, sealed
|
||||||
|
|
||||||
|
SPEECH_PATH = "/audio/speech"
|
||||||
|
HEARING_PATH = "/audio/transcriptions"
|
||||||
|
SPEECH_MODEL = "tts-1"
|
||||||
|
HEARING_MODEL = "whisper-1"
|
||||||
|
SERVICE_TIMEOUT = 120
|
||||||
|
SPOKEN_REPLY = 32 << 20
|
||||||
|
HEARD_REPLY = 1 << 20
|
||||||
|
SCHEMES = frozenset({"http", "https"})
|
||||||
|
|
||||||
|
|
||||||
|
def service_base(url: str) -> bool:
|
||||||
|
"""Returns:
|
||||||
|
Whether url is a printable ASCII http or https URL with a host and a
|
||||||
|
port from 1 to 65535 if any, without whitespace, userinfo, query,
|
||||||
|
fragment or percent escape, so no credential travels in it.
|
||||||
|
"""
|
||||||
|
if not (url.isascii() and url.isprintable()) or any(mark in url for mark in " @?#%"):
|
||||||
|
return False
|
||||||
|
try:
|
||||||
|
parts = urlsplit(url)
|
||||||
|
port_valid = parts.port != 0
|
||||||
|
except ValueError:
|
||||||
|
return False
|
||||||
|
return port_valid and parts.scheme in SCHEMES and bool(parts.hostname)
|
||||||
|
|
||||||
|
|
||||||
|
def form(fields: dict[str, str], name: str, wav: bytes) -> tuple[bytes, str]:
|
||||||
|
"""Args:
|
||||||
|
fields: the plain form fields.
|
||||||
|
name: the field the WAV file travels in.
|
||||||
|
wav: the recording.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
The multipart/form-data body and its content type.
|
||||||
|
"""
|
||||||
|
boundary = secrets.token_hex(16)
|
||||||
|
while boundary.encode() in wav:
|
||||||
|
boundary = secrets.token_hex(16)
|
||||||
|
parts = [f'--{boundary}\r\nContent-Disposition: form-data; name="{key}"\r\n\r\n{value}\r\n'.encode() for key, value in fields.items()]
|
||||||
|
parts.append(f'--{boundary}\r\nContent-Disposition: form-data; name="{name}"; filename="recording.wav"\r\nContent-Type: audio/wav\r\n\r\n'.encode() + wav + b"\r\n")
|
||||||
|
return b"".join(parts) + f"--{boundary}--\r\n".encode(), f"multipart/form-data; boundary={boundary}"
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class Service:
|
||||||
|
"""Args:
|
||||||
|
url: base URL of an OpenAI-compatible audio API with its /v1, such as
|
||||||
|
svc-ai-speech or the LiteLLM route to it; empty keeps the
|
||||||
|
in-container engines.
|
||||||
|
key: the bearer key the service expects; empty sends none. It never
|
||||||
|
shows in the repr or in any message.
|
||||||
|
"""
|
||||||
|
|
||||||
|
url: str
|
||||||
|
key: str = field(repr=False)
|
||||||
|
|
||||||
|
def __post_init__(self) -> None:
|
||||||
|
if self.url and not service_base(self.url):
|
||||||
|
raise ToolError("the speech service URL must be http or https with a host and without userinfo, query or fragment")
|
||||||
|
if self.key and not bearer(self.key):
|
||||||
|
raise ToolError("the speech service key must be printable ASCII without whitespace: check KVB_ORCHESTRATOR_SPEECH_KEY_FILE")
|
||||||
|
if self.url and self.key and not sealed(self.url):
|
||||||
|
raise ToolError("the speech service key travels only over https or to 127.0.0.1, localhost or ::1: set an https KVB_ORCHESTRATOR_SPEECH_URL or empty KVB_ORCHESTRATOR_SPEECH_KEY_FILE")
|
||||||
|
|
||||||
|
def enabled(self) -> bool:
|
||||||
|
return bool(self.url)
|
||||||
|
|
||||||
|
def post(self, path: str, body: bytes, kind: str, limit: int) -> bytes:
|
||||||
|
"""Returns:
|
||||||
|
The body of a 200 answer, within SERVICE_TIMEOUT seconds of wall
|
||||||
|
clock for the whole call.
|
||||||
|
|
||||||
|
Raises:
|
||||||
|
EngineError for every failure, naming a status, a limit or an
|
||||||
|
exception type only.
|
||||||
|
"""
|
||||||
|
headers = {"Content-Type": kind, "Accept": "*/*"}
|
||||||
|
if self.key:
|
||||||
|
headers["Authorization"] = f"Bearer {self.key}"
|
||||||
|
try:
|
||||||
|
return transport.post(f"{self.url.rstrip('/')}{path}", body, headers, limit, SERVICE_TIMEOUT)
|
||||||
|
except transport.Failure as failure:
|
||||||
|
raise EngineError(f"the speech service {failure}") from None
|
||||||
|
|
||||||
|
def speak(self, voice: str, line: str) -> bytes:
|
||||||
|
"""Returns:
|
||||||
|
The WAV the service renders of line in voice, unchecked.
|
||||||
|
"""
|
||||||
|
body = json.dumps({"model": SPEECH_MODEL, "input": line, "voice": voice, "response_format": "wav"}).encode()
|
||||||
|
return self.post(SPEECH_PATH, body, "application/json", SPOKEN_REPLY)
|
||||||
|
|
||||||
|
def hear(self, wav: bytes) -> str:
|
||||||
|
"""Returns:
|
||||||
|
The transcript the service hears in the recording.
|
||||||
|
"""
|
||||||
|
body, kind = form({"model": HEARING_MODEL, "response_format": "json"}, "file", wav)
|
||||||
|
reply = self.post(HEARING_PATH, body, kind, HEARD_REPLY)
|
||||||
|
try:
|
||||||
|
text = json.loads(reply)["text"]
|
||||||
|
except (ValueError, RecursionError, KeyError, IndexError, TypeError):
|
||||||
|
raise EngineError("the speech service answered no transcript") from None
|
||||||
|
if not isinstance(text, str):
|
||||||
|
raise EngineError("the speech service answered no transcript")
|
||||||
|
return text
|
||||||
@@ -0,0 +1,190 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import contextlib
|
||||||
|
import http.client
|
||||||
|
import socket
|
||||||
|
import ssl
|
||||||
|
import threading
|
||||||
|
import time
|
||||||
|
from urllib.parse import urlsplit
|
||||||
|
|
||||||
|
CHUNK = 1 << 16
|
||||||
|
TLS = ssl.create_default_context()
|
||||||
|
|
||||||
|
|
||||||
|
class Failure(Exception):
|
||||||
|
"""Args:
|
||||||
|
reason: what went wrong as a status, a limit or an exception type,
|
||||||
|
never a header or a body of the exchange.
|
||||||
|
"""
|
||||||
|
|
||||||
|
|
||||||
|
class Call:
|
||||||
|
"""Args:
|
||||||
|
seconds: the wall-clock time one call may take, from the name lookup
|
||||||
|
to the last byte of the answer.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self, seconds: float) -> None:
|
||||||
|
self.seconds = seconds
|
||||||
|
self.deadline = time.monotonic() + seconds
|
||||||
|
self.lock = threading.Lock()
|
||||||
|
self.sock: socket.socket | None = None
|
||||||
|
self.over = False
|
||||||
|
|
||||||
|
def late(self) -> Failure:
|
||||||
|
return Failure(f"took longer than {self.seconds:g} seconds")
|
||||||
|
|
||||||
|
def remaining(self) -> float:
|
||||||
|
left = self.deadline - time.monotonic()
|
||||||
|
if left <= 0:
|
||||||
|
raise self.late()
|
||||||
|
return left
|
||||||
|
|
||||||
|
def attach(self, sock: socket.socket | None) -> None:
|
||||||
|
with self.lock:
|
||||||
|
if self.over and sock is not None:
|
||||||
|
raise self.late()
|
||||||
|
self.sock = sock
|
||||||
|
|
||||||
|
def cut(self) -> None:
|
||||||
|
with self.lock:
|
||||||
|
self.over = True
|
||||||
|
if self.sock is not None:
|
||||||
|
with contextlib.suppress(OSError):
|
||||||
|
socket.socket.shutdown(self.sock, socket.SHUT_RDWR)
|
||||||
|
|
||||||
|
def finish(self) -> None:
|
||||||
|
with self.lock:
|
||||||
|
self.over = True
|
||||||
|
self.sock = None
|
||||||
|
|
||||||
|
def resolve(self, host: str, port: int) -> list[tuple]:
|
||||||
|
"""Returns:
|
||||||
|
The addresses of host, looked up in a daemon thread that the call
|
||||||
|
waits for only until its deadline.
|
||||||
|
"""
|
||||||
|
outcome: list[object] = []
|
||||||
|
|
||||||
|
def lookup() -> None:
|
||||||
|
try:
|
||||||
|
outcome.append(socket.getaddrinfo(host, port, type=socket.SOCK_STREAM))
|
||||||
|
except OSError as error:
|
||||||
|
outcome.append(error)
|
||||||
|
|
||||||
|
worker = threading.Thread(target=lookup, name="bounded lookup", daemon=True)
|
||||||
|
worker.start()
|
||||||
|
worker.join(self.remaining())
|
||||||
|
if not outcome:
|
||||||
|
raise self.late()
|
||||||
|
if isinstance(outcome[0], OSError):
|
||||||
|
raise outcome[0]
|
||||||
|
return outcome[0]
|
||||||
|
|
||||||
|
def dial(self, host: str, port: int) -> socket.socket:
|
||||||
|
"""Returns:
|
||||||
|
A socket connected to the first address of host that answers,
|
||||||
|
each attempt within the time left, the watchdog able to cut it.
|
||||||
|
"""
|
||||||
|
failure: OSError = OSError("no address")
|
||||||
|
for family, kind, proto, _, address in self.resolve(host, port):
|
||||||
|
sock = socket.socket(family, kind, proto)
|
||||||
|
try:
|
||||||
|
self.attach(sock)
|
||||||
|
sock.settimeout(self.remaining())
|
||||||
|
sock.connect(address)
|
||||||
|
sock.settimeout(self.remaining())
|
||||||
|
return sock
|
||||||
|
except OSError as error:
|
||||||
|
self.attach(None)
|
||||||
|
sock.close()
|
||||||
|
failure = error
|
||||||
|
except Failure:
|
||||||
|
self.attach(None)
|
||||||
|
sock.close()
|
||||||
|
raise
|
||||||
|
raise failure
|
||||||
|
|
||||||
|
|
||||||
|
class Plain(http.client.HTTPConnection):
|
||||||
|
def __init__(self, host: str, port: int, call: Call) -> None:
|
||||||
|
super().__init__(host, port, timeout=call.seconds)
|
||||||
|
self.call = call
|
||||||
|
|
||||||
|
def connect(self) -> None:
|
||||||
|
self.sock = self.call.dial(self.host, self.port)
|
||||||
|
|
||||||
|
|
||||||
|
class Sealed(Plain):
|
||||||
|
def connect(self) -> None:
|
||||||
|
raw = self.call.dial(self.host, self.port)
|
||||||
|
try:
|
||||||
|
tls = TLS.wrap_socket(raw, server_hostname=self.host, do_handshake_on_connect=False)
|
||||||
|
except BaseException:
|
||||||
|
raw.close()
|
||||||
|
raise
|
||||||
|
self.sock = tls
|
||||||
|
self.call.attach(tls)
|
||||||
|
tls.do_handshake()
|
||||||
|
|
||||||
|
|
||||||
|
CONNECTIONS = {"http": (Plain, 80), "https": (Sealed, 443)}
|
||||||
|
|
||||||
|
|
||||||
|
def drain(response: http.client.HTTPResponse, limit: int, call: Call) -> bytes:
|
||||||
|
chunks: list[bytes] = []
|
||||||
|
size = 0
|
||||||
|
while chunk := response.read1(CHUNK):
|
||||||
|
size += len(chunk)
|
||||||
|
if size > limit:
|
||||||
|
raise Failure(f"answered more than {limit} bytes")
|
||||||
|
call.remaining()
|
||||||
|
chunks.append(chunk)
|
||||||
|
call.remaining()
|
||||||
|
if response.length:
|
||||||
|
raise Failure("answered a truncated reply")
|
||||||
|
return b"".join(chunks)
|
||||||
|
|
||||||
|
|
||||||
|
def post(url: str, body: bytes, headers: dict[str, str], limit: int, seconds: float) -> bytes:
|
||||||
|
"""Args:
|
||||||
|
url: the http or https URL to POST to; no proxy and no redirect.
|
||||||
|
body: the request body.
|
||||||
|
headers: the request headers.
|
||||||
|
limit: the most bytes the answer may carry.
|
||||||
|
seconds: the wall-clock time the whole call may take: name lookup,
|
||||||
|
connect, TLS handshake, request and answer.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
The body of a 200 answer.
|
||||||
|
|
||||||
|
Raises:
|
||||||
|
Failure for every other outcome, naming a status, a limit or an
|
||||||
|
exception type only.
|
||||||
|
"""
|
||||||
|
call = Call(seconds)
|
||||||
|
watchdog = threading.Timer(seconds, call.cut)
|
||||||
|
watchdog.daemon = True
|
||||||
|
connection: Plain | None = None
|
||||||
|
try:
|
||||||
|
watchdog.start()
|
||||||
|
parts = urlsplit(url)
|
||||||
|
kind, port = CONNECTIONS[parts.scheme]
|
||||||
|
connection = kind(parts.hostname or "", parts.port or port, call)
|
||||||
|
connection.request("POST", (parts.path or "/") + (f"?{parts.query}" if parts.query else ""), body=body, headers=headers)
|
||||||
|
call.remaining()
|
||||||
|
with connection.getresponse() as response:
|
||||||
|
if response.status != 200:
|
||||||
|
raise Failure(f"answered HTTP {response.status}")
|
||||||
|
return drain(response, limit, call)
|
||||||
|
except Failure:
|
||||||
|
raise
|
||||||
|
except Exception as error: # noqa: BLE001
|
||||||
|
if isinstance(error, TimeoutError) or time.monotonic() >= call.deadline:
|
||||||
|
raise call.late() from None
|
||||||
|
raise Failure(f"failed: {type(error).__name__}") from None
|
||||||
|
finally:
|
||||||
|
watchdog.cancel()
|
||||||
|
call.finish()
|
||||||
|
if connection is not None:
|
||||||
|
connection.close()
|
||||||
+23
-7
@@ -39,8 +39,8 @@
|
|||||||
--llm-url: base URL of an OpenAI-compatible chat endpoint; empty turns the
|
--llm-url: base URL of an OpenAI-compatible chat endpoint; empty turns the
|
||||||
natural-language command line off.
|
natural-language command line off.
|
||||||
--llm-model: model name sent to that endpoint; empty turns it off.
|
--llm-model: model name sent to that endpoint; empty turns it off.
|
||||||
--llm-key-file: file holding the endpoint's bearer key; empty file or
|
--llm-key-file: file holding the endpoint's bearer key; an empty file
|
||||||
unreadable means no key.
|
means no key, an unreadable one stops the start.
|
||||||
--piper: the Piper executable, run as a separate process; missing turns
|
--piper: the Piper executable, run as a separate process; missing turns
|
||||||
speech output off.
|
speech output off.
|
||||||
--voices: directory holding the Piper voices as <model>.onnx with
|
--voices: directory holding the Piper voices as <model>.onnx with
|
||||||
@@ -51,6 +51,12 @@
|
|||||||
input off.
|
input off.
|
||||||
--whisper-model: the ggml model whisper.cpp loads.
|
--whisper-model: the ggml model whisper.cpp loads.
|
||||||
--scratch: writable directory the engines use as HOME and TMPDIR.
|
--scratch: writable directory the engines use as HOME and TMPDIR.
|
||||||
|
--speech-url: base URL with /v1 of an OpenAI-compatible audio API such as
|
||||||
|
the core role svc-ai-speech; /api/tts and /api/stt then render there
|
||||||
|
instead of in Piper and whisper.cpp, behind the same CPU gate and
|
||||||
|
transcription limits. Empty keeps the in-container engines.
|
||||||
|
--speech-key-file: file holding that service's bearer key; an empty file
|
||||||
|
means no key, an unreadable one stops the start.
|
||||||
--approvers: the pinned passkeys, mounted read-only; the hub lists their
|
--approvers: the pinned passkeys, mounted read-only; the hub lists their
|
||||||
ids and algorithms for the page, never pins, and verifies a signature
|
ids and algorithms for the page, never pins, and verifies a signature
|
||||||
only for the operator messages, denials and rules it acts on itself,
|
only for the operator messages, denials and rules it acts on itself,
|
||||||
@@ -70,9 +76,11 @@ from kvb_orchestrator.errors import ToolError
|
|||||||
from kvb_orchestrator.exchange import HubServer
|
from kvb_orchestrator.exchange import HubServer
|
||||||
from kvb_orchestrator.handler import Handler
|
from kvb_orchestrator.handler import Handler
|
||||||
from kvb_orchestrator.hub import Places, open_hub
|
from kvb_orchestrator.hub import Places, open_hub
|
||||||
from kvb_orchestrator.language import Model, read_key
|
from kvb_orchestrator.keys import key_from
|
||||||
|
from kvb_orchestrator.language import Model
|
||||||
from kvb_orchestrator.redirects import TIMEOUT, Redirects, dial
|
from kvb_orchestrator.redirects import TIMEOUT, Redirects, dial
|
||||||
from kvb_orchestrator.skill_scripts import Scripts
|
from kvb_orchestrator.skill_scripts import Scripts
|
||||||
|
from kvb_orchestrator.speechservice import Service
|
||||||
|
|
||||||
|
|
||||||
def main() -> int:
|
def main() -> int:
|
||||||
@@ -81,9 +89,9 @@ def main() -> int:
|
|||||||
for name in ("--listen-port", "--public-port"):
|
for name in ("--listen-port", "--public-port"):
|
||||||
parser.add_argument(name, type=int, required=True)
|
parser.add_argument(name, type=int, required=True)
|
||||||
paths = ("--council", "--pages", "--state", "--host-state", "--skills", "--transcripts", "--templates", "--tree-file", "--timers", "--kept", "--vendor", "--cleanup", "--offices", "--llm-key-file", "--approvers")
|
paths = ("--council", "--pages", "--state", "--host-state", "--skills", "--transcripts", "--templates", "--tree-file", "--timers", "--kept", "--vendor", "--cleanup", "--offices", "--llm-key-file", "--approvers")
|
||||||
for name in (*paths, "--piper", "--voices", "--whisper", "--whisper-model", "--scratch"):
|
for name in (*paths, "--piper", "--voices", "--whisper", "--whisper-model", "--scratch", "--speech-key-file"):
|
||||||
parser.add_argument(name, type=Path, required=True)
|
parser.add_argument(name, type=Path, required=True)
|
||||||
for name in ("--llm-url", "--llm-model", "--operator-voice"):
|
for name in ("--llm-url", "--llm-model", "--operator-voice", "--speech-url"):
|
||||||
parser.add_argument(name, required=True)
|
parser.add_argument(name, required=True)
|
||||||
args = parser.parse_args()
|
args = parser.parse_args()
|
||||||
try:
|
try:
|
||||||
@@ -97,8 +105,16 @@ def main() -> int:
|
|||||||
scripts=Scripts(skills=args.skills, council=args.council, home=args.state),
|
scripts=Scripts(skills=args.skills, council=args.council, home=args.state),
|
||||||
templates=args.templates,
|
templates=args.templates,
|
||||||
places=Places(tree_file=args.tree_file, timers=args.timers, vendor=args.vendor, cleanup=args.cleanup, offices=args.offices, kept=args.kept),
|
places=Places(tree_file=args.tree_file, timers=args.timers, vendor=args.vendor, cleanup=args.cleanup, offices=args.offices, kept=args.kept),
|
||||||
model=Model(url=args.llm_url.strip(), model=args.llm_model.strip(), key=read_key(args.llm_key_file)),
|
model=Model(url=args.llm_url.strip(), model=args.llm_model.strip(), key=key_from(args.llm_key_file, "model endpoint")),
|
||||||
engines=Engines(piper=args.piper, voices=args.voices, operator_voice=args.operator_voice, whisper=args.whisper, whisper_model=args.whisper_model, scratch=args.scratch),
|
engines=Engines(
|
||||||
|
piper=args.piper,
|
||||||
|
voices=args.voices,
|
||||||
|
operator_voice=args.operator_voice,
|
||||||
|
whisper=args.whisper,
|
||||||
|
whisper_model=args.whisper_model,
|
||||||
|
scratch=args.scratch,
|
||||||
|
service=Service(url=args.speech_url.strip(), key=key_from(args.speech_key_file, "speech service")),
|
||||||
|
),
|
||||||
approvers=args.approvers,
|
approvers=args.approvers,
|
||||||
redirects=Redirects(tls=ssl.create_default_context(), dial=dial, timeout=TIMEOUT),
|
redirects=Redirects(tls=ssl.create_default_context(), dial=dial, timeout=TIMEOUT),
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -55,7 +55,7 @@ def write_engines(base: Path) -> server.Engines:
|
|||||||
(voices / "broken.onnx").write_bytes(b"onnx")
|
(voices / "broken.onnx").write_bytes(b"onnx")
|
||||||
model = base / "ggml-base.bin"
|
model = base / "ggml-base.bin"
|
||||||
model.write_bytes(b"ggml")
|
model.write_bytes(b"ggml")
|
||||||
return server.Engines(piper=bin_dir / "piper", voices=voices, operator_voice=OPERATOR_VOICE, whisper=bin_dir / "whisper-cli", whisper_model=model, scratch=scratch)
|
return server.Engines(piper=bin_dir / "piper", voices=voices, operator_voice=OPERATOR_VOICE, whisper=bin_dir / "whisper-cli", whisper_model=model, scratch=scratch, service=server.Service(url="", key=""))
|
||||||
|
|
||||||
|
|
||||||
def wav_bytes(channels: int, width: int, rate: int, frames: bytes) -> bytes:
|
def wav_bytes(channels: int, width: int, rate: int, frames: bytes) -> bytes:
|
||||||
|
|||||||
@@ -33,7 +33,6 @@ from hub_support import (
|
|||||||
verifier,
|
verifier,
|
||||||
zero_led,
|
zero_led,
|
||||||
)
|
)
|
||||||
from kvb_orchestrator import language
|
|
||||||
|
|
||||||
PASSKEY_PAGE = HUB / "templates" / "page-passkey.js"
|
PASSKEY_PAGE = HUB / "templates" / "page-passkey.js"
|
||||||
PASSKEY_PROBE = """
|
PASSKEY_PROBE = """
|
||||||
@@ -198,10 +197,9 @@ class TestApproval(unittest.TestCase):
|
|||||||
self.addCleanup(server.server_close)
|
self.addCleanup(server.server_close)
|
||||||
threading.Thread(target=server.serve_forever, daemon=True).start()
|
threading.Thread(target=server.serve_forever, daemon=True).start()
|
||||||
self.addCleanup(server.shutdown)
|
self.addCleanup(server.shutdown)
|
||||||
for name, opener in (("client", hub_api.OPENER), ("llm", language.LLM_OPENER)):
|
|
||||||
request = urllib.request.Request(f"{hub_api.base_url(server.server_address[1])}/api/asks")
|
request = urllib.request.Request(f"{hub_api.base_url(server.server_address[1])}/api/asks")
|
||||||
with self.subTest(name=name), self.assertRaises(urllib.error.HTTPError) as caught:
|
with self.assertRaises(urllib.error.HTTPError) as caught:
|
||||||
opener.open(request, timeout=5)
|
hub_api.OPENER.open(request, timeout=5)
|
||||||
self.assertEqual(caught.exception.code, 302)
|
self.assertEqual(caught.exception.code, 302)
|
||||||
|
|
||||||
def test_attestation_parsing_yields_the_pinned_key_of_each_algorithm(self):
|
def test_attestation_parsing_yields_the_pinned_key_of_each_algorithm(self):
|
||||||
|
|||||||
@@ -0,0 +1,133 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import contextlib
|
||||||
|
import dataclasses
|
||||||
|
import http.client
|
||||||
|
import http.server
|
||||||
|
import io
|
||||||
|
import threading
|
||||||
|
import time
|
||||||
|
import unittest
|
||||||
|
from unittest import mock
|
||||||
|
|
||||||
|
from hub_case import HubCase
|
||||||
|
from hub_support import server
|
||||||
|
from kvb_orchestrator import errors, language, transport
|
||||||
|
|
||||||
|
KEY = "sk-model-0123456789abcdef"
|
||||||
|
MALFORMED = "the model endpoint key must be printable ASCII without whitespace: check KVB_ORCHESTRATOR_LLM_KEY_FILE"
|
||||||
|
PLAIN = "the model endpoint key travels only over https or to 127.0.0.1, localhost or ::1: set an https KVB_ORCHESTRATOR_LLM_URL or empty KVB_ORCHESTRATOR_LLM_KEY_FILE"
|
||||||
|
|
||||||
|
|
||||||
|
class Endpoint(http.server.ThreadingHTTPServer):
|
||||||
|
def __init__(self) -> None:
|
||||||
|
super().__init__(("127.0.0.1", 0), Answer)
|
||||||
|
self.mode = "error"
|
||||||
|
threading.Thread(target=self.serve_forever, args=(0.05,), daemon=True).start()
|
||||||
|
|
||||||
|
def handle_error(self, request, client_address):
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
class Answer(http.server.BaseHTTPRequestHandler):
|
||||||
|
def do_POST(self):
|
||||||
|
self.rfile.read(int(self.headers["Content-Length"]))
|
||||||
|
if self.server.mode == "trickle":
|
||||||
|
self.send_response(200)
|
||||||
|
self.send_header("Content-Length", "100")
|
||||||
|
self.end_headers()
|
||||||
|
for _ in range(100):
|
||||||
|
self.wfile.write(b" ")
|
||||||
|
time.sleep(0.2)
|
||||||
|
return
|
||||||
|
redirect = self.server.mode == "redirect"
|
||||||
|
body = b"" if redirect else f"upstream broke, your key {self.headers['Authorization']}".encode()
|
||||||
|
self.send_response(302 if redirect else 500)
|
||||||
|
if redirect:
|
||||||
|
self.send_header("Location", "/elsewhere")
|
||||||
|
self.send_header("Content-Length", str(len(body)))
|
||||||
|
self.end_headers()
|
||||||
|
self.wfile.write(body)
|
||||||
|
|
||||||
|
def log_message(self, format, *args):
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
class TestModelKey(unittest.TestCase):
|
||||||
|
def test_a_key_that_is_no_printable_ascii_token_is_refused_and_never_shown(self):
|
||||||
|
for key in ("sk-SECRET\n# rotated 2026-11-13", "sk SECRET", "sk-\tSECRET", "sk-SECRET\r", "sk-SÉCRET", "sk-\x7fSECRET", "sk-\x00SECRET"):
|
||||||
|
with self.subTest(key=key), self.assertRaises(errors.ToolError) as refused:
|
||||||
|
server.Model(url="http://127.0.0.1:9/v1", model="m", key=key)
|
||||||
|
self.assertEqual(str(refused.exception), MALFORMED)
|
||||||
|
model = server.Model(url="http://127.0.0.1:9/v1", model="m", key=KEY)
|
||||||
|
self.assertEqual(model.key, KEY)
|
||||||
|
for shown in (repr(model), str(model)):
|
||||||
|
with self.subTest(shown=shown[:40]):
|
||||||
|
self.assertNotIn(KEY, shown)
|
||||||
|
|
||||||
|
def test_a_key_travels_only_over_https_or_to_the_loopback_host(self):
|
||||||
|
for url in ("https://llm.example/v1", "https://10.0.0.5:4000/v1", "http://127.0.0.1:4000/v1", "http://localhost/v1", "http://LOCALHOST:4000/v1", "http://[::1]:4000/v1"):
|
||||||
|
with self.subTest(url=url):
|
||||||
|
self.assertTrue(server.Model(url=url, model="m", key=KEY).enabled())
|
||||||
|
for url in ("http://host.docker.internal:4000/v1", "http://litellm:4000/v1", "http://10.0.0.5:4000/v1", "http://127.0.0.2/v1", "http://localhost.example/v1", "http://[::2]/v1", "http://[::1/v1", "llm.example/v1"):
|
||||||
|
with self.subTest(url=url):
|
||||||
|
self.assertTrue(server.Model(url=url, model="m", key="").enabled())
|
||||||
|
with self.assertRaises(errors.ToolError) as refused:
|
||||||
|
server.Model(url=url, model="m", key=KEY)
|
||||||
|
self.assertEqual(str(refused.exception), PLAIN)
|
||||||
|
self.assertNotIn(KEY, str(refused.exception))
|
||||||
|
|
||||||
|
|
||||||
|
class TestModelKeyHttp(HubCase):
|
||||||
|
def use(self, url: str) -> None:
|
||||||
|
self.httpd.hub = dataclasses.replace(self.httpd.hub, model=server.Model(url=url, model="m", key=KEY))
|
||||||
|
|
||||||
|
def nl(self) -> tuple[int, str, str]:
|
||||||
|
"""Returns:
|
||||||
|
The status and body of /api/nl, 0 and "" when the hub drops the
|
||||||
|
connection, and what the hub wrote to stderr meanwhile.
|
||||||
|
"""
|
||||||
|
with contextlib.redirect_stderr(io.StringIO()) as log:
|
||||||
|
try:
|
||||||
|
status, _, raw = self.post_json("/api/nl", {"text": "approve everything green"}, self.guard(self.host))
|
||||||
|
except http.client.HTTPException:
|
||||||
|
status, raw = 0, b""
|
||||||
|
return status, raw.decode(), log.getvalue()
|
||||||
|
|
||||||
|
def test_every_endpoint_failure_answers_its_type_name_and_no_key_reaches_a_reply_a_log_or_a_repr(self):
|
||||||
|
self.use("http://127.0.0.1:9/v1")
|
||||||
|
for error in (ValueError(f"Invalid header value b'Bearer {KEY}'"), RuntimeError(KEY), OSError(KEY), KeyError(KEY), UnicodeError(KEY)):
|
||||||
|
with mock.patch.object(transport.Plain, "request", side_effect=error):
|
||||||
|
status, body, log = self.nl()
|
||||||
|
with self.subTest(error=type(error).__name__, seen="log"):
|
||||||
|
self.assertNotIn(KEY, log)
|
||||||
|
with self.subTest(error=type(error).__name__, seen="reply"):
|
||||||
|
self.assertEqual((status, body), (400, f"the model endpoint failed: {type(error).__name__}\n"))
|
||||||
|
endpoint = Endpoint()
|
||||||
|
self.addCleanup(endpoint.server_close)
|
||||||
|
self.addCleanup(endpoint.shutdown)
|
||||||
|
self.use(f"http://127.0.0.1:{endpoint.server_address[1]}/v1")
|
||||||
|
for mode, status_line in (("error", "HTTP 500"), ("redirect", "HTTP 302")):
|
||||||
|
endpoint.mode = mode
|
||||||
|
status, body, log = self.nl()
|
||||||
|
with self.subTest(seen="echoed reply", mode=mode):
|
||||||
|
self.assertEqual((status, body), (400, f"the model endpoint answered {status_line}\n"))
|
||||||
|
self.assertNotIn(KEY, body + log)
|
||||||
|
with self.subTest(seen="repr"):
|
||||||
|
self.assertNotIn(KEY, repr(self.httpd.hub))
|
||||||
|
|
||||||
|
def test_a_trickling_endpoint_hits_the_wall_clock_deadline(self):
|
||||||
|
endpoint = Endpoint()
|
||||||
|
endpoint.mode = "trickle"
|
||||||
|
self.addCleanup(endpoint.server_close)
|
||||||
|
self.addCleanup(endpoint.shutdown)
|
||||||
|
self.use(f"http://127.0.0.1:{endpoint.server_address[1]}/v1")
|
||||||
|
started = time.monotonic()
|
||||||
|
with mock.patch.object(language, "LLM_TIMEOUT", 0.5):
|
||||||
|
status, body, _ = self.nl()
|
||||||
|
self.assertEqual((status, body), (400, "the model endpoint took longer than 0.5 seconds\n"))
|
||||||
|
self.assertLess(time.monotonic() - started, 2.0)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
@@ -183,13 +183,14 @@ class TestHubSpeech(HubCase):
|
|||||||
"--llm-key-file": os.devnull, "--llm-url": "", "--llm-model": "",
|
"--llm-key-file": os.devnull, "--llm-url": "", "--llm-model": "",
|
||||||
"--piper": str(self.engines.piper), "--voices": str(self.engines.voices), "--operator-voice": "en_US-test-high:9", "--whisper": str(self.engines.whisper),
|
"--piper": str(self.engines.piper), "--voices": str(self.engines.voices), "--operator-voice": "en_US-test-high:9", "--whisper": str(self.engines.whisper),
|
||||||
"--whisper-model": str(self.engines.whisper_model), "--scratch": str(self.engines.scratch), "--approvers": str(self.approvers),
|
"--whisper-model": str(self.engines.whisper_model), "--scratch": str(self.engines.scratch), "--approvers": str(self.approvers),
|
||||||
|
"--speech-url": "", "--speech-key-file": os.devnull,
|
||||||
}
|
}
|
||||||
|
|
||||||
def run(given: dict[str, str]) -> subprocess.CompletedProcess:
|
def run(given: dict[str, str]) -> subprocess.CompletedProcess:
|
||||||
argv = [part for pair in given.items() for part in pair]
|
argv = [part for pair in given.items() for part in pair]
|
||||||
return subprocess.run([sys.executable, str(HUB / "server.py"), *argv], capture_output=True, text=True, timeout=60, check=False)
|
return subprocess.run([sys.executable, str(HUB / "server.py"), *argv], capture_output=True, text=True, timeout=60, check=False)
|
||||||
|
|
||||||
for name in ("--piper", "--voices", "--operator-voice", "--whisper", "--whisper-model", "--scratch", "--approvers", "--vendor", "--cleanup", "--offices"):
|
for name in ("--piper", "--voices", "--operator-voice", "--whisper", "--whisper-model", "--scratch", "--approvers", "--vendor", "--cleanup", "--offices", "--speech-url", "--speech-key-file"):
|
||||||
with self.subTest(name=name):
|
with self.subTest(name=name):
|
||||||
done = run({key: value for key, value in args.items() if key != name})
|
done = run({key: value for key, value in args.items() if key != name})
|
||||||
self.assertEqual(done.returncode, 2, done.stderr)
|
self.assertEqual(done.returncode, 2, done.stderr)
|
||||||
@@ -197,6 +198,34 @@ class TestHubSpeech(HubCase):
|
|||||||
done = run(args)
|
done = run(args)
|
||||||
self.assertEqual(done.returncode, 2, done.stderr)
|
self.assertEqual(done.returncode, 2, done.stderr)
|
||||||
self.assertIn("the operator voice 'en_US-test-high:9' is not installed", done.stderr)
|
self.assertIn("the operator voice 'en_US-test-high:9' is not installed", done.stderr)
|
||||||
|
secret = "op:hunter2"
|
||||||
|
done = run({**args, "--operator-voice": OPERATOR_VOICE, "--speech-url": f"https://{secret}@speech.example/v1"})
|
||||||
|
self.assertEqual(done.returncode, 2, done.stderr)
|
||||||
|
self.assertIn("the speech service URL must be http or https with a host and without userinfo", done.stderr)
|
||||||
|
self.assertNotIn(secret, done.stderr + done.stdout)
|
||||||
|
key_file = self.outside / "speech.key"
|
||||||
|
key_file.write_text("sk-plain-http-0123456789\n", encoding="utf-8")
|
||||||
|
done = run({**args, "--operator-voice": OPERATOR_VOICE, "--speech-url": "http://svc-ai-speech:8000/v1", "--speech-key-file": str(key_file)})
|
||||||
|
self.assertEqual(done.returncode, 2, done.stderr)
|
||||||
|
self.assertIn("the speech service key travels only over https or to 127.0.0.1, localhost or ::1", done.stderr)
|
||||||
|
self.assertNotIn("sk-plain-http", done.stderr + done.stdout)
|
||||||
|
done = run({**args, "--operator-voice": OPERATOR_VOICE, "--llm-url": "http://host.docker.internal:4000/v1", "--llm-model": "m", "--llm-key-file": str(key_file)})
|
||||||
|
self.assertEqual(done.returncode, 2, done.stderr)
|
||||||
|
self.assertIn("the model endpoint key travels only over https or to 127.0.0.1, localhost or ::1", done.stderr)
|
||||||
|
self.assertNotIn("sk-plain-http", done.stderr + done.stdout)
|
||||||
|
key_file.write_text("sk-SECRET-0123\n# rotated 2026-11-13\n", encoding="utf-8")
|
||||||
|
for key_path, message in ((key_file, "the speech service key must be printable ASCII without whitespace"), (self.outside, f"the speech service key file {self.outside} is unreadable")):
|
||||||
|
with self.subTest(key_path=key_path):
|
||||||
|
done = run({**args, "--operator-voice": OPERATOR_VOICE, "--speech-url": "https://speech.example/v1", "--speech-key-file": str(key_path)})
|
||||||
|
self.assertEqual(done.returncode, 2, done.stderr)
|
||||||
|
self.assertIn(message, done.stderr)
|
||||||
|
self.assertNotIn("SECRET", done.stderr + done.stdout)
|
||||||
|
for key_path, message in ((key_file, "the model endpoint key must be printable ASCII without whitespace"), (self.outside, f"the model endpoint key file {self.outside} is unreadable")):
|
||||||
|
with self.subTest(llm_key_path=key_path):
|
||||||
|
done = run({**args, "--operator-voice": OPERATOR_VOICE, "--llm-url": "https://llm.example/v1", "--llm-model": "m", "--llm-key-file": str(key_path)})
|
||||||
|
self.assertEqual(done.returncode, 2, done.stderr)
|
||||||
|
self.assertIn(message, done.stderr)
|
||||||
|
self.assertNotIn("SECRET", done.stderr + done.stdout)
|
||||||
|
|
||||||
|
|
||||||
class TestSpeechImage(unittest.TestCase):
|
class TestSpeechImage(unittest.TestCase):
|
||||||
|
|||||||
@@ -0,0 +1,414 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import dataclasses
|
||||||
|
import http.client
|
||||||
|
import http.server
|
||||||
|
import json
|
||||||
|
import socket
|
||||||
|
import struct
|
||||||
|
import tempfile
|
||||||
|
import threading
|
||||||
|
import time
|
||||||
|
import unittest
|
||||||
|
from pathlib import Path
|
||||||
|
from unittest import mock
|
||||||
|
|
||||||
|
import hub_case
|
||||||
|
from hub_case import HubCase
|
||||||
|
from hub_engines import log_rows, wav_bytes, write_engines
|
||||||
|
from kvb_orchestrator import engines, errors, keys, speechservice, transport
|
||||||
|
from kvb_orchestrator.speechservice import Service
|
||||||
|
|
||||||
|
KEY = "sk-speech-0123456789abcdef"
|
||||||
|
VOICE = "en_US-test-high:2"
|
||||||
|
LINE = "all stations, this is Docs, disk at ninety percent, out"
|
||||||
|
MAYDAY = "MAYDAY MAYDAY MAYDAY"
|
||||||
|
SPOKEN = wav_bytes(1, 2, 22050, b"\x01\x00" * 2205)
|
||||||
|
OVERFLOW = struct.pack("<4sI", b"RIFF", len(SPOKEN) - 108) + SPOKEN[8:-100]
|
||||||
|
CLIP = wav_bytes(1, 2, engines.STT_RATE, b"\x01\x00" * engines.STT_RATE)
|
||||||
|
MALFORMED = "the speech service key must be printable ASCII without whitespace: check KVB_ORCHESTRATOR_SPEECH_KEY_FILE"
|
||||||
|
STREAMING = 0xFFFFFFFF
|
||||||
|
DATA = SPOKEN.index(b"data")
|
||||||
|
|
||||||
|
|
||||||
|
def streamed(riff: int, data: int, body: bytes = SPOKEN) -> bytes:
|
||||||
|
at = body.index(b"data", 12)
|
||||||
|
return body[:4] + struct.pack("<I", riff) + body[8 : at + 4] + struct.pack("<I", data) + body[at + 8 :]
|
||||||
|
|
||||||
|
|
||||||
|
class FakeService(http.server.ThreadingHTTPServer):
|
||||||
|
def __init__(self) -> None:
|
||||||
|
super().__init__(("127.0.0.1", 0), FakeHandler)
|
||||||
|
self.mode = "ok"
|
||||||
|
self.requests: list[dict] = []
|
||||||
|
self.thread = threading.Thread(target=self.serve_forever, args=(0.05,), daemon=True)
|
||||||
|
self.thread.start()
|
||||||
|
|
||||||
|
@property
|
||||||
|
def url(self) -> str:
|
||||||
|
return f"http://127.0.0.1:{self.server_address[1]}/v1"
|
||||||
|
|
||||||
|
def stop(self) -> None:
|
||||||
|
self.shutdown()
|
||||||
|
self.server_close()
|
||||||
|
self.thread.join(10)
|
||||||
|
|
||||||
|
def handle_error(self, request, client_address):
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
class FakeHandler(http.server.BaseHTTPRequestHandler):
|
||||||
|
def log_message(self, format, *args):
|
||||||
|
return None
|
||||||
|
|
||||||
|
def answer(self, status: int, kind: str, body: bytes, headers: dict[str, str]) -> None:
|
||||||
|
self.send_response(status)
|
||||||
|
self.send_header("Content-Type", kind)
|
||||||
|
self.send_header("Content-Length", str(len(body)))
|
||||||
|
for name, value in headers.items():
|
||||||
|
self.send_header(name, value)
|
||||||
|
self.end_headers()
|
||||||
|
self.wfile.write(body)
|
||||||
|
|
||||||
|
def record(self, method: str, body: bytes) -> None:
|
||||||
|
self.server.requests.append({"method": method, "path": self.path, "headers": {name.lower(): value for name, value in self.headers.items()}, "body": body})
|
||||||
|
|
||||||
|
def do_GET(self):
|
||||||
|
self.record("GET", b"")
|
||||||
|
self.answer(200, "audio/wav", SPOKEN, {})
|
||||||
|
|
||||||
|
def trickle(self, data: bytes) -> None:
|
||||||
|
for byte in data:
|
||||||
|
self.wfile.write(bytes([byte]))
|
||||||
|
time.sleep(0.2)
|
||||||
|
|
||||||
|
def do_POST(self):
|
||||||
|
body = self.rfile.read(int(self.headers["Content-Length"]))
|
||||||
|
self.record("POST", body)
|
||||||
|
mode = "trickle-body" if b"trickle" in body else self.server.mode
|
||||||
|
speech = self.path.endswith(speechservice.SPEECH_PATH)
|
||||||
|
if mode == "trickle-body":
|
||||||
|
self.send_response(200)
|
||||||
|
self.send_header("Content-Length", str(len(SPOKEN)))
|
||||||
|
self.end_headers()
|
||||||
|
self.trickle(SPOKEN)
|
||||||
|
elif mode == "trickle-head":
|
||||||
|
self.trickle(b"HTTP/1.1 200 OK\r\n" * 40)
|
||||||
|
elif mode == "short":
|
||||||
|
self.send_response(200)
|
||||||
|
self.send_header("Content-Length", str(len(SPOKEN)))
|
||||||
|
self.end_headers()
|
||||||
|
self.wfile.write(SPOKEN[:1000])
|
||||||
|
self.close_connection = True
|
||||||
|
elif mode == "truncated":
|
||||||
|
self.answer(200, "audio/wav", SPOKEN[:-100], {})
|
||||||
|
elif mode == "overflow":
|
||||||
|
self.answer(200, "audio/wav", OVERFLOW, {})
|
||||||
|
elif mode == "error":
|
||||||
|
self.answer(500, "text/plain", f"upstream broke, your key {self.headers['Authorization']}".encode(), {})
|
||||||
|
elif mode == "redirect":
|
||||||
|
self.answer(302, "text/plain", b"", {"Location": "/elsewhere"})
|
||||||
|
elif mode == "garbage":
|
||||||
|
self.answer(200, "audio/wav", b"RIFF no wave here", {})
|
||||||
|
elif mode == "notext":
|
||||||
|
self.answer(200, "application/json", b'{"words": []}', {})
|
||||||
|
elif mode == "slow":
|
||||||
|
time.sleep(1.5)
|
||||||
|
self.answer(200, "audio/wav", SPOKEN, {})
|
||||||
|
elif mode == "big":
|
||||||
|
self.answer(200, "audio/wav", SPOKEN + b"\x00" * 4096, {})
|
||||||
|
elif mode == "streamed":
|
||||||
|
self.answer(200, "audio/wav", streamed(STREAMING, STREAMING), {})
|
||||||
|
elif mode == "streamed-short":
|
||||||
|
self.answer(200, "audio/wav", streamed(STREAMING, len(SPOKEN) - DATA - 8)[:-100], {})
|
||||||
|
elif speech:
|
||||||
|
self.answer(200, "audio/wav", SPOKEN, {})
|
||||||
|
else:
|
||||||
|
self.answer(200, "application/json", json.dumps({"text": " hello [BLANK_AUDIO] from the service "}).encode(), {})
|
||||||
|
|
||||||
|
|
||||||
|
class TestServiceUrl(unittest.TestCase):
|
||||||
|
def test_only_a_plain_http_or_https_base_is_accepted_and_empty_keeps_the_engines(self):
|
||||||
|
self.assertFalse(Service(url="", key="").enabled())
|
||||||
|
for url in ("http://127.0.0.1:8000/v1", "https://speech.example/v1", "http://svc-ai-speech/v1", "https://[::1]:8443/v1"):
|
||||||
|
with self.subTest(url=url):
|
||||||
|
self.assertTrue(Service(url=url, key="").enabled())
|
||||||
|
for url in (
|
||||||
|
"ftp://speech.example/v1", "speech.example/v1", "https:///v1", "https://user:pass@speech.example/v1", "https://speech.example/v1?key=1",
|
||||||
|
"https://speech.example/v1#top", "https://speech.example:0/v1", "https://speech.example:99999/v1", "https://speech.example/v 1",
|
||||||
|
"https://speech.example/%761", "https://speech.example/v1\n", "https://spéech.example/v1", "file:///etc/passwd",
|
||||||
|
):
|
||||||
|
with self.subTest(url=url), self.assertRaises(errors.ToolError) as refused:
|
||||||
|
Service(url=url, key=KEY)
|
||||||
|
self.assertNotIn("pass", str(refused.exception))
|
||||||
|
|
||||||
|
def test_a_key_travels_only_over_https_or_to_the_loopback_host(self):
|
||||||
|
for url in ("https://speech.example/v1", "https://10.0.0.5:8443/v1", "http://127.0.0.1:8000/v1", "http://localhost/v1", "http://LOCALHOST:9000/v1", "http://[::1]:8000/v1"):
|
||||||
|
with self.subTest(url=url):
|
||||||
|
self.assertTrue(Service(url=url, key=KEY).enabled())
|
||||||
|
for url in ("http://svc-ai-speech/v1", "http://10.0.0.5:8000/v1", "http://127.0.0.2/v1", "http://localhost.example/v1", "http://[::2]/v1", "http://speech.example/v1"):
|
||||||
|
with self.subTest(url=url):
|
||||||
|
self.assertTrue(Service(url=url, key="").enabled())
|
||||||
|
with self.assertRaises(errors.ToolError) as refused:
|
||||||
|
Service(url=url, key=KEY)
|
||||||
|
self.assertEqual(str(refused.exception), "the speech service key travels only over https or to 127.0.0.1, localhost or ::1: set an https KVB_ORCHESTRATOR_SPEECH_URL or empty KVB_ORCHESTRATOR_SPEECH_KEY_FILE")
|
||||||
|
self.assertNotIn(KEY, str(refused.exception))
|
||||||
|
|
||||||
|
|
||||||
|
class TestServiceKey(unittest.TestCase):
|
||||||
|
def test_a_key_that_is_no_printable_ascii_token_is_refused_and_never_shown(self):
|
||||||
|
for key in ("sk-SECRET\n# rotated 2026-11-13", "sk SECRET", "sk-\tSECRET", "sk-SECRET\r", "sk-SÉCRET", "sk-\x7fSECRET", "sk-\x00SECRET"):
|
||||||
|
with self.subTest(key=key), self.assertRaises(errors.ToolError) as refused:
|
||||||
|
Service(url="https://speech.example/v1", key=key)
|
||||||
|
self.assertEqual(str(refused.exception), MALFORMED)
|
||||||
|
service = Service(url="https://speech.example/v1", key=KEY)
|
||||||
|
built = engines.Engines(piper=Path("p"), voices=Path("v"), operator_voice="", whisper=Path("w"), whisper_model=Path("m"), scratch=Path("s"), service=service)
|
||||||
|
for shown in (repr(service), str(service), repr(built)):
|
||||||
|
with self.subTest(shown=shown[:40]):
|
||||||
|
self.assertNotIn(KEY, shown)
|
||||||
|
|
||||||
|
def test_the_key_file_is_read_strictly_and_an_unreadable_one_stops_the_start(self):
|
||||||
|
folder = tempfile.TemporaryDirectory()
|
||||||
|
self.addCleanup(folder.cleanup)
|
||||||
|
base = Path(folder.name)
|
||||||
|
(base / "key").write_text(f" {KEY}\n", encoding="utf-8")
|
||||||
|
(base / "binary").write_bytes(b"\xffsk-SECRET")
|
||||||
|
self.assertEqual((keys.key_from(Path("/dev/null"), "speech service"), keys.key_from(base / "key", "speech service")), ("", KEY))
|
||||||
|
for path in (base, base / "missing", base / "binary"):
|
||||||
|
with self.subTest(path=path), self.assertRaises(errors.ToolError) as refused:
|
||||||
|
keys.key_from(path, "speech service")
|
||||||
|
self.assertEqual(str(refused.exception), f"the speech service key file {path} is unreadable")
|
||||||
|
self.assertIsNone(refused.exception.__cause__)
|
||||||
|
|
||||||
|
def test_a_key_file_over_the_limit_stops_the_start_without_reading_on(self):
|
||||||
|
folder = tempfile.TemporaryDirectory()
|
||||||
|
self.addCleanup(folder.cleanup)
|
||||||
|
base = Path(folder.name)
|
||||||
|
(base / "full").write_text("k" * keys.KEY_LIMIT, encoding="utf-8")
|
||||||
|
(base / "huge").write_text("k" * (keys.KEY_LIMIT + 1), encoding="utf-8")
|
||||||
|
self.assertEqual(keys.key_from(base / "full", "speech service"), "k" * keys.KEY_LIMIT)
|
||||||
|
for path in (base / "huge", Path("/dev/zero")):
|
||||||
|
with self.subTest(path=path):
|
||||||
|
with self.assertRaises(errors.ToolError) as refused:
|
||||||
|
keys.key_from(path, "speech service")
|
||||||
|
self.assertEqual(str(refused.exception), f"the speech service key file {path} holds more than {keys.KEY_LIMIT} bytes")
|
||||||
|
|
||||||
|
def test_every_failure_of_the_request_becomes_its_type_name_only(self):
|
||||||
|
service = Service(url="http://127.0.0.1:9/v1", key=KEY)
|
||||||
|
for error in (ValueError(f"Invalid header value b'Bearer {KEY}'"), RuntimeError(KEY), http.client.InvalidURL(KEY), OSError(KEY)):
|
||||||
|
with self.subTest(error=type(error).__name__), mock.patch.object(http.client.HTTPConnection, "request", side_effect=error):
|
||||||
|
with self.assertRaises(errors.EngineError) as failed:
|
||||||
|
service.speak(VOICE, LINE)
|
||||||
|
self.assertEqual(str(failed.exception), f"the speech service failed: {type(error).__name__}")
|
||||||
|
self.assertTrue(failed.exception.__suppress_context__)
|
||||||
|
|
||||||
|
|
||||||
|
class TestTransportDeadline(unittest.TestCase):
|
||||||
|
def speak(self, url: str) -> tuple[str, float]:
|
||||||
|
started = time.monotonic()
|
||||||
|
with mock.patch.object(speechservice, "SERVICE_TIMEOUT", 0.5), self.assertRaises(errors.EngineError) as failed:
|
||||||
|
Service(url=url, key="").speak(VOICE, LINE)
|
||||||
|
return str(failed.exception), time.monotonic() - started
|
||||||
|
|
||||||
|
def test_a_stalled_name_lookup_ends_at_the_deadline(self):
|
||||||
|
with mock.patch.object(transport.socket, "getaddrinfo", side_effect=lambda *args, **kwargs: time.sleep(3)):
|
||||||
|
message, took = self.speak("http://speech.example/v1")
|
||||||
|
self.assertEqual(message, "the speech service took longer than 0.5 seconds")
|
||||||
|
self.assertLess(took, 1.5)
|
||||||
|
|
||||||
|
def test_a_stalled_connect_gets_only_the_time_left_across_every_address(self):
|
||||||
|
timeouts: list[float] = []
|
||||||
|
|
||||||
|
def stall(sock: socket.socket, address: tuple) -> None:
|
||||||
|
timeouts.append(sock.gettimeout())
|
||||||
|
time.sleep(sock.gettimeout())
|
||||||
|
raise TimeoutError
|
||||||
|
|
||||||
|
addresses = [(socket.AF_INET, socket.SOCK_STREAM, socket.IPPROTO_TCP, "", ("127.0.0.1", 9))] * 3
|
||||||
|
with mock.patch.object(transport.Call, "resolve", return_value=addresses), mock.patch.object(socket.socket, "connect", stall):
|
||||||
|
message, took = self.speak("http://speech.example/v1")
|
||||||
|
self.assertEqual(message, "the speech service took longer than 0.5 seconds")
|
||||||
|
self.assertLess(took, 1.5)
|
||||||
|
self.assertTrue(timeouts and sum(timeouts) <= 0.5, timeouts)
|
||||||
|
|
||||||
|
def test_a_stalled_tls_handshake_ends_at_the_deadline(self):
|
||||||
|
listener = socket.create_server(("127.0.0.1", 0))
|
||||||
|
self.addCleanup(listener.close)
|
||||||
|
message, took = self.speak(f"https://127.0.0.1:{listener.getsockname()[1]}/v1")
|
||||||
|
self.assertEqual(message, "the speech service took longer than 0.5 seconds")
|
||||||
|
self.assertLess(took, 1.5)
|
||||||
|
|
||||||
|
def test_a_watchdog_that_cannot_start_fails_with_its_type_name(self):
|
||||||
|
with mock.patch.object(transport.threading.Timer, "start", side_effect=RuntimeError(KEY)):
|
||||||
|
message, _ = self.speak("http://127.0.0.1:9/v1")
|
||||||
|
self.assertEqual(message, "the speech service failed: RuntimeError")
|
||||||
|
|
||||||
|
|
||||||
|
class TestWholeWav(unittest.TestCase):
|
||||||
|
def test_only_a_wav_whose_sizes_match_the_bytes_received_counts(self):
|
||||||
|
chunks = struct.pack("<4sI", b"LIST", 1) + b"x\x00" + SPOKEN[12:]
|
||||||
|
odd = struct.pack("<4sI4s", b"RIFF", len(chunks) + 4, b"WAVE") + chunks
|
||||||
|
self.assertEqual([engines.whole_wav(body) for body in (SPOKEN, odd)], [SPOKEN, odd])
|
||||||
|
empty = wav_bytes(1, 2, 22050, b"")
|
||||||
|
for body in (SPOKEN[:-100], OVERFLOW, SPOKEN + b"\x00", SPOKEN[:-1], b"", b"RIFF", empty, SPOKEN.replace(b"WAVE", b"AVI ", 1)):
|
||||||
|
with self.subTest(size=len(body)):
|
||||||
|
self.assertIsNone(engines.whole_wav(body))
|
||||||
|
|
||||||
|
def test_a_streamed_wav_counts_only_if_its_data_chunk_is_last_and_runs_to_the_end(self):
|
||||||
|
real = len(SPOKEN) - DATA - 8
|
||||||
|
chunks = struct.pack("<4sI", b"LIST", 1) + b"x\x00" + SPOKEN[12:]
|
||||||
|
odd = struct.pack("<4sI4s", b"RIFF", len(chunks) + 4, b"WAVE") + chunks
|
||||||
|
for riff, data in ((STREAMING, STREAMING), (0, 0), (STREAMING, 0), (0, STREAMING), (STREAMING, real), (0, real)):
|
||||||
|
with self.subTest(riff=riff, data=data):
|
||||||
|
self.assertEqual(engines.whole_wav(streamed(riff, data)), SPOKEN)
|
||||||
|
self.assertEqual(engines.whole_wav(streamed(STREAMING, STREAMING, odd)), odd)
|
||||||
|
trailer = struct.pack("<4sI", b"LIST", 2) + b"xy"
|
||||||
|
for name, body in (
|
||||||
|
("data not last", streamed(STREAMING, real) + trailer),
|
||||||
|
("truncated with a real size", streamed(STREAMING, real)[:-100]),
|
||||||
|
("real size short of the end", streamed(STREAMING, real - 2)),
|
||||||
|
("no audio", streamed(STREAMING, STREAMING)[: DATA + 8]),
|
||||||
|
("no data chunk", streamed(STREAMING, STREAMING)[:DATA]),
|
||||||
|
("real RIFF, placeholder data", streamed(len(SPOKEN) - 8, STREAMING)),
|
||||||
|
("real RIFF, zero data", streamed(len(SPOKEN) - 8, 0)),
|
||||||
|
("mismatched RIFF, placeholder data", streamed(len(SPOKEN), STREAMING)),
|
||||||
|
("placeholder RIFF, truncated fmt", streamed(STREAMING, STREAMING)[:30]),
|
||||||
|
):
|
||||||
|
with self.subTest(name=name):
|
||||||
|
self.assertIsNone(engines.whole_wav(body))
|
||||||
|
|
||||||
|
|
||||||
|
class TestServiceEngines(unittest.TestCase):
|
||||||
|
def test_with_the_service_no_local_piper_or_whisper_is_needed_but_the_voices_are(self):
|
||||||
|
folder = tempfile.TemporaryDirectory()
|
||||||
|
self.addCleanup(folder.cleanup)
|
||||||
|
base = Path(folder.name)
|
||||||
|
built = write_engines(base)
|
||||||
|
gone = {"piper": base / "missing-piper", "whisper": base / "missing-whisper", "whisper_model": base / "missing.bin"}
|
||||||
|
remote = engines.Speech(dataclasses.replace(built, service=Service(url="http://127.0.0.1:9/v1", key=""), **gone), base / "remote")
|
||||||
|
local = engines.Speech(dataclasses.replace(built, **gone), base / "local")
|
||||||
|
self.assertEqual([remote.describe()[key] for key in ("tts", "stt", "tts_off", "stt_off")], [True, True, "", ""])
|
||||||
|
self.assertEqual([local.describe()[key] for key in ("tts", "stt")], [False, False])
|
||||||
|
voiceless = engines.Speech(dataclasses.replace(built, service=Service(url="http://127.0.0.1:9/v1", key=""), voices=base / "none"), base / "voiceless")
|
||||||
|
self.assertEqual((voiceless.describe()["tts"], voiceless.describe()["stt"]), (False, True))
|
||||||
|
|
||||||
|
|
||||||
|
class TestSpeechServiceHttp(HubCase):
|
||||||
|
def setUp(self):
|
||||||
|
self.service = FakeService()
|
||||||
|
self.addCleanup(self.service.stop)
|
||||||
|
original = hub_case.write_engines
|
||||||
|
|
||||||
|
def remote(base: Path) -> engines.Engines:
|
||||||
|
return dataclasses.replace(original(base), service=Service(url=self.service.url, key=KEY))
|
||||||
|
|
||||||
|
with mock.patch.object(hub_case, "write_engines", remote):
|
||||||
|
super().setUp()
|
||||||
|
self.hand_out(VOICE)
|
||||||
|
|
||||||
|
def tts(self, text: str) -> tuple[int, dict, bytes]:
|
||||||
|
return self.post_json("/api/tts", {"text": text, "voice": VOICE}, self.guard(self.host))
|
||||||
|
|
||||||
|
def stt(self, clip: bytes) -> tuple[int, dict, bytes]:
|
||||||
|
return self.call("POST", "/api/stt", clip, {**self.guard(self.host), "Content-Type": "audio/wav"})
|
||||||
|
|
||||||
|
def local_calls(self) -> list[dict]:
|
||||||
|
return log_rows(self.engines.scratch / "piper.jsonl") + log_rows(self.engines.scratch / "whisper.jsonl")
|
||||||
|
|
||||||
|
def test_speech_output_renders_at_the_service_behind_the_cpu_gate_and_the_cache(self):
|
||||||
|
self.host_cpu(10.0, 5)
|
||||||
|
status, headers, body = self.tts(LINE)
|
||||||
|
self.assertEqual((status, headers["content-type"], body), (200, "audio/wav", SPOKEN))
|
||||||
|
[sent] = self.service.requests
|
||||||
|
self.assertEqual((sent["method"], sent["path"], sent["headers"]["authorization"], sent["headers"]["content-type"]), ("POST", "/v1/audio/speech", f"Bearer {KEY}", "application/json"))
|
||||||
|
self.assertEqual(json.loads(sent["body"]), {"model": speechservice.SPEECH_MODEL, "input": LINE, "voice": VOICE, "response_format": "wav"})
|
||||||
|
self.assertEqual(self.tts(LINE)[::2], (200, SPOKEN))
|
||||||
|
self.assertEqual(len(self.service.requests), 1)
|
||||||
|
for cpu, age, reason in ((99.0, 5, "busy"), (None, 5, "unknown"), (10.0, 900, "unknown")):
|
||||||
|
with self.subTest(cpu=cpu, age=age):
|
||||||
|
self.host_cpu(cpu, age)
|
||||||
|
status, _, body = self.tts(f"{LINE} again")
|
||||||
|
self.assertEqual((status, json.loads(body)), (503, {"reason": reason}))
|
||||||
|
self.assertEqual(self.tts(LINE)[::2], (200, SPOKEN))
|
||||||
|
self.assertEqual(len(self.service.requests), 1)
|
||||||
|
self.host_cpu(99.0, 5)
|
||||||
|
self.assertEqual(self.tts(MAYDAY)[0], 200)
|
||||||
|
self.assertEqual(json.loads(self.service.requests[-1]["body"])["input"], MAYDAY)
|
||||||
|
self.assertEqual(self.local_calls(), [])
|
||||||
|
|
||||||
|
def test_a_streamed_wav_with_placeholder_sizes_is_served_and_cached_with_the_real_sizes(self):
|
||||||
|
self.host_cpu(10.0, 5)
|
||||||
|
self.service.mode = "streamed"
|
||||||
|
self.assertEqual(self.tts(LINE)[::2], (200, SPOKEN))
|
||||||
|
self.assertEqual(self.tts(LINE)[::2], (200, SPOKEN))
|
||||||
|
self.assertEqual(len(self.service.requests), 1)
|
||||||
|
|
||||||
|
def test_speech_input_transcribes_at_the_service_within_the_hub_limits(self):
|
||||||
|
status, headers, body = self.stt(CLIP)
|
||||||
|
self.assertEqual((status, headers["content-type"], json.loads(body)), (200, "application/json", {"text": "hello from the service"}))
|
||||||
|
[sent] = self.service.requests
|
||||||
|
kind = sent["headers"]["content-type"]
|
||||||
|
boundary = kind.removeprefix("multipart/form-data; boundary=")
|
||||||
|
self.assertEqual((sent["path"], sent["headers"]["authorization"]), ("/v1/audio/transcriptions", f"Bearer {KEY}"))
|
||||||
|
self.assertNotEqual(boundary, kind)
|
||||||
|
parts = sent["body"].split(f"--{boundary}".encode())
|
||||||
|
self.assertEqual(parts[-1], b"--\r\n")
|
||||||
|
self.assertIn(f'name="model"\r\n\r\n{speechservice.HEARING_MODEL}\r\n'.encode(), parts[1])
|
||||||
|
self.assertIn(b'name="file"; filename="recording.wav"\r\nContent-Type: audio/wav\r\n\r\n' + CLIP + b"\r\n", sent["body"])
|
||||||
|
self.assertEqual(self.stt(b"not a wav")[0], 400)
|
||||||
|
with mock.patch.object(engines, "HEARING_RATE", 1):
|
||||||
|
self.assertEqual(self.stt(CLIP)[0], 503)
|
||||||
|
self.assertEqual(len(self.service.requests), 1)
|
||||||
|
self.assertEqual(self.local_calls(), [])
|
||||||
|
|
||||||
|
def test_a_failing_service_answers_502_without_the_key_and_leaves_no_partial_file(self):
|
||||||
|
self.host_cpu(10.0, 5)
|
||||||
|
cases = (
|
||||||
|
("error", "tts", "the speech service answered HTTP 500"),
|
||||||
|
("redirect", "tts", "the speech service answered HTTP 302"),
|
||||||
|
("garbage", "tts", "the speech service answered no complete WAV"),
|
||||||
|
("truncated", "tts", "the speech service answered no complete WAV"),
|
||||||
|
("overflow", "tts", "the speech service answered no complete WAV"),
|
||||||
|
("streamed-short", "tts", "the speech service answered no complete WAV"),
|
||||||
|
("short", "tts", "the speech service answered a truncated reply"),
|
||||||
|
("big", "tts", f"the speech service answered more than {len(SPOKEN)} bytes"),
|
||||||
|
("slow", "tts", "the speech service took longer than 0.5 seconds"),
|
||||||
|
("error", "stt", "the speech service answered HTTP 500"),
|
||||||
|
("notext", "stt", "the speech service answered no transcript"),
|
||||||
|
("short", "stt", "the speech service answered a truncated reply"),
|
||||||
|
)
|
||||||
|
for number, (mode, route, message) in enumerate(cases):
|
||||||
|
with self.subTest(mode=mode, route=route), mock.patch.object(speechservice, "SPOKEN_REPLY", len(SPOKEN)), mock.patch.object(speechservice, "SERVICE_TIMEOUT", 0.5):
|
||||||
|
self.service.mode = mode
|
||||||
|
status, _, body = self.tts(f"{LINE} {number}") if route == "tts" else self.stt(CLIP)
|
||||||
|
self.assertEqual((status, body.decode()), (502, f"{message}\n"))
|
||||||
|
self.assertNotIn(KEY, body.decode())
|
||||||
|
self.assertNotIn("GET", [sent["method"] for sent in self.service.requests])
|
||||||
|
self.assertEqual([path.name for path in (self.state / "tts").iterdir()], [])
|
||||||
|
self.service.stop()
|
||||||
|
self.service.mode = "ok"
|
||||||
|
self.assertEqual(self.tts(f"{LINE} refused")[::2], (502, b"the speech service failed: ConnectionRefusedError\n"))
|
||||||
|
self.assertEqual(self.local_calls(), [])
|
||||||
|
|
||||||
|
def test_a_trickling_service_hits_the_wall_clock_deadline_and_frees_the_speech_lock(self):
|
||||||
|
self.host_cpu(10.0, 5)
|
||||||
|
late = b"the speech service took longer than 1 seconds\n"
|
||||||
|
with mock.patch.object(speechservice, "SERVICE_TIMEOUT", 1.0):
|
||||||
|
started = time.monotonic()
|
||||||
|
outcome: list[tuple[int, bytes]] = []
|
||||||
|
stuck = threading.Thread(target=lambda: outcome.append(self.tts(f"{LINE} trickle")[::2]))
|
||||||
|
stuck.start()
|
||||||
|
time.sleep(0.3)
|
||||||
|
self.assertEqual(self.tts(MAYDAY)[::2], (200, SPOKEN))
|
||||||
|
stuck.join(10)
|
||||||
|
self.assertEqual(outcome, [(502, late)])
|
||||||
|
self.assertLess(time.monotonic() - started, 3.0)
|
||||||
|
self.service.mode = "trickle-head"
|
||||||
|
started = time.monotonic()
|
||||||
|
self.assertEqual(self.stt(CLIP)[::2], (502, late))
|
||||||
|
self.assertLess(time.monotonic() - started, 3.0)
|
||||||
|
self.assertEqual(sorted(path.name.startswith(engines.WORD_CACHE) for path in (self.state / "tts").iterdir()), [True])
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
@@ -0,0 +1,150 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import re
|
||||||
|
import unittest
|
||||||
|
|
||||||
|
from hub_support import HUB
|
||||||
|
|
||||||
|
NAME = "INFINITO_ARTEFACT_BASE"
|
||||||
|
EXPRESSION = re.compile(r"\$\{([A-Za-z_]\w*)(?::([-+])([^}]*))?\}")
|
||||||
|
COMMIT = "d1be6fde11ac6e0407606b4e42fe72d34add8037"
|
||||||
|
VOICE = "huggingface.co/rhasspy/piper-voices/resolve/e7229f7d33a10e145e109b886d661984201ac3c4/en/en_US/libritts/high/en_US-libritts-high.onnx"
|
||||||
|
PINNED = {
|
||||||
|
VOICE: "9127a559e11603f10b366d1a20ac7426826081dbc521de4c2420c57728d73f0f",
|
||||||
|
f"{VOICE}.json": "2efdc6d7f954588b8180132cbd9b8001933fdd00932c92bc92fd0d2028a9eb3d",
|
||||||
|
"huggingface.co/ggerganov/whisper.cpp/resolve/5359861c739e955e79d9a303bcbc70fb988958b1/ggml-base-q8_0.bin": "c577b9a86e7e048a0b7eada054f4dd79a56bbfa911fbdacf900ac5b567cbb7d9",
|
||||||
|
}
|
||||||
|
ARCHIVE = (f"codeload.github.com/ggml-org/whisper.cpp/tar.gz/{COMMIT}", "645445da884df6dbd52b30f82d7be7c01d1767c938714e477e786b88d529e9f4")
|
||||||
|
CLONE = ("clone", "https://github.com/ggml-org/whisper.cpp", f"v1.9.5@{COMMIT}")
|
||||||
|
|
||||||
|
|
||||||
|
def expand(text: str, values: dict[str, str]) -> str:
|
||||||
|
"""Args:
|
||||||
|
text: one instruction.
|
||||||
|
values: the build args in scope; any other name expands as unset.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
text with every ${name}, ${name:-word} and ${name:+word} expanded as
|
||||||
|
BuildKit does, an empty value counting as unset.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def value(match: re.Match) -> str:
|
||||||
|
name, operator, word = match.groups()
|
||||||
|
current = values.get(name, "")
|
||||||
|
if operator == "-":
|
||||||
|
return current or word
|
||||||
|
if operator == "+":
|
||||||
|
return word if current else ""
|
||||||
|
return current
|
||||||
|
|
||||||
|
return EXPRESSION.sub(value, text)
|
||||||
|
|
||||||
|
|
||||||
|
def instructions(text: str) -> list[str]:
|
||||||
|
joined = text.replace("\\\n", " ")
|
||||||
|
return [" ".join(line.split()) for line in joined.splitlines() if line.strip() and not line.lstrip().startswith("#")]
|
||||||
|
|
||||||
|
|
||||||
|
def stages(text: str) -> tuple[set[str], list[dict]]:
|
||||||
|
"""Returns:
|
||||||
|
The global ARG names and every stage as {"name", "base", "steps"} in
|
||||||
|
file order; a stage without AS is named by its index.
|
||||||
|
"""
|
||||||
|
shared: set[str] = set()
|
||||||
|
found: list[dict] = []
|
||||||
|
for step in instructions(text):
|
||||||
|
word, _, rest = step.partition(" ")
|
||||||
|
if word == "FROM":
|
||||||
|
parts = rest.split()
|
||||||
|
found.append({"name": parts[2] if parts[1:2] == ["AS"] else str(len(found)), "base": parts[0], "steps": []})
|
||||||
|
elif found:
|
||||||
|
found[-1]["steps"].append(step)
|
||||||
|
elif word == "ARG":
|
||||||
|
shared.add(rest.split("=")[0])
|
||||||
|
return shared, found
|
||||||
|
|
||||||
|
|
||||||
|
def rendered(build_args: dict[str, str]) -> tuple[list[tuple[str, str, str]], list[str]]:
|
||||||
|
"""Args:
|
||||||
|
build_args: the --build-arg values; a missing name is unset.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Every fetch of the stages the final image needs, as (kind, url, pin),
|
||||||
|
sorted, and every step of those stages after expansion.
|
||||||
|
"""
|
||||||
|
shared, found = stages((HUB / "Dockerfile").read_text(encoding="utf-8"))
|
||||||
|
index = {stage["name"]: number for number, stage in enumerate(found)}
|
||||||
|
wanted, seen, fetches, steps = [len(found) - 1], set(), [], []
|
||||||
|
while wanted:
|
||||||
|
number = wanted.pop()
|
||||||
|
if number in seen:
|
||||||
|
continue
|
||||||
|
seen.add(number)
|
||||||
|
stage = found[number]
|
||||||
|
base = expand(stage["base"], {name: build_args[name] for name in shared if name in build_args})
|
||||||
|
wanted += [index[base]] if base in index else []
|
||||||
|
scope: dict[str, str] = {}
|
||||||
|
for raw in stage["steps"]:
|
||||||
|
if raw.startswith("ARG "):
|
||||||
|
name = raw[4:].split("=")[0]
|
||||||
|
scope[name] = build_args.get(name, "")
|
||||||
|
continue
|
||||||
|
step = expand(raw, scope)
|
||||||
|
steps.append(step)
|
||||||
|
source = re.search(r"--from=(\S+)", step)
|
||||||
|
wanted += [index[source[1]]] if step.startswith("COPY ") and source else []
|
||||||
|
added = re.fullmatch(r"ADD --checksum=sha256:([0-9a-f]{64}) (?:--chmod=\S+ )?(\S+) (\S+)", step)
|
||||||
|
if added:
|
||||||
|
fetches.append(("add", added[2], added[1]))
|
||||||
|
cloned = re.search(r"git clone --depth 1 --branch (\S+) (\S+) /src && test \"\$\(cd /src && git rev-parse HEAD\)\" = ([0-9a-f]{40})", step)
|
||||||
|
if cloned:
|
||||||
|
fetches.append(("clone", cloned[2], f"{cloned[1]}@{cloned[3]}"))
|
||||||
|
return sorted(fetches), steps
|
||||||
|
|
||||||
|
|
||||||
|
def through(base: str) -> list[tuple[str, str, str]]:
|
||||||
|
return sorted([("add", f"{base}/{ARCHIVE[0]}", ARCHIVE[1]), *(("add", f"{base}/{path}", pin) for path, pin in PINNED.items())])
|
||||||
|
|
||||||
|
|
||||||
|
class TestArtefactBase(unittest.TestCase):
|
||||||
|
def test_unset_the_build_fetches_the_upstream_urls_and_clones_as_before(self):
|
||||||
|
upstream = sorted([CLONE, *(("add", f"https://{path}", pin) for path, pin in PINNED.items())])
|
||||||
|
for build_args in ({}, {NAME: ""}):
|
||||||
|
with self.subTest(build_args=build_args):
|
||||||
|
self.assertEqual(rendered(build_args)[0], upstream)
|
||||||
|
|
||||||
|
def test_set_every_fetch_goes_through_the_base_with_the_same_pins(self):
|
||||||
|
for base in ("http://10.0.0.5", "http://10.0.0.5:8081/cache", "https://cache.example"):
|
||||||
|
with self.subTest(base=base):
|
||||||
|
fetches, steps = rendered({NAME: base})
|
||||||
|
self.assertEqual(fetches, through(base))
|
||||||
|
self.assertEqual([step for step in steps if "https://" in step.replace(base, "") or "git clone" in step], [])
|
||||||
|
self.assertEqual(rendered({NAME: "https:/"})[0], through("https:/"))
|
||||||
|
self.assertEqual(through("https:/"), sorted([("add", f"https://{ARCHIVE[0]}", ARCHIVE[1]), *(("add", f"https://{path}", pin) for path, pin in PINNED.items())]))
|
||||||
|
|
||||||
|
def test_every_add_is_pinned_and_reads_the_base_and_each_stage_declares_it(self):
|
||||||
|
shared, found = stages((HUB / "Dockerfile").read_text(encoding="utf-8"))
|
||||||
|
self.assertIn(NAME, shared)
|
||||||
|
for stage in found:
|
||||||
|
adds = [step for step in stage["steps"] if step.startswith("ADD ")]
|
||||||
|
with self.subTest(stage=stage["name"]):
|
||||||
|
for step in adds:
|
||||||
|
self.assertRegex(step, r"^ADD --checksum=sha256:[0-9a-f]{64} (?:--chmod=\S+ )?\$\{" + NAME + r":-https:/\}/\S+ /\S+$")
|
||||||
|
declared = [number for number, other in enumerate(stage["steps"]) if other == f"ARG {NAME}"]
|
||||||
|
self.assertTrue(declared and declared[0] < stage["steps"].index(step), step)
|
||||||
|
self.assertEqual(sum(step.startswith("ADD ") for stage in found for step in stage["steps"]), len(PINNED) + 1)
|
||||||
|
|
||||||
|
|
||||||
|
class TestComposeWiring(unittest.TestCase):
|
||||||
|
def test_compose_hands_the_base_to_the_build_and_the_speech_service_to_the_hub(self):
|
||||||
|
compose = (HUB / "compose.yml").read_text(encoding="utf-8")
|
||||||
|
self.assertRegex(compose, r"(?m)^ build:\n context: \.\n args:\n - " + NAME + "$")
|
||||||
|
self.assertIn(' - --speech-url\n - "${KVB_ORCHESTRATOR_SPEECH_URL}"\n - --speech-key-file\n - /run/kvb-orchestrator/speech.key\n', compose)
|
||||||
|
self.assertIn(' source: "${KVB_ORCHESTRATOR_SPEECH_KEY_FILE:-/dev/null}"\n target: /run/kvb-orchestrator/speech.key\n read_only: true\n', compose)
|
||||||
|
env = (HUB / ".env").read_text(encoding="utf-8").splitlines()
|
||||||
|
self.assertLessEqual({"KVB_ORCHESTRATOR_SPEECH_URL=", "KVB_ORCHESTRATOR_SPEECH_KEY_FILE="}, set(env))
|
||||||
|
self.assertNotIn(NAME, "\n".join(env))
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
Reference in new issue
Block a user