test(voice): use a German sentence as the German Whisper fixture
The small local model heard the two-word hallo welt clip as English, both the recording and an espeak-ng rendering; the espeak-ng sentence Hallo Welt, heute teste ich die Spracheingabe. is detected as German. The integration test checks the language and the start of the transcript, and 013 decision 8 records the change (operator 2026-11-17). 🎯 Fix: 95 % 💥 Break: 2 % 💸 Tokens: 100 % ⏱️ Velocity: 100 % 🛡️ Security: 100 % (the WAV fixture is binary, not scanned) 🏆 Score: 93 🚦 Gate: ✅ 90/5, tokens and velocity <= 100, security 100 % Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
8b8b82caee
commit
09e3f6dabe
3 files changed
+14
-4
No files matched your search
@@ -28,6 +28,13 @@ keyboard on a desktop and on a small screen.
|
|||||||
works with mouse, touch and keyboard.
|
works with mouse, touch and keyboard.
|
||||||
7. **Order**: built in parallel with 012 in its own sidetree; 012's
|
7. **Order**: built in parallel with 012 in its own sidetree; 012's
|
||||||
feedback popup and marks get the microphone through the shared component.
|
feedback popup and marks get the microphone through the shared component.
|
||||||
|
8. **German fixture (operator-confirmed 2026-11-17)**: the local small model
|
||||||
|
detects a two-word "hallo welt" clip as English, recorded and spoken by
|
||||||
|
espeak-ng alike, while it detects a German sentence as German; the
|
||||||
|
German fixture is the espeak-ng sentence "Hallo Welt, heute teste ich die
|
||||||
|
Spracheingabe." and its criterion checks the language and the start of
|
||||||
|
the transcript. Rejected: a recording by the operator, keeping the
|
||||||
|
two-word fixture with an open criterion.
|
||||||
|
|
||||||
## Approach
|
## Approach
|
||||||
|
|
||||||
@@ -60,8 +67,10 @@ flowchart TD
|
|||||||
|
|
||||||
### Transcription
|
### Transcription
|
||||||
|
|
||||||
- [ ] A POST of the German fixture WAV to `/api/stt` returns "hallo welt",
|
- [ ] A POST of the German fixture WAV, the sentence "Hallo Welt, heute
|
||||||
the English fixture returns "hello world".
|
teste ich die Spracheingabe." spoken by espeak-ng, to `/api/stt`
|
||||||
|
returns language `de` and a transcript starting with "hallo welt"; the
|
||||||
|
English fixture returns language `en` and "hello world".
|
||||||
- [ ] The transcription's temp directory is empty after each request,
|
- [ ] The transcription's temp directory is empty after each request,
|
||||||
including a failed one.
|
including a failed one.
|
||||||
- [ ] A request from another origin, without the CSRF token, over 60 s or
|
- [ ] A request from another origin, without the CSRF token, over 60 s or
|
||||||
|
|||||||
Vendored
BIN
Binary file not shown.
@@ -25,11 +25,12 @@ class TestVoiceModel(HubCase):
|
|||||||
def test_the_real_model_transcribes_the_german_and_the_english_fixture(self):
|
def test_the_real_model_transcribes_the_german_and_the_english_fixture(self):
|
||||||
speech = self.httpd.hub.speech
|
speech = self.httpd.hub.speech
|
||||||
with mock.patch.object(speech, "engines", dataclasses.replace(speech.engines, whisper=WHISPER, whisper_model=MODEL)):
|
with mock.patch.object(speech, "engines", dataclasses.replace(speech.engines, whisper=WHISPER, whisper_model=MODEL)):
|
||||||
for name, words in (("hallo-welt.wav", "hallo welt"), ("hello-world.wav", "hello world")):
|
for name, language, words in (("hallo-welt.wav", "de", "hallo welt"), ("hello-world.wav", "en", "hello world")):
|
||||||
with self.subTest(name=name):
|
with self.subTest(name=name):
|
||||||
status, _, body = self.call("POST", "/api/stt", (FIXTURES / name).read_bytes(), {**self.guard(self.host), "Content-Type": "audio/wav"})
|
status, _, body = self.call("POST", "/api/stt", (FIXTURES / name).read_bytes(), {**self.guard(self.host), "Content-Type": "audio/wav"})
|
||||||
self.assertEqual(status, 200, body)
|
self.assertEqual(status, 200, body)
|
||||||
self.assertEqual(spoken(json.loads(body)["text"]), words)
|
found = json.loads(body)
|
||||||
|
self.assertEqual((found["language"], spoken(found["text"]).startswith(words)), (language, True), found)
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
|
|||||||
Reference in new issue
Block a user