feat(orchestrator): 003 W6 speech bubbles at the agents in every view
A toolbar toggle, a hub setting off by default, shows a speech bubble per agent anchored at its face in every view and at its projected node in 3D; an agent without an anchor speaks from a caption bar. The speaking agent's bubble is the largest and reveals its line word by word over the audio duration, or at 180 words a minute without TTS, a failed TTS request included; queued agents wait as smaller ellipsis bubbles sized by order, and a finished bubble fades out while the next one grows. Comic Neue 400 latin is vendored under SIL OFL 1.1 and loaded through the FontFace API, so the CSP stays unchanged. Residuals: the caption says speaking while bubbles run silently; the word reveal follows the wall clock, not the audio clock; the e2e suite runs once in the unify gate (operator order 2026-11-11 21:15), score gate exit 1 accepted under that order. 🎯 Fix: 82 % 💥 Break: 5 % 💸 Cost: 100 % ⏱️ Time: 100 % 🛡️ Security: 100 % 🏆 Score: 78 🚦 Gate: ⛔ fix 82 % < 90 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Kevin Veen-BirkenbachandClaude Opus 5.5 committed
1 parent
1bcc8c135f
commit
770667ced9
19 files changed
+788
-20
No files matched your search
@@ -435,9 +435,9 @@ Both commands also patch the target's `.claude/settings.local.json` — the mach
|
||||
|
||||
The global install also runs the Agent Orchestrator, a local review page in which the operator approves or rejects the agents' staged commits (the `kvb-git-diff` skill): it copies `orchestrator/` to `~/.local/share/kvb-orchestrator`, starts it there with `docker compose up -d --build` (restart `unless-stopped`, bound to `127.0.0.1` only, state in `~/.local/state/kvb-orchestrator`) and registers its MCP server for Claude Code at user scope. Its port is `KVB_ORCHESTRATOR_PORT` in `orchestrator/.env`; the hub page is `http://localhost:<port>/`, the MCP endpoint `/mcp`. Without docker the hub is skipped and the commit skills ask in chat. The hub reads new radio traffic aloud in maritime phraseology and takes push-to-talk input; its image builds both engines from pinned, sha256-verified sources and runs each as a separate process: Piper (`piper-tts`, GPL-3.0-or-later) with the `en_US-libritts-high` voice (904 speakers, trained on LibriTTS, CC BY 4.0, from `rhasspy/piper-voices`) and whisper.cpp (MIT) with the multilingual `ggml-base-q8_0` model (OpenAI Whisper weights, MIT, 82 MB). The operator's own voice is `KVB_ORCHESTRATOR_OPERATOR_VOICE` in `orchestrator/.env`. Its Tree reads only the file `pkgmgr branch tree` writes: the install clones `PKGMGR_REPO` (default the GitHub repository, never the sandbox-writable local checkout), builds the newest `v*` tag a fully trusted GPG key signed into `~/.local/share/kvb-orchestrator/pkgmgr`, and enables the `kvb-orchestrator-tree.timer` user timer, which writes `~/.local/state/pkgmgr/tree/branch-tree.json` a minute after its previous run while the hub page polls, on at most half a core; the hub mounts that directory read-only and starts no git. A signed tag without `branch tree` leaves the timer off.
|
||||
|
||||
Stream → Radio (`#stream/radio`) is the hub's radio interface. Its station list shows one line per station, the full name in the tooltip, with its state and a play or stop button; when the browser blocks autoplay, the stream fails or the CSP refuses its origin, the row and a toast say so, and one click starts the station. New stations play daily around the clock. While the hub reads a message aloud, the speaking agent pulses there and in the Orbit and Chord views, under the caption "<Agent> speaking" with the message's first line; under `prefers-reduced-motion` the ring stays still. The composer sends on all seven channels: post and call to one agent, newsletter, broadcast, pan-pan, security and mayday to all. "Record" records with MediaRecorder; the hub transcribes the recording with whisper.cpp off the request thread (at most 120 s of audio, a 120 s timeout, four at a time), keeps it for replay in `~/.local/state/kvb-orchestrator/voice` (the newest 50) and puts the transcript into the field to edit; the message goes out as text, with the recording's host path and sha256 only when "Attach recording" is ticked. The hub transcribes at most six recordings a minute and 600 s of audio an hour. Every operator message is passkey-signed, and the hub verifies the signature against the pinned keys, each once, before it relays; `operator_message(id, text, to)` answers `match` only for the signed text and recipient; for a denial the signature covers the recipient the inbox resolved at signing, and the answer's text is only the signed reason, never the agent's own words the radio row quotes. "Cancel mayday", Evening, Sunset, Sunrise and a settings save that adds a stream URL take the passkey the same way.
|
||||
Stream → Radio (`#stream/radio`) is the hub's radio interface. Its station list shows one line per station, the full name in the tooltip, with its state and a play or stop button; when the browser blocks autoplay, the stream fails or the CSP refuses its origin, the row and a toast say so, and one click starts the station. New stations play daily around the clock. While the hub reads a message aloud, the speaking agent pulses there and in the Orbit and Chord views, under the caption "<Agent> speaking" with the message's first line; under `prefers-reduced-motion` the ring stays still. The speech bubble icon in the header (a hub setting, kept across reloads) puts a bubble at each speaking agent in the Agents graph, 3D, Hierarchy, Radio and Stream views, or in a caption bar at the bottom when the view does not show that agent: the agent being read gets the largest bubble and its words appear in time with the audio, waiting messages show a smaller `…` bubble per agent, and without speech output the same rows run at 180 words a minute, at least 1.5 s each. The composer sends on all seven channels: post and call to one agent, newsletter, broadcast, pan-pan, security and mayday to all. "Record" records with MediaRecorder; the hub transcribes the recording with whisper.cpp off the request thread (at most 120 s of audio, a 120 s timeout, four at a time), keeps it for replay in `~/.local/state/kvb-orchestrator/voice` (the newest 50) and puts the transcript into the field to edit; the message goes out as text, with the recording's host path and sha256 only when "Attach recording" is ticked. The hub transcribes at most six recordings a minute and 600 s of audio an hour. Every operator message is passkey-signed, and the hub verifies the signature against the pinned keys, each once, before it relays; `operator_message(id, text, to)` answers `match` only for the signed text and recipient; for a denial the signature covers the recipient the inbox resolved at signing, and the answer's text is only the signed reason, never the agent's own words the radio row quotes. "Cancel mayday", Evening, Sunset, Sunrise and a settings save that adds a stream URL take the passkey the same way.
|
||||
|
||||
The hub page loads nothing from the network: its icons come from Bootstrap Icons 1.13.1 (`orchestrator/vendor/bootstrap-icons`, MIT), inlined per page as the symbols it uses, and its 3D views from `3d-force-graph` 1.80.1 and `three` 0.186.1 (`orchestrator/vendor`, MIT, served by the hub under `/vendor/`). In the Artifacts Tree, "Delete" on a `merged-clean` branch without skip reason, or "Delete all merged", shows the dry-run lines and asks for the passkey signature; the hub then writes a job (repository, branch, worktree and tip per item, no command) into `~/.local/state/kvb-orchestrator/cleanup/jobs`. The `kvb-orchestrator-cleanup.path` user unit runs `client/cleanup.py` on the host, which verifies the signature against the pinned keys, runs `pkgmgr branch cleanup --json --local --branch <name>` as a dry run in the repository, then `--apply` only when that plan names exactly the signed branch as `merged-clean` at the signed tip and worktree, refuses everything else unchanged, writes each result to `cleanup/results` and rewrites the tree file. The buttons stay disabled while the tree file says `delete_ready: false`.
|
||||
The hub page loads nothing from the network: its icons come from Bootstrap Icons 1.13.1 (`orchestrator/vendor/bootstrap-icons`, MIT), inlined per page as the symbols it uses, and its 3D views from `3d-force-graph` 1.80.1 and `three` 0.186.1 (`orchestrator/vendor`, MIT, served by the hub under `/vendor/`); the speech bubbles use Comic Neue 400 latin from `@fontsource/comic-neue` 5.3.0 (`orchestrator/vendor/comic-neue`, SIL OFL 1.1), which the page fetches and adds through the FontFace API, so the CSP needs no `font-src`. In the Artifacts Tree, "Delete" on a `merged-clean` branch without skip reason, or "Delete all merged", shows the dry-run lines and asks for the passkey signature; the hub then writes a job (repository, branch, worktree and tip per item, no command) into `~/.local/state/kvb-orchestrator/cleanup/jobs`. The `kvb-orchestrator-cleanup.path` user unit runs `client/cleanup.py` on the host, which verifies the signature against the pinned keys, runs `pkgmgr branch cleanup --json --local --branch <name>` as a dry run in the repository, then `--apply` only when that plan names exactly the signed branch as `merged-clean` at the signed tip and worktree, refuses everything else unchanged, writes each result to `cleanup/results` and rewrites the tree file. The buttons stay disabled while the tree file says `delete_ready: false`.
|
||||
|
||||
The Agents view "Hierarchy" draws the council's formal offices as a collapsible org chart: Operator, chair, dimensions with their line representatives, organisers (`cluster/<slug>`) and presidents (`work/<slug>`); the chair in every graph is the holder of the `chair` office, not a fixed name. Appointing, handing over or recalling an office there asks for the passkey signature; the hub writes a job (office, signed holder, successor session, work; no command) into `~/.local/state/kvb-orchestrator/offices/jobs`, and the `kvb-orchestrator-offices.path` user unit runs `client/offices.py` on the host, which verifies the signature, refuses a council directory others can write, a link on the path or a holder that changed since the signature. For a takeover or recall it moves the office aside and removes it only while the claim there still names the signed holder; an appoint or takeover then runs a plain `council.py claim <office>` for the successor by argv. council.py itself has no flag that overrides a live holder. The same view holds the operator's agent order as a drag list with up and down buttons: `rasci.py agents order` stores it in `agent-order.json` beside the register, where it breaks ties of equal task priority and orders the chair's grants without touching any task's `--prio` pin.
|
||||
|
||||
|
||||
@@ -41,8 +41,14 @@ OFFICES_ROUTE = "/api/offices"
|
||||
OFFICE_STATE = re.compile(r"/api/offices/([0-9a-f]{16})")
|
||||
TASKS_ROUTE = "/api/tasks"
|
||||
VENDOR_ROUTE = re.compile(r"/vendor/([a-z0-9.-]{1,64})")
|
||||
VENDOR = {"3d-force-graph.min.js": "3d-force-graph/3d-force-graph.min.js", "three.module.js": "three/three.module.js", "three.core.js": "three/three.core.js"}
|
||||
SCRIPT_TYPE = "text/javascript; charset=utf-8"
|
||||
FONT_TYPE = "font/woff2"
|
||||
VENDOR = {
|
||||
"3d-force-graph.min.js": ("3d-force-graph/3d-force-graph.min.js", SCRIPT_TYPE),
|
||||
"three.module.js": ("three/three.module.js", SCRIPT_TYPE),
|
||||
"three.core.js": ("three/three.core.js", SCRIPT_TYPE),
|
||||
"comic-neue.woff2": ("comic-neue/comic-neue-latin-400-normal.woff2", FONT_TYPE),
|
||||
}
|
||||
VOICE_ROUTE = "/api/voice"
|
||||
VOICE_ITEM = re.compile(r"/api/voice/([0-9a-f]{16})(\.wav)?")
|
||||
MESSAGE_SIGN_ROUTE = "/api/messages/sign"
|
||||
@@ -82,7 +88,7 @@ class Handler(Exchange):
|
||||
elif path == "/":
|
||||
self.index()
|
||||
elif vendored and vendored.group(1) in VENDOR:
|
||||
self.vendored(VENDOR[vendored.group(1)])
|
||||
self.vendored(*VENDOR[vendored.group(1)])
|
||||
elif voice and voice.group(2):
|
||||
self.replay_voice(voice.group(1))
|
||||
elif voice:
|
||||
@@ -248,9 +254,9 @@ class Handler(Exchange):
|
||||
headers = {"Content-Security-Policy": csp, "X-Frame-Options": "DENY"}
|
||||
self.send(HTTPStatus.OK, "text/html; charset=utf-8", page.encode("utf-8"), headers)
|
||||
|
||||
def vendored(self, name: str) -> None:
|
||||
def vendored(self, name: str, content_type: str) -> None:
|
||||
headers = {"Content-Security-Policy": PLAIN_CSP, "Cache-Control": "private, max-age=86400"}
|
||||
self.attempt((self.server.hub.vendor / name).read_bytes, lambda data: self.send(HTTPStatus.OK, SCRIPT_TYPE, data, headers))
|
||||
self.attempt((self.server.hub.vendor / name).read_bytes, lambda data: self.send(HTTPStatus.OK, content_type, data, headers))
|
||||
|
||||
def stored(self, review_id: str, name: str) -> None:
|
||||
suffix, content_type, csp = STORED[name]
|
||||
|
||||
@@ -49,6 +49,7 @@ SETTINGS = {
|
||||
},
|
||||
"estimation": "none",
|
||||
"trend": {"mode": "off", "days": 7},
|
||||
"bubbles": False,
|
||||
}
|
||||
SPACE_AXES = ("none", "time", "spawn")
|
||||
SPACE_LIMITS = {
|
||||
@@ -220,6 +221,7 @@ def valid_settings(raw: object) -> dict:
|
||||
"space": space_setting(data["space"]),
|
||||
"estimation": data["estimation"],
|
||||
"trend": trend_setting(data["trend"]),
|
||||
"bubbles": flag(data["bubbles"], "bubbles"),
|
||||
}
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,49 @@
|
||||
const Bubbles = (() => {
|
||||
"use strict";
|
||||
const SCALES = [1, 0.8, 0.65];
|
||||
const SMALLEST = 0.5;
|
||||
const PACE = 180;
|
||||
const SHORTEST = 1.5;
|
||||
|
||||
function scale(position) {
|
||||
return SCALES[position] ?? SMALLEST;
|
||||
}
|
||||
|
||||
function words(text) {
|
||||
return String(text || "").split(/\s+/).filter(Boolean);
|
||||
}
|
||||
|
||||
function reading(count) {
|
||||
return Math.max(SHORTEST, (count * 60) / PACE);
|
||||
}
|
||||
|
||||
function reveals(count, duration) {
|
||||
return Array.from({length: count}, (_, index) => (index / count) * duration);
|
||||
}
|
||||
|
||||
function shown(count, duration, elapsed) {
|
||||
return reveals(count, duration).filter((at) => at <= elapsed).length;
|
||||
}
|
||||
|
||||
function picked(row, speech) {
|
||||
return row.class === "mayday" || speech.classes.includes(row.class);
|
||||
}
|
||||
|
||||
function plan(head, pending) {
|
||||
const queue = head ? [head, ...pending] : pending;
|
||||
const shift = head ? 0 : 1;
|
||||
const seen = new Set();
|
||||
return queue.flatMap((item, index) => {
|
||||
if (seen.has(item.who)) {
|
||||
return [];
|
||||
}
|
||||
seen.add(item.who);
|
||||
return [{who: item.who, item, scale: scale(index + shift)}];
|
||||
});
|
||||
}
|
||||
|
||||
return {SCALES, SMALLEST, PACE, SHORTEST, scale, words, reading, reveals, shown, picked, plan};
|
||||
})();
|
||||
if (typeof module === "object" && module.exports) {
|
||||
module.exports = Bubbles;
|
||||
}
|
||||
@@ -411,6 +411,8 @@ pre.notes { margin: 0; padding: 8px 10px; background: var(--sunken); border-radi
|
||||
{% include "style-ui.css" %}
|
||||
|
||||
{% include "style-tasks.css" %}
|
||||
|
||||
{% include "style-bubbles.css" %}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
@@ -421,6 +423,7 @@ pre.notes { margin: 0; padding: 8px 10px; background: var(--sunken); border-radi
|
||||
<div class="brand"><span class="logo" aria-hidden="true"></span><div class="brand-text"><h1>Agent Orchestrator</h1><span class="sky-note" id="sky-note"></span></div></div>
|
||||
<div class="tools">
|
||||
<button type="button" class="btn small ghost icon-only spot-toggle" id="radio-open" title="Radio: post, call or a message to all, with Record, the radio settings and the stations" aria-label="Radio" aria-controls="radio-spot" aria-expanded="false"><svg class="bi" aria-hidden="true" focusable="false"><use href="#bi-broadcast-pin"></use></svg></button>
|
||||
<button type="button" class="btn small ghost icon-only" id="bubbles-toggle" title="Speech bubbles" aria-label="Speech bubbles" aria-pressed="false" aria-controls="bubbles"><svg class="bi" aria-hidden="true" focusable="false"><use href="#bi-chat-quote"></use></svg></button>
|
||||
<button type="button" class="btn small ghost icon-only" id="palette-open" title="Commands (Ctrl+K)" aria-label="Commands"><svg class="bi" aria-hidden="true" focusable="false"><use href="#bi-command"></use></svg></button>
|
||||
<button type="button" class="btn small ghost icon-only" id="settings-open" title="Settings and ambience" aria-label="Settings"><svg class="bi" aria-hidden="true" focusable="false"><use href="#bi-gear"></use></svg></button>
|
||||
</div>
|
||||
@@ -442,6 +445,7 @@ pre.notes { margin: 0; padding: 8px 10px; background: var(--sunken); border-radi
|
||||
<div class="toasts" id="toasts" aria-live="polite"></div>
|
||||
<div class="palette" id="palette" hidden></div>
|
||||
<div class="sky-panel" id="sky-panel" role="group" aria-label="View of the sky" hidden></div>
|
||||
<div class="bubbles" id="bubbles" aria-hidden="true" hidden><div class="bubble-bar"></div></div>
|
||||
<script nonce="{{ nonce }}" id="state" type="application/json">{{ state | tojson }}</script>
|
||||
<script nonce="{{ nonce }}">
|
||||
{% include "ambience.js" %}
|
||||
@@ -449,6 +453,8 @@ pre.notes { margin: 0; padding: 8px 10px; background: var(--sunken); border-radi
|
||||
<script nonce="{{ nonce }}">
|
||||
{% include "voice.js" %}
|
||||
|
||||
{% include "bubbles.js" %}
|
||||
|
||||
{% include "nav.js" %}
|
||||
|
||||
{% include "popup.js" %}
|
||||
@@ -550,6 +556,8 @@ pre.notes { margin: 0; padding: 8px 10px; background: var(--sunken); border-radi
|
||||
|
||||
{% include "page-speech.js" %}
|
||||
|
||||
{% include "page-bubbles.js" %}
|
||||
|
||||
{% include "page-alerts.js" %}
|
||||
|
||||
{% include "page-rules.js" %}
|
||||
|
||||
@@ -15,6 +15,7 @@
|
||||
});
|
||||
document.getElementById("palette-open").addEventListener("click", openPalette);
|
||||
document.getElementById("settings-open").addEventListener("click", settingsSheet);
|
||||
document.getElementById("bubbles-toggle").addEventListener("click", toggleBubbles);
|
||||
document.addEventListener("click", (event) => closeMenus(event.target instanceof Element ? event.target.closest("details.menu") : null));
|
||||
document.addEventListener("fullscreenchange", () => {
|
||||
if (!document.fullscreenElement) {
|
||||
@@ -37,6 +38,7 @@
|
||||
radio.element = radioElement();
|
||||
drawSky();
|
||||
musicButton();
|
||||
syncBubbles();
|
||||
syncMusic();
|
||||
syncRadio();
|
||||
timeBar();
|
||||
|
||||
@@ -0,0 +1,125 @@
|
||||
const BUBBLE_FONT = "/vendor/comic-neue.woff2";
|
||||
const BUBBLE_FADE = 400;
|
||||
const BUBBLE_EDGE = 8;
|
||||
const bubbles = {shown: new Map(), frame: 0, font: null};
|
||||
|
||||
function bubbleLayer() {
|
||||
return document.getElementById("bubbles");
|
||||
}
|
||||
|
||||
function bubbleFont() {
|
||||
// The CSP has no font-src, so a url() face is refused; a face built from fetched bytes loads nothing itself.
|
||||
if (!bubbles.font) {
|
||||
bubbles.font = fetch(BUBBLE_FONT, {cache: "force-cache"})
|
||||
.then((response) => (response.ok ? response.arrayBuffer() : Promise.reject(new Error(String(response.status)))))
|
||||
.then((data) => new FontFace("Comic Neue", data, {style: "normal", weight: "400"}).load())
|
||||
.then((loaded) => document.fonts.add(loaded))
|
||||
.catch(() => null);
|
||||
}
|
||||
return bubbles.font;
|
||||
}
|
||||
|
||||
function inView(box) {
|
||||
return box.width > 0 && box.height > 0 && box.left + box.width > 0 && box.top + box.height > 0 && box.left < window.innerWidth && box.top < window.innerHeight;
|
||||
}
|
||||
|
||||
function anchorOf(key) {
|
||||
const projected = g3.tab === "agents" ? anchor3d(key)() : null;
|
||||
if (projected && inView(projected)) {
|
||||
return projected;
|
||||
}
|
||||
return [...document.querySelectorAll(`[data-face="${CSS.escape(key)}"]`)].map((node) => node.getBoundingClientRect()).find(inView) || null;
|
||||
}
|
||||
|
||||
function bubbleNode(item) {
|
||||
const words = h("span", {class: "bubble-words"});
|
||||
return {node: h("div", {class: "speech-bubble", "data-who": item.who}, h("strong", {class: "bubble-who"}, nameOf(item.who)), words), words};
|
||||
}
|
||||
|
||||
function placeBubble(view, box, scale) {
|
||||
const layer = bubbleLayer();
|
||||
const bar = layer.querySelector(".bubble-bar");
|
||||
if (!box) {
|
||||
if (view.node.parentElement !== bar) {
|
||||
view.node.style.removeProperty("left");
|
||||
view.node.style.removeProperty("top");
|
||||
bar.append(view.node);
|
||||
}
|
||||
return;
|
||||
}
|
||||
if (view.node.parentElement !== layer) {
|
||||
layer.append(view.node);
|
||||
}
|
||||
const [width, height] = [view.node.offsetWidth * scale, view.node.offsetHeight * scale];
|
||||
const middle = box.left + box.width / 2;
|
||||
const left = Math.min(Math.max(middle, BUBBLE_EDGE + width / 2), window.innerWidth - BUBBLE_EDGE - width / 2);
|
||||
view.node.style.left = `${fixed(left)}px`;
|
||||
view.node.style.top = `${fixed(Math.max(box.top - 4, BUBBLE_EDGE + height))}px`;
|
||||
view.node.style.setProperty("--tail", `${fixed((middle - left) / scale)}px`);
|
||||
}
|
||||
|
||||
function drawBubble(entry, view) {
|
||||
const {item} = entry;
|
||||
const said = item.words || Bubbles.words(item.line);
|
||||
const count = item.started ? Bubbles.shown(said.length, item.duration, (performance.now() - item.started) / 1000) : 0;
|
||||
const text = said.slice(0, count).join(" ") || "…";
|
||||
if (view.words.textContent !== text) {
|
||||
view.words.textContent = text;
|
||||
view.words.scrollTop = view.words.scrollHeight;
|
||||
}
|
||||
Object.assign(view.node.dataset, {scale: String(entry.scale), state: item.started ? "speaking" : "waiting", words: String(count)});
|
||||
view.node.style.setProperty("--scale", String(entry.scale));
|
||||
placeBubble(view, anchorOf(entry.who), entry.scale);
|
||||
}
|
||||
|
||||
function drawBubbles() {
|
||||
if (!settings.bubbles) {
|
||||
return;
|
||||
}
|
||||
const planned = Bubbles.plan(speech.current || speech.loading, speech.pending);
|
||||
const kept = new Set(planned.map((entry) => entry.item));
|
||||
for (const [item, view] of bubbles.shown) {
|
||||
if (!kept.has(item)) {
|
||||
bubbles.shown.delete(item);
|
||||
view.node.classList.add("leaving");
|
||||
window.setTimeout(() => view.node.remove(), BUBBLE_FADE);
|
||||
}
|
||||
}
|
||||
for (const entry of planned) {
|
||||
if (!bubbles.shown.has(entry.item)) {
|
||||
bubbles.shown.set(entry.item, bubbleNode(entry.item));
|
||||
}
|
||||
drawBubble(entry, bubbles.shown.get(entry.item));
|
||||
}
|
||||
if (bubbles.shown.size && !bubbles.frame) {
|
||||
bubbles.frame = window.requestAnimationFrame(() => {
|
||||
bubbles.frame = 0;
|
||||
drawBubbles();
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
function clearBubbles() {
|
||||
window.cancelAnimationFrame(bubbles.frame);
|
||||
bubbles.frame = 0;
|
||||
bubbles.shown.clear();
|
||||
for (const node of bubbleLayer().querySelectorAll(".speech-bubble")) {
|
||||
node.remove();
|
||||
}
|
||||
}
|
||||
|
||||
function syncBubbles() {
|
||||
document.getElementById("bubbles-toggle").setAttribute("aria-pressed", String(settings.bubbles));
|
||||
bubbleLayer().hidden = !settings.bubbles;
|
||||
if (settings.bubbles) {
|
||||
bubbleFont();
|
||||
drawBubbles();
|
||||
} else {
|
||||
clearBubbles();
|
||||
dropSilent();
|
||||
}
|
||||
}
|
||||
|
||||
function toggleBubbles() {
|
||||
saveSettings({...settings, bubbles: !settings.bubbles});
|
||||
}
|
||||
@@ -125,6 +125,7 @@
|
||||
const tasks = node.tasks.length ? ` · ${node.tasks.length} tasks` : "";
|
||||
return s("g", {
|
||||
class: classes, transform: `translate(${fixed(at.x)} ${fixed(at.y)})`, tabindex: 0, role: "button", "aria-label": `${node.label}, ${NODE_KINDS[node.kind]}`, "data-pick": node.id,
|
||||
"data-face": node.kind === "subagent" ? null : node.id,
|
||||
on: {
|
||||
click: () => select(node.id),
|
||||
keydown: (event) => {
|
||||
|
||||
@@ -72,6 +72,7 @@
|
||||
if (ui.screen === "agents" && (ui.views.agents === "orbit" || ui.views.agents === "chord")) {
|
||||
stage();
|
||||
}
|
||||
drawBubbles();
|
||||
}
|
||||
|
||||
function sentRows() {
|
||||
|
||||
@@ -13,6 +13,7 @@
|
||||
syncMusic();
|
||||
syncRadio();
|
||||
musicButton();
|
||||
syncBubbles();
|
||||
}
|
||||
|
||||
async function saveSettings(next) {
|
||||
@@ -187,6 +188,7 @@
|
||||
space: savedSpace(),
|
||||
estimation: settings.estimation,
|
||||
trend: settings.trend,
|
||||
bubbles: settings.bubbles,
|
||||
};
|
||||
if (await saveSettings(next)) {
|
||||
closeSheet();
|
||||
|
||||
@@ -12,30 +12,66 @@
|
||||
function say(item) {
|
||||
speech.pending = Voice.enqueue(speech.pending, item, SPEECH_QUEUE);
|
||||
pump();
|
||||
drawBubbles();
|
||||
}
|
||||
|
||||
function readSilently(item, next) {
|
||||
Object.assign(item, {started: performance.now(), duration: Bubbles.reading(item.words.length)});
|
||||
speech.current = item;
|
||||
speech.timer = window.setTimeout(() => {
|
||||
Object.assign(speech, {current: null, timer: 0});
|
||||
drawSpeaker();
|
||||
next();
|
||||
}, item.duration * 1000);
|
||||
drawSpeaker();
|
||||
}
|
||||
|
||||
function endSilent() {
|
||||
window.clearTimeout(speech.timer);
|
||||
Object.assign(speech, {current: null, timer: 0, busy: false});
|
||||
}
|
||||
|
||||
function dropSilent() {
|
||||
speech.pending = speech.pending.filter((item) => !item.silent);
|
||||
if (speech.timer) {
|
||||
endSilent();
|
||||
drawSpeaker();
|
||||
pump();
|
||||
}
|
||||
}
|
||||
|
||||
async function pump() {
|
||||
if (speech.busy || !speech.pending.length) {
|
||||
return;
|
||||
}
|
||||
const sound = context();
|
||||
if (!sound) {
|
||||
const [item, ...rest] = speech.pending;
|
||||
const sound = item.silent ? null : context();
|
||||
if (!item.silent && !sound) {
|
||||
speech.pending = [];
|
||||
return;
|
||||
}
|
||||
const [item, ...rest] = speech.pending;
|
||||
speech.pending = rest;
|
||||
speech.busy = true;
|
||||
const next = () => {
|
||||
speech.busy = false;
|
||||
pump();
|
||||
};
|
||||
if (item.silent) {
|
||||
readSilently(item, next);
|
||||
return;
|
||||
}
|
||||
speech.loading = item;
|
||||
drawBubbles();
|
||||
try {
|
||||
const response = await fetch("/api/tts", {method: "POST", headers: {"X-Kvb-Orchestrator": "1", "Content-Type": "application/json"}, body: JSON.stringify({text: item.text, voice: item.voice}), cache: "no-store"});
|
||||
if (!response.ok) {
|
||||
throw new Error(`${response.status} ${(await response.text()).trim()}`);
|
||||
}
|
||||
const buffer = await sound.decodeAudioData(await response.arrayBuffer());
|
||||
if (speech.loading !== item) {
|
||||
next();
|
||||
return;
|
||||
}
|
||||
const source = sound.createBufferSource();
|
||||
const gain = sound.createGain();
|
||||
gain.gain.value = loudness(item.tone);
|
||||
@@ -47,29 +83,45 @@
|
||||
next();
|
||||
}, {once: true});
|
||||
duck(buffer.duration);
|
||||
Object.assign(speech, {current: item, source});
|
||||
Object.assign(item, {started: performance.now(), duration: buffer.duration});
|
||||
Object.assign(speech, {current: item, source, loading: null});
|
||||
drawSpeaker();
|
||||
source.start();
|
||||
} catch (error) {
|
||||
const kept = speech.loading === item;
|
||||
speech.loading = null;
|
||||
speechFailed([String(error.message || error)]);
|
||||
if (kept && settings.bubbles && item.words) {
|
||||
readSilently(item, next);
|
||||
return;
|
||||
}
|
||||
next();
|
||||
}
|
||||
}
|
||||
|
||||
function cancelSpeech() {
|
||||
speech.pending = [];
|
||||
if (speech.timer) {
|
||||
endSilent();
|
||||
}
|
||||
Object.assign(speech, {pending: [], current: null, loading: null});
|
||||
if (speech.source) {
|
||||
speech.source.stop();
|
||||
}
|
||||
clearBubbles();
|
||||
drawSpeaker();
|
||||
}
|
||||
|
||||
function speakRows(rows) {
|
||||
if (!settings.speech.on || !state.speech.tts) {
|
||||
const voiced = settings.speech.on && state.speech.tts;
|
||||
if (!voiced && !settings.bubbles) {
|
||||
return;
|
||||
}
|
||||
for (const row of rows.filter((entry) => entry.ts >= state.now - SPEECH_FRESH && Voice.speakable(entry, settings.speech))) {
|
||||
const line = row.text.split("\n").map((part) => part.trim()).find(Boolean) || "";
|
||||
say({text: row.speech, voice: row.voice, tone: Voice.toneOf(row, HUMAN), cls: row.class, who: row.source, line: short(line)});
|
||||
for (const row of rows.filter((entry) => entry.ts >= state.now - SPEECH_FRESH)) {
|
||||
const audible = voiced && Voice.speakable(row, settings.speech);
|
||||
if (audible || (settings.bubbles && Bubbles.picked(row, settings.speech))) {
|
||||
const line = row.text.split("\n").map((part) => part.trim()).find(Boolean) || "";
|
||||
say({text: row.speech, voice: row.voice, tone: Voice.toneOf(row, HUMAN), cls: row.class, who: row.source, line: short(line), words: Bubbles.words(row.text), silent: !audible});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -45,7 +45,7 @@
|
||||
const sky = {canvas: document.getElementById("sky"), stars: Ambience.stars(320, 7), mode: "", frame: 0};
|
||||
const music = {gain: null, filter: null, drone: [], timer: 0, tension: 0, ducked: 0};
|
||||
const radio = {element: null, url: "", level: 0, timer: 0, playing: false, waiting: false, index: -1, pick: null, sig: "", state: "", short: "", note: "", told: null};
|
||||
const speech = {pending: [], busy: false, warned: 0, current: null, source: null};
|
||||
const speech = {pending: [], busy: false, warned: 0, current: null, source: null, loading: null, timer: 0};
|
||||
const mic = {recorder: null, held: false, busy: false, timer: 0};
|
||||
const space = {
|
||||
views: {graph: {...state.settings.space.graph}, sky: {...state.settings.space.sky}}, glides: {graph: null, sky: null}, edits: 0, saved: 0, timer: 0, look: false, compass: null,
|
||||
|
||||
@@ -0,0 +1,19 @@
|
||||
#bubbles-toggle[aria-pressed="true"] { background: var(--accent-soft); color: var(--accent); }
|
||||
.bubbles { position: fixed; inset: 0; z-index: 20; overflow: hidden; pointer-events: none; }
|
||||
.bubbles[hidden] { display: none; }
|
||||
.speech-bubble { --scale: 1; --tail: 0px; position: absolute; display: grid; gap: 2px; width: max-content; max-width: min(280px, 72vw); padding: 8px 13px; border: 2px solid var(--text); border-radius: 18px; background: var(--panel); color: var(--text); box-shadow: var(--shadow-lift); font: 400 var(--fs-md)/1.35 "Comic Neue", "Comic Sans MS", cursive; transform: translate(-50%, -100%) scale(var(--scale)); transform-origin: 50% 100%; transition: transform .3s var(--ease), opacity .4s var(--ease); }
|
||||
.speech-bubble::before, .speech-bubble::after { content: ""; position: absolute; top: 100%; left: calc(50% + var(--tail)); border-style: solid; border-color: transparent; }
|
||||
.speech-bubble::before { margin-left: -9px; border-width: 12px 9px 0; border-top-color: var(--text); }
|
||||
.speech-bubble::after { margin-left: -6px; border-width: 8px 6px 0; border-top-color: var(--panel); }
|
||||
.speech-bubble.leaving { opacity: 0; }
|
||||
.bubble-words { display: block; max-height: calc(6 * 1.35em); overflow: hidden; overflow-wrap: anywhere; scroll-behavior: smooth; }
|
||||
.speech-bubble[data-state="waiting"] .bubble-words { letter-spacing: .12em; }
|
||||
.bubble-who { display: none; color: var(--muted); font: 650 var(--fs-xs)/1.2 var(--sans); }
|
||||
.bubble-bar { position: absolute; right: 0; bottom: calc(var(--sp-4) + env(safe-area-inset-bottom, 0px)); left: 0; display: flex; flex-wrap: wrap-reverse; justify-content: center; align-items: flex-end; gap: var(--sp-3) var(--sp-4); padding: 0 var(--sp-3); }
|
||||
.bubble-bar .speech-bubble { position: relative; transform: scale(var(--scale)); }
|
||||
.bubble-bar .speech-bubble::before, .bubble-bar .speech-bubble::after { display: none; }
|
||||
.bubble-bar .bubble-who { display: block; }
|
||||
@media (prefers-reduced-motion: reduce) {
|
||||
.speech-bubble { transition: opacity .4s var(--ease); }
|
||||
.bubble-words { scroll-behavior: auto; }
|
||||
}
|
||||
+93
@@ -0,0 +1,93 @@
|
||||
Copyright 2014 The Comic Neue Project Authors (https://github.com/crozynski/comicneue) ComicNeue-LightItalic.ttf: Copyright 2014 The Comic Neue Project Authors (https://github.com/crozynski/comicneue) ComicNeue-Regular.ttf: Copyright 2014 The Comic Neue Project Authors (https://github.com/crozynski/comicneue) ComicNeue-Italic.ttf: Copyright 2014 The Comic Neue Project Authors (https://github.com/crozynski/comicneue) ComicNeue-Bold.ttf: Copyright 2014 The Comic Neue Project Authors (https://github.com/crozynski/comicneue) ComicNeue-BoldItalic.ttf: Copyright 2014 The Comic Neue Project Authors (https://github.com/crozynski/comicneue)
|
||||
|
||||
This Font Software is licensed under the SIL Open Font License, Version 1.1.
|
||||
This license is copied below, and is also available with a FAQ at:
|
||||
http://scripts.sil.org/OFL
|
||||
|
||||
|
||||
-----------------------------------------------------------
|
||||
SIL OPEN FONT LICENSE Version 1.1 - 26 February 2007
|
||||
-----------------------------------------------------------
|
||||
|
||||
PREAMBLE
|
||||
The goals of the Open Font License (OFL) are to stimulate worldwide
|
||||
development of collaborative font projects, to support the font creation
|
||||
efforts of academic and linguistic communities, and to provide a free and
|
||||
open framework in which fonts may be shared and improved in partnership
|
||||
with others.
|
||||
|
||||
The OFL allows the licensed fonts to be used, studied, modified and
|
||||
redistributed freely as long as they are not sold by themselves. The
|
||||
fonts, including any derivative works, can be bundled, embedded,
|
||||
redistributed and/or sold with any software provided that any reserved
|
||||
names are not used by derivative works. The fonts and derivatives,
|
||||
however, cannot be released under any other type of license. The
|
||||
requirement for fonts to remain under this license does not apply
|
||||
to any document created using the fonts or their derivatives.
|
||||
|
||||
DEFINITIONS
|
||||
"Font Software" refers to the set of files released by the Copyright
|
||||
Holder(s) under this license and clearly marked as such. This may
|
||||
include source files, build scripts and documentation.
|
||||
|
||||
"Reserved Font Name" refers to any names specified as such after the
|
||||
copyright statement(s).
|
||||
|
||||
"Original Version" refers to the collection of Font Software components as
|
||||
distributed by the Copyright Holder(s).
|
||||
|
||||
"Modified Version" refers to any derivative made by adding to, deleting,
|
||||
or substituting -- in part or in whole -- any of the components of the
|
||||
Original Version, by changing formats or by porting the Font Software to a
|
||||
new environment.
|
||||
|
||||
"Author" refers to any designer, engineer, programmer, technical
|
||||
writer or other person who contributed to the Font Software.
|
||||
|
||||
PERMISSION & CONDITIONS
|
||||
Permission is hereby granted, free of charge, to any person obtaining
|
||||
a copy of the Font Software, to use, study, copy, merge, embed, modify,
|
||||
redistribute, and sell modified and unmodified copies of the Font
|
||||
Software, subject to the following conditions:
|
||||
|
||||
1) Neither the Font Software nor any of its individual components,
|
||||
in Original or Modified Versions, may be sold by itself.
|
||||
|
||||
2) Original or Modified Versions of the Font Software may be bundled,
|
||||
redistributed and/or sold with any software, provided that each copy
|
||||
contains the above copyright notice and this license. These can be
|
||||
included either as stand-alone text files, human-readable headers or
|
||||
in the appropriate machine-readable metadata fields within text or
|
||||
binary files as long as those fields can be easily viewed by the user.
|
||||
|
||||
3) No Modified Version of the Font Software may use the Reserved Font
|
||||
Name(s) unless explicit written permission is granted by the corresponding
|
||||
Copyright Holder. This restriction only applies to the primary font name as
|
||||
presented to the users.
|
||||
|
||||
4) The name(s) of the Copyright Holder(s) or the Author(s) of the Font
|
||||
Software shall not be used to promote, endorse or advertise any
|
||||
Modified Version, except to acknowledge the contribution(s) of the
|
||||
Copyright Holder(s) and the Author(s) or with their explicit written
|
||||
permission.
|
||||
|
||||
5) The Font Software, modified or unmodified, in part or in whole,
|
||||
must be distributed entirely under this license, and must not be
|
||||
distributed under any other license. The requirement for fonts to
|
||||
remain under this license does not apply to any document created
|
||||
using the Font Software.
|
||||
|
||||
TERMINATION
|
||||
This license becomes null and void if any of the above conditions are
|
||||
not met.
|
||||
|
||||
DISCLAIMER
|
||||
THE FONT SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
||||
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTIES OF
|
||||
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT
|
||||
OF COPYRIGHT, PATENT, TRADEMARK, OR OTHER RIGHT. IN NO EVENT SHALL THE
|
||||
COPYRIGHT HOLDER BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
|
||||
INCLUDING ANY GENERAL, SPECIAL, INDIRECT, INCIDENTAL, OR CONSEQUENTIAL
|
||||
DAMAGES, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
|
||||
FROM, OUT OF THE USE OR INABILITY TO USE THE FONT SOFTWARE OR FROM
|
||||
OTHER DEALINGS IN THE FONT SOFTWARE.
|
||||
Binary file not shown.
@@ -1766,7 +1766,7 @@ class TestOrchestratorE2E(HubCase):
|
||||
with self.subTest(url=step["url"], width=step["width"]):
|
||||
self.assertNotIn("error", probe)
|
||||
self.assertEqual((probe["title"], probe["h1"]), ("(2) Agent Orchestrator", "Agent Orchestrator"))
|
||||
self.assertEqual(probe["header"], ["radio-open", "palette-open", "settings-open"])
|
||||
self.assertEqual(probe["header"], ["radio-open", "bubbles-toggle", "palette-open", "settings-open"])
|
||||
self.assertEqual((probe["tabs"], probe["subs"]), (TABS, SUBS))
|
||||
self.assertEqual(probe["badges"], {"tab-stream": None, "tab-inbox": "2", "tab-rules": None, "tab-agents": "1", "tab-artifacts": None, "tab-tasks": None})
|
||||
self.assertEqual(probe["banner"], "MAYDAY · Docs: disk full")
|
||||
@@ -1974,7 +1974,7 @@ class TestOrchestratorE2E(HubCase):
|
||||
jump, *walked = data["results"]
|
||||
self.assertEqual(jump, {
|
||||
"hash": "#stream", "focused": True, "tabs": [[tab, True, True] for tab in ("#stream/inbox", "#agents/orbit", "#artifacts/tree", "#tasks/list")], "closed": [True, "false"],
|
||||
"header": [["radio-open", "Radio", ""], ["palette-open", "Commands", ""], ["settings-open", "Settings", ""]], "radioTab": False,
|
||||
"header": [["radio-open", "Radio", ""], ["bubbles-toggle", "Speech bubbles", ""], ["palette-open", "Commands", ""], ["settings-open", "Settings", ""]], "radioTab": False,
|
||||
})
|
||||
for index, (width, _) in enumerate(WIDTHS):
|
||||
sent, saved, read = walked[3 * index : 3 * (index + 1)]
|
||||
|
||||
@@ -0,0 +1,220 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import sys
|
||||
import time
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from unittest import mock
|
||||
|
||||
HERE = Path(__file__).resolve().parent
|
||||
sys.path[:0] = [str(HERE.parent), str(HERE)]
|
||||
|
||||
import test_orchestrator_e2e as base
|
||||
from hub_case import HubCase
|
||||
from hub_support import iso
|
||||
|
||||
OUT = Path(os.environ.get("E2E_OUT", "/tmp/workbench-w6-shots"))
|
||||
CUE = "e2e-bubbles"
|
||||
FIRST = "Disk almost full on the build host, the gate waits for space now"
|
||||
SECOND = "Gate green for the merge"
|
||||
VIEWS = {1920: "agents/orbit", 1366: "stream/radio", 390: "agents/hierarchy"}
|
||||
SECOND_FROM = {1920: "Skills session", 1366: "Skills session", 390: "Docs"}
|
||||
CALM = [{"name": "prefers-reduced-motion", "value": "reduce"}]
|
||||
GUARD = """() => {
|
||||
window.e2eViolations = [];
|
||||
document.addEventListener("securitypolicyviolation", (event) => window.e2eViolations.push(`${event.violatedDirective} ${event.blockedURI}`));
|
||||
}"""
|
||||
READ = """
|
||||
const toggle = document.getElementById("bubbles-toggle");
|
||||
const layer = document.getElementById("bubbles");
|
||||
const live = () => all(".speech-bubble:not(.leaving)", layer);
|
||||
const speaking = () => live().find((node) => node.dataset.state === "speaking") || null;
|
||||
const waiting = () => live().find((node) => node.dataset.state === "waiting") || null;
|
||||
const read = (node) => {
|
||||
if (!node) {
|
||||
return null;
|
||||
}
|
||||
const style = getComputedStyle(node);
|
||||
const box = node.getBoundingClientRect();
|
||||
return {
|
||||
who: node.dataset.who, scale: Number(node.dataset.scale), state: node.dataset.state, words: Number(node.dataset.words), text: text(node.querySelector(".bubble-words")),
|
||||
anchored: node.parentElement === layer, rendered: Math.round(new DOMMatrix(style.transform).a * 100) / 100, transition: style.transitionProperty, font: style.fontFamily,
|
||||
box: {left: box.left, top: box.top, right: box.right, bottom: box.bottom},
|
||||
};
|
||||
};
|
||||
const cue = (word) => fetch("/api/tts", {method: "POST", headers: {"X-Kvb-Orchestrator": "1", "Content-Type": "application/json"}, body: JSON.stringify({text: word, voice: ""}), cache: "no-store"});
|
||||
"""
|
||||
BUBBLES_ON = """(async () => {@HELPERS@@READ@
|
||||
const before = [toggle.getAttribute("aria-pressed"), layer.hidden];
|
||||
toggle.click();
|
||||
const on = Boolean(await until(() => toggle.getAttribute("aria-pressed") === "true" && !layer.hidden, 8000));
|
||||
await cue("@CUE@");
|
||||
if (!(await until(() => speaking() && waiting(), 25000))) {
|
||||
return {before, on, error: "no pair of bubbles", bubbles: live().map(read), toasts: all("#toasts .toast").map(text)};
|
||||
}
|
||||
await sleep(400);
|
||||
return {before, on, first: read(speaking()), second: read(waiting())};
|
||||
})()"""
|
||||
BUBBLES_GROW = """(async () => {@HELPERS@@READ@
|
||||
const who = speaking() ? speaking().dataset.who : null;
|
||||
const seen = [];
|
||||
let [grown, faded, gone] = [null, null, null];
|
||||
const end = Date.now() + 15000;
|
||||
while (who && !grown && Date.now() < end) {
|
||||
const now = speaking();
|
||||
if (now && now.dataset.who === who) {
|
||||
seen.push(Number(now.dataset.words));
|
||||
} else if (now) {
|
||||
faded = Boolean(layer.querySelector(`.speech-bubble.leaving[data-who="${CSS.escape(who)}"]`));
|
||||
await sleep(450);
|
||||
gone = !layer.querySelector(`.speech-bubble[data-who="${CSS.escape(who)}"]`);
|
||||
grown = read(speaking());
|
||||
}
|
||||
await sleep(40);
|
||||
}
|
||||
return {who, seen: seen.filter((value, index) => value !== seen[index - 1]), faded, gone, grown};
|
||||
})()"""
|
||||
BUBBLES_DONE = """(async () => {@HELPERS@@READ@
|
||||
const cleared = Boolean(await until(() => !layer.querySelector(".speech-bubble"), 10000));
|
||||
const faces = [...document.fonts].filter((face) => face.family.replace(/["']/g, "") === "Comic Neue").map((face) => face.status);
|
||||
return {cleared, faces, hidden: layer.hidden, violations: window.e2eViolations || null};
|
||||
})()"""
|
||||
BUBBLES_3D = """(async () => {@HELPERS@@READ@
|
||||
const canvas = Boolean(await until(() => document.querySelector("#main .graph3d canvas"), 15000));
|
||||
await sleep(1500);
|
||||
await cue("@CUE@");
|
||||
if (!(await until(speaking, 25000))) {
|
||||
return {canvas, error: "no bubble in 3D", bubbles: live().map(read)};
|
||||
}
|
||||
const first = read(speaking());
|
||||
await sleep(500);
|
||||
const later = read(speaking());
|
||||
const graph = document.querySelector("#main .graph3d").getBoundingClientRect();
|
||||
return {canvas, first, later, graph: {left: graph.left, top: graph.top, right: graph.right, bottom: graph.bottom}, mode: text(document.querySelector("#main .space-note"))};
|
||||
})()"""
|
||||
BUBBLES_KEPT = """(async () => {@HELPERS@@READ@
|
||||
await sleep(300);
|
||||
const kept = [toggle.getAttribute("aria-pressed"), layer.hidden, toggle.title, toggle.getAttribute("aria-label")];
|
||||
toggle.click();
|
||||
const off = Boolean(await until(() => toggle.getAttribute("aria-pressed") === "false" && layer.hidden, 8000));
|
||||
return {kept, off, display: getComputedStyle(layer).display, bubbles: layer.querySelectorAll(".speech-bubble").length, violations: window.e2eViolations || null};
|
||||
})()"""
|
||||
|
||||
|
||||
def probe(template: str, cue: str = CUE) -> str:
|
||||
return template.replace("@HELPERS@", base.HELPERS).replace("@READ@", READ).replace("@CUE@", cue)
|
||||
|
||||
|
||||
@unittest.skipUnless(shutil.which(os.environ.get("CHROMIUM", "chromium")), "no Chromium")
|
||||
class TestWorkbenchBubbles(HubCase):
|
||||
drive = base.TestOrchestratorE2E.drive
|
||||
|
||||
def setUp(self):
|
||||
super().setUp()
|
||||
OUT.mkdir(parents=True, exist_ok=True)
|
||||
now = time.time()
|
||||
self.write_council({
|
||||
"work/s2/claim.json": {"office": "work/s2", "session": "s2", "runtime": "claude", "name": "Stage", "since": iso(now - 7200), "beat": now - 60, "work": "x"},
|
||||
"chair/claim.json": {"office": "chair", "session": "s1", "runtime": "claude", "name": "Skills session", "since": iso(now - 7200), "beat": now, "work": "allocate"},
|
||||
"news.jsonl": self.jsonl([
|
||||
{"ts": now - 600, "from": "Skills session", "to": ["Stage"], "text": "rebase please", "class": "post", "v": 2},
|
||||
{"ts": now - 500, "from": "Stage", "to": ["Skills session"], "text": "gate red", "class": "call", "v": 2},
|
||||
{"ts": now - 400, "from": "Docs", "to": [], "text": "docs are green", "class": "newsletter", "v": 2},
|
||||
]),
|
||||
})
|
||||
self.keys = {row["from"]: row["source"] for row in self.snapshot()["radio"]}
|
||||
hub = self.httpd.hub
|
||||
spoken = base.tone(1, 8000)
|
||||
|
||||
def synthesize(body: bytes) -> bytes:
|
||||
word = json.loads(body)["text"]
|
||||
if word.startswith(CUE):
|
||||
now = time.time()
|
||||
rows = [
|
||||
{"ts": now - 0.01, "from": "Stage", "to": [], "text": FIRST, "class": "security", "v": 2},
|
||||
{"ts": now, "from": "Docs" if word.endswith("-docs") else "Skills session", "to": [], "text": SECOND, "class": "security", "v": 2},
|
||||
]
|
||||
with (self.council / "news.jsonl").open("a", encoding="utf-8") as handle:
|
||||
handle.write(self.jsonl(rows))
|
||||
return spoken
|
||||
|
||||
patch = mock.patch.object(hub.speech, "synthesize", synthesize)
|
||||
patch.start()
|
||||
self.addCleanup(patch.stop)
|
||||
self.base = f"http://{self.host}/"
|
||||
|
||||
def steps(self, browser: str, calm: bool, three: bool) -> list[dict]:
|
||||
steps = []
|
||||
for width, height in base.WIDTHS:
|
||||
view = VIEWS[width]
|
||||
media = CALM if calm and width == 390 else []
|
||||
cue = f"{CUE}-docs" if SECOND_FROM[width] == "Docs" else CUE
|
||||
shot = str(OUT / f"{browser}-{width}-bubbles")
|
||||
steps += [
|
||||
{"width": width, "height": height, "url": f"{self.base}?{width}#{view}", "wait": 1200, "probe": probe(BUBBLES_ON, cue), "shot": f"{shot}-pair.png", "media": media},
|
||||
{"width": width, "height": height, "url": None, "wait": 0, "probe": probe(BUBBLES_GROW), "shot": f"{shot}-grown.png", "media": media},
|
||||
{"width": width, "height": height, "url": None, "wait": 0, "probe": probe(BUBBLES_DONE), "shot": None, "media": media},
|
||||
]
|
||||
if three and width == 1366:
|
||||
steps.append({"width": width, "height": height, "url": f"{self.base}?3d#agents/3d", "wait": 1500, "probe": probe(BUBBLES_3D), "shot": f"{shot}-3d.png"})
|
||||
steps.append({"width": width, "height": height, "url": f"{self.base}?{width}-reloaded#{view}", "wait": 1200, "probe": probe(BUBBLES_KEPT), "shot": f"{shot}-off.png", "media": media})
|
||||
return steps
|
||||
|
||||
def check(self, data: dict, browser: str, calm: bool, three: bool) -> None:
|
||||
results = iter(data["results"])
|
||||
for width, _ in base.WIDTHS:
|
||||
on, grow, done = next(results), next(results), next(results)
|
||||
solid = next(results) if three and width == 1366 else None
|
||||
kept = next(results)
|
||||
first_key, second_key = self.keys["Stage"], self.keys[SECOND_FROM[width]]
|
||||
with self.subTest(browser=browser, width=width):
|
||||
self.assertNotIn("error", on, on)
|
||||
self.assertEqual((on["before"], on["on"]), (["false", True], True))
|
||||
first, second = on["first"], on["second"]
|
||||
self.assertEqual((first["who"], first["scale"], first["rendered"], first["state"], first["anchored"]), (first_key, 1, 1, "speaking", True), first)
|
||||
self.assertTrue(0 < first["words"] < len(FIRST.split()), first)
|
||||
self.assertEqual(first["text"], " ".join(FIRST.split()[: first["words"]]))
|
||||
self.assertTrue(first["font"].startswith('"Comic Neue"'), first["font"])
|
||||
self.assertEqual((second["who"], second["scale"], second["rendered"], second["state"], second["text"], second["words"]), (second_key, 0.8, 0.8, "waiting", "…", 0), second)
|
||||
self.assertEqual(second["anchored"], SECOND_FROM[width] != "Docs", second)
|
||||
motion = "opacity" if calm and width == 390 else "transform, opacity"
|
||||
self.assertEqual((first["transition"], second["transition"]), (motion, motion))
|
||||
self.assertEqual(grow["who"], first_key)
|
||||
self.assertGreaterEqual(len(grow["seen"]), 3, grow)
|
||||
self.assertEqual(grow["seen"], sorted(grow["seen"]), grow)
|
||||
self.assertEqual(grow["seen"][-1], len(FIRST.split()), grow)
|
||||
self.assertEqual((grow["faded"], grow["gone"]), (True, True), grow)
|
||||
grown = grow["grown"]
|
||||
self.assertEqual((grown["who"], grown["scale"], grown["rendered"], grown["state"]), (second_key, 1, 1, "speaking"), grown)
|
||||
self.assertEqual(grown["text"], " ".join(SECOND.split()[: grown["words"]]))
|
||||
self.assertEqual(done, {"cleared": True, "faces": ["loaded"], "hidden": False, "violations": []})
|
||||
self.assertEqual(kept, {"kept": ["true", False, "Speech bubbles", "Speech bubbles"], "off": True, "display": "none", "bubbles": 0, "violations": []})
|
||||
if solid is not None:
|
||||
with self.subTest(browser=browser, view="3d"):
|
||||
self.assertNotIn("error", solid, solid)
|
||||
graph = solid["graph"]
|
||||
for read in (solid["first"], solid["later"]):
|
||||
self.assertEqual((read["who"], read["anchored"]), (first_key, True), solid)
|
||||
middle = (read["box"]["left"] + read["box"]["right"]) / 2
|
||||
self.assertTrue(graph["left"] <= middle <= graph["right"] and graph["top"] - 16 <= read["box"]["bottom"] <= graph["bottom"], solid)
|
||||
self.assertFalse(self.httpd.hub.settings.get()["bubbles"])
|
||||
|
||||
def test_chromium_reads_two_rows_without_tts_as_growing_bubbles_keeps_the_switch_and_hides_the_layer_at_three_widths(self):
|
||||
data = self.drive(self.steps("chromium", True, True), None, init=GUARD)
|
||||
(OUT / "chromium-bubbles.json").write_text(json.dumps(data, indent=1), encoding="utf-8")
|
||||
self.assertEqual(data["errors"], [])
|
||||
self.check(data, "chromium", True, True)
|
||||
|
||||
@unittest.skipUnless(shutil.which(os.environ.get("FIREFOX", "firefox")), "no Firefox")
|
||||
def test_firefox_reads_two_rows_without_tts_as_growing_bubbles_keeps_the_switch_and_hides_the_layer_at_three_widths(self):
|
||||
data = self.drive(self.steps("firefox", False, False), None, "bidi.mjs", os.environ.get("FIREFOX", "firefox"), init=GUARD)
|
||||
(OUT / "firefox-bubbles.json").write_text(json.dumps(data, indent=1), encoding="utf-8")
|
||||
self.assertEqual([error for error in data["errors"] if "/favicon.ico" not in error], [])
|
||||
self.check(data, "firefox", False, False)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,188 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import re
|
||||
import subprocess
|
||||
import unittest
|
||||
|
||||
from hub_case import HubCase
|
||||
from hub_support import HUB
|
||||
from kvb_orchestrator import exchange, settings, view
|
||||
|
||||
BUBBLES = HUB / "templates" / "bubbles.js"
|
||||
LAYER = HUB / "templates" / "page-bubbles.js"
|
||||
SPEECH = HUB / "templates" / "page-speech.js"
|
||||
STYLE = HUB / "templates" / "style-bubbles.css"
|
||||
FONT = HUB / "vendor" / "comic-neue" / "comic-neue-latin-400-normal.woff2"
|
||||
FONT_SHA256 = "b7753f30f8bd4925cf48a2ffe936c52ff1f16fe441f8c0245acf046296ac590f"
|
||||
HOLDER = "Copyright 2014 The Comic Neue Project Authors (https://github.com/crozynski/comicneue)"
|
||||
BUBBLES_PROBE = """
|
||||
const B = require(process.argv[1]);
|
||||
const item = (who, n) => ({who, n});
|
||||
const [a1, b1, c1, a2, b2, d1] = [item("A", 1), item("B", 1), item("C", 1), item("A", 2), item("B", 2), item("D", 1)];
|
||||
const planOf = (head, pending) => B.plan(head, pending).map((entry) => [entry.who, entry.item.n, entry.scale]);
|
||||
const speech = {classes: ["security", "post"]};
|
||||
console.log(JSON.stringify({
|
||||
scales: [0, 1, 2, 3, 4, 9].map(B.scale),
|
||||
reveals: [B.reveals(4, 2), B.reveals(5, 1), B.reveals(0, 3)],
|
||||
shown: [0, 0.49, 0.5, 1.49, 1.5, 9].map((at) => B.shown(4, 2, at)),
|
||||
reading: [0, 4, 5, 9, 180].map(B.reading),
|
||||
words: [B.words(" Disk almost\\nfull\\t now "), B.words(""), B.words(null)],
|
||||
plans: [planOf(a1, [b1, c1, a2, b2, d1]), planOf(null, [b1, a1, b2]), planOf(a1, []), planOf(null, [])],
|
||||
picked: [{class: "mayday"}, {class: "security"}, {class: "post"}, {class: "broadcast"}].map((row) => B.picked(row, speech)),
|
||||
}));
|
||||
"""
|
||||
SPEECH_PROBE = """
|
||||
const fs = require("fs");
|
||||
const vm = require("vm");
|
||||
const Bubbles = require(process.argv[1]);
|
||||
const code = fs.readFileSync(process.argv[2], "utf8");
|
||||
const settle = () => new Promise((resolve) => setImmediate(resolve));
|
||||
async function failed(bubbles, item, cancel) {
|
||||
const shown = [];
|
||||
const speech = {pending: [], busy: false, current: null, loading: null, timer: 0, warned: 0};
|
||||
const fetch = async () => {
|
||||
if (cancel) {
|
||||
speech.loading = null;
|
||||
}
|
||||
return {ok: false, status: 503, text: async () => "tts down "};
|
||||
};
|
||||
vm.runInNewContext(`${code}\\nsay(item);`, {
|
||||
Bubbles, speech, fetch, item, Date, settings: {bubbles}, SPEECH_QUEUE: 9, SPEECH_QUIET: 0,
|
||||
Voice: {enqueue: (pending, next) => [...pending, next]},
|
||||
toast: (title, lines) => shown.push(`${title}: ${lines.join(" ")}`),
|
||||
context: () => ({}), drawBubbles: () => {}, drawSpeaker: () => {},
|
||||
performance: {now: () => 1000}, window: {setTimeout: () => 7, clearTimeout: () => {}},
|
||||
});
|
||||
for (let round = 0; round < 5; round += 1) {
|
||||
await settle();
|
||||
}
|
||||
return {current: speech.current === item, busy: speech.busy, timer: speech.timer, started: item.started ?? null, duration: item.duration ?? null, shown};
|
||||
}
|
||||
(async () => {
|
||||
const row = () => ({text: "t", who: "A", line: "Disk almost full", words: ["Disk", "almost", "full"]});
|
||||
console.log(JSON.stringify({
|
||||
fallback: await failed(true, row(), false),
|
||||
off: await failed(false, row(), false),
|
||||
sample: await failed(true, {text: "t", who: "Operator", line: "radio check"}, false),
|
||||
cancelled: await failed(true, row(), true),
|
||||
}));
|
||||
})();
|
||||
"""
|
||||
|
||||
|
||||
class TestBubbleSchedule(unittest.TestCase):
|
||||
@classmethod
|
||||
def setUpClass(cls):
|
||||
done = subprocess.run(["node", "-e", BUBBLES_PROBE, str(BUBBLES)], capture_output=True, text=True, timeout=60, check=True)
|
||||
cls.data = json.loads(done.stdout)
|
||||
|
||||
def test_the_scale_falls_with_the_queue_position_to_at_least_a_half(self):
|
||||
self.assertEqual(self.data["scales"], [1, 0.8, 0.65, 0.5, 0.5, 0.5])
|
||||
|
||||
def test_word_i_of_n_appears_at_i_over_n_of_the_duration(self):
|
||||
self.assertEqual(self.data["reveals"], [[0, 0.5, 1, 1.5], [0, 0.2, 0.4, 0.6, 0.8], []])
|
||||
self.assertEqual(self.data["shown"], [1, 1, 2, 3, 4, 4])
|
||||
|
||||
def test_without_audio_a_row_lasts_its_reading_time_at_180_words_a_minute_and_at_least_one_and_a_half_seconds(self):
|
||||
self.assertEqual(self.data["reading"], [1.5, 1.5, 5 / 3, 3, 60])
|
||||
self.assertEqual(self.data["words"], [["Disk", "almost", "full", "now"], [], []])
|
||||
|
||||
def test_each_agent_gets_one_bubble_sized_by_its_earliest_item(self):
|
||||
self.assertEqual(self.data["plans"], [
|
||||
[["A", 1, 1], ["B", 1, 0.8], ["C", 1, 0.65], ["D", 1, 0.5]],
|
||||
[["B", 1, 0.8], ["A", 1, 0.65]],
|
||||
[["A", 1, 1]],
|
||||
[],
|
||||
])
|
||||
|
||||
def test_bubbles_follow_the_read_aloud_classes_and_mayday_always(self):
|
||||
self.assertEqual(self.data["picked"], [True, True, True, False])
|
||||
|
||||
|
||||
class TestSpeechFallback(unittest.TestCase):
|
||||
def test_a_failed_tts_request_reads_the_row_at_reading_pace_only_while_bubbles_are_on(self):
|
||||
done = subprocess.run(["node", "-e", SPEECH_PROBE, str(BUBBLES), str(SPEECH)], capture_output=True, text=True, timeout=60, check=True)
|
||||
data = json.loads(done.stdout)
|
||||
failed = ["Reading radio aloud failed: 503 tts down"]
|
||||
self.assertEqual(data["fallback"], {"current": True, "busy": True, "timer": 7, "started": 1000, "duration": 1.5, "shown": failed})
|
||||
for case in ("off", "sample", "cancelled"):
|
||||
with self.subTest(case=case):
|
||||
self.assertEqual(data[case], {"current": False, "busy": False, "timer": 0, "started": None, "duration": None, "shown": failed})
|
||||
|
||||
|
||||
class TestBubbleAssets(unittest.TestCase):
|
||||
def test_the_font_is_pinned_and_carries_the_ofl_without_a_reserved_name(self):
|
||||
data = FONT.read_bytes()
|
||||
self.assertEqual((data[:4], hashlib.sha256(data).hexdigest()), (b"wOF2", FONT_SHA256))
|
||||
licence = (FONT.parent / "LICENSE").read_text(encoding="utf-8")
|
||||
self.assertIn("SIL OPEN FONT LICENSE Version 1.1 - 26 February 2007", licence)
|
||||
self.assertIn(HOLDER, licence)
|
||||
self.assertNotIn("with Reserved Font Name", licence)
|
||||
|
||||
def test_bubble_text_is_set_as_text_only_in_the_comic_neue_stack_and_reduced_motion_drops_the_motion(self):
|
||||
layer = LAYER.read_text(encoding="utf-8")
|
||||
for markup in ("innerHTML", "outerHTML", "insertAdjacentHTML", "document.write", "DOMParser"):
|
||||
with self.subTest(markup=markup):
|
||||
self.assertNotIn(markup, layer)
|
||||
self.assertIn("view.words.textContent = text;", layer)
|
||||
style = STYLE.read_text(encoding="utf-8")
|
||||
self.assertIn('"Comic Neue", "Comic Sans MS", cursive', style)
|
||||
self.assertIn("pointer-events: none", style)
|
||||
calm = re.search(r"@media \(prefers-reduced-motion: reduce\) \{(.*?)\n\}", style, re.DOTALL).group(1)
|
||||
self.assertIn(".speech-bubble { transition: opacity .4s var(--ease); }", calm)
|
||||
self.assertIn(".bubble-words { scroll-behavior: auto; }", calm)
|
||||
|
||||
def test_the_schedule_module_parses_on_its_own(self):
|
||||
done = subprocess.run(["node", "--check", str(BUBBLES)], capture_output=True, text=True, timeout=60, check=False)
|
||||
self.assertEqual(done.returncode, 0, done.stderr)
|
||||
|
||||
|
||||
class TestHubBubbles(HubCase):
|
||||
def test_the_toolbar_icon_and_the_layer_ship_in_the_page_under_the_unchanged_csp(self):
|
||||
status, headers, body = self.get("/")
|
||||
self.assertEqual(status, 200)
|
||||
html = body.decode("utf-8")
|
||||
header = re.search(r'<header class="top">(.*?)</header>', html, re.DOTALL).group(1)
|
||||
toggle = re.search(r'<button [^>]*id="bubbles-toggle"[^>]*>(.*?)</button>', header).group(0)
|
||||
for part in ('title="Speech bubbles"', 'aria-label="Speech bubbles"', 'aria-pressed="false"', '<use href="#bi-chat-quote"></use>'):
|
||||
with self.subTest(part=part):
|
||||
self.assertIn(part, toggle)
|
||||
self.assertIn('<symbol id="bi-chat-quote"', html)
|
||||
self.assertIn('<div class="bubbles" id="bubbles" aria-hidden="true" hidden><div class="bubble-bar"></div></div>', html)
|
||||
self.assertIn(BUBBLES.read_text(encoding="utf-8").strip(), html)
|
||||
nonce = re.search(r"script-src 'nonce-([^']+)'", headers["content-security-policy"]).group(1)
|
||||
self.assertEqual(headers["content-security-policy"], view.INDEX_CSP.format(nonce, "'self'"))
|
||||
self.assertNotIn("font-src", headers["content-security-policy"])
|
||||
self.assertNotIn("@font-face", html)
|
||||
|
||||
def test_the_vendor_route_serves_the_font_as_woff2_under_the_plain_csp(self):
|
||||
status, headers, body = self.get("/vendor/comic-neue.woff2")
|
||||
self.assertEqual((status, headers["content-type"], headers["content-security-policy"]), (200, "font/woff2", exchange.PLAIN_CSP))
|
||||
self.assertEqual(body, FONT.read_bytes())
|
||||
for path in ("/vendor/comic-neue-latin-400-normal.woff2", "/vendor/comic-neue", "/vendor/LICENSE"):
|
||||
with self.subTest(path=path):
|
||||
self.assertEqual(self.get(path)[0], 404)
|
||||
|
||||
def test_the_bubble_switch_is_a_hub_setting_saved_without_a_passkey_that_survives_a_reload(self):
|
||||
self.assertEqual((settings.SETTINGS["bubbles"], self.snapshot()["settings"]["bubbles"]), (False, False))
|
||||
good = self.guard(self.host)
|
||||
current = self.snapshot()["settings"]
|
||||
for wrong in ("true", 1, None, [True]):
|
||||
with self.subTest(wrong=wrong):
|
||||
self.assertEqual(self.post_json("/api/settings", {**current, "bubbles": wrong}, good)[0], 400)
|
||||
self.assertEqual(self.post_json("/api/settings", {key: value for key, value in current.items() if key != "bubbles"}, good)[0], 400)
|
||||
status, _, body = self.post_json("/api/settings/sign", {**current, "bubbles": True}, good)
|
||||
self.assertEqual((status, json.loads(body)["payload"]), (200, None))
|
||||
status, _, body = self.post_json("/api/settings", {**current, "bubbles": True, "proof": None}, good)
|
||||
self.assertEqual((status, json.loads(body)["settings"]), (200, {**current, "bubbles": True}), body)
|
||||
self.assertTrue(settings.Settings(self.state / "settings.json").get()["bubbles"])
|
||||
self.assertTrue(self.snapshot()["settings"]["bubbles"])
|
||||
older = {key: value for key, value in current.items() if key != "bubbles"}
|
||||
(self.state / "settings.json").write_text(json.dumps(older), encoding="utf-8")
|
||||
self.assertFalse(settings.Settings(self.state / "settings.json").get()["bubbles"])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -12,12 +12,12 @@ from kvb_orchestrator import council, events, settings, skill_scripts, view
|
||||
|
||||
|
||||
class TestHubSurface(HubCase):
|
||||
def test_the_header_holds_only_the_title_the_radio_the_palette_and_the_settings(self):
|
||||
def test_the_header_holds_only_the_title_the_radio_the_bubbles_the_palette_and_the_settings(self):
|
||||
html = self.index()
|
||||
self.assertIn("<title>Agent Orchestrator</title>", html)
|
||||
header = re.search(r'<header class="top">(.*?)</header>', html, re.DOTALL).group(1)
|
||||
self.assertIn("<h1>Agent Orchestrator</h1>", header)
|
||||
self.assertEqual(re.findall(r'<button[^>]*id="([^"]+)"', header), ["radio-open", "palette-open", "settings-open"])
|
||||
self.assertEqual(re.findall(r'<button[^>]*id="([^"]+)"', header), ["radio-open", "bubbles-toggle", "palette-open", "settings-open"])
|
||||
self.assertEqual(re.findall(r"<(?:nav|select|input|textarea)\b", header), [])
|
||||
for gone in ('id="rail"', 'id="inbox-bar"', 'id="status"', 'id="board-open"', 'id="rules-open"', 'id="timebar"'):
|
||||
with self.subTest(gone=gone):
|
||||
|
||||
Reference in new issue
Block a user