fix(kvb-org-queue): a hotfix held back by host load holds back the entries after it

The grant order ignored live load: a prio-1 hotfix that fit the budget but waited on RAM or CPU counted as granted for the entries after it, so CPU-light jobs behind it could start, keep the load up and starve it. Such a hotfix now holds back every entry after it until it starts (operator pick 2026-11-11 on Main's report). Each holder stores the reason of its last poll, and list shows it, so a job the load holds back no longer reads as granted, without list sampling the CPU.

🎯 Fix: 92 %
💥 Break: 4 %
💸 Cost: 100 %
⏱️ Time: 100 %
🛡️ Security: 100 %
🏆 Score: 88
🚦 Gate: ✅ 90/5, cost and time <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Kevin Veen-BirkenbachandClaude Opus 5.5 committed 2026-11-11 16:34:37 +00:00
1 parent 5053c6a9be
commit 134b1d30f1
2 files changed
+55 -12

No files matched your search

+18
View File
@@ -31,6 +31,7 @@ BASE = {
"budget": {"ram": 75, "cpu": 100},
"ranks": {"work/a": 0, "work/b": 1, "work/c": 2},
"agents": {},
"loaded": {},
"now": NOW,
}
CHILD = (
@@ -253,6 +254,16 @@ class TestEvaluate(unittest.TestCase):
])
self.assertEqual(verdicts, {"j2": "/w runs another job", "j3": None, "j4": "/x runs another job"})
def test_a_hotfix_the_host_load_holds_back_holds_back_every_entry_after_it_and_a_normal_one_does_not(self):
load = "RAM 55 % used + 42 % still to come reach the limit 90 %"
hotfix, light = job(1, hotfix="main-red"), job(2, task="work/b")
self.assertEqual(
self.decide([hotfix, light], loaded={"j1": load}),
{"j1": load, "j2": "waits behind hotfix j1, which the host load holds back"},
)
self.assertEqual(self.decide([job(1), light], loaded={"j1": load}), {"j1": None, "j2": None})
self.assertEqual(self.decide([hotfix, light]), {"j1": None, "j2": None})
class TestHistory(unittest.TestCase):
@classmethod
@@ -514,6 +525,13 @@ class TestQueueCommands(unittest.TestCase):
)
self.assertEqual(self.entries(), [])
def test_list_shows_the_host_load_the_holder_saw_at_its_last_poll(self):
self.cfg["load_limits"]["mem_percent"] = 0
self.write_config()
self.blocked_hold("true")
reason = self.run_queue("list").stdout.splitlines()[1].split("\t")[-1]
self.assertRegex(reason, r"^RAM \d+ % used \+ \d+ % still to come reach the limit 0 %$")
def test_hold_refuses_bad_input_with_exit_2_and_enqueues_nothing(self):
def hold(*args: str) -> list[str]:
return ["hold", "--session", "s1", *args]