review: fix 26 findings from a 4-agent audit

CORRECTNESS
- lost-lease race: complete()/fail() did not check ownership, so a worker whose
  lease expired could mark a task done while another worker was running it, or
  requeue a task someone else owned. Reproduced, fixed with a CAS on
  (state, locked_by), pinned by two regression tests.
- worker died on report failure: _run_one's docstring claimed no exception
  escapes the TaskGroup; fail()/complete() were outside the guarded block, so a
  DB blip cancelled every sibling provision on the pod.
- claim query used an INNER join, which could strand a just-claimed task and
  report 'queue empty'. LEFT join.
- InstanceRepo.set_error bypassed the state machine and had no callers. Deleted.
- handle_deprovision ignored its CAS result, so a wrong-state instance kept a
  dangling endpoint and got re-provisioned by the drift check 60s later.
- handle_verify re-notified on every retry: five pages for one halt.

DEPLOY-BREAKING
- the migration Job could never succeed: no Dockerfile copied migrations/, and
  migrate.py resolved the path relative to the source tree, which only works for
  an editable install. Added COPY + SVCFORGE_MIGRATIONS_DIR.
- ServiceMonitor selector did not match the Service: API metrics never scraped.
- SvcforgeReconcilerStale fired permanently from every pod, because the gauge is
  module-level and every service exports it as 0. Scoped to the reconciler job.
- SvcforgeTaskFailed latched forever on a monotonic counter. Now increase()[15m].
- the digest guard accepted the all-zeros placeholder.
- worker terminationGracePeriodSeconds was 60s against a 600s helm timeout.

DEAD CODE THAT SHOULD NOT HAVE BEEN
- adapters/k8s.py was never called, so tenant namespaces were never created and
  the first provision for a new team would fail. Wired into handle_provision.
- adapters/redis.py was never imported by any service. Rate limiting is now wired
  into the API, failing open.
- Settings.check_production() had no callers. Given an explicit environment and
  called from every entrypoint.

OBSERVABILITY
- the API never called obs.setup(): no JSON logs, no trace correlation, log_json
  silently inert.
- LogNotifier's structured fields were discarded by the stdlib->structlog bridge.
- bind_task_context cleared the 'service' binding for the life of every task.
- split tasks_failed into task_attempts_failed and tasks_dead_lettered.

SECURITY
- trivy correctly blocked the worker/reconciler images: helm 3.16.2 and kubectl
  1.31.2 carry CRITICAL Go stdlib CVEs. Bumped to helm 3.21.3 and kubectl 1.35.3,
  which also closes a four-minor skew against the v1.35.3 cluster.

TESTS THAT COULD NOT FAIL
- the concurrency cap test passed on a fully serial worker.
- the alert/metric cross-check asserted a hardcoded list instead of reading the
  chart, so it could not catch a rename on the chart side.
- fixed OTel tracer-provider pollution between test files.

DOCS
- ARCHITECTURE.md: mermaid diagrams, user stories, and the helm-vs-ArgoCD
  guarantee (verified with --dry-run=server).
- AGENTS.md + CLAUDE.md.
- prose sweep for back-and-forth phrasing across 19 files.

scripts/redis_budget.py

@@ -3,8 +3,17 @@
 
     $ python3 scripts/redis_budget.py
     $ python3 scripts/redis_budget.py --url http://localhost:8000/metrics --budget 500000
+    $ python3 scripts/redis_budget.py --url http://api:8000/metrics \
+                                      --url http://worker:9100/metrics \
+                                      --url http://reconciler:9100/metrics
 
-Upstash's free tier is 500,000 commands/month, which sounds enormous and is not:
+The budget is per Upstash database; the counters are per process. api, worker and
+reconciler each keep their own registry (see obs.py — one process per pod, no multiproc
+directory), so scraping one endpoint measures one third of the burn. Repeat `--url` to sum
+them; a run that covers fewer than three sources says so in its output rather than printing
+a reassuring number derived from one process.
+
+Upstash's free tier is 500,000 commands/month, which is far smaller than it sounds:
 
     500,000 / month = 16,129 / day = 11 / minute = 0.19 / second, sustained
 
@@ -14,7 +23,7 @@ Redis is only ever on the request path here, and why this script exists: the rul
 to state and invisible to violate. A `cache.get()` added inside the reconciler's
 per-instance loop is one line in review and 2,160,000 commands/month in production.
 
-**Why a projection and not an alarm on the counter.** Exhausting the budget is a slow,
+**This projects the burn rather than alarming on the counter.** Exhausting the budget is a slow,
 silent failure with a cliff at the end: nothing degrades, nothing pages, every call
 succeeds, and then the month rolls over and every Redis call starts erroring at once. By
 then the fix is a bill or an outage. A burn rate extrapolated from the counter is visible
@@ -113,7 +122,23 @@ def collect(text: str) -> tuple[dict[str, float], float]:
     return per_op, started_at
 
 
-def report(per_op: dict[str, float], started_at: float, budget: int, now: float) -> int:
+def merge(scrapes: list[tuple[dict[str, float], float]]) -> tuple[dict[str, float], float]:
+    """Fold several processes' scrapes into one budget view.
+
+    The budget is per-Upstash-database, but the counters are per-process: api, worker and
+    reconciler each hold their own prometheus_client registry, so scraping one of them
+    projects a third of the truth. Command totals sum across processes; the window is the
+    EARLIEST start time, because a counter that has been running longest bounds how far back
+    the summed total can be attributed — using the latest would inflate the rate.
+    """
+    per_op: dict[str, float] = {}
+    for scraped, _ in scrapes:
+        for op, value in scraped.items():
+            per_op[op] = per_op.get(op, 0.0) + value
+    return per_op, min(started for _, started in scrapes)
+
+
+def report(per_op: dict[str, float], started_at: float, budget: int, now: float, sources: int = 1) -> int:
     """Print the projection. Returns the process exit code."""
     elapsed_s = max(1.0, now - started_at)
     total = sum(per_op.values())
@@ -128,6 +153,17 @@ def report(per_op: dict[str, float], started_at: float, budget: int, now: float)
     print(f"rate            {rate:.4f} /s   (budget allows {budget / _MONTH_S:.4f} /s sustained)")
     print(f"projected       {projected:,.0f} / month")
     print(f"budget          {budget:,} / month")
+    print(f"sources         {sources} process(es) scraped")
+    if sources < 3:
+        # Be honest about what was measured. api/worker/reconciler each keep their own
+        # in-process registry (obs.py: one process per pod, no multiproc dir), so a
+        # single-endpoint run undercounts the shared Upstash budget by however many
+        # processes were left out. An optimistic verdict here is worse than no verdict.
+        print(
+            "                UNDERCOUNT: svcforge runs api + worker + reconciler, each with "
+            "its own\n                registry. Pass --url once per process for the real "
+            "total; the numbers\n                above cover only what was scraped."
+        )
 
     if total < 100:
         # Extrapolating a month from a handful of commands is astrology. Say so rather than
@@ -147,20 +183,32 @@ def main(argv: list[str] | None = None) -> int:
     parser = argparse.ArgumentParser(
         description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter
     )
-    parser.add_argument("--url", default="http://localhost:8000/metrics", help="Prometheus endpoint")
+    parser.add_argument(
+        "--url",
+        action="append",
+        dest="urls",
+        metavar="URL",
+        help="Prometheus endpoint; repeat once per process (api, worker, reconciler)",
+    )
     parser.add_argument("--budget", type=int, default=_FREE_TIER_BUDGET, help="commands per month")
     args = parser.parse_args(argv)
 
-    try:
-        per_op, started_at = collect(scrape(args.url))
-    except BudgetError as exc:
-        print(f"error: {exc}", file=sys.stderr)
-        return 2
-    except OSError as exc:
-        print(f"error: cannot scrape {args.url}: {exc}", file=sys.stderr)
-        return 2
+    # action="append" cannot carry a default (argparse appends to it), so apply it here.
+    urls: list[str] = args.urls or ["http://localhost:8000/metrics"]
 
-    return report(per_op, started_at, args.budget, time.time())
+    scrapes: list[tuple[dict[str, float], float]] = []
+    for url in urls:
+        try:
+            scrapes.append(collect(scrape(url)))
+        except BudgetError as exc:
+            print(f"error: {url}: {exc}", file=sys.stderr)
+            return 2
+        except OSError as exc:
+            print(f"error: cannot scrape {url}: {exc}", file=sys.stderr)
+            return 2
+
+    per_op, started_at = merge(scrapes)
+    return report(per_op, started_at, args.budget, time.time(), sources=len(scrapes))
 
 
 if __name__ == "__main__":