review: fix 26 findings from a 4-agent audit

CORRECTNESS
- lost-lease race: complete()/fail() did not check ownership, so a worker whose
  lease expired could mark a task done while another worker was running it, or
  requeue a task someone else owned. Reproduced, fixed with a CAS on
  (state, locked_by), pinned by two regression tests.
- worker died on report failure: _run_one's docstring claimed no exception
  escapes the TaskGroup; fail()/complete() were outside the guarded block, so a
  DB blip cancelled every sibling provision on the pod.
- claim query used an INNER join, which could strand a just-claimed task and
  report 'queue empty'. LEFT join.
- InstanceRepo.set_error bypassed the state machine and had no callers. Deleted.
- handle_deprovision ignored its CAS result, so a wrong-state instance kept a
  dangling endpoint and got re-provisioned by the drift check 60s later.
- handle_verify re-notified on every retry: five pages for one halt.

DEPLOY-BREAKING
- the migration Job could never succeed: no Dockerfile copied migrations/, and
  migrate.py resolved the path relative to the source tree, which only works for
  an editable install. Added COPY + SVCFORGE_MIGRATIONS_DIR.
- ServiceMonitor selector did not match the Service: API metrics never scraped.
- SvcforgeReconcilerStale fired permanently from every pod, because the gauge is
  module-level and every service exports it as 0. Scoped to the reconciler job.
- SvcforgeTaskFailed latched forever on a monotonic counter. Now increase()[15m].
- the digest guard accepted the all-zeros placeholder.
- worker terminationGracePeriodSeconds was 60s against a 600s helm timeout.

DEAD CODE THAT SHOULD NOT HAVE BEEN
- adapters/k8s.py was never called, so tenant namespaces were never created and
  the first provision for a new team would fail. Wired into handle_provision.
- adapters/redis.py was never imported by any service. Rate limiting is now wired
  into the API, failing open.
- Settings.check_production() had no callers. Given an explicit environment and
  called from every entrypoint.

OBSERVABILITY
- the API never called obs.setup(): no JSON logs, no trace correlation, log_json
  silently inert.
- LogNotifier's structured fields were discarded by the stdlib->structlog bridge.
- bind_task_context cleared the 'service' binding for the life of every task.
- split tasks_failed into task_attempts_failed and tasks_dead_lettered.

SECURITY
- trivy correctly blocked the worker/reconciler images: helm 3.16.2 and kubectl
  1.31.2 carry CRITICAL Go stdlib CVEs. Bumped to helm 3.21.3 and kubectl 1.35.3,
  which also closes a four-minor skew against the v1.35.3 cluster.

TESTS THAT COULD NOT FAIL
- the concurrency cap test passed on a fully serial worker.
- the alert/metric cross-check asserted a hardcoded list instead of reading the
  chart, so it could not catch a rename on the chart side.
- fixed OTel tracer-provider pollution between test files.

DOCS
- ARCHITECTURE.md: mermaid diagrams, user stories, and the helm-vs-ArgoCD
  guarantee (verified with --dry-run=server).
- AGENTS.md + CLAUDE.md.
- prose sweep for back-and-forth phrasing across 19 files.

tests/unit/test_obs.py

@@ -13,11 +13,13 @@ import json
 import logging
 import re
 from collections.abc import Iterator
+from pathlib import Path
 from typing import Any
 from uuid import uuid4
 
 import pytest
 import structlog
+import yaml
 from opentelemetry import trace
 from opentelemetry.sdk.trace import TracerProvider
 from prometheus_client import REGISTRY
@@ -89,8 +91,8 @@ def test_provision_histogram_has_a_bucket_for_a_thirty_minute_provision() -> Non
 
     With the library defaults the top finite bucket is 10s, every provision lands in +Inf,
     and histogram_quantile interpolates across a bucket spanning 10s to infinity. The p95
-    it returns is not slow or fast, it is meaningless — and it is meaningless silently,
-    which is why this is asserted rather than eyeballed on a dashboard.
+    it returns is meaningless, and it is meaningless silently, which is why this is
+    asserted rather than eyeballed on a dashboard.
     """
     bounds = obs.PROVISION_TIME._upper_bounds
     assert 1800.0 in bounds
@@ -105,21 +107,34 @@ def test_provision_histogram_exports_the_1800_bucket() -> None:
     assert bucket is not None
 
 
-def test_metric_names_are_the_ones_the_alerts_query() -> None:
-    """The four alerts are PromQL strings in values.yaml; nothing type-checks them.
+def test_every_metric_the_alerts_query_actually_exists() -> None:
+    """Parse the alerts out of values.yaml and check each metric is really registered.
 
-    A rename here is a silent alert that never fires again. This test is the link between
-    the chart's rules and the code.
+    Reading the chart is the whole point. An earlier version of this test asserted against
+    a hardcoded list, which meant it could only catch a rename on the *code* side — rename
+    a metric in values.yaml and it stayed green while the alert silently never fired again.
+    A test that duplicates the thing it is meant to cross-check is not a cross-check.
     """
-    for name in (
-        "svcforge_tasks_claimed_total",
-        "svcforge_tasks_failed_total",
-        "svcforge_provision_duration_seconds",
-        "svcforge_queue_depth",
-        "svcforge_instances",
-        "svcforge_reconciler_last_tick_timestamp_seconds",
-    ):
-        assert REGISTRY._names_to_collectors.get(name) is not None, f"{name} is queried by an alert"
+    values = yaml.safe_load(
+        (Path(__file__).resolve().parents[2] / "deploy" / "chart" / "values.yaml").read_text()
+    )
+    rules = (values.get("prometheusRule") or {}).get("rules") or []
+    assert rules, "no alert rules found under prometheusRule.rules — this test would silently pass"
+
+    # Prometheus appends _bucket/_sum/_count to a histogram's series; the collector is
+    # registered under the base name.
+    suffixes = ("_bucket", "_sum", "_count")
+    registered = set(REGISTRY._names_to_collectors)
+
+    checked = 0
+    for rule in rules:
+        for token in re.findall(r"\bsvcforge_[a-z0-9_]+\b", str(rule.get("expr", ""))):
+            base = next((token[: -len(s)] for s in suffixes if token.endswith(s)), token)
+            assert base in registered, (
+                f"alert {rule.get('alert')!r} queries {token!r}, but no such metric is registered"
+            )
+            checked += 1
+    assert checked >= len(rules), "expected at least one metric reference per alert"
 
 
 # --- Trace context across the queue -------------------------------------------------------
@@ -232,7 +247,7 @@ def test_json_renderer_is_last_and_output_is_one_object_per_line(logs: io.String
 
 
 def test_setup_is_idempotent(logs: io.StringIO) -> None:
-    """Called twice, one handler, one line. Not a nicety: two handlers is two of every line.
+    """Called twice, one handler, one line. Two handlers would be two of every line.
 
     Each service calls setup() from its entrypoint, and an entrypoint that imports another
     entrypoint (the CLI shelling into the reconciler) calls it twice.