6 Commits

Author	SHA1	Message	Date
Nguyen Minh Phuc	c53734d2bc	docs: add USER_GUIDE.md, tighten comments, fix CLI needing a DSN ... The comment pass is prose-only: every distinct "why" is kept, the narration around it is not. Verified by AST-comparing each changed file against HEAD with docstrings stripped — only the two files below differ in executable code. Two real fixes fell out of the read-through: * The CLI documented itself as never touching the database, then called load_settings(), which requires SVCFORGE_PG_DSN. It refused to start without a Postgres URL it never opens. It now has its own two-field ClientSettings; the orphaned api_url/api_token are dropped from Settings, where nothing else read them. * repo/db.py had the DictRow alias comment and the ERROR_MAX_CHARS comment run together above the wrong symbol. USER_GUIDE.md is the caller-facing guide the README only gestured at: auth, catalog, every endpoint with curl, the lifecycle, the error table, rate limiting, the CLI, client generation, an end-to-end poll loop. It records two facts about the live deployment rather than documenting a flow nobody can run. SVCFORGE_JWKS_URL points at a realm with no IdP behind it, so the API logs "JWKS warm-up failed" at startup and every /v1 request is a 401. And `helm repo list` in the worker returns no repositories, so the three bitnamilegacy/ catalog entries cannot resolve at provision time; only the oci:// entries can. make lint clean, 76 unit + 111 integration tests pass.	2026-07-21 14:51:26 +00:00
Nguyen Minh Phuc	e971e04d75	fix: shared runtime helper must live where every image packages it ... services/_runtime.py crashed the worker and reconciler on boot with `ModuleNotFoundError: No module named 'services._runtime'`, while every unit and integration gate was green and the API rolled out fine. The cause is packaging, not code. Each service Dockerfile copies only its own `services/<svc>/` subdir — `services/` itself is a namespace package with no __init__.py, so a file added at the `services/` root is never copied into any image. Tests import from the source tree, where the file exists, so nothing below the image boundary could catch it. The API survived only because it does not import the helper. Moved to svcforge_core.runtime, which `COPY libs/ libs/` packages into every image, next to adapters/tempyaml.py for the same reason. Added an import smoke test to the image job: `docker run --entrypoint python <image> -c "import services.<svc>.main"` loads the whole transitive graph inside the built image and fails the build before the digest is pushed. This is the one check the test suite structurally cannot perform — it runs against source, the image is a different filesystem — and it is exactly the gap this bug fell through.	2026-07-21 02:40:38 +00:00
Nguyen Minh Phuc	66eb6cb0ee	refactor: converge the patterns multiple authors left divergent ... The codebase was written by several agents and had the same concept done more than one way. This makes it read as one voice, with no behaviour change. Dedup, each to a single canonical form: - INSTANCE_COLUMNS: the 13-column instances SELECT list existed as _COLUMNS in instances.py and reconcile.py (byte-identical) and inlined a third time in the worker. One exported constant now. - Settings.runtime_dsn: the three entrypoints each chose between str(pg_dsn) and pg_dsn.unicode_string(). One property. - yaml_tempfile: helm._ValuesFile and k8s._ManifestFile were the same write-yaml-to-a-temp-dir context manager. One helper in adapters/tempyaml.py. - services/_runtime.py: sleep_or_stop and install_stop_signals were copied between the worker and reconciler loops. One module, so shutdown behaviour cannot drift between them. - k8s.ensure_namespace used MANAGED_BY_LABEL/VALUE from helm.py instead of a hardcoded literal, so the managed-by label has one definition. - SvcforgeError is now the root of every svcforge exception (CatalogError, IllegalTransition, BadWindow, HandlerError), keeping each stdlib base in the MRO, so `except SvcforgeError` means what errors.py says it does. - ERROR_MAX_CHARS replaces the repeated `[-2000:]` truncation feeding the same error columns. - the reconciler reads settings.metrics_port like the worker, dropping its duplicate DEFAULT_METRICS_PORT and redundant --metrics-port option; the SVCFORGE_METRICS_PORT env override still applies through pydantic. Two smaller correctness/consistency fixes: - RateLimitResult.retry_after_s computed its delta against datetime.now(UTC) while the limiter runs on an injectable clock, so it was meaningless under a FakeClock and drifted by request latency in production. It now carries a checked_at from the same clock as reset_at. - handle_provision's notifier.send is wrapped like the reconciler's: a flaky webhook after the READY CAS would fail the task, and the retry would hit the READY early-return and drop the notification, turning a good provision into a failed one.	2026-07-21 01:46:54 +00:00
Nguyen Minh Phuc	4a426dbe50	worker: create namespaces with helm, drop the kubectl binary ... trivy took the worker image 39 -> 18 -> 5 findings across two version bumps, and the last 5 (4x golang.org/x/net, 1x Go stdlib) live in kubectl v1.36.2 — the newest kubectl that exists. No release clears them; upstream has not rebuilt against the patched Go yet. Chasing the version further has no end. kubectl was in that image for exactly one call: ensure_namespace before helm. `helm upgrade --install --create-namespace` does the same thing, idempotently, as part of the install it already runs. So the binary goes, and its vendored CVEs go with it. read_secret had no callers. Tradeoff recorded: the namespace no longer gets an svcforge.io/team label, since --create-namespace makes a bare one. Nothing reads that label today.	2026-07-18 13:05:22 +00:00
Nguyen Minh Phuc	c76154aeaa	review: fix 26 findings from a 4-agent audit ... CORRECTNESS - lost-lease race: complete()/fail() did not check ownership, so a worker whose lease expired could mark a task done while another worker was running it, or requeue a task someone else owned. Reproduced, fixed with a CAS on (state, locked_by), pinned by two regression tests. - worker died on report failure: _run_one's docstring claimed no exception escapes the TaskGroup; fail()/complete() were outside the guarded block, so a DB blip cancelled every sibling provision on the pod. - claim query used an INNER join, which could strand a just-claimed task and report 'queue empty'. LEFT join. - InstanceRepo.set_error bypassed the state machine and had no callers. Deleted. - handle_deprovision ignored its CAS result, so a wrong-state instance kept a dangling endpoint and got re-provisioned by the drift check 60s later. - handle_verify re-notified on every retry: five pages for one halt. DEPLOY-BREAKING - the migration Job could never succeed: no Dockerfile copied migrations/, and migrate.py resolved the path relative to the source tree, which only works for an editable install. Added COPY + SVCFORGE_MIGRATIONS_DIR. - ServiceMonitor selector did not match the Service: API metrics never scraped. - SvcforgeReconcilerStale fired permanently from every pod, because the gauge is module-level and every service exports it as 0. Scoped to the reconciler job. - SvcforgeTaskFailed latched forever on a monotonic counter. Now increase()[15m]. - the digest guard accepted the all-zeros placeholder. - worker terminationGracePeriodSeconds was 60s against a 600s helm timeout. DEAD CODE THAT SHOULD NOT HAVE BEEN - adapters/k8s.py was never called, so tenant namespaces were never created and the first provision for a new team would fail. Wired into handle_provision. - adapters/redis.py was never imported by any service. Rate limiting is now wired into the API, failing open. - Settings.check_production() had no callers. Given an explicit environment and called from every entrypoint. OBSERVABILITY - the API never called obs.setup(): no JSON logs, no trace correlation, log_json silently inert. - LogNotifier's structured fields were discarded by the stdlib->structlog bridge. - bind_task_context cleared the 'service' binding for the life of every task. - split tasks_failed into task_attempts_failed and tasks_dead_lettered. SECURITY - trivy correctly blocked the worker/reconciler images: helm 3.16.2 and kubectl 1.31.2 carry CRITICAL Go stdlib CVEs. Bumped to helm 3.21.3 and kubectl 1.35.3, which also closes a four-minor skew against the v1.35.3 cluster. TESTS THAT COULD NOT FAIL - the concurrency cap test passed on a fully serial worker. - the alert/metric cross-check asserted a hardcoded list instead of reading the chart, so it could not catch a rename on the chart side. - fixed OTel tracer-provider pollution between test files. DOCS - ARCHITECTURE.md: mermaid diagrams, user stories, and the helm-vs-ArgoCD guarantee (verified with --dry-run=server). - AGENTS.md + CLAUDE.md. - prose sweep for back-and-forth phrasing across 19 files.	2026-07-18 12:13:49 +00:00
Nguyen Minh Phuc	50c2fe2a1e	svcforge: reference implementation ... Complete working build of the system learn-python/ teaches. 164 tests, mypy --strict clean, domain coverage 99%.	2026-07-17 10:44:54 +00:00