Nguyen Minh Phuc
c76154aeaa
ci / lint (push) Successful in 34s
ci / unit (push) Successful in 1m41s
ci / types (push) Successful in 1m41s
ci / dockerfile (push) Successful in 18s
ci / security (push) Successful in 1m27s
ci / chart (push) Failing after 1m11s
ci / integration (push) Successful in 1m10s
ci / image (api) (push) Has been skipped
ci / image (reconciler) (push) Has been skipped
ci / image (worker) (push) Has been skipped
ci / bump (push) Has been skipped
CORRECTNESS - lost-lease race: complete()/fail() did not check ownership, so a worker whose lease expired could mark a task done while another worker was running it, or requeue a task someone else owned. Reproduced, fixed with a CAS on (state, locked_by), pinned by two regression tests. - worker died on report failure: _run_one's docstring claimed no exception escapes the TaskGroup; fail()/complete() were outside the guarded block, so a DB blip cancelled every sibling provision on the pod. - claim query used an INNER join, which could strand a just-claimed task and report 'queue empty'. LEFT join. - InstanceRepo.set_error bypassed the state machine and had no callers. Deleted. - handle_deprovision ignored its CAS result, so a wrong-state instance kept a dangling endpoint and got re-provisioned by the drift check 60s later. - handle_verify re-notified on every retry: five pages for one halt. DEPLOY-BREAKING - the migration Job could never succeed: no Dockerfile copied migrations/, and migrate.py resolved the path relative to the source tree, which only works for an editable install. Added COPY + SVCFORGE_MIGRATIONS_DIR. - ServiceMonitor selector did not match the Service: API metrics never scraped. - SvcforgeReconcilerStale fired permanently from every pod, because the gauge is module-level and every service exports it as 0. Scoped to the reconciler job. - SvcforgeTaskFailed latched forever on a monotonic counter. Now increase()[15m]. - the digest guard accepted the all-zeros placeholder. - worker terminationGracePeriodSeconds was 60s against a 600s helm timeout. DEAD CODE THAT SHOULD NOT HAVE BEEN - adapters/k8s.py was never called, so tenant namespaces were never created and the first provision for a new team would fail. Wired into handle_provision. - adapters/redis.py was never imported by any service. Rate limiting is now wired into the API, failing open. - Settings.check_production() had no callers. Given an explicit environment and called from every entrypoint. OBSERVABILITY - the API never called obs.setup(): no JSON logs, no trace correlation, log_json silently inert. - LogNotifier's structured fields were discarded by the stdlib->structlog bridge. - bind_task_context cleared the 'service' binding for the life of every task. - split tasks_failed into task_attempts_failed and tasks_dead_lettered. SECURITY - trivy correctly blocked the worker/reconciler images: helm 3.16.2 and kubectl 1.31.2 carry CRITICAL Go stdlib CVEs. Bumped to helm 3.21.3 and kubectl 1.35.3, which also closes a four-minor skew against the v1.35.3 cluster. TESTS THAT COULD NOT FAIL - the concurrency cap test passed on a fully serial worker. - the alert/metric cross-check asserted a hardcoded list instead of reading the chart, so it could not catch a rename on the chart side. - fixed OTel tracer-provider pollution between test files. DOCS - ARCHITECTURE.md: mermaid diagrams, user stories, and the helm-vs-ArgoCD guarantee (verified with --dry-run=server). - AGENTS.md + CLAUDE.md. - prose sweep for back-and-forth phrasing across 19 files.
68 lines
3.4 KiB
Docker
68 lines
3.4 KiB
Docker
# syntax=docker/dockerfile:1.10
|
|
#
|
|
# svcforge worker. Build from the REPO ROOT:
|
|
# docker buildx build -f services/worker/Dockerfile -t svcforge/worker:dev .
|
|
#
|
|
# The only service that shells out to helm/kubectl, so the only one carrying those two
|
|
# binaries. They are copied from pinned images rather than curl'd, so the version is a
|
|
# reviewable line in a Dockerfile instead of a network call at build time.
|
|
|
|
FROM python:3.12-slim@sha256:57cd7c3a7a273101a6485ba99423ee568157882804b1124b4dd04266317710de AS builder
|
|
|
|
COPY --from=ghcr.io/astral-sh/uv:0.5.11@sha256:0ac957607303916420297a4c9c213bb33fbd3c888f9cd7f4f7273596ebf42b85 /uv /usr/local/bin/uv
|
|
|
|
ENV UV_COMPILE_BYTECODE=1 UV_LINK_MODE=copy UV_PYTHON_DOWNLOADS=never
|
|
WORKDIR /app
|
|
|
|
COPY pyproject.toml uv.lock ./
|
|
COPY libs/svcforge_core/pyproject.toml libs/svcforge_core/
|
|
RUN --mount=type=cache,target=/root/.cache/uv \
|
|
uv sync --frozen --no-dev --no-editable \
|
|
--no-install-project --no-install-package svcforge-core
|
|
|
|
COPY libs/ libs/
|
|
COPY services/worker/ services/worker/
|
|
COPY catalog.yaml ./
|
|
RUN --mount=type=cache,target=/root/.cache/uv \
|
|
uv sync --frozen --no-dev --no-editable && \
|
|
/app/.venv/bin/python -c 'import svcforge_core, sys; \
|
|
p = svcforge_core.__file__; \
|
|
sys.exit(0) if "site-packages" in p else sys.exit("not a wheel install: " + p)'
|
|
|
|
# --- runtime --------------------------------------------------------------------------
|
|
FROM python:3.12-slim@sha256:57cd7c3a7a273101a6485ba99423ee568157882804b1124b4dd04266317710de
|
|
|
|
ARG BUILD_SHA=unknown
|
|
LABEL org.opencontainers.image.title="svcforge-worker" \
|
|
org.opencontainers.image.source="https://gitea.oci-oci.duckdns.org/gitea_admin/svcforge" \
|
|
org.opencontainers.image.revision="${BUILD_SHA}"
|
|
|
|
RUN useradd -u 10001 -m -s /usr/sbin/nologin svcforge
|
|
WORKDIR /app
|
|
COPY --from=builder --chown=10001:10001 /app /app
|
|
|
|
# helm 3.21.3, not 3.16.2. 3.16.2 was built with Go 1.22.9 and carries CRITICAL
|
|
# CVE-2025-68121 (crypto/tls) and CVE-2026-33186 (grpc), plus HIGH CVE-2026-35469
|
|
# (spdystream, fixed in 0.5.1) — trivy fails the build on them and is right to.
|
|
# Deliberately 3.x: helm 4 is a breaking change and is not a CVE fix.
|
|
COPY --from=alpine/helm:3.21.3@sha256:35da09ba0716fc7c3cd63b6b31ee380a9c7662e95f29ab0e4ae962420afd315b /usr/bin/helm /usr/local/bin/helm
|
|
# kubectl 1.35.3 from the OFFICIAL registry.k8s.io image, replacing
|
|
# bitnamilegacy/kubectl:1.31.2. Two reasons, either sufficient:
|
|
# - CVEs: the bitnami image is Go 1.22.9 and ships containerd < 1.7.29 (HIGH
|
|
# CVE-2024-25621) along with the same crypto/tls and grpc CRITICALs as helm above.
|
|
# - Skew: the target cluster runs v1.35.3. 1.31.2 is four minors behind, well outside
|
|
# kubectl's supported +/-1 window, so it was unsupported against this cluster.
|
|
# bitnamilegacy publishes no 1.35 tag. registry.k8s.io/kubectl is the upstream-maintained
|
|
# image, is a manifest list with linux/arm64 (this cluster is Ampere), and puts the binary
|
|
# at /bin/kubectl rather than bitnami's /opt/bitnami path.
|
|
COPY --from=registry.k8s.io/kubectl:v1.35.3@sha256:8dad99b604a2c0bafe17f53cadf78482d6f667a6da687f385508f5f4e4696d37 /bin/kubectl /usr/local/bin/kubectl
|
|
|
|
ENV PATH="/app/.venv/bin:$PATH" \
|
|
PYTHONUNBUFFERED=1 \
|
|
PYTHONDONTWRITEBYTECODE=1 \
|
|
HELM_CACHE_HOME=/tmp/helm/cache \
|
|
HELM_CONFIG_HOME=/tmp/helm/config \
|
|
HELM_DATA_HOME=/tmp/helm/data
|
|
USER 10001
|
|
ENTRYPOINT ["python", "-m", "services.worker.main"]
|