Get off Docker Hub, add a catalog values passthrough, fix the dind prune
ci / lint (push) Successful in 24s
ci / types (push) Successful in 34s
ci / unit (push) Successful in 26s
ci / security (push) Successful in 37s
ci / dockerfile (push) Successful in 6s
ci / chart (push) Successful in 7s
ci / integration (push) Successful in 41s
ci / image (api) (push) Successful in 2m9s
ci / image (reconciler) (push) Successful in 2m1s
ci / image (worker) (push) Successful in 2m7s
ci / bump (push) Successful in 13s
ci / lint (push) Successful in 24s
ci / types (push) Successful in 34s
ci / unit (push) Successful in 26s
ci / security (push) Successful in 37s
ci / dockerfile (push) Successful in 6s
ci / chart (push) Successful in 7s
ci / integration (push) Successful in 41s
ci / image (api) (push) Successful in 2m9s
ci / image (reconciler) (push) Successful in 2m1s
ci / image (worker) (push) Successful in 2m7s
ci / bump (push) Successful in 13s
Docker Hub rate-limits anonymous pulls per source IP and every node here shares one NAT address, so a busy afternoon fails an unrelated build with `toomanyrequests`. Nothing in this repo needs to be there. Every base image now comes from mirror.gcr.io (python, alpine/helm, postgres) or ghcr.io (uv, trivy). Verified digest-for-digest against Docker Hub before switching, including the superseded postgres digest this repo still pins, so every existing pin stays valid — same bytes, different transport. catalog.yaml: the three bitnami entries named `bitnamilegacy/<chart>`, a repo alias nothing in the worker image configures, so they could never resolve at provision time. All five entries are now `oci://` refs, which need no `helm repo add`, and all are on latest stable: elasticsearch 21.3.15 -> 22.1.6 redis 20.6.2 -> 27.0.15 postgresql 16.4.5 -> 18.8.0 podinfo 6.7.1 -> 6.14.0 Moving the chart pull is only half of it, though: a bitnami chart defaults its own images to registry-1.docker.io. CatalogEntry gains a `values:` dict, merged under the size's replicas and resources, so an entry can set `global.imageRegistry` and move the image pull too. Size wins on conflict — otherwise an entry setting replicaCount would make every size deploy the same shape. Deep merge, because a shallow one drops sibling keys of a shared nested map. Bitnami charts reject a substituted registry unless `global.security.allowInsecureImages` is set. That check is about provenance, and the mirror serves byte-identical manifests, so it is set deliberately and only for entries whose digests were verified. The dind prune had `--filter until=168h` on both prunes, and it got both cases exactly backwards. `until` reads an image's CREATED time, so it deleted trivy every leg (a released tool image is always older than any window) while protecting the dangling build layers it existed to remove. Measured on node0: 21 dangling images / 5.96GB, and exactly 1 of them older than 168h. Trivy is protected by a tag now, so the image prune drops the filter; buildx keeps it, where age genuinely matters. Tests: +10 unit (deep merge, precedence, no-mutation, and a guard that fails if any catalog entry points at Docker Hub). Both new guards were control-tested by breaking the code and watching them fail. The API test that hardcoded `21.3.15` now reads the catalog — its subject is where the value comes from, not what it is.
This commit is contained in:
@@ -7,7 +7,7 @@
|
||||
# binaries. They are copied from pinned images rather than curl'd, so the version is a
|
||||
# reviewable line in a Dockerfile instead of a network call at build time.
|
||||
|
||||
FROM python:3.14-slim@sha256:cea0e6040540fb2b965b6e7fb5ffa00871e632eef63719f0ea54bca189ce14a6 AS builder
|
||||
FROM mirror.gcr.io/library/python:3.14-slim@sha256:cea0e6040540fb2b965b6e7fb5ffa00871e632eef63719f0ea54bca189ce14a6 AS builder
|
||||
|
||||
COPY --from=ghcr.io/astral-sh/uv:0.11.29@sha256:eb2843a1e56fd9e30c7276ce1a52cba86e64c7b385f5e3279a0e08e02dd058fc /uv /usr/local/bin/uv
|
||||
|
||||
@@ -30,7 +30,7 @@ p = svcforge_core.__file__; \
|
||||
sys.exit(0) if "site-packages" in p else sys.exit("not a wheel install: " + p)'
|
||||
|
||||
# --- runtime --------------------------------------------------------------------------
|
||||
FROM python:3.14-slim@sha256:cea0e6040540fb2b965b6e7fb5ffa00871e632eef63719f0ea54bca189ce14a6
|
||||
FROM mirror.gcr.io/library/python:3.14-slim@sha256:cea0e6040540fb2b965b6e7fb5ffa00871e632eef63719f0ea54bca189ce14a6
|
||||
|
||||
ARG BUILD_SHA=unknown
|
||||
LABEL org.opencontainers.image.title="svcforge-worker" \
|
||||
@@ -45,7 +45,7 @@ COPY --from=builder --chown=10001:10001 /app /app
|
||||
# CVE-2025-68121 (crypto/tls) and CVE-2026-33186 (grpc), plus HIGH CVE-2026-35469
|
||||
# (spdystream, fixed in 0.5.1) — trivy fails the build on them and is right to.
|
||||
# Deliberately 3.x: helm 4 is a breaking change and is not a CVE fix.
|
||||
COPY --from=alpine/helm:3.21.3@sha256:35da09ba0716fc7c3cd63b6b31ee380a9c7662e95f29ab0e4ae962420afd315b /usr/bin/helm /usr/local/bin/helm
|
||||
COPY --from=mirror.gcr.io/alpine/helm:3.21.3@sha256:35da09ba0716fc7c3cd63b6b31ee380a9c7662e95f29ab0e4ae962420afd315b /usr/bin/helm /usr/local/bin/helm
|
||||
|
||||
ENV PATH="/app/.venv/bin:$PATH" \
|
||||
PYTHONUNBUFFERED=1 \
|
||||
|
||||
@@ -40,12 +40,35 @@ async def _load_instance(task: Task, deps: WorkerDeps) -> Instance:
|
||||
return Instance.model_validate(row)
|
||||
|
||||
|
||||
def _deep_merge(base: dict[str, Any], override: dict[str, Any]) -> dict[str, Any]:
|
||||
"""`override` wins, except where both sides hold a dict — then merge those too.
|
||||
|
||||
Shallow `base | override` would be wrong the moment two layers touch different keys of
|
||||
the same nested map: `{"global": {"imageRegistry": ...}}` overridden by
|
||||
`{"global": {"storageClass": ...}}` silently drops the registry, and the pod pulls from
|
||||
somewhere nobody chose.
|
||||
"""
|
||||
out = dict(base)
|
||||
for key, value in override.items():
|
||||
current = out.get(key)
|
||||
if isinstance(current, dict) and isinstance(value, dict):
|
||||
out[key] = _deep_merge(current, value)
|
||||
else:
|
||||
out[key] = value
|
||||
return out
|
||||
|
||||
|
||||
def _values_for(inst: Instance, entry: CatalogEntry) -> dict[str, Any]:
|
||||
"""Turn a catalog size into helm values."""
|
||||
"""Catalog values, with the requested size's replicas and resources on top.
|
||||
|
||||
Size last, deliberately. An entry that sets `replicaCount` in its own `values:` would
|
||||
otherwise beat the size the tenant actually asked for, and every size would deploy the
|
||||
same shape.
|
||||
"""
|
||||
size = entry.sizes.get(inst.size)
|
||||
if size is None:
|
||||
raise HandlerError(f"size {inst.size!r} not in catalog for {inst.service_type!r}")
|
||||
return {"replicaCount": size.replicas, "resources": size.resources}
|
||||
return _deep_merge(entry.values, {"replicaCount": size.replicas, "resources": size.resources})
|
||||
|
||||
|
||||
async def handle_provision(task: Task, deps: WorkerDeps) -> None:
|
||||
|
||||
Reference in New Issue
Block a user