fix: shared runtime helper must live where every image packages it
ci / lint (push) Successful in 28s
ci / types (push) Successful in 37s
ci / unit (push) Successful in 27s
ci / security (push) Successful in 38s
ci / dockerfile (push) Successful in 6s
ci / chart (push) Successful in 9s
ci / integration (push) Successful in 49s
ci / image (api) (push) Successful in 2m25s
ci / image (reconciler) (push) Successful in 2m45s
ci / image (worker) (push) Successful in 2m37s
ci / bump (push) Successful in 24s

services/_runtime.py crashed the worker and reconciler on boot with
`ModuleNotFoundError: No module named 'services._runtime'`, while every unit and
integration gate was green and the API rolled out fine.

The cause is packaging, not code. Each service Dockerfile copies only its own
`services/<svc>/` subdir — `services/` itself is a namespace package with no
__init__.py, so a file added at the `services/` root is never copied into any
image. Tests import from the source tree, where the file exists, so nothing
below the image boundary could catch it. The API survived only because it does
not import the helper.

Moved to svcforge_core.runtime, which `COPY libs/ libs/` packages into every
image, next to adapters/tempyaml.py for the same reason.

Added an import smoke test to the image job: `docker run --entrypoint python
<image> -c "import services.<svc>.main"` loads the whole transitive graph inside
the built image and fails the build before the digest is pushed. This is the
one check the test suite structurally cannot perform — it runs against source,
the image is a different filesystem — and it is exactly the gap this bug fell
through.
This commit is contained in:
Nguyen Minh Phuc
2026-07-21 02:40:37 +00:00
parent 9c8d10ce1f
commit e971e04d75
4 changed files with 17 additions and 2 deletions
+15
View File
@@ -373,6 +373,21 @@ jobs:
-t svcforge/${{ matrix.svc }}:ci \ -t svcforge/${{ matrix.svc }}:ci \
. .
- name: import smoke test
# Import the service's entrypoint module INSIDE the built image, which every unit and
# integration test that passes cannot do: they import from the source tree, where
# every file exists. The image is a different filesystem — each Dockerfile copies only
# its own `services/<svc>/`, so a shared module added at the `services/` root, or any
# dependency the Dockerfile forgets, is present in the tests and absent in the image.
#
# That gap shipped a reconciler that crashed on boot with
# `ModuleNotFoundError: No module named 'services._runtime'` while every gate was
# green. Importing main here loads the whole transitive graph and fails the build
# before the digest is pushed, instead of after ArgoCD has rolled it out.
run: |
docker run --rm --entrypoint python svcforge/${{ matrix.svc }}:ci \
-c "import services.${{ matrix.svc }}.main"
- name: trivy - name: trivy
# Run trivy directly rather than via aquasecurity/trivy-action, for the same reason # Run trivy directly rather than via aquasecurity/trivy-action, for the same reason
# gitleaks is run directly above: the command is the documented one, pinned by # gitleaks is run directly above: the command is the documented one, pinned by
+1 -1
View File
@@ -34,7 +34,6 @@ from dataclasses import dataclass
import typer import typer
from services._runtime import install_stop_signals, sleep_or_stop
from svcforge_core.adapters.clock import Clock, SystemClock from svcforge_core.adapters.clock import Clock, SystemClock
from svcforge_core.adapters.helm import HelmProvisioner, Provisioner from svcforge_core.adapters.helm import HelmProvisioner, Provisioner
from svcforge_core.adapters.notify import LogNotifier, Notifier from svcforge_core.adapters.notify import LogNotifier, Notifier
@@ -54,6 +53,7 @@ from svcforge_core.repo.db import DictPool, make_pool
from svcforge_core.repo.instances import InstanceRepo from svcforge_core.repo.instances import InstanceRepo
from svcforge_core.repo.reconcile import ReconcileRepo from svcforge_core.repo.reconcile import ReconcileRepo
from svcforge_core.repo.tasks import TaskRepo from svcforge_core.repo.tasks import TaskRepo
from svcforge_core.runtime import install_stop_signals, sleep_or_stop
from svcforge_core.settings import Settings, load_settings from svcforge_core.settings import Settings, load_settings
log = get_logger("svcforge.reconciler") log = get_logger("svcforge.reconciler")
+1 -1
View File
@@ -15,7 +15,6 @@ from collections.abc import Awaitable
from opentelemetry import trace from opentelemetry import trace
from services._runtime import install_stop_signals, sleep_or_stop
from services.worker.deps import WorkerDeps from services.worker.deps import WorkerDeps
from services.worker.handlers import HANDLERS from services.worker.handlers import HANDLERS
from svcforge_core import obs from svcforge_core import obs
@@ -27,6 +26,7 @@ from svcforge_core.domain.models import Task, TaskKind
from svcforge_core.repo.db import make_pool from svcforge_core.repo.db import make_pool
from svcforge_core.repo.instances import InstanceRepo from svcforge_core.repo.instances import InstanceRepo
from svcforge_core.repo.tasks import TaskRepo from svcforge_core.repo.tasks import TaskRepo
from svcforge_core.runtime import install_stop_signals, sleep_or_stop
from svcforge_core.settings import Settings, load_settings from svcforge_core.settings import Settings, load_settings
log = obs.get_logger("svcforge.worker") log = obs.get_logger("svcforge.worker")