refactor: converge the patterns multiple authors left divergent
The codebase was written by several agents and had the same concept done more
than one way. This makes it read as one voice, with no behaviour change.
Dedup, each to a single canonical form:
- INSTANCE_COLUMNS: the 13-column instances SELECT list existed as _COLUMNS in
instances.py and reconcile.py (byte-identical) and inlined a third time in
the worker. One exported constant now.
- Settings.runtime_dsn: the three entrypoints each chose between str(pg_dsn)
and pg_dsn.unicode_string(). One property.
- yaml_tempfile: helm._ValuesFile and k8s._ManifestFile were the same
write-yaml-to-a-temp-dir context manager. One helper in adapters/tempyaml.py.
- services/_runtime.py: sleep_or_stop and install_stop_signals were copied
between the worker and reconciler loops. One module, so shutdown behaviour
cannot drift between them.
- k8s.ensure_namespace used MANAGED_BY_LABEL/VALUE from helm.py instead of a
hardcoded literal, so the managed-by label has one definition.
- SvcforgeError is now the root of every svcforge exception (CatalogError,
IllegalTransition, BadWindow, HandlerError), keeping each stdlib base in the
MRO, so `except SvcforgeError` means what errors.py says it does.
- ERROR_MAX_CHARS replaces the repeated `[-2000:]` truncation feeding the same
error columns.
- the reconciler reads settings.metrics_port like the worker, dropping its
duplicate DEFAULT_METRICS_PORT and redundant --metrics-port option; the
SVCFORGE_METRICS_PORT env override still applies through pydantic.
Two smaller correctness/consistency fixes:
- RateLimitResult.retry_after_s computed its delta against datetime.now(UTC)
while the limiter runs on an injectable clock, so it was meaningless under a
FakeClock and drifted by request latency in production. It now carries a
checked_at from the same clock as reset_at.
- handle_provision's notifier.send is wrapped like the reconciler's: a flaky
webhook after the READY CAS would fail the task, and the retry would hit the
READY early-return and drop the notification, turning a good provision into a
failed one.
This commit is contained in:
@@ -18,18 +18,21 @@ from typing import Any
|
||||
from services.worker.deps import WorkerDeps
|
||||
from svcforge_core.domain.models import CatalogEntry, Instance, Task, TaskKind
|
||||
from svcforge_core.domain.states import InstanceState
|
||||
from svcforge_core.errors import SvcforgeError
|
||||
from svcforge_core.obs import get_logger
|
||||
from svcforge_core.repo.instances import INSTANCE_COLUMNS
|
||||
|
||||
log = get_logger("svcforge.worker")
|
||||
|
||||
|
||||
class HandlerError(RuntimeError):
|
||||
class HandlerError(SvcforgeError, RuntimeError):
|
||||
"""A task failed in a way worth retrying. The message lands in tasks.last_error."""
|
||||
|
||||
|
||||
async def _load_instance(task: Task, deps: WorkerDeps) -> Instance:
|
||||
async with deps.pool.connection() as conn, conn.cursor() as cur:
|
||||
await cur.execute(
|
||||
"""select id, team, service_type, size, state, namespace, release_name,
|
||||
chart_version, endpoint, error, expires_at, created_at, updated_at
|
||||
from instances where id = %s""",
|
||||
f"select {INSTANCE_COLUMNS} from instances where id = %s", # noqa: S608 - module constant
|
||||
(task.instance_id,),
|
||||
)
|
||||
row = await cur.fetchone()
|
||||
@@ -75,11 +78,19 @@ async def handle_provision(task: Task, deps: WorkerDeps) -> None:
|
||||
inst.id, InstanceState.PROVISIONING, InstanceState.READY, endpoint=endpoint
|
||||
)
|
||||
if ok:
|
||||
await deps.notifier.send(
|
||||
"instance.ready",
|
||||
f"instance {inst.id} is ready at {endpoint}",
|
||||
{"instance_id": str(inst.id), "team": inst.team, "service_type": inst.service_type},
|
||||
)
|
||||
try:
|
||||
await deps.notifier.send(
|
||||
"instance.ready",
|
||||
f"instance {inst.id} is ready at {endpoint}",
|
||||
{"instance_id": str(inst.id), "team": inst.team, "service_type": inst.service_type},
|
||||
)
|
||||
except Exception:
|
||||
# The provision succeeded and the row is already READY; the notification is a
|
||||
# courtesy. Letting a webhook timeout propagate would fail the task, and the
|
||||
# retry would hit the READY early-return and drop the notification anyway — so a
|
||||
# flaky notifier would turn every provision into a "failed" task. Same guard the
|
||||
# reconciler puts around its own notify.
|
||||
log.exception("notify.failed", instance_id=str(inst.id))
|
||||
|
||||
|
||||
async def handle_deprovision(task: Task, deps: WorkerDeps) -> None:
|
||||
|
||||
Reference in New Issue
Block a user