svcforge/USER_GUIDE.md

# svcforge — user guide

Ask for a managed service, get one. svcforge provisions Elasticsearch, Redis, Postgres and
a couple of tiny test services into Kubernetes, one namespace per team, and keeps them
matching what the database says they should be.

**Base URL:** `https://svcforge.oci-oci.duckdns.org`

| | |
|---|---|
| Swagger UI (send requests from the browser) | [`/docs`](https://svcforge.oci-oci.duckdns.org/docs) |
| ReDoc (nicer to read) | [`/redoc`](https://svcforge.oci-oci.duckdns.org/redoc) |
| Raw OpenAPI spec | [`/openapi.json`](https://svcforge.oci-oci.duckdns.org/openapi.json) |

The spec is generated from the same models and routes the server runs, so it cannot drift
from the implementation. Generate a client from it rather than hand-rolling one.

---

## The three things that will surprise you

1. **Writes are asynchronous.** `POST` and `DELETE` return **202 Accepted**. Nothing is
   provisioned when you get the response — you have a row and a queued task. Poll
   `GET /v1/instances/{id}` and watch `state`.
2. **Authorisation is a WHERE clause.** Another team's instance returns **404**, not 403.
   The API never confirms that an id you cannot access exists.
3. **Every non-2xx body is `{"code", "message"}`** — including the 404s and 405s raised by
   the framework itself. Never branch on the body's shape.

---

## Getting a token

Every `/v1` route needs `Authorization: Bearer <jwt>`. The token is verified against the
configured JWKS (RS256 only), and its **`team` claim** decides which instances you see.
`aud`, `iss` and `exp` are all required and all checked.

> **The public deployment currently issues no tokens.** `SVCFORGE_JWKS_URL` points at
> `https://auth.oci-oci.duckdns.org/realms/svcforge/...`, and no identity provider is
> deployed there — the hostname resolves to the ingress, which answers with its default
> self-signed certificate. The API logs `JWKS warm-up failed` at startup and every `/v1`
> request returns `401`. Unauthenticated routes (`/healthz`, `/readyz`, `/metrics`,
> `/docs`, `/openapi.json`) work normally. To make the live API usable, either deploy an
> OIDC provider at that realm URL or repoint `auth.jwksUrl` / `auth.issuer` in
> `deploy/chart/values.yaml` at one that exists.

Any OIDC provider works. With Keycloak, the client-credentials flow is:

```bash
TOKEN=$(curl -s -X POST \
  https://auth.example.com/realms/svcforge/protocol/openid-connect/token \
  -d grant_type=client_credentials \
  -d client_id=svcforge-cli \
  -d client_secret="$CLIENT_SECRET" | jq -r .access_token)
```

The provider must put a `team` claim in the token (a Keycloak protocol mapper, or the
equivalent) and set `aud: svcforge`. A token without a non-empty string `team` is a 401.

### Running it locally instead

The fastest way to actually drive the API is to run it yourself with auth off:

```bash
export SVCFORGE_PG_DSN="postgresql://svcforge:svcforge@127.0.0.1:5432/svcforge"
export SVCFORGE_AUTH_DISABLED=true          # refused unless SVCFORGE_ENVIRONMENT=local
uv run uvicorn services.api.main:app --factory
```

Every request is then team `platform` and no header is needed. `check_production()` refuses
this flag whenever `SVCFORGE_ENVIRONMENT` is anything but `local`, so it cannot escape a
laptop.

---

## The catalog

`service_type` and `size` must both exist in the catalog. An unknown `service_type` is a
**404**; a real service type with an unknown size is a **422** that lists the valid sizes.

| `service_type` | `size` | Memory request | Notes |
|---|---|---|---|
| `elasticsearch` | `small`, `medium` | 1Gi / 4Gi per replica | 1 or 3 replicas |
| `redis` | `small`, `medium` | 256Mi / 1Gi | 1 or 3 replicas |
| `postgres` | `small`, `medium` | 512Mi / 2Gi | 1 or 2 replicas |
| `podinfo` | `small`, `medium` | 16Mi / 32Mi | tiny, for exercising the platform |
| `nginx` | `small`, `medium` | 32Mi / 64Mi | tiny, for exercising the platform |

Use `podinfo` or `nginx` to exercise the control plane: they are single small pods and fit
on a node with no room for a real Elasticsearch. Both are pulled straight from an OCI
registry. The three larger entries reference a `bitnamilegacy/` chart repo that the worker
image does not currently configure, so they will fail at provision time until it is added.

---

## Endpoints

### `POST /v1/instances` → 202

```bash
curl -X POST https://svcforge.oci-oci.duckdns.org/v1/instances \
  -H "Authorization: Bearer $TOKEN" \
  -H 'Content-Type: application/json' \
  -d '{"service_type": "podinfo", "size": "small", "ttl_days": 7}'
```

```json
{
  "id": "0f8b7d3e-1c2a-4f5b-9e6d-7a8b9c0d1e2f",
  "state": "requested",
  "service_type": "podinfo",
  "size": "small",
  "endpoint": null,
  "chart_version": "6.7.1",
  "error": null
}
```

The response carries a `Location` header pointing at the instance. `ttl_days` (1–30,
optional) deletes the instance automatically; omit it for no expiry. Unknown body fields
are rejected with a 422 rather than ignored.

### `GET /v1/instances/{id}` → 200

The polling endpoint. Repeat until `state` is `ready` or `failed`.

```bash
curl -H "Authorization: Bearer $TOKEN" \
  https://svcforge.oci-oci.duckdns.org/v1/instances/$ID
```

### `GET /v1/instances` → 200

Your team's instances, newest first. `?limit=` accepts 1–200 and defaults to 50. No
endpoint here returns "all rows".

### `DELETE /v1/instances/{id}` → 202

```bash
curl -X DELETE -H "Authorization: Bearer $TOKEN" \
  https://svcforge.oci-oci.duckdns.org/v1/instances/$ID
```

Moves the instance to `deleting` and queues the teardown; the helm uninstall has not
happened when this returns. Deleting something already `deleting` or `deleted` is a **409**.

### Unauthenticated

`GET /healthz` (liveness, no I/O) · `GET /readyz` (readiness, checks Postgres only) ·
`GET /metrics` (Prometheus exposition).

---

## The lifecycle

```
requested ──> provisioning ──> ready ──> deleting ──> deleted
     │              │            
     └──────────────┴──> failed
```

| State | What it means |
|---|---|
| `requested` | The row exists and a provision task is queued. |
| `provisioning` | A worker is running `helm upgrade --install`. |
| `ready` | The release is up. **Only this state carries a usable `endpoint`.** |
| `failed` | The provision exhausted its retries. `error` says why. |
| `deleting` | Teardown queued or running. |
| `deleted` | Terminal. |

`endpoint` is in-cluster DNS —
`http://<release>.tenant-<team>.svc.cluster.local` — reachable from inside the cluster, not
from your laptop.

A few things happen without you asking:

- **Drift repair.** If a `ready` instance's helm release disappears, the reconciler notices
  within ~60s and re-provisions it. You may see `ready → failed → provisioning → ready`.
- **TTL.** An instance past `expires_at` is torn down automatically.
- **Upgrades.** When the catalog pins a newer chart version, instances are upgraded inside
  their maintenance window, one at a time. Entries marked `security: true` skip the window.

A `failed` instance is not retried automatically — it needs a human.

---

## Errors

Every non-2xx body is the same shape. `code` is stable and meant for machines; `message` is
for humans and must not be parsed.

```json
{"code": "unknown_service_type", "message": "no such service_type: mongodb"}
```

| Status | `code` | Cause |
|---|---|---|
| 401 | `unauthorized` | Missing, expired, malformed or unverifiable token. Never says which. |
| 404 | `unknown_service_type` | Not in the catalog. |
| 404 | `not_found` | No such instance — **or it belongs to another team**. |
| 409 | `illegal_transition` | e.g. deleting something already deleted. |
| 409 | `conflict` | The row changed between the read and the write. Retry. |
| 422 | `unknown_size` | The message lists the sizes that exist. |
| 422 | `validation_error` | Malformed body: bad type, extra field, `ttl_days` out of range. |
| 429 | `rate_limited` | Over the per-team budget. Honour `Retry-After`. |
| 503 | `not_ready` | `/readyz` only: Postgres is unreachable. |

**Rate limiting** is per team, 60 requests/minute by default, and it fails *open* — if the
limiter's Redis is down you are unmetered rather than refused. A 429 always means a real,
counted overage.

---

## The CLI

`services/cli/` is a thin API client. It holds a URL and a token and never touches the
database.

```bash
export SVCFORGE_API_URL=https://svcforge.oci-oci.duckdns.org
export SVCFORGE_API_TOKEN="$TOKEN"

svcforge create podinfo --size small --ttl 7d --wait
svcforge list --state ready
svcforge status <instance-id>
svcforge delete <instance-id> --yes
```

`--wait` polls until the instance reaches `ready` or `failed`.

---

## Generating a client

```bash
curl -s https://svcforge.oci-oci.duckdns.org/openapi.json > openapi.json
openapi-generator-cli generate -i openapi.json -g python -o ./client
```

The error models are declared on every route, so a generated client gets typed 401/404/409/
422 bodies rather than guessing. `tests/integration/test_api.py` pins the description, the
tags and the bearer security scheme, so these docs fail CI if they rot.

---

## Polling, end to end

```bash
ID=$(curl -s -X POST https://svcforge.oci-oci.duckdns.org/v1/instances \
  -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
  -d '{"service_type":"podinfo","size":"small","ttl_days":1}' | jq -r .id)

while :; do
  BODY=$(curl -s -H "Authorization: Bearer $TOKEN" \
    https://svcforge.oci-oci.duckdns.org/v1/instances/$ID)
  STATE=$(jq -r .state <<<"$BODY")
  echo "$STATE"
  case "$STATE" in
    ready)  jq -r .endpoint <<<"$BODY"; break ;;
    failed) jq -r .error    <<<"$BODY"; exit 1 ;;
  esac
  sleep 5
done
```

Poll every few seconds, not every few milliseconds. A provision is a helm install against a
StatefulSet; `podinfo` takes seconds, Elasticsearch takes minutes.