Public ports & bind policy

Every Dataland service runs as a Docker-Compose container on the Spark DGX VDS (ege@100.124.170.43). Internal service-to-service traffic stays on the dataland-network Docker bridge and uses container DNS names (http://dataland-agent:4141, http://dataland-rag:4143, dataland-auth:9000, dataland-redis:6379, …) — never host-published ports. Only the host port mappings described below are reachable from off-container, and only a small public subset leaves the host at all.

The one rule: never 0.0.0.0 for data-plane or ops services

Binding a stateful or unauthenticated service to 0.0.0.0 republishes it on the host's public Spectrum IP, not just the tailnet. That regression once exposed Postgres (5432) and Qdrant (no API key) to the open internet. The fix: bind those services to 127.0.0.1 plus the tailnet IP, explicitly, and let the workflow's post-deploy /health poll catch a bad value.

The *_PUBLIC_BIND variables let each data-plane and ops service publish on two host IPs: 127.0.0.1 (local host + SSH-tunnel workflow) and the host's tailnet IP (direct access from Tailscale peers, default 100.124.170.43). Never 0.0.0.0, and never 127.0.0.1 for the bind var (it would collide with the loopback line already present in compose.yml).

Bind policy at a glance

flowchart LR
  Internet([Public internet]) -->|HTTPS| CF[Cloudflare Tunnel<br/>cloudflared on host systemd]
  Peers([Tailscale peers]) -->|tailnet 100.124.170.43| TN
  Op([Operator workstation]) -->|SSH -L tunnel| LB

  subgraph Host["Spark VDS — host network namespace"]
    CF -->|127.0.0.1| LB[127.0.0.1 loopback binds]
    TN[Tailnet-IP binds<br/>*_PUBLIC_BIND]
    LB --> DKR
    TN --> DKR
  end

  DKR[(Docker bridge<br/>dataland-network)]

Three publishing patterns are used in compose.yml:

Pattern	Host binding(s)	Used by	Why
Cloudflare-fronted	0.0.0.0 (single mapping) — but only ever reached through cloudflared	Agent, Museum API/dashboard, Knowledge	Public hostnames terminate TLS at Cloudflare; the app enforces its own auth (JWT / shared-password / API token).
Dual-bind (loopback + tailnet)	127.0.0.1 and ${X_PUBLIC_BIND}	Postgres, Qdrant (HTTP+gRPC), Redis, RAG, Notification API, Auth	Stateful or weakly/un-authenticated: the tailnet is the trust boundary. Two explicit lines, never 0.0.0.0.
Loopback-only / no host port	127.0.0.1 only, or no ports: at all	sim redis (dev overlay)	Dev-only sidecar; loopback keeps the sim stream off the tailnet.

Cloudflare-fronted ≠ 0.0.0.0 is safe

The agent, museum and knowledge containers publish with a single ${X_PUBLIC_PORT}:container mapping, which Docker binds on 0.0.0.0. That is acceptable only because each enforces application-layer auth (RS256 JWT, shared-password session, API token) and the intended path is the Cloudflare tunnel. Do not copy this pattern to a service without auth.

Application services

Service	Container	Host port (var)	Container port	Bind policy	Cloudflare hostname
Agent	dataland-agent	4141 (AGENT_PUBLIC_PORT)	4141	0.0.0.0 (Cloudflare-fronted; RS256 JWT via JWKS)	dataland.chat
Museum API + dashboard	dataland-museum	4144 (MUSEUM_PUBLIC_PORT)	5001	0.0.0.0 (Cloudflare-fronted; dashboard shared-password gate)	museum dashboard hostname
Knowledge ("Catalog Studio")	dataland-knowledge	4152 (KNOWLEDGE_PUBLIC_PORT)	4152	0.0.0.0 (Cloudflare-fronted; shared-password gate)	knowledge.dataland.chat
Auth (JWKS)	dataland-auth	9000 (AUTH_PUBLIC_PORT)	9000	dual-bind 127.0.0.1 + AUTH_PUBLIC_BIND; internal JWKS (agent reaches it at http://dataland-auth:9000)	— (internal)
RAG	dataland-rag	4143 (RAG_PUBLIC_PORT)	4143	dual-bind 127.0.0.1 + RAG_PUBLIC_BIND; X-API-Key auth	— (internal)
Notification API	dataland-notification-api	8080 (NOTIFICATION_PUBLIC_PORT)	8080	dual-bind 127.0.0.1 + NOTIFICATION_PUBLIC_BIND; ops-token on writes	— (internal)

Notification worker has no host port

dataland-notification-worker is a pure stream consumer of museum:telemetry — it publishes nothing and exposes no host port. Only dataland-notification-api (the DLQ + state inspector) is published, and only on loopback + tailnet.

The docs site is not a container

This documentation site is built by Cloudflare Pages from docs/src/; it is not a compose service and has no host port. Updating the docs means editing files under docs/src/ and letting Cloudflare Pages rebuild.

Cloudflare hostnames

Hostname	Fronts	Container target (host)	App-layer gate
dataland.chat	Chat agent (SSE chat, conversations, service endpoints)	dataland-agent → 127.0.0.1:4141	RS256 JWT (JWKS)
knowledge.dataland.chat	Knowledge (Catalog Studio CMS)	dataland-knowledge → 127.0.0.1:4152	Shared-password session + optional API token
museum dashboard	Live museum monitor UI	dataland-museum → 127.0.0.1:4144	Shared-password session (MUSEUM_PASSWORD)

How the tunnel reaches the container

cloudflared runs as a host systemd unit (not a compose service) and ingresses each public hostname to localhost:<host-port> on the VDS. TLS terminates at Cloudflare. Adding a new public service means registering a route in the Zero Trust dashboard (Tunnels → tunnel → Public Hostname → Add), as described in Deploy.

Data plane (dual-bind: loopback + tailnet)

These are stateful or carry weak/no built-in auth, so they are published on 127.0.0.1 and their *_PUBLIC_BIND tailnet IP — two explicit lines in compose.yml, never 0.0.0.0.

Service	Container	Loopback host port (var)	Container port	Tailnet bind var	Trust boundary
Postgres	dataland-postgres	5432 (POSTGRES_PORT)	5432	POSTGRES_PUBLIC_BIND	Tailnet (password auth only)
Redis	dataland-redis	4145 (REDIS_PUBLIC_PORT)	6379	REDIS_PUBLIC_BIND	Tailnet + --requirepass (REDIS_PASSWORD)
Qdrant HTTP	dataland-qdrant	4146 (QDRANT_HTTP_PORT)	6333	QDRANT_PUBLIC_BIND	Tailnet (Qdrant runs with no API key)
Qdrant gRPC	dataland-qdrant	4147 (QDRANT_GRPC_PUBLIC_PORT)	6334	QDRANT_PUBLIC_BIND	Tailnet (no API key)

Qdrant has no auth — the tailnet IS the auth

QDRANT_API_KEY is empty in the deploy env. The only thing between an attacker and the knowledge / images / scenes collections is the bind policy. Setting QDRANT_PUBLIC_BIND=0.0.0.0 would publish a fully open vector store on the public IP. Redis is safer (--requirepass) but the same rule holds: tailnet, not world.

Host binding does not affect service-to-service traffic

rag talks to qdrant, the agent talks to postgres and redis, etc., all over the dataland-network Docker bridge. Changing or removing a host port binding never breaks internal calls — those resolve container-to-container, independent of the host publish lines.

Dev & simulator overlays

Service	Container	Host port	Bind	Notes
Museum simulator	dataland-simulator	—	—	compose.sim.yml overlay (started by service name). Publishes synthetic museum:telemetry into dataland-redis. Production runs without it — the RDC bridge in museum-api feeds telemetry instead.
Sim redis	dataland-redis-sim	4149	127.0.0.1 only	compose.sim.yml overlay. Separate volume + password (SIM_REDIS_PASSWORD). Kept on loopback so the operator can XINFO from the VDS without exposing the sim stream to the tailnet.
Telemetry sim publisher	dataland-telemetry-sim	—	—	compose.sim.yml. Emits museum:telemetry shaped like the prod RDC → museum bridge into dataland-redis-sim, redirecting the notification worker/api at the sim redis.

Isolate playback from live telemetry

The compose.sim.yml overlay exists precisely so synthetic events land on a dedicated dataland-redis-sim, never on the live dataland-redis. Real visitors are not disturbed. Tear down with scripts/stop-simulator.sh.

Port allocation reference

All internally-allocated host ports live in the 414x range plus a few upstream-conventional ports. The canonical values live in .env.example under "Public host ports".

Port	Service	Source var
4141	Agent	AGENT_PUBLIC_PORT
4143	RAG	RAG_PUBLIC_PORT
4144	Museum API + dashboard	MUSEUM_PUBLIC_PORT
4145	Redis	REDIS_PUBLIC_PORT
4146	Qdrant HTTP	QDRANT_HTTP_PORT
4147	Qdrant gRPC	QDRANT_GRPC_PUBLIC_PORT
4149	Sim redis (dev)	hard-coded (compose.sim.yml)
4152	Knowledge	KNOWLEDGE_PUBLIC_PORT
5432	Postgres	POSTGRES_PORT
8080	Notification API	NOTIFICATION_PUBLIC_PORT
9000	Auth (JWKS)	AUTH_PUBLIC_PORT

Health-check cheat sheet

Run on the VDS (everything resolves on localhost there). Off-host, open the SSH tunnel for the loopback-only services first, or hit the tailnet IP (100.124.170.43) for the dual-bound ones.

for url in \
  http://localhost:4141/health \
  http://localhost:4143/health \
  http://localhost:4144/health \
  http://localhost:4152/health \
  http://localhost:8080/health \
  http://localhost:9000/.well-known/jwks.json
do
  printf '%-55s ' "$url"
  curl -fsS -o /dev/null -w '%{http_code}\n' "$url" || echo down # (1)!
done

1. -f makes curl exit non-zero on HTTP 4xx/5xx so the || echo down branch fires; -sS silences the progress meter but keeps real errors; -o /dev/null discards the body and -w '%{http_code}\n' prints only the status code. The auth probe hits /.well-known/jwks.json (the JWKS endpoint, not a /health route).

Related pages

• Deploy — Coolify, image pinning, Cloudflare ingress, smoke tests.
• Observability — Logfire traces + structured JSONL logs.
• Service hosting & relocation — what runs on the host and what stays on dataland-network.
• Auth — JWKS, RS256 validation, the CMS signing-key mirror.