Services

This is the catalog of every container in the Dataland stack: what each one does, which repo builds it, the endpoints and ports it exposes, and which datastores it depends on. The whole stack runs as one Docker Compose project (name: dataland, file compose.yml) on the Spark DGX VDS, on a single bridge network — dataland-network — fronted by Cloudflare Tunnels and a Tailscale tailnet.

For the wiring between services (who calls whom over the docker network, public ingress paths, the ring layout) see Architecture. This page is the reference table; each row links to a detail page.

How the stack fits together

graph LR
  subgraph clients["Clients"]
    MOB["Mobile app"]
    CUR["Curators"]
    OPS["Operators"]
  end

  subgraph edge["Edge"]
    CF["Cloudflare Tunnel<br/>(host systemd cloudflared)"]
    TS["Tailscale tailnet<br/>(spark 100.124.170.43)"]
  end

  MOB --> CF
  CUR --> CF
  OPS --> TS

  CF -->|dataland.chat| AG["agent :4141"]
  CF -->|"dataland.chat (dashboard)"| MU["museum-api :4144→5001"]
  CF -->|knowledge.dataland.chat| WU["knowledge :4152"]

  AG -->|"HTTP /search, /images/*"| RG["rag :4143"]
  AG -->|"HTTP /api/*"| MU
  AG -->|"HTTP /v1/ops/*"| NA["notification-api :8080"]
  AG -.->|JWKS RS256| AU["auth :9000<br/>dataland-auth"]
  WU -->|"HTTP /ingest/*"| RG

  MU -->|"read (mixed msgpack)"| RDC[("RDC redis<br/>external")]
  MU -->|"publish museum:telemetry"| RD[("redis")]
  NW["notification-worker"] -->|consume telemetry| RD
  NW -->|resolve ticket→user| AG
  NW -->|OneSignal push| OS["OneSignal"]
  NW -->|ops alerts| DSC["Discord / Slack"]

  AG --> PG[("postgres")]
  AU --> PG
  RG --> QD[("qdrant")]
  RG --> GCS["GCS buckets"]

The recurring pattern: mobile + visitor traffic enters over Cloudflare, operator/curator traffic enters over the tailnet (or an SSH tunnel for loopback-only ports), service-to-service calls stay on the docker network using container DNS names (e.g. http://dataland-rag:4143) and never touch host-published ports.

App services

Business-logic containers. All are FastAPI/uvicorn unless noted; all build from a sibling repo under /home/cobanov/DATALAND/ on the host.

Service	Container	Repo	Role	Host port → internal	Datastore deps
Agent	dataland-agent	dataland-agent	AI guide: museum + general SSE chat, conversation history, internal service API	4141 → 4141	Postgres, Redis, RAG, museum-api, notification-api, auth (JWKS)
Museum API + dashboard	dataland-museum	dataland-museum	Bridge to external RDC redis; publishes museum:telemetry; mirrors Visitors:ActiveTicketIDs; serves dashboard + chapters.json	4144 → 5001	Redis (write), RDC redis (read)
Museum simulator	dataland-simulator	dataland-museum	Synthetic telemetry producer for dev/testing (publishes to museum:telemetry)	— (no HTTP)	Redis (write)
RAG	dataland-rag	dataland-rag	Retrieval: hybrid dense+BM25 search + rerank, file/image ingest, Gemini captioning	4143 → 4143 (loopback + tailnet)	Qdrant, GCS
Knowledge	dataland-knowledge	dataland-knowledge	"Catalog Studio" CMS: Refik artwork catalog + Museum sections/scenes, RAG live-sync	4152 → 4152	SQLite (/app/data), GCS, RAG
Notification worker	dataland-notification-worker	dataland-notification	Telemetry rules engine → OneSignal pushes + ops alerts; DLQ + replay	— (no HTTP)	Redis, agent (ticket resolve)
Notification API	dataland-notification-api	dataland-notification	Ops surface: rules reload, ticket state, /v1/ops/* ingress, /metrics	8080 → 8080 (loopback + tailnet)	Redis, agent
Auth	dataland-auth	dataland-agent	RS256 JWT issuer + JWKS (mirrors the CMS signing key); runs auth_server.py from the agent image	9000 (internal)	Postgres (auth_server_users)

Auth runs on Spark

auth is a compose service (container dataland-auth, port 9000) built from the agent image (uv run python auth_server.py). The agent verifies chat JWTs against its JWKS over the docker network at http://dataland-auth:9000; accounts live in the on-host dataland Postgres (auth_server_users). See Auth.

The docs site (docs.dataland.chat) is not a compose service — it is built and served by Cloudflare Pages from docs/src/.

Data plane

Stateful stores. None are published on 0.0.0.0; each is bound to 127.0.0.1 (host tooling + SSH tunnel) and to its *_PUBLIC_BIND tailnet IP (default 100.124.170.43) for direct peer access — never the public Spectrum IP.

Service	Container	Image	Host port → internal	Auth	Holds
Redis	dataland-redis	redis:7-alpine	4145 → 6379	requirepass (REDIS_PASSWORD)	museum:telemetry stream, ticket state, dedup keys, rate-limit state
Postgres	dataland-postgres	postgres:16-alpine	5432 → 5432	user/password	Agent conversations DB and the auth server's accounts (auth_server_users), all in the dataland database — see Auth
Qdrant (HTTP)	dataland-qdrant	qdrant/qdrant	4146 → 6333	none (tailnet is the boundary)	Vector collections (see below)
Qdrant (gRPC)	dataland-qdrant	qdrant/qdrant	4147 → 6334	none	Same collections, gRPC client path

RDC redis is external

The Refik Anadol data center (RDC) redis — the wearable/sensor source of truth — is not part of this Compose stack. dataland-museum reads it (mixed msgpack + plain-UTF-8 encoding) and bridges what it needs onto the local dataland-redis. Configure it via RDC_REDIS_URL. Never replay museum-simulation-playback Avro into dataland-redis; use a dedicated container instead.

Qdrant collections (RAG)

Collection	Approx points	Source	Vectors
knowledge	~4969	Documents + museum sections/scenes/overview (/ingest/file)	gemini-embedding (dense) + BM25 (sparse)
images	~1485	Artwork + chapter images, Gemini-captioned (/ingest/image)	gemini-embedding
scenes	—	Scene JSON (/ingest/sync)	gemini-embedding

Knowledge re-ingest

The 20 museum sections + scenes + overview were re-ingested today, taking knowledge from ~4839 → ~4969 points. The RAG /search client read timeout in the agent was raised 10s → 25s after museum-knowledge queries surfaced a 10s-timeout → 3× retry → agent_timeout chain.

Observability

App-level traces and AI spans go to Logfire. The self-hosted metrics stack (Prometheus / Grafana / Alertmanager / exporters) was turned off (2026-07) and removed from compose.yml — it remains in git history if reinstated. Services still expose /metrics text, but nothing scrapes it currently. See Observability.

Endpoint quick-reference

The endpoints you reach for most often. Internal/service-to-service paths are marked; everything else is client-facing.

Agent (dataland-agent, port 4141)

uvicorn app.main:app. All /v1 endpoints require a mobile access JWT (Authorization: Bearer <token>, RS256, validated against the auth JWKS). /v1/service/* is service-to-service, gated by AGENT_SERVICE_TOKEN.

Method	Path	Purpose
GET	/health, /health/full	Liveness; composite dependency check (RAG + museum)
GET	/v1/auth/me	Resolve the JWT to the Dataland user record
POST	/v1/auth/logout	Stateless logout acknowledgement
POST	/v1/chat/museum	Museum-mode SSE chat. First call registers the ticket; empty first message = instant welcome
POST	/v1/chat/museum/multimodal	Museum chat with an optional image (multipart)
POST	/v1/chat/general	General assistant SSE chat
POST	/v1/chat/general/multimodal	General chat with an optional image (multipart)
GET	/v1/conversations	List the user's conversations
GET	/v1/conversations/{id}/messages	Mobile message timeline (?raw=true for provider-native)
DELETE	/v1/conversations/{id}	Delete a conversation
POST	/v1/conversations/{id}/messages/{mid}/feedback	Like / dislike an assistant message
GET	/v1/service/tickets/{ticket_id}/user	Internal: resolve ticket → user + conversation (used by notification)
POST	/v1/service/chat/museum	Internal: notification-triggered museum chat
GET	/metrics	Prometheus (multiprocess; aggregated across uvicorn workers)

Ticket endpoints removed

/v1/tickets/register and /v1/tickets/current no longer exist. The first POST /v1/chat/museum (or /multimodal) with a ticket_id registers the ticket implicitly, and the conversation id is permanently equal to the ticket id. /v1/ops/complaint and /v1/ops/welcome are not agent endpoints — the agent posts to them on the notification service.

The agent's chat tools: get_visitor_vitals, get_room_info, get_scene_flow, search_knowledge, search_artwork_images. It speaks real room/section names — Data Pavilion (GA), Latent Gallery (GB), Infinity Room (GC), The Sanctuary (GD), Discovery Portal (ON) — never bare codes.

Museum API (dataland-museum, host 4144 → container 5001)

uvicorn run:app. Dashboard is gated by a shared password (MUSEUM_PASSWORD) + signed session cookie.

Method	Path	Purpose
GET	/health	Liveness (pings Redis)
GET	/	Dashboard (auth)
POST	/api/auth/login, /api/auth/logout, GET /api/auth/session	Dashboard auth
GET	/api/active-tickets	Mirror of Visitors:ActiveTicketIDs
GET	/api/tickets/{ticket_id}/vitals	Live vitals for a ticket
GET	/api/tickets/{ticket_id}/session-status	Session state
GET	/api/visitors/{visitor_id}/vitals	Vitals by visitor
GET	/api/rooms/{room_code}/chapters, /api/chapters	Chapter catalog
GET	/api/bridge/metrics	Telemetry-bridge stats
GET	/api/aqi, /api/qsys, /api/audio, /api/onboarding/kiosks	RDC-sourced environment + Q-SYS/audio (plain-UTF-8 decode)
GET	/metrics	Prometheus

RAG (dataland-rag, port 4143)

FastAPI dataland-rag v3.0.0. Write/search endpoints require X-API-Key (RAG_API_KEY). Reranker: jina-reranker-v2-base-multilingual (ONNX, the heavy tenant — 12 GB / 12 cores).

Method	Path	Purpose
GET	/health, /health/full	Liveness; full dependency check
POST	/search	Hybrid dense+BM25 search + rerank (knowledge)
POST	/ingest/file	Ingest a document into knowledge
POST	/ingest/image	Ingest an image into images (Gemini captioning)
POST	/ingest/sync	Ingest scene JSON into scenes
POST	/ingest/by-project-slug/{slug}	Project-scoped ingest
POST	/images/search/text, /images/search/image	Image search by text / by image
GET	/images/{filepath:path}, /images/extracted/{filepath:path}	Serve images
POST	/admin/sparse-backfill	Ops-driven BM25 backfill
GET	/metrics	Prometheus

GCS buckets: dataland-public/artworks (+ cobanov-public/chapters for chapter images), dataland-private (documents, museum/scenes).

Knowledge (dataland-knowledge, port 4152)

The Catalog Studio CMS. Auth-gated (KNOWLEDGE_PASSWORD + session cookie); all /api/* routes sit behind require_auth. Persists to SQLite (/app/data) + GCS, and live-syncs to RAG (text → /ingest/file, images → /ingest/image; rag slugs museum-section-<slug> / museum-scene-<slug>, replace-by-slug with UUIDv5 ids).

Method	Path (prefix /api)	Purpose
GET/POST/PUT/DELETE	/projects, /projects/{id}	Refik artwork catalog CRUD
GET	/projects/search, /catalog, /projects/{id}/catalog	Search + catalog export
POST/PUT/DELETE	/projects/{id}/images...	Artwork image upload + management
GET/PUT	/museum, /museum/catalog	Museum overview
GET/POST/PUT/DELETE	/museum/sections, /museum/sections/{identifier}	Sections CRUD (+ images, catalog)
GET/POST/PUT/DELETE	/museum/scenes, /museum/scenes/{identifier}	Scenes CRUD (+ images, catalog)

Notification API (dataland-notification-api, port 8080)

Ops surface. Write/inspect endpoints require the ops bearer token; /health + /metrics are open (and readable by tailnet peers).

Method	Path	Purpose
GET	/health	Liveness (pings Redis)
GET	/metrics	Prometheus (incl. DLQ + telemetry stream XLEN gauges)
GET	/rules, POST /rules/reload, POST /rules/validate	Inspect / hot-swap / dry-run the rules TOML
POST	/events	Process a raw telemetry event (debug)
GET	/state/{ticket_id}	Per-ticket notification state
POST	/v1/ops/complaint	Receive a silent visitor-complaint event from the agent
POST	/v1/ops/welcome	Send the museum welcome push for a ticket

Telemetry rules: visit_started (welcome), heart_rate, skin_conductance, artwork_engagement, room_transition, session_flow, experience_tip, visit_ended. Content pushes are gated until Gallery A (welcome exempt); room-transition pushes bypass the 240s cooldown. Ops alerts fan out to Discord/Slack via a swappable OpsNotifier (OPS_NOTIFIER_PROVIDER, comma-separated multi-provider).

Auth (dataland-auth, port 9000)

auth_server.py running from the agent image as an on-Spark compose service — the RS256 signing authority for mobile JWTs. Internal only (reached at http://dataland-auth:9000 over the docker network).

Method	Path	Purpose
GET	/.well-known/jwks.json	Public JWKS the agent validates tokens against
POST	/api/auth/signup, /api/auth/login	Mobile account creation + login
GET	/api/auth/me	Verify token, return user
GET	/	Test/login HTML

JWKS mirror

auth mirrors the CMS signing key (kid dataland-rs256-1) via AUTH_EXTRA_JWKS_JSON (or data/extra_jwks.json), removing the chat-auth single point of failure. The agent logs a WARN when a fallback JWKS provider is the sole validator. Re-run provisioning (update the key + restart auth) after a CMS key rotation.

A note on "loopback" ports

A port marked loopback is bound to 127.0.0.1:<port> on the host. Service-to-service traffic uses the docker network and is unaffected. Most data-plane and ops ports are additionally published on the tailnet IP (*_PUBLIC_BIND, default 100.124.170.43) for direct peer access. To reach a loopback-only port from your workstation, open an SSH tunnel:

ssh -L 4143:127.0.0.1:4143 \
    -L 4145:127.0.0.1:4145 \
    -L 4146:127.0.0.1:4146 \
    -L 8080:127.0.0.1:8080 \
    ege@100.124.170.43      # (1)!

1. Tunnel terminates on the tailnet IP (100.124.170.43), never the public Spectrum IP — the data plane is bound only to loopback + the *_PUBLIC_BIND tailnet address. Each -L forwards a loopback-only port: RAG (4143), Redis (4145), Qdrant HTTP (4146), notification-api (8080). Cloudflare-fronted services (4141/4144/4152) need no tunnel.

Cloudflare-fronted services (agent 4141, museum 4144, knowledge 4152) are reachable through their public *.dataland.chat hostname. Tailnet peers can also hit any *_PUBLIC_BIND port directly as spark:<port> without a tunnel.

Detail pages

• Agent
• Museum
• RAG
• Knowledge
• Notification
• Auth
• Redis
• Postgres
• Qdrant

See also: Architecture · Public ports · Deploy · Observability · Service hosting & relocation