Run the full stack from source

Running a service from source — go run / uv run on the host instead of a container — is what you do when you’re actually changing that service: fast reload, a debugger attached, breakpoints. You rarely run the whole fleet this way. The usual shape is one service from source against the rest in Docker: bring the stack up with make up, stop the one container you’re editing, and run its repo directly.

This page gives the full manual bring-up order so you understand the dependency chain, then the practical “one from source, rest in Docker” recipe.

Just want it running? Use make up

If you don’t need to edit a service, the one-command docker-compose quick start builds and starts the entire backend (schema-init and wiring included). Come back here when you need a service running on the host.

Why bottom-up

Services dial downward: the frontend needs the gateway; the gateway needs core-database + core-identity; the tool workers need core-mcp + core-storage; and everything needs Postgres and Redis. Start from the bottom so each process finds its dependencies already listening — otherwise the gRPC clients spin on connect.

Canonical local ports

Run each source process on the same host port make up publishes, so one .env works whether a dependency is in Docker or on the host. These are verified against docker-compose.yml:

Service	Shape	Listens on (from source)	Others reach it via
postgres	infra	localhost:5432	DATABASE_DSN
redis	infra	localhost:6379	REDIS_ADDR
azurite	infra (blob emulator)	localhost:10000	AZURE_CONNECTION_STRING
core-database	Go, gRPC/h2c	:8080	CORE_DATABASE_GRPC
core-identity	Python, HTTP	:8001	CORE_IDENTITY_GRPC
core-storage	Python, gRPC	:8081	CORE_STORAGE_GRPC
core-mcp	Go, gRPC/h2c	:8090	CORE_MCP_GRPC / CORE_MCP_URL
core-gateway-consumer	Python, gRPC	:8082	CORE_GATEWAY_CONSUMER_GRPC
a tool worker	Python, no ingress	— (polls the queue)	—
soundverse-saas-2.0	Node, Next.js	:3000	— (the browser)

Defaults already match — except two ports

core-database defaults to LISTEN_ADDR :8080 and core-mcp defaults to :8090, so those need no override. core-identity, core-storage, and core-gateway-consumer all default to port 80 in code; the table above bumps each to a non-privileged port (APP_PORT / hypercorn bind) so you don’t need root.

Bring-up order

Each backend service reads its config from a local .env in its own repo (cp .env.example .env) — see Configuring .env for the shared-secret and TLS-flag details. Every step below assumes that file exists.

1. Postgres + Redis + Azurite. The pragmatic path is to run just the stateful containers from the super-repo and leave them up:

   docker compose up -d postgres redis azurite
   make azurite-init          # create the private/public/live blob containers (once)

   On a first boot (empty data volume) Postgres auto-runs the schema init, so step 2 is already done for you. Endpoints: Postgres localhost:5432, Redis redis://localhost:6379/0, Azurite blob on localhost:10000.

2. Apply the schema — only if you’re using a native Postgres. core-database runs no migrations; it expects the schema to already exist. Load, in FK-safe order: first compose/postgres/00-prelude.sql (extensions, schema namespaces, the storage.uuid_generate_v7() stub that breaks the storage⇄generation cycle), then the per-domain DDL files under soundverse-proto/proto/soundverse_proto/<domain>/v1/schema.sql (e.g. identity, storage, chat, generation/task, generation/tool, billing/tokens). The compose loader 10-load-schemas.sh is the reference for the exact order.

3. core-database Go — the single Postgres door; everything else is a client of it.

   cd core-database
   go mod tidy
   go run ./cmd/server        # LISTEN_ADDR defaults to :8080

   Needs Postgres (and Redis for caching). It pings the database at startup and exits if the schema is missing — that’s your signal step 2 didn’t run.

4. core-identity Python — validates Logto OIDC tokens (JWKS) and just-in-time provisions users into core-database. It’s a FastAPI HTTP app, so drive it with hypercorn (no Makefile in this repo):

   cd core-identity
   uv sync
   uv run hypercorn app.main:app -b 0.0.0.0:8001

   Needs core-database (CORE_DATABASE_GRPC). Browser login needs a real Logto OIDC app; token validation works offline. See Configure Logto.

5. core-storage Python — content-deduped blob uploads and the FLAC → CMAF/HLS transcode. Also no Makefile; run the module directly:

   cd core-storage
   uv sync
   APP_PORT=8081 uv run python -m app.main

   Needs core-database and a blob backend. Point AZURE_CONNECTION_STRING at your Azurite from step 1, or core-storage falls back to Azure managed identity and fails on a laptop. Uploads 404 until you’ve run make azurite-init.

6. core-mcp Go — the MCP tool registry + the second billing pipeline the agent runs through.

   cd core-mcp
   go mod tidy
   go run ./cmd/server        # LISTEN_ADDR defaults to :8090

   Needs core-database, core-storage, and Redis.

7. core-gateway-consumer Python — the authenticated edge the frontend talks to (authz → rate-limit → price → reserve → queue). This repo ships a Makefile:

   cd core-gateway-consumer
   make sync
   APP_PORT=8082 make run     # make run = uv run python -m app.main

   Needs core-database, core-identity, and Redis.

8. A tool worker Python — pick one, e.g. core-tool-sansaarm (song-gen) or core-tool-agent. Workers have no ingress — they poll the task queue — so there’s no port to set:

   cd core-tool-sansaarm
   make sync
   make run                   # uv run python -m app.main

   Needs core-database (queue + registry), core-storage, and Redis. Nothing generates until at least one worker is up: queued tasks sit at “planning…”, and because the AI Tools panel is registry-only (no curated fallback), it stays empty until a worker boots and self-registers its tools.

9. soundverse-saas-2.0 Node — the Next.js SaaS UI.

   cd soundverse-saas-2.0
   npm install
   npm run dev                # next dev on :3000

   Needs core-gateway-consumer (CORE_GATEWAY_CONSUMER_GRPC). Full frontend details, including the standalone UI2.0 studio, are in Run the frontends.

The traps that bite bring-up

Set *_USE_TLS=false locally or every dial fails

In staging/prod the Go data services sit behind a TLS-only ingress on :443, so the secure default for CORE_DATABASE_USE_TLS / CORE_STORAGE_USE_TLS (and the gateway/worker equivalents) is true. Locally you speak plaintext h2c — leave them at the default and the gRPC dial fails with an opaque transport error. The shipped .env.example files already set them to false; just don’t override. Details in Configuring .env.

INTERNAL_RPC_SECRET and ENVIRONMENT must match everywhere

core-database verifies the shared bearer INTERNAL_RPC_SECRET; every other service and worker sends it. If they disagree, calls 401 and a worker can’t register — it looks like a task stuck at “planning…”. ENVIRONMENT (local) must also be identical across gateway, core-mcp, and the workers, because it selects the registry environment: a mismatch means the gateway looks up tools in a different env than the worker registered into, and the AI Tools panel is empty. One value each, set once, in every .env.

core-identity and core-storage have no Makefile

Only the gateway and the tool-worker repos ship the four-target Makefile (sync / run / test / check — see Make targets). Drive core-identity and core-storage with uv directly, as in steps 4–5. The Go services (core-database, core-mcp) have no Makefile either — use go run ./cmd/server.

Import crash: 'Descriptors cannot be created directly'

If a Python service dies at import with this TypeError, uv dragged the OTLP exporter back to a 2022-era release whose pre-3.20 generated code crashes the protobuf-7 runtime that soundverse-proto requires. The fix is per-repo in pyproject.toml: pin opentelemetry-exporter-otlp-proto-http (a recent release) and add a [tool.uv] override-dependencies entry for protobuf. It’s already in the fleet repos; copy the block if you fork one. Full backstory in Troubleshooting.

Refreshing stubs after a proto change

If you edited the contract, each repo pulls the regenerated SDKs with its own update-proto.sh (Go: go get …@staging + go mod tidy; Python: uv sync --upgrade). See Refresh proto stubs.

What needs a real credential

The whole stack boots and stays healthy with zero external accounts. Only features that reach outside the fleet need a secret in .env: agent LLM turns (an LLM proxy key), Sansaarm song generation (its API creds), and browser login (a Logto OIDC app). The full matrix is in What works offline vs. what needs secrets.

Related

• Quick start with docker-compose — the one-command path
• Configuring .env — the shared secret, TLS flags, injection
• Run the frontends — saas-2.0 and the standalone UI2.0
• Prerequisites & toolchains — Go, Python + uv, Node versions
• Service catalog — every service, its shape, and its port
• Troubleshooting — the failures you’ll actually hit