The 60-second mental model

A user types a prompt in the browser; a minute later a finished master lands in their library. Between those two moments, the ~20 services of the Soundverse 2.0 backend authenticate the request, charge tokens, queue a job, run an LLM or a model provider, stream live progress back, store the result, and reconcile the bill. This page is that whole journey on one screen — read it once and every other page slots into place.

The whole system on one screen

flowchart TB
  U(["Browser / User"]):::frontend
  FE["soundverse-saas-2.0<br/>Next.js SPA + BFF"]:::frontend

  subgraph GWTIER["Trust boundary · gateways"]
    GW["core-gateway-consumer<br/>authz · price · reserve · queue"]:::gateway
    ID["core-identity<br/>OIDC validate · JIT provision"]:::gateway
    MCP["core-mcp<br/>MCP registry · 2nd billing pipeline"]:::gateway
  end

  subgraph DATA["Data plane"]
    DB[("core-database<br/>sole Postgres door")]:::data
    PG[("PostgreSQL")]:::data
    RDS[("Redis<br/>cache · queue signal · events")]:::data
    ST["core-storage<br/>blobs · SAS · HLS"]:::data
  end

  subgraph WK["Tool workers · soundverse-py WorkerFleet"]
    TA["core-tool-agent"]:::worker
    TS["core-tool-sansaarm"]:::worker
    TST["core-tool-stitch"]:::worker
  end

  subgraph EXT["External"]
    LOGTO["Logto OIDC IdP"]:::external
    BLOB[("Azure Blob Storage")]:::external
    PROV["LLM + model providers<br/>litellm · Sansaarm"]:::external
  end

  U --> FE
  FE -->|OIDC login| LOGTO
  FE -->|gRPC consumer/v1| GW
  GW <-->|SSE stream| FE
  GW -->|validate token| ID
  ID -->|JWKS| LOGTO
  ID -->|gRPC| DB
  GW -->|price · reserve · QueueTask| DB
  GW -.->|publish task:wake| RDS
  GW -.->|tail task stream| RDS
  DB --> PG
  DB <-->|cache| RDS
  WK -->|ClaimNextTask| DB
  RDS -.->|wake + progress events| WK
  TA -->|MCP sub-tools| MCP
  MCP -->|price · reserve · QueueTask| DB
  WK -->|UploadFile| ST
  ST --> BLOB
  TA --> PROV
  TS --> PROV

  classDef frontend fill:#6366f1,color:#fff,stroke:#4338ca
  classDef gateway  fill:#0ea5e9,color:#fff,stroke:#0369a1
  classDef data     fill:#8b5cf6,color:#fff,stroke:#6d28d9
  classDef worker   fill:#10b981,color:#fff,stroke:#047857
  classDef external fill:#f59e0b,color:#111,stroke:#b45309

Colour is the tier, and the same five-tier legend is used on every architecture diagram: indigo frontend, blue gateways, purple data plane, green workers, amber external.

Follow one generation, start to finish

1. Browser → BFF. The prompt goes to the Next.js app’s own server-side BFF (Backend-For-Frontend) — route handlers under /api in soundverse-saas-2.0. The BFF holds the session, attaches the Logto access token, and speaks gRPC on the browser’s behalf. The browser never gets a raw backend credential.

2. BFF → gateway. The BFF calls CreateGeneration on core-gateway-consumer over the consumer/v1 contract. This gateway is the trust boundary — the only backend service the outside world may reach.

3. Validate identity (never trust the client). The gateway hands the token to core-identity, whose POST /v1/auth/validate verifies it against Logto’s JWKS and returns our internal user_id. First time a valid token is seen, identity provisions the user (JIT) and grants a starting token balance. The caller identity comes only from the validated token, never from a request field.

4. Rate-limit → price → reserve. Inside the gateway, the enforcement pipeline (app/enforcement/pipeline.py) runs resolve tool → rate-limit → price → reserve. Pricing produces an estimate; reserve deducts that estimate up front so a user can’t launch jobs they can’t pay for. The hold is keyed by a billing idempotency key so a retry can’t double-charge.

5. Queue and acknowledge. The gateway calls QueueTask, which inserts a row into the generation.tasks Postgres table with status = 'queued' — the queue is a database table, not a separate broker. It then publishes a tiny task:wake message on Redis (env-namespaced channel {env}:common:task:wake) to nudge idle workers, and returns a task_id immediately. Nothing blocks on the generation itself.

6. Open the stream. The BFF reconnects with StreamGeneration; the gateway relays everything to the browser as Server-Sent Events (SSE). Long work and live updates are decoupled.

7. Worker claims the task. A tool worker (built on the soundverse-py WorkerFleet) wakes and calls ClaimNextTask — an UPDATE ... WHERE id = (SELECT ... FOR UPDATE SKIP LOCKED) so a hundred workers grab different rows without contention. The claim sets a 2-minute lease (claim_expires_at) the worker must renew via HeartbeatTask or the task becomes re-claimable.

8. Run, stream, upload, complete. The worker calls the upstream provider (an LLM via litellm, or a model service like Sansaarm), XADDs progress events onto a per-task Redis stream (which the gateway tails and forwards as SSE), streams the finished master to core-storage via UploadFile, then calls CompleteTask with the output and its real reported_usage.

9. Settle the bill. On the terminal event the gateway compares the reserved estimate against actual usage and either deducts the shortfall or refunds the surplus, then marks the task settled. Settlement is idempotent (the token ledger has a unique idempotency index), so background reconcilers can re-run it safely if the browser closed early.

The blow-by-blow version — with the full sequence diagram — lives in Request lifecycle.

Five ideas that explain everything else

1. The proto is the source of truth, not the code. One repo, soundverse-proto, holds every RPC, message, and field. buf generate fans it out into Go, Python, Swift, Kotlin, and TypeScript SDKs. The clever twist: the SQL for the data layer lives in the proto as an annotation (sql_query), and codegen emits the handler that binds request fields to the $1…$N placeholders. Change a proto and effects cascade everywhere — see the contract.

2. One door to the data. core-database is the only service that holds a Postgres credential. Every gateway, worker, and tool reaches data through its gRPC contract — never a raw connection string. That is what makes the SQL-in-proto approach safe: exactly one place runs SQL. See the data model.

3. The gateway is the trust boundary; identity is server-derived. Everything outside the gateway is untrusted; everything past it is authenticated with an internal shared secret (env-var name INTERNAL_RPC_SECRET). Ownership of workspaces, projects, and files is always checked against the token-derived user_id. See identity & auth.

4. Money moves as reserve → settle on an append-only ledger. Estimate up front, hold the estimate, true up at completion, all keyed by idempotency so retries and reconcilers are no-ops. The token ledger & money path is the full story; a tool that registers with no pricing row fails every call with NotFound.

5. Tool workers do the heavy lifting, and there are two request paths. Workers are long-running queue consumers with no HTTP ingress — they poll generation.tasks, run a model, and drain on SIGTERM.

Two request paths — and the billing pipeline runs twice

Everything above is the consumer path: a human in the browser drives one generation through the gateway. The agent path starts when core-tool-agent runs its LLM turn-loop and calls sub-tools through core-mcp — the MCP tool registry. When core-mcp dispatches a sub-tool it re-runs the same rate-limit → price → reserve → QueueTask pipeline, only re-implemented in Go. So the reserve/price/queue/settle logic exists twice (Python in the gateway, Go in core-mcp) — intentional today, but real duplication and a known debt. Change billing behaviour and you must touch both. Details in the agent + MCP path.

Where to go next

System overviewThe same hero diagram, in full detail: trust boundary, single data door, worker fleet.

Request lifecycleThe 19-step consumer generation sequence, end to end.

Backend at a glanceEvery service is one of three shapes: Go data plane, Python gateway, or queue worker.

Add a tool workerThe most common change to the fleet — write three things and deploy.

GlossaryBFF, reserve/settle, ClaimNextTask, SAS, JIT, proto cascade — the terms you'll meet constantly.