core-database

core-database is the one and only thing in the fleet that talks to Postgres. Every other service — the gateway, identity, core-mcp, tool workers — reaches data by calling its Connect/gRPC handlers. Nobody else holds DB credentials or opens a connection. It is a Go service, but you will not find a hand-written RPC handler in the repo: the entire service surface is code-generated from soundverse-proto and mounted with a single call. This repo is the deployment wrapper — config, connection pooling, a health check, telemetry, and a graceful drain.

You don't add services here

To add or change an RPC, edit the proto + schema.sql in soundverse-proto, regenerate, then pull the stubs with ./update-proto.sh. The new handler is mounted automatically. See Add a field or RPC to a data service and How the DB codegen plugin works.

What it does

The whole service is one function call in cmd/server/main.go:

cmd/server/main.go (the entire mount)

registry.RegisterAllDBServices(mux, dbPools, redisClient, cfg.InternalRPCSecret, cfg.CacheKeyPrefix())

RegisterAllDBServices lives in the generated soundverse-proto-go registry (produced by gen-db-registry in soundverse-proto) and mounts 15 Connect handlers onto one http.ServeMux — 14 product DataServices plus one codegen cache-test fixture.

Count drift

Older onboarding material says “eleven services.” The deployed registry now mounts 15 (14 real + the CacheEdgeCaseService fixture). The list grows every time a new DB-backed proto service lands; it is not maintained by hand here.

Handler	Domain	Backs
GenerationDataService	generation/task	The Postgres task queue — claim / lease / heartbeat / settle
GenerationToolDataService	generation/tool	The tool registry + pricing, rate-limits, credentials
TokenService	billing/tokens	The append-only token ledger and balances
BillingDataService	billing/subscriptions	Plans, plan prices, subscriptions
ChatDataService	chat	Workspaces, projects, messages
CollabStudioDataService	collab-studio	Studio timeline / session rows
IdentityDataService	identity	Users and identity links
StorageDataService	storage	File and blob rows — not the blob bytes (see below)
DnaDataService	generation/dna	DNA plan data
MigrationService	migration	The legacy_file_map spine + 1.0→2.0 migration
NotificationDataService	notifications	Notification rows
AudioDataService	compat/audios	Legacy 1.0 audio shape (read path)
ProfileDataService	compat/profile	Legacy 1.0 profile shape
UserDataService	compat/user	Legacy 1.0 user shape
CacheEdgeCaseService	tests	Codegen cache-aside fixture — not a product surface

Reads that carry cache annotations go through Redis cache-aside (optional; skipped if REDIS_ADDR is unset). Cache keys are namespaced {env}:database: from ENVIRONMENT, so environments sharing one Redis instance never collide. Every RPC is gated: the generated handlers reject any call that does not present Authorization: Bearer carrying the value of INTERNAL_RPC_SECRET (the proto require_internal_auth option).

It runs no migrations

core-database only serves data — it never applies DDL. The authoritative schema lives in the schema.sql files next to each proto in soundverse-proto (see the data schema catalog); those are applied out of band by the compose Postgres init scripts and by core-migration. The service boots against a database it assumes is already migrated.

StorageDataService is not StorageService

Two different services, easy to misroute

core-database serves StorageDataService — the row operations (InsertFile, ListFiles, InsertBlob, …). The blob-handling RPCs (UploadFile, CreateDownloadLink, IngestStream) belong to StorageService, which is served by the separate Python core-storage app. A caller that needs blob RPCs must dial core-storage on its own endpoint (CORE_STORAGE_GRPC) — dialing core-database for them returns UNIMPLEMENTED or stalls. This exact misroute once broke library upload and signed playback while library listing (a StorageDataService call) kept working.

The single-pool / target_db nuance

The proto target_db method option (value "prod" on every RPC today) is used as the Postgres database name, not a physical environment selector — same host, user, and password, only the database path in the DSN changes per RPC. The pool map the server builds is therefore prod-only:

cmd/server/main.go

dbPools := map[string]*pgxpool.Pool{
    "prod": dbPool,
}

Environment isolation (local / staging / prod) comes from pointing at different Postgres servers, not different database names — the logical database is called prod in every environment. On boot the server asserts the prod pool exists and Pings every configured pool, so a misconfigured deploy crash-loops immediately instead of booting green and 500-ing on first traffic.

Transport: native h2c, no in-process TLS

core-database speaks HTTP/1.1 and unencrypted HTTP/2 (h2c) on the same port so both Connect-Web (HTTP/1.1) and gRPC (h2c) clients hit it without any TLS handshake at this layer. It uses Go’s native cleartext-HTTP/2 support, not the older golang.org/x/net/http2/h2c wrapper:

cmd/server/main.go

server.Protocols = new(http.Protocols)
server.Protocols.SetHTTP1(true)
server.Protocols.SetUnencryptedHTTP2(true)

TLS is terminated by the platform at the edge. In staging the container app is exposed externally over TLS on :443 while the process itself listens plaintext on :8080; consumers flip CORE_DATABASE_USE_TLS to match their route. See environment & service discovery.

Run locally

1. Have a reachable Postgres (already migrated). Redis is optional.

2. Set the minimum env — DATABASE_DSN (or the POSTGRESQL_* parts) and INTERNAL_RPC_SECRET. Names only; never commit real values.

3. Build and run:

   core-database

   go mod tidy
   go run ./cmd/server

4. It listens on :8080 with native h2c. Health-check it:

   curl -s localhost:8080/healthz

   /healthz pings Postgres + Redis and reports the soundverse-proto-go build provenance (Version / SourceCommit / SourceRef) and the discovered registries — use it to confirm which proto version a running instance was built against.

Key files

• Directorycore-database/

  • Directorycmd/server/

    • main.go boot, pool wiring + fail-fast validation, /healthz, native h2c, graceful drain
  • Directoryinternal/

    • Directoryconfig/

      • config.go env parsing, POSTGRESQL_* DSN fallback, DatabaseDSNForName
    • Directorytelemetry/

      • telemetry.go OTel init
      • dbmetrics.go queue-depth / oldest-unsettled-reservation gauges
  • update-proto.sh pull regenerated stubs (go get …@staging && go mod tidy)
  • Dockerfile

The generated handlers and the RegisterAllDBServices registry are not in this repo — they are published to soundverse-proto-go by the gen-db-registry command in soundverse-proto.

Key environment variables

Names and purpose only — never real values.

Name	Purpose
DATABASE_DSN	Full Postgres URL. If unset, assembled from the POSTGRESQL_* parts.
POSTGRESQL_HOST / _DB / _USER / _PASS / _PORT (or _FALLBACK_PORT) / _SSLMODE	Component fallback for the DSN (_SSLMODE defaults to require).
INTERNAL_RPC_SECRET	Required. The Bearer secret every caller must present.
LISTEN_ADDR	Listen address; defaults to :8080.
REDIS_ADDR / REDIS_PASSWORD	Cache-aside Redis (URL form; the logical DB index lives in the URL path — there is no separate REDIS_DB knob).
ENVIRONMENT	Namespaces all Redis cache keys as {env}:database: (default local).
DATABASE_MIN_CONNS / DATABASE_MAX_CONNS	pgx pool bounds (defaults 2 / 12).
DATABASE_MAX_CONN_IDLE	Max idle time per pooled conn (Go duration; default 5m).

Tests & deploy

There is no Go test suite — confidence comes from the typed proto contract and the codegen’s build-time guardrails (no SELECT *, db_column coverage checks) that fail generation on a bad annotation. Deployment is Azure Container Apps via the shared deploy-aca-template.yml (older README/CLAUDE notes that mention AKS + k8s/ manifests are stale). Push the staging or prod branch to deploy.

The proto → Go cascade is manual

core-database’s proto_updated auto-deploy trigger is intentionally commented out. A proto change does not redeploy this service automatically — you pull stubs with ./update-proto.sh, commit, and push. See Drive the proto → deploy cascade and Refresh proto stubs.

Related

• Data model overview — the one-database-one-door picture
• How the DB codegen plugin works — how these handlers are generated
• Add a field or RPC to a data service — the change you actually make
• core-storage — the StorageService blob plane
• Proto contract reference — the DB annotations and error mapping
• Service catalog — every service, port, and health path