The agent + MCP path (second billing pipeline)

There are two ways a generation gets billed and queued in this fleet. The consumer path is a human in the browser driving one generation through core-gateway-consumer. This page is the other path: the agent runs an LLM loop, decides it needs a sub-tool (write lyrics, generate a song, name it), and reaches that sub-tool through core-mcp — the MCP (Model Context Protocol) tool registry.

The load-bearing fact: when core-mcp dispatches a sub-tool it re-runs the entire scope → rate-limit → price → reserve → QueueTask → settle pipeline that the gateway runs, only re-implemented in Go. So an agent turn produces a nested generation that is independently priced, reserved, queued, and settled. This is the second billing pipeline, and it is the single most important thing to understand about the agent.

The path, end to end

sequenceDiagram
  autonumber
  participant W as core-tool-agent (worker)
  participant LLM as LLM provider
  participant MCP as core-mcp (Go)
  participant DB as core-database
  participant R as Redis
  participant SUB as sub-tool worker
  W->>LLM: chat turn (litellm)
  LLM-->>W: tool_call(sub_tool, args)
  W->>MCP: POST /mcp callTool(sub_tool, args)
  MCP->>MCP: resolve scope + clamp license to plan
  MCP->>DB: FetchResolvedRateLimits / FetchResolvedPricing
  MCP->>MCP: rate-limit, estimate, Reserve (2nd pipeline)
  MCP->>DB: QueueTask (nested sub-generation)
  Note over DB,R: task enqueued; task:wake published
  SUB->>DB: ClaimNextTask (SKIP LOCKED lease)
  SUB->>R: XADD task:{id}:events (progress)
  MCP-->>W: relay PROGRESS / STREAMING_URL (PARTIAL_OUTPUT skipped)
  alt terminal before CALL_TIMEOUT
    MCP->>DB: Settle (or RefundAll on failure)
    MCP-->>W: result + _meta.sv_credit_cost
  else still running at CALL_TIMEOUT
    MCP-->>W: handle {status:running, task_id}
    loop every AGENT_POLL_INTERVAL until terminal
      W->>MCP: callTool(get_generation_result, task_id)
      MCP->>DB: GetTask (read-only, no billing)
      MCP-->>W: running | final result
    end
    Note over MCP,DB: reconciler settles out-of-band
  end
  W->>LLM: feed tool result back into the model

The agent is a normal soundverse-py queue worker (core-tool-agent). Its process() runs an LLM turn-loop; each tool the model picks becomes one MCP callTool over POST /mcp. The handler that fields that call is core-mcp/internal/toolcall/handler.go.

Inside the second billing pipeline

Handler.Handle runs the same shape as the gateway, in this order:

1. Scope + entitlement clamp. Identity is resolved from request headers into a ScopeContext (X-Soundverse-User-Id / -Workspace-Id / -Project-Id). The handler then resolves the user’s subscription entitlements and clamps the requested license down to what the plan allows — silently, because the agent path never lets a user type a license (the gateway, by contrast, hard-rejects an un-entitled explicit pick).

2. Rate-limit. FetchResolvedRateLimits from core-database, scaled by the plan multiplier, enforced in Redis on a user:workspace:project key.

3. Price estimate. FetchResolvedPricing. A NotFound here means the tool was never priced — the handler returns an actionable tool error, not a 500. A quantity-aware hold uses the tool’s declared usage_field (see tool pricing).

4. Reserve. h.billing.Reserve(...) holds the estimate against the token ledger. A non-OK status (e.g. INSUFFICIENT_FUNDS) rejects the call cleanly.

5. Queue. h.clients.Task.QueueTask(...) persists the nested task with its scope + billing context, sets MaxRetries explicitly, carries the subscription-tier queue Priority, and injects the W3C traceparent so the child generation’s span continues this tool call’s trace across the async queue hop. The sub-tool worker claims it off the Postgres task queue.

MaxRetries must be set explicitly — a 0 strands the task

QueueTask binds max_retries literally. Leaving it the proto3 default of 0 overrides the column’s DEFAULT 3 and makes ClaimNextTask’s retry_count < max_retries predicate (0 < 0) false — the worker can never claim the task and it sits QUEUED forever. core-mcp sets defaultMaxRetries = 3 for exactly this reason. Same footgun bites anyone calling QueueTask directly.

Fire-and-poll: why the call doesn’t just block

A naive handler would hold the /mcp POST open until the sub-tool finishes. That breaks on a slow generation: ACA ingress idle-times-out a held-open POST at roughly 240s, and a long song render or a text-streaming lyric tool easily exceeds that. The 504 surfaces in the agent’s MCP client at transport teardown as unhandled errors in a TaskGroup, the fleet treats it as a retryable error, and the whole generation restarts from scratch.

The design that avoids this is fire-and-poll — decouple connection duration from work duration:

1. Bounded in-band wait. Handle races two observers against CALL_TIMEOUT (default 180s, deliberately below the ~240s ingress limit): the task event stream (instant fast-path, also relaying live progress as MCP notifications) and a periodic authoritative GetTask poll (COMPLETION_POLL_INTERVAL, default 4s) that bounds worst-case completion observation even if the stream misses its terminal entry. Whichever first sees a terminal DB state wins; the loser is cancelled; settlement runs exactly once.

2. Relay skips PARTIAL_OUTPUT. A text-streaming sub-tool (lyric-writer) emits one PARTIAL_OUTPUT per LLM token-chunk — hundreds per call. Relaying each as a synchronous MCP progress notification across the ingress hop is slow enough to delay reaching the terminal event past the idle timeout. The handler forwards PROGRESS %, STREAMING_URL (live preview), and the terminal event, and drops PARTIAL_OUTPUT (callers discard the payload anyway).

3. Still running at the timeout → hand back a handle. If the sub-tool hasn’t finished at CALL_TIMEOUT, the handler does not settle. It returns {status: running, task_id} and leaves the money to the reconciler.

4. The agent polls itself. core-tool-agent’s loop takes that handle and calls the read-only get_generation_result tool every AGENT_POLL_INTERVAL (default ~5–10s), bounded by AGENT_POLL_MAX_WAIT (default 1200s), until terminal. The wait now lives on the agent — a worker whose own fleet lease is heartbeated — not on an open connection, and the polls keep the MCP session warm. The model never sees a running envelope; it only ever sees the final result.

get_generation_result is read-only by contract

The poll endpoint (Handler.Poll) is a pure GetTask read — no QueueTask, no event stream, no billing, no rate-limit — with an owner check (a task_id is a guessable UUIDv7, so any caller could otherwise read another user’s generation). It must stay read-only: settling there would race the reconciler’s idempotency keys and could double-move tokens. It is registered directly on the MCP server (not via the DB-synced registry) so the registry’s Sync never removes it.

Settlement and the reconciler backstop

On the in-band terminal path, Handler.Handle settles once:

• COMPLETED → Settle(reserved, actual) deducts the shortfall (actual > reserved) or refunds the surplus (actual < reserved), then marks the task settled and surfaces the cost on the result’s _meta.sv_credit_cost side-channel so the agent can record per-step “tokens used” without a second billing lookup.
• FAILED → RefundAll returns the whole reservation.

Settlement is idempotent via derived keys. The reservation uses the per-call callID; settlement derives suffixes off it — billing.go uses callID:settle-extra, callID:settle-refund, and callID:refund-all. Because the task persists that same billing_idempotency_key, the reconciler (RECONCILE_INTERVAL, default 60s) reuses byte-identical keys when it sweeps ListUnsettledTasks. So a task that returned a running handle — or whose in-band settle failed — is settled out-of-band exactly once. See billing reconciliation.

Known debt: the billing pipeline exists twice

Architectural debt The reserve / price / queue / settle logic lives once in Python (core-gateway-consumer) and again in Go (core-mcp). This is intentional today — the two paths evolved separately — but it means two pricing implementations, two reconcilers, and two places to keep in sync. When you change billing behaviour, change both. This is tracked as a P0-adjacent risk in known limitations.

Nested-task concurrency: orchestrator vs leaf lanes

The agent worker hosts both the orchestrators (the agent tiers) and several leaf sub-tools in one fleet. Currently ALL_TOOLS in core-tool-agent is: agent-flash / agent-standard / agent-pro (orchestrators) plus lyric-writer, song-namer, file-manager, and wait_for (leaves). Song generation (song_gen_v5 in core-tool-sansaarm) and album art (core-tool-media) run in separate fleets, so the agent never deadlocks against them — only the co-hosted leaves can.

A single shared concurrency pool would deadlock: an agent task holds a slot for its whole run while it waits on a leaf, and that leaf needs a slot from the same pool. Enough concurrent agents hold every slot, the leaves sit QUEUED and unclaimable, and every agent waits forever.

The fix in WorkerFleet is two separate lanes, keyed off the declarative BaseTool.is_orchestrator class attr (the agent tiers set it True):

• an orchestrator lane (ORCHESTRATOR_CONCURRENCY, agent default 16 — agents mostly idle-wait, so it is sized generously), and
• a leaf lane (MAX_CONCURRENCY, agent default 10).

An orchestrator never draws from the leaf pool, so leaves always have slots and make progress regardless of orchestrator load. The waits-for graph (orchestrator → leaf → external API) becomes a DAG with no resource cycle, so it is deadlock-free by construction. The orchestrator lane auto-zeroes when a fleet hosts no orchestrator, so leaf-only workers (sansaarm) spend no extra threads. Critically, a saturated orchestrator lane does continue, never break, in the poll loop — it must never stop claiming leaves.

Providers

In code, not deployed The agent tiers dial their models directly (not through a proxy), routed by litellm model-string prefix: agent-standard and agent-pro are Anthropic Claude (anthropic/…), and agent-flash is Kimi-K2 on an OpenAI-compatible host (openai/…). The leaf lyric-writer and song-namer reuse the standard tier (Claude). Deploy requires the provider-key org secrets (by name: ANTHROPIC_API_KEY and OPENAI_API_KEY, prefixed STAGING_/PROD_). See core-tool-agent for the full tier map.

The traps that will actually bite you

Never settle in the poll path

get_generation_result and the reconciler both touch handle-returned tasks. If you add settlement to the poll path you race the reconciler’s idempotency keys and can double-move tokens. Keep the poll a pure read.

CALL_TIMEOUT must stay below the ingress idle timeout

CALL_TIMEOUT (180s) is chosen to fire before the ~240s ACA ingress idle timeout so the handler can gracefully hand back a running handle instead of getting 504’d mid-flight. If you raise it above the ingress limit, long sub-calls fail as TaskGroup errors and the fleet regenerates from scratch. Likewise wait_for / AGENT_POLL_MAX_WAIT bound the agent-side wait.

A schema change needs a core-mcp restart

core-mcp caches tool input/output schemas in memory. Redeploying a sub-tool worker updates the DB registry row but the changed schema is not picked up until core-mcp restarts. See the core-mcp field guide.

Related

• Life of a generation — the consumer path this one mirrors
• core-mcp — the registry + schema cache + auth caveats
• core-tool-agent — the LLM worker and its tiers
• The token ledger & money path — reserve/settle mechanics
• Task queue on Postgres — claim/lease the nested task
• Billing reconciliation — the out-of-band settle sweep
• Known limitations — the dup-billing debt in context