LLM providers

Core rule: deterministic code finds the evidence. Mastra agents and workflows explain a packed digest. They must never invent causality, files, or PRs, or change scorer ranks.

Orchestration is the in-repo Mastra-compatible runtime (src/lib/agents/mastra-runtime, package name @mastra/core). Agent.generate throws; model HTTP lives in src/lib/llm/providers.ts. AgentConfig.model is a string or a function that returns a string, not { url, id, apiKey }. Public APIs stay explain() and runInvestigation() so UI/MCP callers do not change.

Investigation always goes through workflows explain_incident and investigate_incident on that runtime. Configure at least one provider. Missing keys fail closed as not_configured.

Context packing SoT: Agent context packing. OSS catalog: Open source tools. Workflows run on the in-repo runtime; provider HTTP is the ChangeGraph router in src/lib/llm/providers.ts.

Configure a provider

Set one or more keys (see product .env.example):

Pipeline (explain_incident workflow):

1. Build the evidence package (scorer ranks frozen)
2. Redact secrets, then pack compact JSON (caps on stacks, logs, paths, timeline)
3. Optional Headroom compress (see below)
4. Resolve a model id string and call the ChangeGraph provider router (src/lib/llm/providers.ts)
5. Call the explain agent
6. Validate structured output (drop invented PRs/files; copy ranks/scores from the package; soften absolute causation)
7. Persist summary status: ok | not_configured | error

Without a provider, the UI shows a configure an LLM provider state (not_configured). Scoring and evidence remain deterministic.

Headroom

Optional post-pack compress after redaction. The packer works without it. Packer + explain/investigate work without Headroom. If the proxy is down, the packed prompt is used (fallback: true).

Upstream: github.com/headroomlabs-ai/headroom. Catalog: Open source tools.

Adding OpenAI-compatible providers

Any OpenAI-style /chat/completions endpoint:

LLM_COMPAT_BASE_URL=https://api.together.xyz/v1
LLM_COMPAT_API_KEY=...
LLM_COMPAT_MODEL=meta-llama/Llama-3.3-70B-Instruct-Turbo
LLM_DEFAULT_PROVIDER=compat

AgentConfig.model is a provider/model string or a function that returns one. The runtime does not accept an object model (url, id, apiKey). OpenAI-compatible vendors use the LLM_COMPAT_* settings above. Do not add a new core domain type per vendor.

What never goes to the model

• Integration secrets / tokens
• Raw webhook payloads with credentials
• Pretty-printed full evidence dumps (the packer sends a budgeted digest)
• Scorer internals beyond the packaged breakdown fields

Investigation workflows

Neither path may change deterministic scores. Set INVESTIGATION_AGENT_ENABLED=0 only to opt out of the multi-step workflow.

Failure modes

• Missing keys → not_configured (configure a provider; not a success skip)
• Provider error → error with a safe message; UI still shows evidence
• Hallucinated citations → stripped by validation
• Headroom down → packed prompt is used (fallback: true). Explain/investigate do not depend on Headroom.

Engineering SoT: repo docs/MASTRA_AGENTS.md, docs/AGENT_CONTEXT.md, and docs/INTEGRATIONS_OSS.md. Checklist: repo docs/SECURITY.md.