How to export Mastra agent metrics to Prometheus (no database required)
Mastra derives per-agent metrics (latency, tokens, cost) from its traces, and shows them well in Studio. When you want those metrics in Prometheus so your production monitoring and alerting can use them, you need a scrape endpoint that Mastra doesn’t ship. You can add one without running an analytics database. Mastra puts every derived metric onto an observability event bus, so a small exporter can listen, feed prom-client, and serve /metrics from memory, with no analytics database (DuckDB or ClickHouse) to run. Here is how I wired it up, and the open-source exporter
that does it.
Every code block below links to the exact line in the exporter’s v0.1.0
tag so you can follow along.
What Mastra gives you, and the one piece to add #
Mastra already has strong observability. For Prometheus-based monitoring and alerting, you add one piece: a metrics endpoint to scrape.
Mastra extracts metrics from spans automatically when a span ends, so you get per-agent latency, token, and cost numbers with no manual instrumentation. Studio renders them in a dashboard that’s genuinely useful for watching an agent’s behavior. And the built-in exporters send trace data to systems like Datadog LLM Observability and Langfuse, which are good tools for trace-based analysis. I reach for all of these.
What Mastra doesn’t ship is a Prometheus exporter: a /metrics endpoint that a Prometheus server scrapes, so you can compute a windowed p95 with histogram_quantile, wire it into Alertmanager, and page someone when an agent slows down or starts failing. If Prometheus is already how your team monitors production, that endpoint is the piece to add. This article adds it.
Where Mastra’s metrics live #
Mastra’s metrics query API reads from an analytics store : DuckDB in development, ClickHouse in production. Run one and you get the Studio metrics view and a queryable metric history.
This is a reasonable design. OLAP stores are built for aggregating measurements over time, which is what a metrics history is for. If you already operate ClickHouse, that path gives you the Studio dashboard and the getMetric* query API
for free.
The query API is one way to reach the metrics, but it isn’t the only one. Alongside storage, Mastra pushes each derived metric onto its observability bus as the metric is produced. That path is live, and it doesn’t involve the analytics store at all. So if you don’t run DuckDB or ClickHouse, the metrics are still reachable on the bus.
The exporter: listen on the bus, serve /metrics #
The ObservabilityExporter interface
has an onMetricEvent callback that fires for every derived metric, independent of storage. An exporter that implements it turns those events into prom-client metrics.
Here is the dispatch at the center of it (src/index.ts ):
onMetricEvent(event: unknown): void {
const metric = this.#readMetric(event)
if (typeof metric.name !== 'string' || !isMeasurement(metric.value)) {
return this.#drop('invalid_payload')
}
if (DURATION_METRIC.test(metric.name)) return this.#recordDuration(metric.name, metric)
const token = parseTokenName(metric.name)
if (token) return this.#recordTokens(token, metric)
this.#drop('unmapped_metric')
}
A duration metric becomes a histogram observation, and a token metric becomes a counter increment. Anything it can’t map increments a dropped-events counter, so a miss shows up as a number instead of silence.
Registering it takes a few lines, and the storage stays a plain LibSQL file (this is the README usage ; a complete runnable version lives in examples/basic.ts ):
import { Mastra } from '@mastra/core'
import { LibSQLStore } from '@mastra/libsql'
import { Observability } from '@mastra/observability'
import { PrometheusExporter } from 'mastra-prometheus-exporter'
const prom = new PrometheusExporter({ version: '1.0.0' })
export const mastra = new Mastra({
storage: new LibSQLStore({ id: 'app', url: 'file:./app.db' }), // no OLAP store
observability: new Observability({
configs: { default: { serviceName: 'my-service', exporters: [prom] } },
}),
})
prom.serve(9464) // http://0.0.0.0:9464/metrics
There’s no DuckDB and no ClickHouse in the process. prom.serve(9464) starts an HTTP server that exposes the metrics in the format Prometheus scrapes. The path from an agent run to a firing alert looks like this:
The solid path is the exporter. The dashed path is the OLAP query route that powers Studio. They run off the same spans, so you can use both.
The details that make it production-grade #
A working exporter is short. A correct one follows a few Prometheus conventions that a first draft tends to miss.
The shipped version handles them:
- Durations are converted from milliseconds to seconds, the Prometheus base unit, so
histogram_quantilereturns a number you can read. - The buckets are tuned for model latency.
prom-client’s default histogram tops out at 10 seconds, and agent calls run longer, so the defaults extend to 120 seconds and stay configurable. - Counters end in
_total, following Prometheus naming, so tooling recognizes them. - High-cardinality fields ride as exemplars, not labels.
trace_idand run identifiers would blow up label cardinality, so they attach as OpenMetrics exemplars, and a latency spike links straight to the exact trace. - The exporter keeps its own registry and reports its own dropped-event counter, so it never touches
prom-client’s global default and you can see when it fails to map something.
The full metric table is in the README , and Mastra documents the upstream metric names and cost fields in its automatic metrics reference .
Wiring Prometheus: scrape, p95, alert #
Prometheus scrapes the exporter’s /metrics, computes a real p95 with histogram_quantile, and fires an alert, all on LibSQL.
Point Prometheus at the exporter:
scrape_configs:
- job_name: mastra
static_configs:
- targets: ['host.docker.internal:9464']
Record the p95 as a rule, then alert on it (from the shipped recording-rules.yml and alerts.yml ):
groups:
- name: mastra
rules:
- record: mastra:agent_latency_p95_seconds:5m
expr: histogram_quantile(0.95, sum by (le, entity) (rate(mastra_agent_duration_seconds_bucket[5m])))
- alert: MastraAgentLatencyP95High
expr: mastra:agent_latency_p95_seconds:5m > 10
for: 5m
labels: { severity: warning }
annotations:
summary: "Agent p95 latency high"
Because the exporter emits real histogram buckets, this is a windowed percentile, not a per-scrape average. That’s the reason to build histograms rather than read a stored average out of a table.
Here’s the scrape from my run, trimmed to the interesting lines:
mastra_agent_duration_seconds_bucket{le="5",entity="Weather Agent",status="ok"} 2 # {trace_id="f919..."} 3.069
mastra_model_tokens_total{direction="input",type="total",provider="openrouter",model="gpt-4o-mini"} 1859
mastra_model_cost_usd_total{provider="openrouter",model="gpt-4o-mini"} 0.00027885
The # {trace_id="f919..."} on the bucket line is the exemplar. Point Grafana at the same Prometheus for dashboards, where the exemplar turns a latency spike into a one-click jump to that trace. Prometheus stored the exemplars and returned them from query_exemplars, and the alert fired against a demo threshold. The whole pipeline ran on a single LibSQL file, with no analytics database. Those counts are from one lab run, so treat them as an example rather than a benchmark.
The gotchas that cost me time #
Provider, model, and cost are nested in costContext. On Mastra 1.49, a token event carries them in a costContext object, not at the top level (automatic metrics reference
):
// the shape Mastra attaches to a token metric
metric.costContext = { provider: 'openrouter', model: 'gpt-4o-mini', estimatedCost, costUnit: 'USD' }
Read the wrong field (metric.provider or metric.estimatedCost) and token metrics come back with provider="unknown" and no cost, even on a successful call. The exporter reads costContext for you; if you write your own, read there.
The rest:
- A metric doesn’t exist until its first event. Token and cost series appear after the first successful model call. On error-only traffic you correctly see only the duration histograms, with
status="error". - Alert on the agent status. Mastra reports the model span
status="ok"when a provider call fails upstream (an OpenRouter 429, for example); the agent span carries the error. The shipped error-ratio rule keys off the agent metric for that reason.
When you also want Studio and the OLAP store #
The exporter gives you Prometheus alerting. It doesn’t replace Studio or the stored-metric history, and it isn’t meant to. DuckDB and ClickHouse buy you the Studio metrics view and a queryable history; the exporter buys you live scrape-based alerting and Grafana. They read from the same spans, so run both when you want exploration in Studio and paging in Prometheus. If all you need is a monitor and a pager, the exporter on its own is the shorter path.
Common questions #
Do I need ClickHouse or DuckDB to get Mastra metrics into Prometheus?
No. You can get Mastra metrics without ClickHouse or DuckDB. Mastra pushes every derived metric onto its observability bus viaonMetricEvent, independent of storage, so an exporter that listens there turns each event into an in-memory prom-client metric it serves at /metrics. The analytics store (DuckDB or ClickHouse) is only for Mastra’s own metrics query API and Studio history; the exporter path never touches it.Does Mastra ship a Prometheus exporter?
No. Mastra’s built-in exporters send trace data to systems like Datadog and Langfuse. To get a Prometheus metrics endpoint, you add a small exporter against theObservabilityExporter interface, or use the open-source mastra-prometheus-exporter.How do I get p95 latency?
Usehistogram_quantile over the _bucket series: histogram_quantile(0.95, sum by (le, entity) (rate(mastra_agent_duration_seconds_bucket[5m]))). Because the exporter emits real histograms, this is a windowed percentile rather than a per-scrape average.How do I track cost?
Track tokens as the source of truth and derive cost with a recording rule over tokens times price. The exporter also emits a best-effortmastra_model_cost_usd_total from Mastra’s estimated cost, but prices drift, so the recording rule is the authoritative path.What about Datadog?
Datadog is a different path. Mastra’s Datadog exporter sends traces to Datadog LLM Observability, which is a strong fit for trace-based analysis. The Prometheus exporter here is for scrape-based metrics and alerting. Use whichever matches how your team already monitors.How do I visualize Mastra metrics in Grafana?
Point Grafana at the Prometheus that scrapes the exporter, then build panels over themastra_* series: latency percentiles with histogram_quantile, token usage, and cost. The trace_id exemplars let you click from a latency spike to the exact Mastra trace.How does this relate to OpenTelemetry?
Mastra ships an OpenTelemetry exporter that sends traces to an OTel collector. This is a different, direct path: it turns Mastra’s metric events into a Prometheus/metrics endpoint you scrape. Reach for OTel when you’re standardizing on a collector, and for this when Prometheus is already your metrics stack.Can I use a SQL exporter over Mastra's tables instead?
Yes, if you already run ClickHouse. The trade-off is coupling: a SQL exporter depends on Mastra’s internal table schema, which can change between releases. TheonMetricEvent exporter depends on the public interface instead.What I’d do Monday morning #
If you run Mastra agents in production and page on metrics, add a Prometheus scrape endpoint off the observability bus and alert on the agent metric. You don’t need a new database to do it.
You can find the exporter at github.com/charlesgwilson/mastra-prometheus-exporter . Point it at your agents, and your existing Prometheus setup can monitor and alert on them.
Practical AI #
I co-contribute to Practical AI , Damian Galarza’s newsletter for builders working with AI.
It covers the patterns, tradeoffs, and lessons that show up when AI moves from prototype to real work.