Agent Observability

Store traces, run history, and audit logs in your database.

AgentOS comes with a built-in tracing provider that routes traces to your database. Every run produces a trace tree: spans for the LLM call, each tool, hook, retrieval, and team delegation. These can be stored in agno_traces and agno_spans in the same db you use for sessions.

1from kern.os import AgentOS
2
3agent_os = AgentOS(
4    agents=[agent],
5    db=db,
6    tracing=True,
7)

Multi-database tracing

Traces are high-volume and write-heavy. For production deployments, you'll often want them in a separate database from sessions and memory.

Traces generally have a different cost profile, different retention and different access patterns:

1from kern.db.postgres import PostgresDb
2
3primary_db = PostgresDb(db_url="postgresql://primary/...")
4trace_db = PostgresDb(db_url="postgresql://traces/...")
5
6agent_os = AgentOS(
7    agents=[agent],
8    db=primary_db,
9    tracing=True,
10    trace_db=trace_db,
11)

See Multi-DB tracing for the full setup.

What gets captured

Span	Attributes
Run	`agent_id`, `user_id`, `session_id`, `model`, latency, status
LLM call	Model, prompt tokens, completion tokens, temperature, tool calls returned
Tool	Tool name, arguments, result, duration, exception (if any)
Pre/post hook	Hook name, duration, modified input/output
Retrieval	Query, vector store, k, returned docs, scores
Team delegation	Member name, mode, sub-run trace

Traces follow OpenTelemetry semantic conventions and you can query them directly:

1-- Top-10 slowest span types by average duration
2SELECT
3    name,
4    AVG(duration_ms) AS avg_ms,
5    COUNT(*) AS calls
6FROM agno_spans
7GROUP BY name
8ORDER BY avg_ms DESC
9LIMIT 10;

In the AgentOS UI

The control plane renders the same traces visually. Click a run and see the full tree: LLM hops, tool calls with their inputs and outputs, hooks, sub-agent traces. Filter by user, session, time range.

External providers

Send traces to Langfuse, Langsmith, Arize, or any OpenTelemetry endpoint by adding an exporter. Check out the Observability section: Langfuse, Langsmith, Arize, Logfire, MLflow.