Observability records the full execution path of an agent so you can understand its internal behavior, debug issues, analyze performance, and keep optimizing. VeADK’s built-in tracing captures each request — from receiving user input, through model inference, tool calls, memory and knowledge-base reads/writes, to producing a response — as structured span data, and reports it to Volcengine or third-party platforms through exporters.
VeADK tracing follows the OpenTelemetry generative-AI semantic conventions, with field names aligned to standard span attributes. Trace data can therefore be imported directly into any OpenTelemetry-compatible system for analysis and visualization.
Runtime logs use Python’s standard logging package. See Application logging for levels, formatting, and trace correlation.
Core concepts
span
A span is a traceable unit of execution that records a name, start/end times, attributes, and parent-child relationships. VeADK builds a tree of spans for each request, covering nodes such as agent runs, model calls, and tool executions, and annotates them per the generative-AI semantic conventions with attributes like gen_ai.operation.name and gen_ai.span.kind.
The tracer OpentelemetryTracer
OpentelemetryTracer is the unified tracing entry point. It holds a list of exporters, wires each one into the tracing pipeline on initialization, and automatically attaches an in-memory exporter for local retention and dumping.
exporter
An exporter sends span data to a specific backend platform. Each exporter targets one backend and can be used alone or combined. VeADK ships the following exporters:
Attaching a tracer to an agent
Tracing is wired in through the tracers field of Agent. Construct an exporter for the target backend, hand it to OpentelemetryTracer, then attach the tracer to the agent:
A single tracer can hold multiple exporters, reporting the same spans to several platforms:
Tracer parameters
OpentelemetryTracer accepts the following fields:
InMemoryExporter is managed automatically by OpentelemetryTracer and records every span. Do not add it to the exporters list manually, or initialization fails.
Enabling exporters via environment variables
Besides constructing exporters explicitly, Agent attaches the matching exporter at runtime based on the following environment variables (enabled when set to true). If the agent provides no tracers, an OpentelemetryTracer is created automatically:
When all three variables are false (the default), Agent creates no tracer. To trace in that case, construct an OpentelemetryTracer explicitly and pass it via tracers.
Content tracing
By default, spans record the input and output content of the agent, model, and tools. In scenarios involving sensitive data, you can disable content capture and keep only the non-content span structure and timing. This is controlled by OpenTelemetryConfig.trace_content.
Local dumping
The in-memory exporter built into OpentelemetryTracer retains spans by session_id. Call dump to export the current session’s spans to a local JSON file for offline analysis:
The exported JSON contains, for each span, its name, span_id, trace_id, start/end times, attributes, and parent span. See In-memory exporter for details.