Traces & Spans

A trace is the full record of one AI interaction — from the user’s prompt through every LLM call, retrieval, tool execution, and final response. Traces are the raw material that Failure Analysis, Evaluations, and Security Audits operate on.

Trace Structure

Trace (top-level)
├── metadata          # project, environment, timing, cost
├── messages[]        # user ↔ assistant conversation turns
├── spans{}           # execution tree — nested operations
├── retrievals[]      # RAG document retrievals (if any)
└── execution_tree    # hierarchical span visualization

Every trace has a unique trace_id and belongs to a project_id. The SDK assigns these automatically when you use auto-instrumentation.

Trace Metadata

Each trace captures operational metadata automatically:

Field	Type	Description
`trace_id`	`str`	Unique identifier
`project_id`	`str`	Owning project
`status`	`str`	Processing status
`session_id`	`str`	Groups traces in a conversation
`app_name`	`str`	Application name
`app_version`	`str`	Application version
`environment`	`str`	e.g. `"production"`, `"staging"`
`duration_ms`	`float`	Total execution time
`total_cost`	`float`	Estimated LLM API cost (USD)
`total_tokens`	`int`	Total tokens consumed
`input_tokens`	`int`	Prompt tokens
`output_tokens`	`int`	Completion tokens
`llm_calls`	`int`	Number of LLM invocations
`tool_calls`	`int`	Number of tool executions
`retrieval_calls`	`int`	Number of retrieval operations
`message_count`	`int`	Conversation turns
`span_count`	`int`	Total spans in execution tree

Messages

Messages represent the conversation between the user and the AI. Each message has:

Field	Type	Description
`role`	`str`	`"user"`, `"assistant"`, `"system"`, or `"tool"`
`content`	`str`	The message text
`timestamp`	`str`	When the message was sent
`tool_calls`	`list`	Tool/function calls made (if any)
`tool_call_id`	`str`	ID linking a tool response to its call
`name`	`str`	Function/tool name (for tool messages)

Spans

A span is a single unit of work inside a trace — an LLM call, a retrieval lookup, a tool execution, etc. Spans form a tree via parent_span_id:

Field	Type	Description
`span_id`	`str`	Unique span identifier
`name`	`str`	Human-readable name (e.g. `"ChatCompletion"`)
`kind`	`str`	Span kind (see table below)
`parent_span_id`	`str`	Parent span — forms the execution tree
`start_time`	`str`	ISO timestamp when span started
`end_time`	`str`	ISO timestamp when span ended
`duration_ms`	`float`	Execution time in milliseconds
`status`	`str`	`"ok"`, `"error"`, etc.
`attributes`	`dict`	Span-specific data (model, tokens, etc.)
`events`	`list`	Discrete events within the span

Span Kinds

The kind field classifies what a span represents:

Kind	Value	Description
Workflow Node	`workflow_node`	A named step in a workflow (e.g. LangGraph node)
LLM Call	`llm_call`	An LLM API invocation
Retriever	`retriever`	A retrieval / search operation
Tool	`tool`	A tool or function execution
Evaluator	`evaluator`	An evaluation or judging operation
Embedding	`embedding`	An embedding computation
System	`system`	Internal / framework span
Unknown	`unknown`	Unclassified span (fallback)

When using auto-instrumentation (import valiqor.auto), span kinds are assigned automatically based on the provider library being instrumented.

RAG Pipeline Stages

For RAG applications, Valiqor further classifies spans by their pipeline stage:

Stage	Value	Description
Retrieval	`retrieval`	Document / knowledge retrieval
Evaluation	`evaluation`	Document grading, relevance checking
Synthesis	`synthesis`	Answer generation, response synthesis
Routing	`routing`	Query routing, decision making
Orchestration	`orchestration`	Workflow coordination, graph execution
LLM Call	`llm_call`	Direct LLM invocations
Tool Execution	`tool_execution`	Tool / function calls
Embedding	`embedding`	Embedding generation
Reranking	`reranking`	Result reranking
Preprocessing	`preprocessing`	Query preprocessing, transformation
Postprocessing	`postprocessing`	Output formatting, filtering
Unknown	`unknown`	Unclassified stage

RAG stages are used by Failure Analysis to pinpoint where in the pipeline a failure occurred — for example, distinguishing a retrieval failure from a synthesis hallucination.

RAG-Specific Data

When tracing RAG applications, Valiqor captures additional structured data:

Retrieval Info

Field	Type	Description
`retriever`	`str`	Retriever type: `"vectordb"`, `"bm25"`, `"api"`, `"hybrid"`
`embedding_model`	`str`	Embedding model used
`chunk_size`	`int`	Document chunk size
`chunk_overlap`	`int`	Overlap between chunks
`top_k`	`int`	Number of documents retrieved
`similarity_threshold`	`float`	Minimum similarity score
`query`	`str`	The retrieval query
`hits`	`list`	Retrieved documents with scores
`latency_ms`	`float`	Retrieval latency

Document Hits

Each retrieved document is recorded with:

Field	Type	Description
`doc_id`	`str`	Document identifier
`rank`	`int`	Position in result set
`score`	`float`	Similarity / relevance score
`snippet_preview`	`str`	First 200 characters of content

Citations

Field	Type	Description
`doc_id`	`str`	Referenced document
`confidence`	`float`	Citation confidence
`relevance_score`	`float`	How relevant the citation is

Conversation Threads

Multiple traces can be grouped into a conversation using session_id. This lets Valiqor track multi-turn interactions:

from valiqor.trace import start_conversation, end_conversation

# Group related traces into a conversation
start_conversation("session-abc-123")

# ... each traced call within this block shares the session_id ...

end_conversation()

When you query traces by session_id, you get the full conversation history with all traces in chronological order.

Execution Tree

The execution tree is the hierarchical view of all spans in a trace. It shows the parent-child relationships, making it easy to understand the flow of execution:

workflow: "answer_question"          # root span (workflow_node)
├── retriever: "search_docs"         # retrieval span
│   └── embedding: "embed_query"     # embedding span
├── llm_call: "ChatCompletion"       # LLM span
└── tool: "format_response"          # tool span

The execution tree is what you see in the Valiqor dashboard’s trace viewer and is also available via TraceFullResult.execution_tree in the SDK.

SDK Data Classes

The SDK provides typed dataclasses for working with trace data:

Class	Description
`TraceRecord`	Single trace in list views — ID, status, duration, token counts
`TraceListResponse`	Paginated list of `TraceRecord` objects
`TraceOverview`	Metadata for a single trace
`TraceSummary`	Rich summary with all metrics, diagnostics, and counts
`TraceMessage`	Single conversation turn (role, content, tool calls)
`TraceSpan`	Single span with kind, timing, and attributes
`TraceFullResult`	Complete trace JSON — messages, spans, retrievals, execution tree
`TraceEvalStep`	Per-metric evaluation result against a trace

All classes include .from_dict() and .to_dict() methods for easy serialization.

Where Traces Are Used

Traces are the input for every Valiqor analysis workflow:

Failure Analysis

Pass a trace_id to run() — FA extracts messages, spans, and retrievals to classify failures across 15 subcategories.

Evaluations

Pass a trace dict to evaluate_trace() — metrics are computed on the extracted input/output/context.

Security Audits

Pass a trace dict to audit_trace() — user/assistant pairs are extracted and scanned against S1–S23 categories.

Dashboard

Browse traces visually — see the execution tree, messages, spans, costs, and evaluation results.

Start Here

Core Workflows

Concepts

Integrations

Resources

Trace Structure

Trace Metadata

Messages

Spans

Span Kinds

RAG Pipeline Stages

RAG-Specific Data

Retrieval Info

Document Hits

Citations

Conversation Threads

Execution Tree

SDK Data Classes

Where Traces Are Used

Failure Analysis

Evaluations

Security Audits

Dashboard

​Trace Structure

​Trace Metadata

​Messages

​Spans

​Span Kinds

​RAG Pipeline Stages

​RAG-Specific Data

​Retrieval Info

​Document Hits

​Citations

​Conversation Threads

​Execution Tree

​SDK Data Classes

​Where Traces Are Used

Failure Analysis

Evaluations

Security Audits

Dashboard

Trace Structure

Trace Metadata

Messages

Spans

Span Kinds

RAG Pipeline Stages

RAG-Specific Data

Retrieval Info

Document Hits

Citations

Conversation Threads

Execution Tree

SDK Data Classes

Where Traces Are Used