Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.valiqor.com/llms.txt

Use this file to discover all available pages before exploring further.

A trace is the full record of one AI interaction — from the user’s prompt through every LLM call, retrieval, tool execution, and final response. Traces are the raw material that Failure Analysis, Evaluations, and Security Audits operate on.

​
Trace Structure

Trace (top-level)
├── metadata          # project, environment, timing, cost
├── messages[]        # user ↔ assistant conversation turns
├── spans{}           # execution tree — nested operations
├── retrievals[]      # RAG document retrievals (if any)
└── execution_tree    # hierarchical span visualization
Every trace has a unique trace_id and belongs to a project_id. The SDK assigns these automatically when you use auto-instrumentation.

​
Trace Metadata

Each trace captures operational metadata automatically:
FieldTypeDescription
trace_idstrUnique identifier
project_idstrOwning project
statusstrProcessing status
session_idstrGroups traces in a conversation
app_namestrApplication name
app_versionstrApplication version
environmentstre.g. "production", "staging"
duration_msfloatTotal execution time
total_costfloatEstimated LLM API cost (USD)
total_tokensintTotal tokens consumed
input_tokensintPrompt tokens
output_tokensintCompletion tokens
llm_callsintNumber of LLM invocations
tool_callsintNumber of tool executions
retrieval_callsintNumber of retrieval operations
message_countintConversation turns
span_countintTotal spans in execution tree

​
Messages

Messages represent the conversation between the user and the AI. Each message has:
FieldTypeDescription
rolestr"user", "assistant", "system", or "tool"
contentstrThe message text
timestampstrWhen the message was sent
tool_callslistTool/function calls made (if any)
tool_call_idstrID linking a tool response to its call
namestrFunction/tool name (for tool messages)

​
Spans

A span is a single unit of work inside a trace — an LLM call, a retrieval lookup, a tool execution, etc. Spans form a tree via parent_span_id:
FieldTypeDescription
span_idstrUnique span identifier
namestrHuman-readable name (e.g. "ChatCompletion")
kindstrSpan kind (see table below)
parent_span_idstrParent span — forms the execution tree
start_timestrISO timestamp when span started
end_timestrISO timestamp when span ended
duration_msfloatExecution time in milliseconds
statusstr"ok", "error", etc.
attributesdictSpan-specific data (model, tokens, etc.)
eventslistDiscrete events within the span

​
Span Kinds

The kind field classifies what a span represents:
KindValueDescription
Workflow Nodeworkflow_nodeA named step in a workflow (e.g. LangGraph node)
LLM Callllm_callAn LLM API invocation
RetrieverretrieverA retrieval / search operation
TooltoolA tool or function execution
EvaluatorevaluatorAn evaluation or judging operation
EmbeddingembeddingAn embedding computation
SystemsystemInternal / framework span
UnknownunknownUnclassified span (fallback)
When using auto-instrumentation (import valiqor.auto), span kinds are assigned automatically based on the provider library being instrumented.

​
RAG Pipeline Stages

For RAG applications, Valiqor further classifies spans by their pipeline stage:
StageValueDescription
RetrievalretrievalDocument / knowledge retrieval
EvaluationevaluationDocument grading, relevance checking
SynthesissynthesisAnswer generation, response synthesis
RoutingroutingQuery routing, decision making
OrchestrationorchestrationWorkflow coordination, graph execution
LLM Callllm_callDirect LLM invocations
Tool Executiontool_executionTool / function calls
EmbeddingembeddingEmbedding generation
RerankingrerankingResult reranking
PreprocessingpreprocessingQuery preprocessing, transformation
PostprocessingpostprocessingOutput formatting, filtering
UnknownunknownUnclassified stage
RAG stages are used by Failure Analysis to pinpoint where in the pipeline a failure occurred — for example, distinguishing a retrieval failure from a synthesis hallucination.

​
RAG-Specific Data

When tracing RAG applications, Valiqor captures additional structured data:

​
Retrieval Info

FieldTypeDescription
retrieverstrRetriever type: "vectordb", "bm25", "api", "hybrid"
embedding_modelstrEmbedding model used
chunk_sizeintDocument chunk size
chunk_overlapintOverlap between chunks
top_kintNumber of documents retrieved
similarity_thresholdfloatMinimum similarity score
querystrThe retrieval query
hitslistRetrieved documents with scores
latency_msfloatRetrieval latency

​
Document Hits

Each retrieved document is recorded with:
FieldTypeDescription
doc_idstrDocument identifier
rankintPosition in result set
scorefloatSimilarity / relevance score
snippet_previewstrFirst 200 characters of content

​
Citations

FieldTypeDescription
doc_idstrReferenced document
confidencefloatCitation confidence
relevance_scorefloatHow relevant the citation is

​
Conversation Threads

Multiple traces can be grouped into a conversation using session_id. This lets Valiqor track multi-turn interactions: 
from valiqor.trace import start_conversation, end_conversation

# Group related traces into a conversation
start_conversation("session-abc-123")

# ... each traced call within this block shares the session_id ...

end_conversation()
When you query traces by session_id, you get the full conversation history with all traces in chronological order.

​
Execution Tree

The execution tree is the hierarchical view of all spans in a trace. It shows the parent-child relationships, making it easy to understand the flow of execution: 
workflow: "answer_question"          # root span (workflow_node)
├── retriever: "search_docs"         # retrieval span
│   └── embedding: "embed_query"     # embedding span
├── llm_call: "ChatCompletion"       # LLM span
└── tool: "format_response"          # tool span
The execution tree is what you see in the Valiqor dashboard’s trace viewer and is also available via TraceFullResult.execution_tree in the SDK.

​
SDK Data Classes

The SDK provides typed dataclasses for working with trace data:
ClassDescription
TraceRecordSingle trace in list views — ID, status, duration, token counts
TraceListResponsePaginated list of TraceRecord objects
TraceOverviewMetadata for a single trace
TraceSummaryRich summary with all metrics, diagnostics, and counts
TraceMessageSingle conversation turn (role, content, tool calls)
TraceSpanSingle span with kind, timing, and attributes
TraceFullResultComplete trace JSON — messages, spans, retrievals, execution tree
TraceEvalStepPer-metric evaluation result against a trace
All classes include .from_dict() and .to_dict() methods for easy serialization.

​
Where Traces Are Used

Traces are the input for every Valiqor analysis workflow:

Failure Analysis

Pass a trace_id to run() — FA extracts messages, spans, and retrievals to classify failures across 15 subcategories.

Evaluations

Pass a trace dict to evaluate_trace() — metrics are computed on the extracted input/output/context.

Security Audits

Pass a trace dict to audit_trace() — user/assistant pairs are extracted and scanned against S1–S23 categories.

Dashboard

Browse traces visually — see the execution tree, messages, spans, costs, and evaluation results.