> ## Documentation Index
> Fetch the complete documentation index at: https://docs.valiqor.com/llms.txt
> Use this file to discover all available pages before exploring further.

# See a Failure in 5 Minutes

> Go from zero to your first failure analysis result in under 5 minutes — no tracing or instrumentation required.

Run Failure Analysis on your existing AI data in three steps. No tracing, no instrumentation, no OpenAI key needed.

<Info>
  This quickstart uses **dataset mode** — you pass your AI inputs and outputs directly. If you already have tracing set up, see [Failure Analysis from traces](/workflows/failure-analysis) instead.
</Info>

***

## Step 1: Install the SDK

```bash theme={"system"}
pip install valiqor
```

## Step 2: Set your API key

```bash theme={"system"}
export VALIQOR_API_KEY="vq_your_api_key_here"
```

<Tip>
  Get your API key from the [Valiqor Dashboard](https://app.valiqor.com). You can also pass it directly via `ValiqorClient(api_key="vq_...")`.
</Tip>

## Step 3: Run Failure Analysis

Create a file called `quickstart.py` and paste:

```python quickstart.py theme={"system"}
from valiqor import ValiqorClient

client = ValiqorClient(api_key="vq_your_api_key_here", project_name="quickstart")

# Run FA on existing data — no tracing required
result = client.failure_analysis.run(
    dataset=[
        {
            "input": "What is the capital of France?",
            "output": "The capital of France is Berlin.",
            "context": [
                "France is a country in Western Europe.",
                "The capital of France is Paris.",
                "Paris is known as the City of Light.",
            ],
        }
    ]
)

# Print the summary
print(f"Status: {result.status}")
print(f"Failures detected: {result.summary.total_failures_detected}")
print(f"Overall severity: {result.summary.overall_severity}/5")
print(f"Should alert: {result.summary.should_alert}")
print()

# Print each failure
for tag in result.failure_tags:
    if tag.decision == "fail":
        print(f"❌ [{tag.bucket_name}] {tag.subcategory_name}")
        print(f"   Severity: {tag.severity}/5  Confidence: {tag.confidence}")
        if tag.judge_rationale:
            print(f"   Rationale: {tag.judge_rationale}")
        print()
```

Run it:

```bash theme={"system"}
python quickstart.py
```

***

## What you'll see

The output intentionally passes a **wrong answer** ("Berlin" instead of "Paris") so you can see a real failure detected:

```text theme={"system"}
Status: completed
Failures detected: 1
Overall severity: 4/5
Should alert: True

❌ [Hallucination] Entity Fabrication
   Severity: 4/5  Confidence: 0.95
   Rationale: The output states Berlin is the capital of France,
   which directly contradicts the provided context that identifies
   Paris as the capital. This is a factual fabrication.
```

<Note>
  The exact bucket names, severity scores, and rationale text may vary slightly depending on your backend configuration and LLM judge version.
</Note>

***

## Understanding the result

The `result` object is an `FARunResult` with these key fields:

| Field                                    | Type               | What it means                                                           |
| ---------------------------------------- | ------------------ | ----------------------------------------------------------------------- |
| `result.summary.total_failures_detected` | `int`              | Total number of failure tags with `decision == "fail"`                  |
| `result.summary.overall_severity`        | `float`            | Worst-case severity across all failures (0–5)                           |
| `result.summary.should_alert`            | `bool`             | Whether this result warrants an alert                                   |
| `result.failure_tags`                    | `list[FailureTag]` | Individual failure classifications                                      |
| `result.failure_tags[i].decision`        | `str`              | `"pass"`, `"fail"`, or `"unsure"`                                       |
| `result.failure_tags[i].judge_rationale` | `Optional[str]`    | Human-readable explanation of the failure (may be `None` for pass tags) |

<Card title="Next: Fix the failure →" icon="wrench" href="/start-here/fix-the-failure">
  Learn how to interpret the result, understand root causes, and fix your prompt.
</Card>

***

## Analyze multiple items at once

You can pass multiple input/output pairs in a single call:

```python theme={"system"}
result = client.failure_analysis.run(
    dataset=[
        {
            "input": "What is the capital of France?",
            "output": "The capital of France is Paris.",
            "context": ["The capital of France is Paris."],
        },
        {
            "input": "Summarize the article about climate change.",
            "output": "The article discusses economic policy in the 1990s.",
            "context": [
                "Climate change is causing rising sea levels worldwide.",
                "The Paris Agreement aims to limit warming to 1.5°C.",
            ],
        },
        {
            "input": "What medications interact with warfarin?",
            "output": "Warfarin interacts with aspirin and ibuprofen.",
            "context": [
                "Warfarin interacts with aspirin, ibuprofen, and many antibiotics.",
                "Always consult a healthcare provider for drug interactions.",
            ],
        },
    ]
)

print(f"Items analyzed: {result.summary.total_items}")
print(f"Items with failures: {result.summary.items_with_failures}")
print(f"Items all passed: {result.summary.items_all_passed}")
```

***

## Optional: Add tracing later

Once you're comfortable with dataset mode, you can add auto-tracing to capture LLM calls in production and run Failure Analysis on traces:

```python theme={"system"}
import valiqor.auto  # Auto-instruments OpenAI, Anthropic, LangChain

# Your normal LLM calls are now traced automatically
# Then run FA on the captured trace:
result = client.failure_analysis.run(trace_id="tr_abc123")
```

See the [Tracing guide](/workflows/tracing) for full setup instructions.