> ## Documentation Index
> Fetch the complete documentation index at: https://docs.valiqor.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Root Cause Detection

> How Valiqor traces failures back to root causes with severity scoring, confidence, and actionable remediation.

Valiqor's Failure Analysis doesn't just detect failures — it traces each
failure back to its **root cause**, assigns a **severity score**, computes
a **confidence level**, and suggests **remediation**.

***

## How It Works

Every dataset item or trace is analyzed against Valiqor's
[failure taxonomy](/concepts/failure-taxonomy). For each applicable
subcategory, a classification decision is produced along with a severity
score and confidence level.

```
Input (prompt, response, context)
        ↓
   Evidence Collection
        ↓
   Failure Classification
        ↓
   FATag per subcategory
   (decision + severity + confidence + evidence)
```

***

## Failure Decisions

Each subcategory check produces one of four decisions:

| Decision         | Meaning                                                  |
| ---------------- | -------------------------------------------------------- |
| `fail`           | Failure detected — evidence supports this classification |
| `pass`           | No failure — the system behaved correctly                |
| `unsure`         | Ambiguous — insufficient evidence to decide              |
| `not_applicable` | This subcategory doesn't apply to this app type          |

***

## Severity

Severity measures **how bad** a failure is, on a **0–5 scale**.

Higher severity means greater potential impact on the user, the business, or
safety. Valiqor computes severity automatically based on the failure type,
its context, and how frequently it recurs.

| Severity | Interpretation                                              |
| -------- | ----------------------------------------------------------- |
| **0–1**  | Minor — cosmetic or low-risk issue                          |
| **2–3**  | Moderate — user-visible failure that should be investigated |
| **4–5**  | Critical — potential financial, legal, or safety impact     |

<Note>
  Failures associated with high-risk [security categories](/concepts/security-categories)
  (e.g. self-harm, PII exposure, hate speech) are automatically escalated
  to critical severity.
</Note>

### Frequency Amplification

When the same failure type recurs across multiple items in a dataset,
severity is amplified. Isolated issues score lower than systemic patterns.

***

## Confidence

Confidence measures **how certain** Valiqor is about a classification,
on a **0.0–1.0 scale**.

Confidence increases when multiple independent signals agree:

* **Rule-based detectors** confirm the failure
* **LLM judge** classifies the failure
* **Evaluation metrics** corroborate the finding
* **Security classifiers** flag related content

When signals **disagree**, confidence is reduced and the result may be
flagged for human review.

| Confidence  | Interpretation                                 |
| ----------- | ---------------------------------------------- |
| **0.8–1.0** | High — multiple signals agree                  |
| **0.5–0.7** | Moderate — some evidence, but not conclusive   |
| **\< 0.5**  | Low — limited evidence, consider manual review |

***

## Reading FATag Results

Every failure is returned as an `FATag` with these key fields:

```python theme={"system"}
tag = result.failure_tags[0]

print(tag.decision)            # "fail", "pass", "unsure", "not_applicable"
print(tag.bucket_name)         # e.g. "Hallucination & Grounding"
print(tag.subcategory_name)    # e.g. "Unsupported factual claim"
print(tag.severity)            # 0.0 – 5.0
print(tag.confidence)          # 0.0 – 1.0
print(tag.detector_type_used)  # "deterministic", "llm_judge", or "hybrid"
```

### Judge Rationale

For LLM-judge-detected failures, the `judge_rationale` field contains
the judge's explanation:

```python theme={"system"}
if tag.judge_rationale:
    print(f"Why: {tag.judge_rationale}")
```

### Evidence

Each tag includes structured evidence linking back to the original data:

```python theme={"system"}
for evidence in tag.evidence_items:
    print(f"  [{evidence.evidence_type}] {evidence.description}")
    if evidence.content_snippet:
        print(f"    → {evidence.content_snippet}")
```

| Field             | Description                                             |
| ----------------- | ------------------------------------------------------- |
| `evidence_type`   | Kind of evidence (e.g. `"span"`, `"claim"`, `"metric"`) |
| `description`     | Human-readable explanation                              |
| `source`          | Where it came from (`"trace"`, `"eval"`, `"security"`)  |
| `content_snippet` | Relevant text excerpt                                   |

### Eval Metric Values

The `eval_metric_values` dict shows which evaluation metrics were used
as supporting evidence:

```python theme={"system"}
for metric, score in tag.eval_metric_values.items():
    print(f"  {metric}: {score:.3f}")
# e.g. hallucination: 0.85, context_precision: 0.42
```

***

## Automation Flags

The `FARunResult` summary includes built-in flags for automation:

```python theme={"system"}
result = client.failure_analysis.run(dataset=data)

if result.summary.should_alert:
    # Trigger Slack / PagerDuty notification
    send_alert(result)

if result.summary.should_gate_ci:
    # Block the CI/CD pipeline
    sys.exit(1)

if result.summary.needs_human_review:
    # Queue for manual inspection
    create_review_ticket(result)
```

| Flag                 | When It Triggers                                   |
| -------------------- | -------------------------------------------------- |
| `should_alert`       | Critical failures with high confidence             |
| `should_gate_ci`     | Failures severe enough to block deployment         |
| `needs_human_review` | High-severity failures where evidence is ambiguous |

### Summary Statistics

```python theme={"system"}
summary = result.summary
print(f"Failures: {summary.total_failures_detected}")
print(f"Passes: {summary.total_passes}")
print(f"Items with failures: {summary.items_with_failures}")
print(f"Overall severity: {summary.overall_severity:.1f}")
print(f"Buckets affected: {summary.buckets_affected}")
```

***

## Interpreting Results

### High Severity + High Confidence → Act Now

Reliable, serious failures. Set up automated alerts and CI gates.

### High Severity + Low Confidence → Review

The system suspects a serious failure but evidence is ambiguous. Queue for
human review — the `needs_human_review` flag catches these automatically.

### Low Severity + High Confidence → Monitor

Real but minor issues. Track trends with `get_trends()` — if frequency
increases, severity will be amplified.

### Unsure Decision → Investigate

The detector couldn't reach a conclusion. This typically means the input
data is insufficient for classification (e.g. missing context for RAG
checks). Provide richer data for better results.

***

## Complete Example

```python theme={"system"}
from valiqor import ValiqorClient

client = ValiqorClient()
result = client.failure_analysis.run(
    dataset=[{
        "input": "What are the side effects of aspirin?",
        "output": "Aspirin can cause liver failure in most patients.",
        "context": ["Aspirin may cause stomach upset and rare allergic reactions."]
    }]
)

# Check summary
print(f"Failures found: {result.summary.total_failures_detected}")
print(f"Should alert: {result.summary.should_alert}")

# Inspect individual failures
for tag in result.failure_tags:
    if tag.decision == "fail":
        print(f"\n🔴 {tag.subcategory_name}")
        print(f"   Severity: {tag.severity:.1f} | Confidence: {tag.confidence:.2f}")
        print(f"   Detector: {tag.detector_type_used}")
        if tag.judge_rationale:
            print(f"   Rationale: {tag.judge_rationale}")
```

See the [Failure Analysis workflow](/workflows/failure-analysis) for
end-to-end usage and the [Failure Taxonomy](/concepts/failure-taxonomy)
for the full list of detectable failure types.