> ## Documentation Index
> Fetch the complete documentation index at: https://docs.valiqor.com/llms.txt
> Use this file to discover all available pages before exploring further.

# FailureAnalysisClient

> Run failure analysis, explore failure taxonomy, and retrieve insights and trends.

## Overview

`ValiqorFAClient` is the core of Valiqor — it finds *why* your AI app fails, not just *that* it fails. It analyzes AI inputs and outputs to detect failure patterns, categorize them into a taxonomy of buckets, and provide root cause analysis with severity scoring.

Two modes of operation:

* **Dataset mode** — pass existing inputs/outputs directly as a list of dicts
* **Trace mode** — pass a `trace_id` from a traced execution

```python theme={"system"}
from valiqor import ValiqorClient

client = ValiqorClient(api_key="your-api-key")
fa = client.failure_analysis
```

Or standalone:

```python theme={"system"}
from valiqor.failure_analysis import ValiqorFAClient

fa = ValiqorFAClient(api_key="your-api-key")
```

<Tip>
  Supports context manager protocol: `with ValiqorFAClient(...) as fa:`
</Tip>

***

## Constructor

```python theme={"system"}
ValiqorFAClient(
    api_key: Optional[str] = None,
    project_name: Optional[str] = None,
    base_url: Optional[str] = None,
    timeout: int = 300,
    openai_api_key: Optional[str] = None,
)
```

| Parameter        | Type            | Default | Description                     |
| ---------------- | --------------- | ------- | ------------------------------- |
| `api_key`        | `Optional[str]` | `None`  | Valiqor API key.                |
| `project_name`   | `Optional[str]` | `None`  | Default project name.           |
| `base_url`       | `Optional[str]` | `None`  | Backend URL override.           |
| `timeout`        | `int`           | `300`   | Request timeout in seconds.     |
| `openai_api_key` | `Optional[str]` | `None`  | OpenAI key for LLM judge calls. |

***

## Core Methods

### `run()`

Run failure analysis. Auto-polls if the backend returns an async response.

```python theme={"system"}
def run(
    self,
    trace_id: Optional[str] = None,
    dataset: Optional[List[Dict[str, Any]]] = None,
    project_name: Optional[str] = None,
    feature_kind: Optional[str] = None,
    run_eval: bool = True,
    run_security: bool = True,
    run_scan: bool = True,
    mandatory_eval_metrics: Optional[List[str]] = None,
    mandatory_security_categories: Optional[List[str]] = None,
    subcategories: Optional[List[str]] = None,
    buckets: Optional[List[str]] = None,
    openai_api_key: Optional[str] = None,
) -> FARunResult
```

| Parameter                       | Type                   | Default | Description                                                           |
| ------------------------------- | ---------------------- | ------- | --------------------------------------------------------------------- |
| `trace_id`                      | `Optional[str]`        | `None`  | Trace ID for trace mode. Mutually exclusive with `dataset`.           |
| `dataset`                       | `Optional[List[Dict]]` | `None`  | Items with `input`, `output`, and optionally `context`, `tool_calls`. |
| `feature_kind`                  | `Optional[str]`        | `None`  | One of `"rag"`, `"agent"`, `"agentic_rag"`, `"generic_llm"`.          |
| `run_eval`                      | `bool`                 | `True`  | Run evaluation metrics as part of FA.                                 |
| `run_security`                  | `bool`                 | `True`  | Run security checks as part of FA.                                    |
| `run_scan`                      | `bool`                 | `True`  | Run code scanning as part of FA.                                      |
| `mandatory_eval_metrics`        | `Optional[List[str]]`  | `None`  | Specific eval metrics to always include.                              |
| `mandatory_security_categories` | `Optional[List[str]]`  | `None`  | Specific security categories to always check.                         |
| `subcategories`                 | `Optional[List[str]]`  | `None`  | Filter to specific failure subcategories.                             |
| `buckets`                       | `Optional[List[str]]`  | `None`  | Filter to specific failure buckets.                                   |

**Returns:** [`FARunResult`](/sdk/models/failure-analysis#farunresult)

<CodeGroup>
  ```python Dataset Mode theme={"system"}
  result = client.failure_analysis.run(
      dataset=[
          {
              "input": "What is the capital of France?",
              "output": "The capital of France is London.",
              "context": "Paris is the capital of France."
          }
      ],
      feature_kind="rag"
  )
  print(f"Failures: {result.summary.total_failures_detected}")
  print(f"Primary failure: {result.summary.primary_failure_name}")
  for tag in result.failure_tags:
      print(f"  [{tag.decision}] {tag.subcategory_name} (severity: {tag.severity})")
  ```

  ```python Trace Mode theme={"system"}
  result = client.failure_analysis.run(
      trace_id="trace-abc-123",
      feature_kind="agent"
  )
  ```
</CodeGroup>

***

### `run_async()`

Always returns an `AsyncJobHandle`, even if the backend responds synchronously.

```python theme={"system"}
def run_async(
    self,
    trace_id: Optional[str] = None,
    dataset: Optional[List[Dict[str, Any]]] = None,
    project_name: Optional[str] = None,
    feature_kind: Optional[str] = None,
    run_eval: bool = True,
    run_security: bool = True,
    run_scan: bool = True,
    mandatory_eval_metrics: Optional[List[str]] = None,
    mandatory_security_categories: Optional[List[str]] = None,
    subcategories: Optional[List[str]] = None,
    buckets: Optional[List[str]] = None,
    openai_api_key: Optional[str] = None,
) -> AsyncJobHandle
```

**Returns:** [`AsyncJobHandle`](/sdk/models/traces#asyncjobhandle)

```python theme={"system"}
handle = client.failure_analysis.run_async(dataset=large_dataset)
handle.wait(on_progress=lambda s: print(f"{s.progress_percent}%"))
result = handle.result()
```

***

### `playground()`

Lightweight single-item failure analysis for quick testing.

```python theme={"system"}
def playground(
    self,
    input: str,
    output: str,
    context: Optional[str] = None,
    tool_calls: Optional[List[Dict[str, Any]]] = None,
) -> FAPlaygroundResult
```

<Warning>
  Rate limited: **10 calls/day**, **2 calls/minute**.
</Warning>

```python theme={"system"}
result = client.failure_analysis.playground(
    input="What is 2+2?",
    output="2+2 is 5",
    context="Basic arithmetic"
)
print(f"Checks passed: {result.checks_passed}")
print(f"Remaining today: {result.playground_limit_per_day - result.playground_runs_today}")
```

***

## Result Retrieval

### `get_run()`

Get a completed FA result by run ID.

```python theme={"system"}
def get_run(self, run_id: str) -> FARunResult
```

### `get_run_inputs()`

Get original input items with per-item failure statistics.

```python theme={"system"}
def get_run_inputs(self, run_id: str) -> List[FARunInput]
```

```python theme={"system"}
inputs = client.failure_analysis.get_run_inputs("run-id-123")
for item in inputs:
    if item.has_failures:
        print(f"Item {item.item_index}: {item.failure_count} failures, max severity {item.max_severity}")
```

### `get_tags()`

Get failure tags for a run, with optional filtering.

```python theme={"system"}
def get_tags(
    self,
    run_id: str,
    decision: Optional[str] = None,
    bucket_id: Optional[str] = None,
    min_severity: Optional[float] = None,
) -> List[FATag]
```

| Parameter      | Type              | Description                                  |
| -------------- | ----------------- | -------------------------------------------- |
| `decision`     | `Optional[str]`   | Filter by `"pass"`, `"fail"`, or `"unsure"`. |
| `bucket_id`    | `Optional[str]`   | Filter to a specific failure bucket.         |
| `min_severity` | `Optional[float]` | Minimum severity threshold (0.0–5.0).        |

### `update_tag()`

Update a failure tag's review status or link it to an external issue tracker.

```python theme={"system"}
def update_tag(
    self,
    tag_id: str,
    is_reviewed: Optional[bool] = None,
    issue_url: Optional[str] = None,
) -> Dict[str, Any]
```

| Parameter     | Type             | Description                                                     |
| ------------- | ---------------- | --------------------------------------------------------------- |
| `tag_id`      | `str`            | The failure tag ID (required).                                  |
| `is_reviewed` | `Optional[bool]` | Set review status — `True` to mark reviewed, `False` to unmark. |
| `issue_url`   | `Optional[str]`  | URL to an external issue (empty string to unlink).              |

**Returns:** Dict with `tag_id`, `is_reviewed`, `reviewed_at`, `issue_url`, `message`.

```python theme={"system"}
# Mark a failure as reviewed
client.failure_analysis.update_tag("tag-id-123", is_reviewed=True)

# Link a failure to a GitHub issue
client.failure_analysis.update_tag(
    "tag-id-123",
    issue_url="https://github.com/org/repo/issues/42"
)

# Mark reviewed and link issue in one call
client.failure_analysis.update_tag(
    "tag-id-123",
    is_reviewed=True,
    issue_url="https://github.com/org/repo/issues/42"
)
```

<Tip>
  At least one of `is_reviewed` or `issue_url` must be provided.
</Tip>

***

## Run History

### `list_runs()`

List FA runs with pagination and filtering.

```python theme={"system"}
def list_runs(
    self,
    project_name: Optional[str] = None,
    status: Optional[str] = None,
    page: int = 1,
    page_size: int = 20,
) -> FARunListPage
```

### `get_run_count()`

```python theme={"system"}
def get_run_count(self, project_name: Optional[str] = None) -> int
```

***

## Job Management

### `poll_status()`

Check the status of an async FA run.

```python theme={"system"}
def poll_status(self, run_id: str) -> FAJobStatus
```

### `cancel_run()`

Cancel a running FA job.

```python theme={"system"}
def cancel_run(self, run_id: str) -> Dict[str, str]
```

***

## Taxonomy

### `get_taxonomy()`

Get the full failure taxonomy — all buckets with their subcategories.

```python theme={"system"}
def get_taxonomy(self, project_name: Optional[str] = None) -> List[FABucket]
```

```python theme={"system"}
taxonomy = client.failure_analysis.get_taxonomy()
for bucket in taxonomy:
    print(f"\n{bucket.bucket_name}: {bucket.description}")
    for sub in bucket.subcategories:
        print(f"  - {sub.subcategory_name} ({sub.detection_approach})")
```

### `get_subcategories()`

Get subcategories with optional filtering.

```python theme={"system"}
def get_subcategories(
    self,
    project_name: Optional[str] = None,
    bucket_id: Optional[str] = None,
    detection_approach: Optional[str] = None,
    applies_to: Optional[str] = None,
) -> List[FASubcategory]
```

| Parameter            | Type            | Description                                      |
| -------------------- | --------------- | ------------------------------------------------ |
| `bucket_id`          | `Optional[str]` | Filter to a specific bucket.                     |
| `detection_approach` | `Optional[str]` | `"deterministic"`, `"llm_judge"`, or `"hybrid"`. |
| `applies_to`         | `Optional[str]` | Filter by applicable feature kind.               |

### `get_bucket()`

Get details for a specific failure bucket.

```python theme={"system"}
def get_bucket(self, bucket_id: str) -> FABucket
```

***

## Analytics

### `get_insights()`

Get aggregated failure insights for a project over a time period.

```python theme={"system"}
def get_insights(
    self,
    project_name: Optional[str] = None,
    days: int = 30,
) -> FAInsightsSummary
```

```python theme={"system"}
insights = client.failure_analysis.get_insights(days=7)
print(f"Failure rate: {insights.overall_failure_rate:.1%}")
print(f"Avg severity: {insights.average_severity}")
for failure in insights.top_recurring_failures:
    print(f"  {failure.subcategory_name}: {failure.occurrence_count} occurrences")
```

### `get_trends()`

Get failure trends over time.

```python theme={"system"}
def get_trends(
    self,
    project_name: Optional[str] = None,
    days: int = 30,
) -> FATrends
```

### `get_security_insights()`

Get security-specific insights from FA runs.

```python theme={"system"}
def get_security_insights(
    self,
    project_name: Optional[str] = None,
    days: int = 30,
) -> FASecurityInsightsSummary
```

### `get_projects()`

List all projects that have FA runs.

```python theme={"system"}
def get_projects(self) -> List[str]
```

***

## Related

* [Models Reference](/sdk/models) — `FARunResult`, `FATag`, `FASummary`, `FABucket`
* [Failure Analysis Workflow](/workflows/failure-analysis) — Step-by-step guide
* [Failure Taxonomy](/concepts/failure-taxonomy) — Understanding failure buckets
* [Root Cause Detection](/concepts/root-cause-detection) — How detection works