> ## Documentation Index
> Fetch the complete documentation index at: https://docs.valiqor.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Evaluation

> CLI commands for running evaluations, checking status, retrieving results, and listing metrics.

All evaluation commands live under the `eval` group.

```bash theme={"system"}
valiqor eval <run|status|result|list|metrics>
```

### Common Flags

Every `eval` subcommand accepts:

| Flag               | Type     | Default | Description                      |
| ------------------ | -------- | ------- | -------------------------------- |
| `--api-key`        | `string` | —       | API key override                 |
| `--project`        | `string` | —       | Project name override            |
| `--base-url`       | `string` | —       | Backend URL override             |
| `--openai-api-key` | `string` | —       | OpenAI key for LLM-based metrics |

***

## eval run

Run an evaluation on a dataset or trace file.

```bash theme={"system"}
valiqor eval run --metrics <metrics> [--dataset <path> | --trace-file <path>] [flags]
```

| Flag           | Type     | Default      | Description                               |
| -------------- | -------- | ------------ | ----------------------------------------- |
| `--dataset`    | `string` | —            | Path to a dataset JSON file               |
| `--trace-file` | `string` | —            | Path to a trace JSON file                 |
| `--metrics`    | `string` | **required** | Comma-separated metric keys               |
| `--run-name`   | `string` | —            | Name for this evaluation run              |
| `--model`      | `string` | —            | Model name metadata                       |
| `--async`      | `flag`   | `false`      | Submit asynchronously and return a job ID |

<Warning>
  `--dataset` and `--trace-file` are mutually exclusive. At least one is required.
</Warning>

### Dataset Format

Each item in the dataset JSON array must include `input` and `output` fields:

```json theme={"system"}
[
  {
    "input": "What is the capital of France?",
    "output": "The capital of France is Paris."
  }
]
```

### Examples

```bash theme={"system"}
# Synchronous evaluation
valiqor eval run --dataset data.json --metrics factual_accuracy,coherence

# Evaluate a trace file
valiqor eval run --trace-file trace.json --metrics hallucination

# Async evaluation (returns job ID)
valiqor eval run --dataset data.json --metrics coherence --async
```

***

## eval status

Check the status of an evaluation run.

```bash theme={"system"}
valiqor eval status --run-id <id> [flags]
```

| Flag            | Type     | Default      | Description                                    |
| --------------- | -------- | ------------ | ---------------------------------------------- |
| `--run-id`      | `string` | **required** | Evaluation run or job ID                       |
| `--wait` / `-w` | `flag`   | `false`      | Poll until completion with a live progress bar |

### Examples

```bash theme={"system"}
# Check status once
valiqor eval status --run-id abc123

# Wait for completion with progress bar
valiqor eval status --run-id abc123 --wait
```

<Info>
  In wait mode, the CLI polls every 3 seconds until the run completes, fails, or is cancelled.
</Info>

***

## eval result

Fetch and display evaluation results.

```bash theme={"system"}
valiqor eval result --run-id <id> [flags]
```

| Flag       | Type     | Default      | Description                                  |
| ---------- | -------- | ------------ | -------------------------------------------- |
| `--run-id` | `string` | **required** | Evaluation run ID                            |
| `--json`   | `flag`   | `false`      | Output raw JSON instead of formatted display |

### Output

The formatted display includes:

* Run ID, project, status, items evaluated
* Overall score with indicator (≥0.8 ✅, ≥0.5 ⚠️, \<0.5 ❌)
* Per-metric aggregate scores
* Per-metric detail scores for each item

```bash theme={"system"}
# Formatted output
valiqor eval result --run-id abc123

# Raw JSON for scripting
valiqor eval result --run-id abc123 --json
```

***

## eval list

List evaluation runs for the configured project.

```bash theme={"system"}
valiqor eval list
```

Displays run ID, status, score, items count, name, and created date.

***

## eval metrics

List all available evaluation metric templates.

```bash theme={"system"}
valiqor eval metrics
```

Shows each metric's key, display name, value type, and category. Use metric keys with `--metrics` in `eval run`.