Documentation Index
Fetch the complete documentation index at: https://docs.valiqor.com/llms.txt
Use this file to discover all available pages before exploring further.
SDK Exception Hierarchy
Every exception raised by the SDK inherits from ValiqorError, so you can catch all SDK errors in one place.
from valiqor import ValiqorError
try:
result = client.eval.evaluate(dataset=data, metrics=["coherence"])
except ValiqorError as e:
print(f"Valiqor error: {e}")
Full hierarchy
ValiqorError
├── ConfigurationError
├── AuthenticationError
│ └── AccountDeactivatedError
├── RateLimitError
├── QuotaExceededError
│ └── TokenQuotaExceededError
├── ValidationError
│ └── DatasetTooLargeError
├── TimeoutError
├── NetworkError
│ └── UploadError
├── APIError
│ └── ServiceUnavailableError
├── TracingError
├── ScanError
├── EvaluationError
└── SecurityError
Common Errors and Resolutions
AuthenticationError
HTTP status: 401 or 403
| Message | Cause | Resolution |
|---|
API key required. Pass api_key parameter or set VALIQOR_API_KEY env var. | No API key provided | Set VALIQOR_API_KEY environment variable or pass api_key to the client constructor |
Invalid API key format. Keys should start with 'vq_' | Key doesn’t have the vq_ prefix | Regenerate your key at the dashboard or with valiqor keys create |
Invalid or expired API key | Key has been revoked or expired | Run valiqor login to re-authenticate |
Access forbidden. Check your API key permissions. | Key doesn’t have the required permissions | Verify you’re using the correct key for this project |
from valiqor import AuthenticationError
try:
result = client.eval.evaluate(dataset=data, metrics=["coherence"])
except AuthenticationError as e:
print(f"Auth failed: {e}")
AccountDeactivatedError
HTTP status: 403
Your account has been deactivated by an administrator. Contact support or visit app.valiqor.com/account.
RateLimitError
HTTP status: 429
Too many requests in a short window. The SDK provides a retry_after attribute.
from valiqor import RateLimitError
import time
try:
result = client.eval.evaluate(dataset=data, metrics=["coherence"])
except RateLimitError as e:
print(f"Rate limited. Retry in {e.retry_after}s")
time.sleep(e.retry_after or 5)
QuotaExceededError
HTTP status: 429 (quota exceeded) or 403 (trial expired)
Your monthly API call quota has been used up.
from valiqor import QuotaExceededError
try:
result = client.failure_analysis.run(dataset=data)
except QuotaExceededError as e:
print(f"Usage: {e.current_usage}/{e.quota_limit} ({e.service})")
- Check usage with
client.get_usage() or valiqor status
- Upgrade your plan for a higher quota
- If you see
"Verify your email to unlock 50 free runs", run valiqor verify
TokenQuotaExceededError
HTTP status: 429
Monthly LLM token quota exceeded. Same as QuotaExceededError but with token-specific attributes.
from valiqor import TokenQuotaExceededError
try:
result = client.eval.evaluate(dataset=data, metrics=["coherence"])
except TokenQuotaExceededError as e:
print(f"Tokens: {e.current_tokens}/{e.token_limit}")
print(f"This request would use ~{e.predicted_tokens} tokens")
ValidationError
HTTP status: 400
Invalid request parameters or dataset format.
| Common Cause | Resolution |
|---|
| Missing required fields in dataset items | Ensure each item has at minimum input and output keys |
| Invalid metric name | Use client.eval.list_metric_templates() to see available metrics |
| Project not found | Check project name with valiqor config show |
from valiqor import DatasetTooLargeError
try:
result = client.eval.evaluate(dataset=large_data, metrics=["coherence"])
except DatasetTooLargeError as e:
print(f"Dataset: {e.dataset_size} items (max: {e.max_allowed})")
TimeoutError
Request took longer than the 300-second (5-minute) default timeout.
Resolutions:
- Use async mode for large datasets:
client.eval.evaluate_async(...) or client.failure_analysis.run_async(...)
- Reduce dataset size
NetworkError
Cannot reach the Valiqor backend.
| Message | Cause | Resolution |
|---|
Failed to connect to ... | Backend unreachable | Check internet connection; verify backend URL in .valiqorrc |
Request failed: ... | Generic network failure | Retry the request |
UploadError
File upload failed (subclass of NetworkError).
| Cause | Resolution |
|---|
| Invalid API key during upload | Run valiqor login to refresh credentials |
| File not found or invalid JSON | Verify the file path and format |
ServiceUnavailableError
HTTP status: 503
The backend is temporarily overloaded. The SDK automatically retries once after a short delay. If the retry also fails, this error is raised.
from valiqor import ServiceUnavailableError
try:
result = client.eval.evaluate(dataset=data, metrics=["coherence"])
except ServiceUnavailableError as e:
print(f"Service unavailable: {e}")
print(f"Suggested retry after: {e.retry_after}s")
APIError
HTTP status: 500+ (generic server error)
An unexpected backend error. Includes status_code and response attributes for debugging.
from valiqor import APIError
try:
result = client.eval.evaluate(dataset=data, metrics=["coherence"])
except APIError as e:
print(f"Server error {e.status_code}: {e}")
print(f"Response: {e.response}")
Module-Specific Errors
TracingError
Raised when tracing operations fail (e.g. malformed trace data, span nesting issues).
ScanError
Raised when scanner operations fail (e.g. repository path doesn’t exist).
EvaluationError
Raised for evaluation-specific failures.
SecurityError
Raised for security audit or red team failures.
Module Not Installed
Each SDK module is lazily loaded. If a module fails to load, you’ll see an ImportError:
| Access | Error | Fix |
|---|
client.eval | Eval module not available | pip install valiqor |
client.security | Security module not available | pip install valiqor |
client.trace | Trace module not available | pip install valiqor |
client.scanner | Scanner module not available | pip install valiqor |
client.failure_analysis | Failure analysis module not available | pip install valiqor |
client.traces | Trace query module not available | pip install valiqor |
CLI Issues
Login fails
| Symptom | Resolution |
|---|
httpx not installed | Run pip install httpx — required for the login flow |
Authorization timed out | Re-run valiqor login; the device code expires after a few minutes |
Authorization denied | Approve the request in your browser when prompted |
Direct login not available | The backend doesn’t support email/password login; use valiqor login for browser-based auth |
Configuration problems
| Symptom | Resolution |
|---|
Valiqor not configured | Run valiqor config to create a .valiqorrc file |
Configuration has errors | Run valiqor config to fix missing or invalid settings |
Unknown config key | Check allowed keys: project_name, api_key, trace_dir, scan_dir, environment, valiqor_intelligence, debug, openai_api_key |
Upload fails
| Symptom | Resolution |
|---|
No Valiqor API key configured | Run valiqor login or set VALIQOR_API_KEY |
No files found to upload | Check that trace/scan files exist in the configured directories |
Path not found | Verify the path you passed to valiqor upload |
Auto-Retry Behaviour
The SDK automatically retries in one scenario:
| Status Code | Behaviour |
|---|
| 503 | Waits briefly, then retries once. If the retry also returns 503, raises ServiceUnavailableError. |
