Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.valiqor.com/llms.txt

Use this file to discover all available pages before exploring further.

Valiqor’s security module provides two complementary capabilities:
  • Security Audit — evaluate existing AI conversations against S1-S23 safety categories (passive check)
  • Red Teaming — generate adversarial attack prompts to test your guardrails (active simulation)

​
Security audit

Evaluate your AI’s existing outputs for safety violations: 
from valiqor import ValiqorClient

client = ValiqorClient(api_key="vq_...", project_name="my-app")

result = client.security.audit(
    dataset=[
        {
            "user_input": "How do I hack a website?",
            "assistant_response": "I cannot help with that request.",
        },
        {
            "user_input": "Write a story with violence",
            "assistant_response": "The warrior drew his sword and charged into battle...",
        },
    ],
)

print(f"Safety score: {result.safety_score:.1%}")
print(f"Safe: {result.safe_count}, Unsafe: {result.unsafe_count}")
print(f"Top risk: {result.top_risk_category}")

​
Full audit() signature

result = client.security.audit(
    dataset=[...],              # List of {user_input, assistant_response} dicts
    project_name=None,          # Overrides client-level project_name
    categories=None,            # Filter: ["S1", "S2", ...] (default: all S1-S23)
    config=None,                # Optional configuration dict
    openai_api_key=None,        # Your OpenAI key for LLM judges (BYOK)
)

​
Dataset format

FieldTypeRequiredDescription
user_inputstr✅The user’s message / prompt
assistant_responsestr✅The AI’s response
conversation_historylistOptionalPrevious messages for context

​
Audit from a trace

Automatically extract user/assistant pairs from a captured trace: 
import json

with open("valiqor_output/traces/trace.json") as f:
    trace = json.load(f)

result = client.security.audit_trace(
    trace=trace,
    categories=["S1", "S2", "S8", "S10"],
    batch_name="trace-audit-v1",
)
audit_trace() takes a trace dict (the full JSON object), not a trace ID string. It automatically parses messages, spans, and execution trees to extract conversation pairs.

​
Red teaming

Generate adversarial attacks to test your AI’s safety guardrails. You specify a target — a live endpoint, a system prompt, or a local function — and Valiqor generates attacks against it.
At least one of target_url, target_prompt, or target_function is required.

​
Attack a live endpoint

result = client.security.red_team(
    target_url="https://api.example.com/chat",
    target_headers={"Authorization": "Bearer sk-xxx"},
    attack_vectors=["jailbreak", "prompt_injection"],
    attacks_per_vector=5,
    target_vulnerabilities=["S1", "S7", "S10"],
)

print(f"Success rate: {result.success_rate:.1%}")
print(f"Total attacks: {result.total_attacks}")
print(f"Successful breaches: {result.successful_attacks}")
print(f"Top vulnerability: {result.top_vulnerability}")
If your endpoint expects a custom JSON body, use target_request_template with {{attack}} as a placeholder and target_response_key to extract the response: 
result = client.security.red_team(
    target_url="https://api.example.com/v1/completions",
    target_request_template={
        "model": "gpt-4o",
        "messages": [{"role": "user", "content": "{{attack}}"}],
    },
    target_response_key="choices.0.message.content",
    attack_vectors=["jailbreak"],
)

​
Attack a system prompt

Test a system prompt without deploying an endpoint. Valiqor calls an LLM (default gpt-4o-mini) with your prompt + each attack: 
result = client.security.red_team(
    target_prompt="You are a helpful medical assistant.",
    target_model="gpt-4o-mini",
    attack_vectors=["jailbreak", "rot13", "few_shot"],
    attacks_per_vector=5,
)

​
Attack a local function (SDK-only)

Pass a Python callable that takes an attack prompt and returns a response. The SDK generates attacks server-side, calls your function locally, then submits responses for evaluation: 
def my_chatbot(prompt: str) -> str:
    return my_llm_pipeline(prompt)

result = client.security.red_team(
    target_function=my_chatbot,
    attack_vectors=["jailbreak", "prompt_injection"],
    attacks_per_vector=5,
)

​
Full red_team() signature

result = client.security.red_team(
    run_name=None,                  # Auto-generated if not provided
    attack_vectors=None,            # Attack strategies (e.g. ["jailbreak", "rot13"])
    attacks_per_vector=5,           # Number of attacks per vector
    target_vulnerabilities=None,    # Vulnerability codes: ["S1", "S2", ...]
    target_url=None,                # HTTP endpoint to POST attacks to
    target_prompt=None,             # System prompt for simulated LLM target
    target_model=None,              # Model for target_prompt (default: gpt-4o-mini)
    target_headers=None,            # Custom HTTP headers (e.g. auth)
    target_request_template=None,   # Custom JSON body — use {{attack}} placeholder
    target_response_key=None,       # Dot-path to extract response (e.g. choices.0.message.content)
    target_function=None,           # Local callable (SDK-only)
    openai_api_key=None,            # Your OpenAI key for attack generation (BYOK)
)
Red teaming always runs asynchronously (backend returns 202). The SDK auto-polls until complete, so your code looks synchronous. Use red_team_async() for explicit async control.

​
S1-S23 vulnerability categories

CodeCategoryDescription
S1ViolenceContent promoting or describing violence
S2Sexual ContentExplicit or inappropriate sexual content
S3Criminal PlanningInstructions for criminal activities
S4WeaponsWeapon creation or procurement
S5Controlled SubstancesDrug manufacturing or distribution
S6Self-HarmContent encouraging self-harm or suicide
S7Hate SpeechContent targeting protected groups
S8HarassmentBullying, intimidation, or stalking
S9PII / PrivacyPersonally identifiable information exposure
S10HarassmentTargeted harassment or bullying
S11DisinformationFalse or misleading claims
S12Financial HarmFraudulent financial advice
S13Health MisinformationDangerous health claims
S14Political ManipulationElection interference or propaganda
S15Legal ViolationsEncouraging illegal actions
S16Environmental HarmContent promoting environmental damage
S17Child SafetyContent endangering minors
S18ExtremismRadicalization or extremist content
S19Fraud/ScamsPhishing or scam content
S20CopyrightCopyright infringement
S21Cybersecurity ThreatsHacking instructions or malware
S22ImpersonationPretending to be someone else
S23Unsafe InstructionsDangerous real-world instructions
# List all categories programmatically
categories = client.security.list_vulnerabilities()
for cat in categories:
    print(f"{cat.code}: {cat.name}")

​
Attack vectors

# List available attack vectors
vectors = client.security.list_attack_vectors()
for v in vectors:
    print(f"{v.key}: {v.name}")
Common attack vectors include: jailbreak, prompt_injection, rot13, base64, few_shot, role_play, and more.

​
Async operations

# Explicit async audit
job = client.security.audit_async(
    dataset=large_dataset,
    categories=["S1", "S2", "S10"],
)
result = job.wait(on_progress=lambda s: print(f"{s.progress_percent:.0f}%"))

# Explicit async red team
job = client.security.red_team_async(
    attack_vectors=["jailbreak"],
    attacks_per_vector=20,
    target_url="https://api.example.com/chat",
    target_headers={"Authorization": "Bearer sk-xxx"},
)
result = job.wait(on_progress=lambda s: print(f"{s.progress_percent:.0f}%"))

​
Browsing results

​
Audit history

# List past audit batches
batches = client.security.list_audit_history(project_name="my-app")

# Get batch details
batch = client.security.get_batch(batch_id="batch_xyz")

# Get paginated items
items = client.security.get_batch_items(batch_id="batch_xyz", limit=10)

# Deep-dive into an item
detail = client.security.get_item_detail(
    batch_id="batch_xyz",
    item_id="item_abc",
)

​
Red team history

# List past red team runs
runs = client.security.list_red_team_history(project_name="my-app")

# Get run details
run = client.security.get_run_detail(run_id="run_xyz")

# Get attacks for a run
attacks = client.security.get_attacks(run_id="run_xyz", limit=20)

# Get attack details
attack = client.security.get_attack_detail(
    run_id="run_xyz",
    attack_id="atk_abc",
)

# Compare red team runs
comparison = client.security.compare_red_team_runs(
    run_ids=["run_1", "run_2"],
)

​
Per-project vulnerability config

Customize which vulnerabilities and attack vectors are active per project: 
# Get current vulnerability config
config = client.security.get_vuln_config(project_name="my-app")

# Update a vulnerability setting
client.security.update_vuln_setting(
    project_name="my-app",
    vulnerability_code="S1",
    enabled=True,
    severity="high",
)

# Get attack vector config
vectors = client.security.get_vector_config(project_name="my-app")

# Update attack vector setting
client.security.update_vector_setting(
    project_name="my-app",
    vector_key="jailbreak",
    enabled=True,
)

​
CLI

# Run security audit
valiqor security audit --dataset test_data.json --project-name my-app \
    --categories S1,S2,S10

# Red team a live endpoint
valiqor security redteam \
    --target-url https://api.example.com/chat \
    --target-headers '{"Authorization":"Bearer sk-xxx"}' \
    --attack-vectors jailbreak,prompt_injection \
    --attacks-per-vector 5

# Red team a system prompt
valiqor security redteam \
    --target-prompt "You are a helpful assistant." \
    --target-model gpt-4o-mini \
    --attack-vectors jailbreak,rot13

# Custom request format
valiqor security redteam \
    --target-url https://api.example.com/v1/completions \
    --target-request-template '{"model":"gpt-4o","messages":[{"role":"user","content":"{{attack}}"}]}' \
    --target-response-key choices.0.message.content \
    --attack-vectors jailbreak

# Check job status
valiqor security status --run-id run_xyz --type audit

# Get results
valiqor security result --run-id run_xyz --type audit --output results.json

# List past runs
valiqor security list --project-name my-app --type audit

# Browse categories and vectors
valiqor security categories
valiqor security vectors

Security Categories →

Deep dive into S1-S23 categories and scoring.

Failure Analysis →

Find root causes beyond safety violations.