Skip to main content
POST
/
api
/
dataframer
/
evaluations
Python
import os
from dataframer import Dataframer

client = Dataframer(
    api_key=os.environ.get("DATAFRAMER_API_KEY"),  # This is the default and can be omitted
)
evaluation = client.dataframer.evaluations.create(
    run_id="a98715da-921d-4326-bbf8-208f8bcc2956",
)
print(evaluation.id)
{
  "id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
  "run_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
  "status": "PENDING",
  "conformance_score": 123,
  "conformance_explanation": "<string>",
  "distribution_analysis": [
    {
      "property_name": "<string>",
      "total_samples": 123,
      "requested_distributions": {
        "positive": 40,
        "negative": 30,
        "neutral": 30
      },
      "expected_distributions": {
        "positive": 42,
        "negative": 30,
        "neutral": 28
      },
      "evaluated_distributions": {
        "positive": 45,
        "negative": 28,
        "neutral": 27
      },
      "observed_distributions": {},
      "total_samples_analyzed": 123
    }
  ],
  "sample_classifications": [
    {
      "id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
      "evaluation_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
      "sample_identifier": "<string>",
      "classifications": {
        "sentiment": "positive",
        "topic": "technology",
        "length": "medium"
      },
      "created_at": "2023-11-07T05:31:56Z"
    }
  ],
  "started_at": "2023-11-07T05:31:56Z",
  "completed_at": "2023-11-07T05:31:56Z",
  "error_message": "<string>",
  "created_by_email": "<string>",
  "created_at": "2023-11-07T05:31:56Z",
  "duration_seconds": 123,
  "company_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
  "status_display": "<string>",
  "conformant_areas": "<string>",
  "non_conformant_areas": "<string>",
  "trace": {},
  "created_by": 123,
  "updated_at": "2023-11-07T05:31:56Z"
}

Documentation Index

Fetch the complete documentation index at: https://docs.dataframer.ai/llms.txt

Use this file to discover all available pages before exploring further.

Async operation: This endpoint returns immediately with an evaluation ID. Poll GET /api/dataframer/evaluations/{evaluation_id}/ until status is COMPLETED or FAILED.
The run must be in SUCCEEDED status before an evaluation can be created.

Authorizations

Authorization
string
header
required

API Key authentication. Format: "Bearer YOUR_API_KEY"

Body

application/json

Request body for creating an evaluation

run_id
string<uuid>
required

ID of the completed run to evaluate. Run must be in SUCCEEDED status.

evaluation_model
enum<string>
default:anthropic/claude-sonnet-4-6-thinking

AI model to use for evaluation. Defaults to anthropic/claude-sonnet-4-5.

Available options:
anthropic/claude-opus-4-6,
anthropic/claude-opus-4-6-thinking,
anthropic/claude-sonnet-4-6,
anthropic/claude-sonnet-4-6-thinking,
anthropic/claude-haiku-4-5,
anthropic/claude-haiku-4-5-thinking,
openai/gpt-5.4,
openai/gpt-5.4-thinking

Response

Evaluation started successfully

Full evaluation details including distribution analysis and sample classifications

id
string<uuid>
read-only

Unique identifier for the evaluation

run_id
string<uuid>
read-only

ID of the run being evaluated

status
enum<string>

Current status of the evaluation

Available options:
PENDING,
PROCESSING,
SUCCEEDED,
FAILED
conformance_score
number | null

Overall conformance score (0-100) measuring how well generated samples match the spec's expected distributions. Null until evaluation completes.

conformance_explanation
string | null

Human-readable explanation of the conformance score and any notable deviations

distribution_analysis
object[] | null

Per-property comparison of expected vs observed distributions. Null until evaluation completes.

sample_classifications
object[]
read-only

Classification results for each generated sample. Empty until evaluation completes.

started_at
string<date-time> | null
read-only

When evaluation processing started

completed_at
string<date-time> | null
read-only

When evaluation completed

error_message
string | null
read-only

Error message if evaluation failed

created_by_email
string
read-only

Email of the user who created the evaluation

created_at
string<date-time>
read-only

When the evaluation was created

duration_seconds
number | null
read-only

Time taken to complete the evaluation in seconds

company_id
string<uuid>
read-only

ID of the company that owns this evaluation

status_display
string
read-only

Human-readable status display

conformant_areas
string | null
read-only

Description of areas where samples conform well to the spec

non_conformant_areas
string | null
read-only

Description of areas where samples deviate from the spec

trace
object
read-only

Internal trace information including task_id and evaluation model used

created_by
integer
read-only

ID of the user who created this evaluation

updated_at
string<date-time>
read-only

When the evaluation was last updated