Define outcomes

An outcome is a session’s definition of done: an objective and a rubric. Each time the agent finishes its work, an independent grader checks it against every criterion in the rubric and either accepts it or sends the agent back with the failed criteria and the reasons. Use outcomes when “the agent stopped” is not proof enough that the work is right. A session with an outcome only reports execution_state completed after the grader accepts the work. The TypeScript samples assume the rl client from Client setup.

Before you begin

You need the organization developer or admin role to start a session or add an outcome to one. The organization user role can read outcomes and grading results. See Organizations and roles and API keys.
You need an agent and an environment. See Agents and Environments.
Decide what the agent must produce. If it is a file, pick its name now (for example release-notes.md). You will use that name in the task and in the rubric.

How grading works

The agent works until it ends a turn with nothing queued. That is its claim that the work is done. A turn you stop with an interrupt or a cancel is not graded.
Deliverables the agent saved are kept, then the grader starts a grading pass. The grader is a separate model run on the session’s model, with its own context. It sees the objective and the rubric, never the agent’s reasoning. It works in the same sandbox with the same tools and credentials, so it can open files, run the project’s tests, and check external systems the agent changed.
The grader gives each criterion exactly one verdict. The result of the pass is computed from those verdicts, not chosen by the grader: one fail is enough to send the work back.
On needs_revision, the agent receives the full scorecard, with a reason for every failed criterion, and starts another turn. Then step 1 repeats.

What success means: the outcome’s status is terminal with terminal_result satisfied, and the session’s execution_state is completed with stop_reason outcome_satisfied. Nothing else marks a graded session as completed.

Write a rubric

A rubric is Markdown. The grader owes a verdict on every list item and on nothing else.

Markdown	How it is read
List item (`-`, `*`, `+`, `1.`, `1)`)	One criterion. A task box such as `- [ ]` is stripped. Nested list items are criteria too.
Heading (`#` to `######`)	The section for the criteria below it, returned as `section`.
Paragraph	Context for the grader. Not scored, and not shown to the agent.
Fenced code block	An example. List items inside a fence are not criteria.
Horizontal rule (`---`)	Ignored.

Write each criterion on one line. A wrapped second line is read as a paragraph, so the criterion loses that text.

# Release notes for v4.12.0

The notes cover every pull request merged between the tags v4.11.0 and v4.12.0.
Customers read them, so internal tracker ids do not belong in the text.

## Deliverable
- The notes are saved as a deliverable named `release-notes.md`.

## Coverage
- Every merged pull request in the range appears exactly once.
- No entry describes a pull request outside the range.

## Format
- Breaking changes are listed first, under a heading named "Breaking changes".
- Each entry is one sentence and ends with the pull request number in parentheses.

This rubric has five criteria in three sections. The two paragraphs tell the grader how to find the range and who reads the notes. All criteria count the same. You cannot set weights (every criterion reports weight: 1), and a strong result on one criterion never offsets a failure on another. not_applicable does not count as a failure.

Name the deliverable

Only files the agent saves under /workspace/.managed-agents/outputs are kept as deliverables. Files anywhere else, including paths you invent in a prompt, are not kept. Agents already know where to save deliverables, so you do not need to give a path. Name the file in the task and in the rubric, for example “Save the notes as release-notes.md” and ”- The notes are saved as a deliverable named release-notes.md.” Deliverables are saved before every grading pass, so the grader checks the version the agent just finished. See Deliverables and artifacts.

Tips for writing rubrics

Make each criterion checkable on its own. “The CSV has a numeric price column” can be checked. “The data looks good” cannot, and a criterion like that causes revision loops that do not end.
Give one requirement to each list item. When one item holds two requirements, a failure does not say which one the work missed.
Put facts the agent needs where it can see them. The agent sees the objective and the criteria, not the paragraphs. Put background in the task message.
Start from a good example. Ask a model what makes a known-good result good, and turn the answer into criteria.
Cap a new rubric. Set max_iterations until you know the rubric can be met. A criterion that can never pass keeps an uncapped session revising.
Keep the text stable. Criterion ids stay the same only while the rubric text is identical. See Follow a criterion across sessions.

Outcome fields

These fields go in outcome on startSession, or in the body of defineSessionOutcome.

Field	Required	Description
`description`	Yes	The objective. The grader measures the work against it. If the start request has no `message`, the agent also starts from it.
`rubric`	One of `rubric` or `rubric_ref`	The rubric as one Markdown string, with 1 to 200 criteria.
`rubric_ref`	One of `rubric` or `rubric_ref`	The rubric as a file from your library: `{"type": "file", "file_id": "..."}`.
`max_iterations`	No	The largest number of grading passes. Omit it or send `0` to revise until the rubric is satisfied (the default). Negative values are rejected.

To keep a long or shared rubric in one place, upload it as a text or YAML file of up to 256 KiB and send rubric_ref instead of rubric. Its text is read once, when the outcome is defined, and returned as the outcome’s rubric, so later edits to the file don’t change a running session. An unknown or expired file returns 404; a file that isn’t UTF-8 text, is too large, or holds no criterion returns 400.

Start a session with an outcome

Send outcome in the start request. Grading is on from the first turn.

Console
TypeScript
cURL

Open the agent and click Start session, or click Launch session on the Sessions tab.
In Launch a session, choose the Agent and Environment.
Under Task, select Grade this session against a rubric.
Write the criteria in Rubric.
Write the task in Opening message. The console sends it as the objective.
Click Launch session.

The console never sets a cap, so the session revises until the rubric is satisfied.

const { session_id } = await rl.managedAgents.startSession({
  'Idempotency-Key': 'release-notes-v4-12-0',
  body: {
    agent_id: '5f0c2a1e-8b7d-4c3a-9e21-6d4f0b9a7c55',
    environment_id: '9d3e7b52-1a4c-4f80-b6e9-2c8a5d0f7e13',
    message: 'The repository is at /workspace/app. Compare the tags v4.11.0 and v4.12.0.',
    outcome: {
      description: 'Release notes for v4.12.0, saved as release-notes.md.',
      rubric: rubricMarkdown,
      max_iterations: 4,
    },
  },
});

curl -X POST 'https://api.recursion.labelbox.com/managed-agents/v1/sessions' \
  -H "Authorization: Bearer $RECURSION_API_KEY" \
  -H 'Idempotency-Key: release-notes-v4-12-0' \
  -H 'Content-Type: application/json' \
  -d '{
    "agent_id": "5f0c2a1e-8b7d-4c3a-9e21-6d4f0b9a7c55",
    "environment_id": "9d3e7b52-1a4c-4f80-b6e9-2c8a5d0f7e13",
    "message": "The repository is at /workspace/app. Compare the tags v4.11.0 and v4.12.0.",
    "outcome": {
      "description": "Release notes for v4.12.0, saved as release-notes.md.",
      "rubric": "## Deliverable\n- The notes are saved as a deliverable named `release-notes.md`.\n\n## Format\n- Breaking changes are listed first, under a heading named \"Breaking changes\".",
      "max_iterations": 4
    }
  }'

The API answers 202 Accepted. The session is accepted, not finished. Keep session_id to read the grading.

{
  "session_id": "b7e1c9a4-3d62-4f15-8a07-5c2e9f6d1b38",
  "status_path": "/v1/sessions/b7e1c9a4-3d62-4f15-8a07-5c2e9f6d1b38"
}

With max_iterations: 4, the grader runs at most four passes, so the agent gets up to three revisions.

Choose the message and the objective

message and outcome.description are separate. The agent starts from the message. The grader measures against the objective and never sees the message. Use the message for facts the agent needs that are not part of the bar, such as where the repository is.

You send	The agent starts from	The grader measures against
`outcome` only	`outcome.description`	`outcome.description`
`outcome` and `message`	`message`	`outcome.description`
`message` only, agent has a default rubric	`message`	`message`, with the agent’s default rubric
`message` only, agent has no default rubric	`message`	Nothing. The session is not graded.

Keep the two consistent. If they disagree, the agent works toward one target and is graded against another. The agent can see the objective and the criteria for the whole session, not just in its first turn.

Grade every session of an agent

Store a rubric on the agent as default_rubric to grade every session started from it. In the console, the field is Default rubric in the Outcome section of the agent’s Configuration tab. Through the API, send default_rubric on createAgent or createAgentVersion. It must have 1 to 200 criteria. See Agents.

Start request	Result
`message`, no `outcome`	Graded against the default rubric. The message is the objective.
No `message` and no `outcome`	Not graded. There is no task to grade.
An `outcome`	Graded against that outcome. The default rubric is ignored as a whole, never merged.
`skip_default_outcome: true`	Not graded.

In Launch a session, an agent with a default rubric has Grade this session against a rubric selected and shows the stored rubric. Clear the box to run ungraded. An edit in the form applies to that session only and gives its criteria new ids.

Add an outcome to a running session

Call defineSessionOutcome to grade a session that started without an outcome, or to set the next objective after an outcome ended. The body has the same fields as outcome. There is no console action for this. A session runs one outcome at a time. While an outcome is not terminal, a second one is rejected.

TypeScript
cURL

const outcome = await rl.managedAgents.defineSessionOutcome({
  session_id: 'b7e1c9a4-3d62-4f15-8a07-5c2e9f6d1b38',
  body: {
    description: 'A one-page summary of v4.12.0 for the support team, saved as support-summary.md.',
    rubric:
      '- The summary is saved as a deliverable named `support-summary.md`.\n- It lists every breaking change from release-notes.md.',
  },
});

curl -X POST 'https://api.recursion.labelbox.com/managed-agents/v1/sessions/b7e1c9a4-3d62-4f15-8a07-5c2e9f6d1b38/outcomes' \
  -H "Authorization: Bearer $RECURSION_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{
    "description": "A one-page summary of v4.12.0 for the support team, saved as support-summary.md.",
    "rubric": "- The summary is saved as a deliverable named `support-summary.md`.\n- It lists every breaking change from release-notes.md."
  }'

The response is the stored outcome. Some fields are omitted.

{
  "outcome_id": "4a9d2f70-6c1b-4e83-9f25-8b0e3a7c5d14",
  "session_id": "b7e1c9a4-3d62-4f15-8a07-5c2e9f6d1b38",
  "agent_id": "5f0c2a1e-8b7d-4c3a-9e21-6d4f0b9a7c55",
  "description": "A one-page summary of v4.12.0 for the support team, saved as support-summary.md.",
  "rubric": "- The summary is saved as a deliverable named `support-summary.md`.\n- It lists every breaking change from release-notes.md.",
  "max_iterations": 0,
  "status": "pending",
  "created_at": "2026-09-17T11:40:02Z",
  "updated_at": "2026-09-17T11:40:02Z"
}

The new objective and criteria join the agent’s instructions, and a message naming the objective appears in the transcript. Defining an outcome does not start a turn by itself. If the session is idle, send a message so the agent starts working toward it. See Session operations.

Read the grading

listSessionOutcomes returns every outcome of a session, oldest first, each with all of its grading passes.

Console
TypeScript
cURL

On the Sessions tab, open the session.
Read the Outcome line above the transcript: the current result, the number of unmet criteria, and the start of the objective.
Click Details to see the objective, the cap (revises until satisfied, or at most N revisions where N is max_iterations), the Rubric, and one block per pass, newest first (Pass 1 is the first pass). Each pass shows its result, how many criteria were met, its cost and tokens, and every criterion with its verdict. A failed criterion also shows the grader’s reason.

const { outcomes } = await rl.managedAgents.listSessionOutcomes({
  session_id: 'b7e1c9a4-3d62-4f15-8a07-5c2e9f6d1b38',
});
const latest = outcomes?.at(-1);
console.log(latest?.status, latest?.terminal_result);

curl 'https://api.recursion.labelbox.com/managed-agents/v1/sessions/b7e1c9a4-3d62-4f15-8a07-5c2e9f6d1b38/outcomes' \
  -H "Authorization: Bearer $RECURSION_API_KEY"

In this response, the first pass sent the work back and the second accepted it. Some fields are omitted.

{
  "outcomes": [
    {
      "outcome_id": "e5c07a3b-1f94-4d26-b8a0-6d2c9e4f7b51",
      "session_id": "b7e1c9a4-3d62-4f15-8a07-5c2e9f6d1b38",
      "description": "Release notes for v4.12.0, saved as release-notes.md.",
      "rubric": "## Deliverable\n- The notes are saved as a deliverable named `release-notes.md`.\n\n## Format\n- Breaking changes are listed first, under a heading named \"Breaking changes\".",
      "max_iterations": 4,
      "status": "terminal",
      "terminal_result": "satisfied",
      "created_at": "2026-09-17T10:15:03Z",
      "updated_at": "2026-09-17T10:31:47Z",
      "ended_at": "2026-09-17T10:31:47Z",
      "evaluations": [
        {
          "outcome_id": "e5c07a3b-1f94-4d26-b8a0-6d2c9e4f7b51",
          "iteration": 0,
          "result": "needs_revision",
          "explanation": "1 of 2 criteria not met.",
          "criteria": [
            {
              "criterion_id": "r3fa91c07b2de.c001",
              "criterion_text": "The notes are saved as a deliverable named `release-notes.md`.",
              "section": "Deliverable",
              "weight": 1,
              "verdict": "pass",
              "rationale": "release-notes.md is in the outputs folder and has 41 lines.",
              "evidence_event_ids": ["7b2e4c10-5a9d-4f63-8e01-3c6d9a2f5b74"]
            },
            {
              "criterion_id": "r3fa91c07b2de.c002",
              "criterion_text": "Breaking changes are listed first, under a heading named \"Breaking changes\".",
              "section": "Format",
              "weight": 1,
              "verdict": "fail",
              "rationale": "The removal of the v1 export endpoint appears under \"API\", after two feature sections. There is no \"Breaking changes\" heading.",
              "evidence_event_ids": ["7b2e4c10-5a9d-4f63-8e01-3c6d9a2f5b74"]
            }
          ],
          "grader_thread_id": "a3d5e7f9-0b2c-4d6e-8f1a-3b5c7d9e1f20",
          "input_tokens": 18420,
          "output_tokens": 912,
          "cost_micros": 68950,
          "started_at": "2026-09-17T10:21:30Z",
          "ended_at": "2026-09-17T10:23:02Z"
        },
        {
          "outcome_id": "e5c07a3b-1f94-4d26-b8a0-6d2c9e4f7b51",
          "iteration": 1,
          "result": "satisfied",
          "explanation": "All 2 criteria met.",
          "grader_thread_id": "c8e2a4f6-1d3b-4e5f-9a7c-2b4d6f8a0c13",
          "started_at": "2026-09-17T10:30:11Z",
          "ended_at": "2026-09-17T10:31:47Z"
        }
      ]
    }
  ]
}

Field	What it tells you
`status`, `terminal_result`	Whether grading is finished and how it ended. Poll until `status` is `terminal`.
`evaluations`	The grading passes, oldest first. `iteration` starts at `0`. Empty until the first pass ends.
`evaluations[].result`	The result of that pass. See the tables below.
`evaluations[].explanation`	A one-line summary, such as `1 of 2 criteria not met.` The agent receives the full scorecard, with every verdict and the reason for each failure.
`criteria[].verdict`, `criteria[].rationale`	The verdict for one criterion and the grader’s reason. A `fail` always has a reason. Criteria come back in rubric order.
`criteria[].evidence_event_ids`	The grader’s own tool calls in that pass, such as commands it ran and files it read. They are attached by the service, not written by the grader, so they cannot be invented.
`grader_thread_id`	The grader’s thread for that pass. It appears in the session tree, so you can read everything the grader did. See Events.
`cost_micros` and token fields	What the pass cost, in millionths of a US dollar, and its tokens. See Cost of grading.

When a session has no outcome, outcomes is null. Each pass also writes an outcome_evaluation event to the session, so you can follow the loop live on the event stream.

Results and statuses

Criterion verdict	Meaning
`pass`	The work meets the criterion.
`fail`	The work does not meet it. `rationale` says why.
`not_applicable`	The criterion does not apply to this work. Not a failure.

Pass result	When	What happens next
`needs_revision`	At least one criterion failed and passes remain.	The agent revises, then the grader runs another pass.
`satisfied`	No criterion failed.	The outcome ends. The session is `completed` with `stop_reason` `outcome_satisfied`.
`max_iterations_reached`	A criterion failed on the last pass `max_iterations` allows.	The outcome ends. The session goes `idle` with `stop_reason` `max_iterations`. The last pass keeps its verdicts.
`failed`	The rubric could not be graded.	The outcome ends. The session goes `idle` with `stop_reason` `outcome_unmet`. Fix the rubric and define a new outcome.

Every result except needs_revision is final, and becomes the outcome’s terminal_result.

Outcome `status`	Meaning
`pending`	No grading pass has run yet.
`running`	The agent is working toward the outcome.
`evaluating`	A grading pass is in progress.
`terminal`	Grading is finished. Read `terminal_result`.

If the grader cannot produce a usable verdict, the session goes idle with stop_reason grader_failed. The work is not marked as failed and the outcome stays open. Send a message to continue; the next time the agent finishes, grading runs again. After a final result, the session still accepts follow-up messages. Those turns are not graded. To grade more work, add a new outcome.

Follow a criterion across sessions

A criterion id looks like r3fa91c07b2de.c001. The first part identifies the rubric and the last part is the criterion’s position in it. The ids are stable: the same rubric text gives the same ids in every session, so you can track how one criterion fares across runs and agent versions. Any edit to the rubric, even a typo fix, gives every criterion a new id. A default rubric stored on the agent keeps its ids for as long as you do not change it.

Cost of grading

Each grading pass is a model run with its own tokens, and its cost is counted like the agent’s own work. Every pass reports its cost_micros and token counts. Grading passes are child sessions of the graded session, so their cost appears in the subtree and tree scopes of the model-cost reads, not in self. See Usage and cost. To keep grading costs down, write criteria the grader can check quickly, and set max_iterations on rubrics you have not tested yet. An uncapped outcome whose rubric can never pass keeps revising and grading.

What can go wrong

Symptom or code	Cause	Fix
`400 invalid_request` on `outcome.description`	The objective is empty.	Send `description`.
`400 invalid_request` on `outcome.rubric`	The rubric is missing, has no list item, or has more than 200 criteria.	Write each criterion as a Markdown list item. Paragraphs alone are not criteria. Split very large rubrics.
`400 invalid_request` on `outcome.max_iterations`	The value is negative.	Send `0`, a positive integer, or nothing.
`400 invalid_request` on `outcome` from `defineSessionOutcome`	The session already has an outcome that is not `terminal`.	Wait for `terminal_result`, then define the next one.
`403 forbidden`	Your role can only read sessions.	See Organizations and roles. Use an account or key with the developer or admin role.
`404 not_found`	The session does not exist or is outside your organization.	Check the session id and your API key’s organization.
The deliverable criterion fails because the file can’t be found	The agent saved the file outside the outputs folder, or under another name.	Name the deliverable file in both the task and the rubric. See Deliverables and artifacts.
The session keeps revising	A criterion is vague or impossible, so it never passes.	Read the failing `rationale`, rewrite the criterion, and set `max_iterations`.
`stop_reason` is `grader_failed`	The grader did not return a usable verdict. The work was not judged.	Send a message to resume; grading runs again when the agent finishes.
`outcomes` is `null`	The session was started without an outcome, or with a default rubric but no `message`.	Send `outcome`, or send a `message` when relying on the agent’s default rubric.
Criterion history does not line up across sessions	The rubric text differed, so the ids differ.	Store the rubric as the agent’s `default_rubric` and reuse it unchanged.
`429 rate_limit_exceeded`	Too many requests.	Wait for the time in `Retry-After`, then retry.

The full list of codes is in Errors.

Limits

Limit	Value
Criteria in one rubric	1 to 200
Open outcomes in one session	1
`max_iterations`	`0` (no cap, the default) or any positive integer
Criterion weights	Not settable. Every criterion has weight `1`.

Other limits are on Limits.

Next steps

Deliverables and artifacts

Ask for named deliverables the grader can check.

Events

Follow each grading pass live and read what the grader did.

Agents

Store a default rubric so every session is graded.

Usage and cost

See what the agent and the grader cost.

Get started

Account and access

Build agents

Coordinate multiple agents

Configure environments

Run and observe sessions

Manage agent context

Automate

Tutorials

Before you begin

How grading works

Write a rubric

Name the deliverable

Outcome fields

Start a session with an outcome

Choose the message and the objective

Grade every session of an agent

Add an outcome to a running session

Read the grading

Results and statuses

Follow a criterion across sessions

Cost of grading

What can go wrong

Limits

Next steps

Deliverables and artifacts

Events

Agents

Usage and cost

​Before you begin

​How grading works

​Write a rubric

​Name the deliverable

​Outcome fields

​Start a session with an outcome

​Choose the message and the objective

​Grade every session of an agent

​Add an outcome to a running session

​Read the grading

​Results and statuses

​Follow a criterion across sessions

​Cost of grading

​What can go wrong

​Limits

​Next steps

Deliverables and artifacts

Events

Agents

Usage and cost

Before you begin

How grading works

Write a rubric

Name the deliverable

Outcome fields

Start a session with an outcome

Choose the message and the objective

Grade every session of an agent

Add an outcome to a running session

Read the grading

Results and statuses

Follow a criterion across sessions

Cost of grading

What can go wrong

Limits

Next steps