> ## Documentation Index
> Fetch the complete documentation index at: https://docs.veadk.xyz/llms.txt
> Use this file to discover all available pages before exploring further.

# Manage evaluators

The `eval evaluator` command group manages evaluators: browse built-in templates, list and show existing evaluators, create evaluators, update TEA drafts, submit TEA versions, and delete evaluators. An evaluator is typically a rubric prompt backed by a judge model; it receives input fields and produces a score or distribution.

<Note>
  `--project` applies only to the Coze evaluation backend; the TEA backend ignores it. TEA version commands are available only on the TEA evaluation backend.
</Note>

## evaluator template list

List built-in evaluator templates. `evaluator templates` is a compatibility entry point equivalent to `evaluator template list`.

| Flag / Argument | Description | Default |
| - | - | - |
| `--type <type>` | TEA built-in template type; use `prompt` or a numeric type. | `prompt` |
| `--locale <locale>` | TEA template locale; `cn` or `zh-CN` appends the Chinese locale. | `zh-CN` |
| `-p, --project <name>` | Coze project name; ignored on TEA. | `default` |
| `--json` | Output raw JSON. | `false` |

```bash lines theme={null}
agentkit eval evaluator template list
agentkit eval evaluator templates --type prompt
```

## evaluator template show

Show a built-in evaluator template, including its prompt and input fields.

| Flag / Argument | Description | Default |
| - | - | - |
| `<key>` | TEA template key, or Coze template ID / name. | Required |
| `--type <type>` | TEA built-in template type; use `prompt` or a numeric type. | `prompt` |
| `--language <type>` | TEA code template `language_type`. | — |
| `--locale <locale>` | TEA template locale; `cn` or `zh-CN` appends the Chinese locale. | `zh-CN` |
| `-p, --project <name>` | Coze project name; ignored on TEA. | `default` |
| `--json` | Output raw JSON. | `false` |

```bash lines theme={null}
agentkit eval evaluator template show relevance
```

<span id="evaluator-template-list-2" />

## evaluator list

List evaluators in the current project or workspace.

| Flag / Argument | Description | Default |
| - | - | - |
| `-p, --project <name>` | Coze project name; ignored on TEA. | `default` |
| `--json` | Output raw JSON. | `false` |

```bash lines theme={null}
agentkit eval evaluator list
```

## evaluator show

Show an evaluator's details: rubric prompt, model configuration, and input fields. On TEA, you can select a specific evaluator version.

| Flag / Argument | Description | Default |
| - | - | - |
| `<id\|name>` | Evaluator ID or exact name. | Required |
| `--evaluator-version <id\|name>` | TEA evaluator version ID or version string, such as `0.0.1`. | — |
| `-p, --project <name>` | Coze project name; ignored on TEA. | `default` |
| `--json` | Output raw JSON. | `false` |

```bash lines theme={null}
agentkit eval evaluator show relevance

agentkit eval evaluator show relevance --evaluator-version 0.0.1
```

## evaluator create

Create an evaluator. You can clone rubric and input schema from a built-in template, or define the evaluator with a custom prompt file.

| Flag / Argument | Description | Default |
| - | - | - |
| `--name <name>` | Evaluator name. | Required |
| `--from-template <key\|id\|name>` | Clone the rubric and schema from a built-in template. TEA uses the template key; Coze uses the template ID or name. | — |
| `--model <name\|id>` | Judge model. TEA accepts `Doubao 2.0 Lite`, `Doubao 2.0 Pro`, `Doubao 2.0 Mini`, or model IDs `1`, `2`, `3`; Coze uses an Ark endpoint ID. | — |
| `--prompt-file <path>` | Custom rubric text file. | — |
| `--description <text>` | Evaluator description. | — |
| `--input-schemas <keys>` | Comma-separated input field keys; used only on TEA. | — |
| `-p, --project <name>` | Coze project name; ignored on TEA. | `default` |
| `--json` | Output raw JSON. | `false` |

On Coze, provide at least one of `--from-template` or `--prompt-file`. On TEA, you can create from a template or from a custom prompt and input fields.

```bash lines theme={null}
agentkit eval evaluator create \
  --name relevance \
  --from-template relevance \
  --model "Doubao 2.0 Pro"
```

Create `rubric.txt` before running the custom evaluator example:

```text title="rubric.txt" lines theme={null}
Question: {{input}}
Reference answer: {{reference_output}}
Actual answer: {{output}}
Check whether the actual answer correctly answers the question and matches the reference
Score 1 for a correct and complete answer, otherwise 0, and briefly explain the score
```

```bash lines theme={null}
agentkit eval evaluator create \
  --name custom-score \
  --prompt-file ./rubric.txt \
  --input-schemas input,output,reference_output \
  --model 2
```

<Tip>
  Reference input fields in the prompt with placeholders like <code>{'{{input}}'}</code>, <code>{'{{output}}'}</code>, and <code>{'{{reference_output}}'}</code>; these field names are the evaluator's input schema and are mapped automatically during [`eval run`](/productions/agentkit-cli/preview/en/commands/eval/run).
</Tip>

## evaluator update-draft

Update a TEA evaluator draft. When `--evaluator-content-file` is passed, that JSON file is used as the complete evaluator content and overrides `--prompt-file`, `--model`, and `--input-schemas-json`.

<Note>
  `evaluator update-draft` is supported only on the TEA evaluation backend.
</Note>

| Flag / Argument | Description | Default |
| - | - | - |
| `<id>` | Evaluator ID. | Required |
| `--prompt-file <path>` | Custom rubric text file. | — |
| `--model <name\|id>` | Judge model: `Doubao 2.0 Lite`, `Doubao 2.0 Pro`, `Doubao 2.0 Mini`, or model IDs `1`, `2`, `3`. | — |
| `--input-schemas-json <json>` | JSON array of input field schemas. | — |
| `--evaluator-content-file <path>` | Complete `evaluator_content` JSON file. | — |
| `--evaluator-type <type>` | Evaluator type; unchanged when omitted. | Unchanged |
| `--json` | Output raw JSON. | `false` |

```bash lines theme={null}
agentkit eval evaluator update-draft <evaluator-id> \
  --prompt-file ./rubric.txt \
  --input-schemas-json '[{"key":"input"},{"key":"output"},{"key":"reference_output"}]'
```

## evaluator version list

List TEA evaluator versions.

| Flag / Argument | Description | Default |
| - | - | - |
| `--evaluator <id\|name>` | Evaluator ID or exact name. | Required |
| `-r, --region <region>` | Volcengine region. | Environment |
| `--json` | Output raw JSON. | `false` |

```bash lines theme={null}
agentkit eval evaluator version list --evaluator relevance
```

## evaluator version submit

Submit a TEA evaluator draft as a version.

| Flag / Argument | Description | Default |
| - | - | - |
| `<version>` | Version string, such as `0.0.2`. | Required |
| `--evaluator <id\|name>` | Evaluator ID or exact name. | Required |
| `--description <text>` | Version description. | — |
| `--desc <text>` | Alias for `--description`. | — |
| `-r, --region <region>` | Volcengine region. | Environment |
| `--json` | Output raw JSON. | `false` |

```bash lines theme={null}
agentkit eval evaluator version submit 0.0.2 --evaluator relevance --description "Update rubric"
```

## evaluator delete

Delete an evaluator. (Alias: `rm`)

<Warning>
  This command deletes the evaluator. Check the target first; the CLI prompts for confirmation unless you pass `--yes`.
</Warning>

| Flag / Argument | Description | Default |
| - | - | - |
| `<id\|name>` | Evaluator ID or exact name. | Required |
| `-p, --project <name>` | Coze project name; ignored on TEA. | `default` |
| `-y, --yes` | Skip the confirmation prompt. | `false` |

```bash lines theme={null}
agentkit eval evaluator delete relevance -y
```

After creation, use `evaluator show` to inspect the model, rubric, and input fields. Submit a TEA draft before using an explicit version in an experiment. Submit a new version after further draft edits; editing a draft does not update an existing version

```bash lines theme={null}
agentkit eval evaluator show relevance
agentkit eval evaluator version submit 0.0.1 --evaluator relevance
agentkit eval evaluator version list --evaluator relevance
```

Template keys and models must be available on the selected backend. Coze model arguments use an Ark endpoint ID available to your account, not a TEA numeric model ID