eval evaluator command group manages evaluators (the judging models that assign scores): browse built-in templates, list and show existing evaluators, derive a new evaluator from a template, and delete evaluators. An evaluator is typically a prompt with a scoring rubric backed by a large-model endpoint; it takes several input fields (such as input, output, reference_output) and produces a score.
All subcommands share
-p, --project <name> (project name, default default) and --json (output raw JSON).evaluator templates
Browse the platform’s built-in evaluator templates as a starting point for creating your own evaluators.evaluator list
List the evaluators in your current project.evaluator show
Show details of an evaluator: the rubric prompt, model configuration, and input field schema.evaluator create
Create an evaluator. The simplest way is to clone the rubric and input schema from a built-in template, then point it at your own model endpoint.
You must provide at least one of
--from-template or --prompt-file.
evaluator delete
Delete an evaluator. (Alias:rm)