Guides

Create an Evaluation Preset

Save a reusable, task-compatible scoring configuration.

An Evaluation Preset stores Performance metrics, Product module methods and metrics, robustness transforms, and thresholds for one task.

A Preset does not own a Dataset, Model, API Profile, Prediction Set, inference settings, or sampling policy.

Create the Preset

  1. Open Evaluation work -> Evaluation Presets.
  2. Create a Preset and select its task.
  3. Choose Performance metrics.
  4. Add compatible Robustness, Fairness, or Explainability methods and metrics.
  5. Configure transforms and thresholds where required.
  6. Give the Preset a name that describes its scientific purpose.
  7. Save and review the final selection.

Object detection Evaluation Preset editor with Performance and Fog settings.

Watch: Review an Evaluation PresetOpen Helmet Fog 3 and inspect its metrics, transform, and criterion.Silent demo · Subtitles follow page language · Full screen for detail

When a Preset is used before Prediction Generation, Knotest projects only its required prediction coverage. When it is used for Evaluation, Knotest applies its scoring choices and validates them against the selected Prediction Set.

Configure an API Profile

Use a synchronous HTTP JSON inference operation as a reusable Inference Source.

Evaluate Robustness

Measure how reproducible transforms affect clean-input Performance.

On this page

Create the Preset