Guides

Evaluate Robustness

Measure how reproducible transforms affect clean-input Performance.

Robustness reruns selected Baseline metrics on the same samples after each configured transform. The Evaluation compares each transformed value directly with its matching clean-input Baseline.

Before scoring

The selected Prediction Set must already contain Prediction Evidence for every transform and severity you intend to score. Generate a new Prediction Set if coverage is missing; an existing sealed set cannot be extended.

Configure the limit

Each transform has one Maximum allowed degradation (%) value from 0 to 100. It defaults to 20 and applies to every selected severity for that transform. A degradation equal to the maximum passes.

  • Higher-is-better metric: (Baseline - Transformed) / abs(Baseline) * 100
  • Lower-is-better metric: (Transformed - Baseline) / abs(Baseline) * 100
  • A negative value means the transformed result improved.
  • If both values are zero, degradation is zero.
  • If the Baseline is zero and the transformed value changes, comparison is N/A.

The captured tutorial contains one Fog severity 3 comparison. Add additional severities to Prediction Generation before comparing multiple levels.

Expanded Fog severity 3 result showing Baseline, transformed value, degradation, 100 validation samples, and the configured threshold.

Watch: Inspect the Fog comparisonRead the baseline, transformed score, sample count, and criterion in a completed run.Silent demo · Subtitles follow page language · Full screen for detail

Severity is a selector, not the full reproduction contract. The PDF Report's exact settings show the resolved transform parameters and base seed.

Create an Evaluation Preset

Save a reusable, task-compatible scoring configuration.

Build a PDF Report

Review, customize, and export a module report from a completed Evaluation.

On this page

Before scoringConfigure the limit