Model evaluation

Ask a fair question and examine the measurements behind an answer.

Start with a common input basis, choose a comparison that answers your question, and retain the evidence behind the conclusion. These workflows connect preparation, testing, and review.

Read the guide

Prepare a fair comparison

Check the Dataset and task before choosing settings. When a candidate runs through an API, verify its mapping with a sample request first.

Prepare a Dataset for testing

Bring samples and labels into Vault with a clear task and version.

Prepare data

Test a model behind an API

Verify the request and prediction mapping before running the Dataset.

Prepare data

Test the question that matters

Compare model candidates on a shared basis, or examine a supported change in conditions. Keep each Run and its settings identifiable.

Compare model candidates on the same question

Use a common Dataset and test basis to examine candidate differences.

Test models

Test a model under changing conditions

Compare the clean reference with conditions that matter to the intended use.

Test models

Read the examples behind the measurements and carry their limits into the report. Hand over the saved evidence and the questions that remain.

Share findings with their evidence

Give the recipient a result they can understand in context.

Share evidence

Hand an Assessment to another reviewer

Share the saved case, exact evidence, and the work that remains.