A useful candidate comparison starts with the same question and a fair basis. Keep each candidate’s source, settings, results, and costs of interpretation identifiable.
A comparison tied to the recorded measurements and limits
Try this with the assistant
Add the relevant saved records to your chat, then adapt this request to your work.
Help me plan a fair comparison of these model candidates for the stated use. Identify which Dataset version, tests, metrics, and settings must stay common, and which source settings differ. Prepare separate Runs for review and explain limitations before drawing a conclusion.
Define the decision question
State the task, intended use, and criterion. Make the choice depend on the needed behavior rather than selecting a convenient aggregate metric after seeing results.
Hold the test basis common
Use the same reviewed Dataset version, Test plan, and relevant conditions. Preserve the configuration needed for each Model or API profile. Prepare a separate Run for each candidate.
Examine the results together
Review each Run’s measurements, examples, coverage, and report. Explain differences in the source settings or omitted tests when interpreting the comparison.
Record the selected use
If the comparison informs an Assessment, link the selected evidence and record the human conclusion and decision there. Keep the other candidate’s result available for review.
