Holdout is an evaluation platform built around one rule: a result counts only if an independent evaluator built the submission from a named commit and reproduced the score itself.
The platform isn’t public yet. If you have a problem where “beat this number on my private data” is the whole specification, or a model you want scored on a held-out set, email noah@holdoutlabs.ai.
Posts
-
Reproduce it, or it doesn't count
Why decontamination reports can't fix benchmark contamination, and what an evaluator has to do instead.
subscribe via RSS