Assay concordance sets exact ceilings on what one biological score can predict

In plain English

A model cannot consistently predict experiments that do not consistently agree with one another.

A plain-language walk through the work behind the result.

Derived a metric-dependent ceiling from agreement among repeated biological measurements.
Measured concordance across MaveDB, ProteinGym, drug-screen, CRISPR-cell-line, and protease datasets.
Compared published predictor scores with the performance available below each estimated ceiling.

Two assays of one target agreed at correlations of 0.56 to 0.68 in the analyzed registries.
Published predictors reached 63% of achievable correlation performance and 18% of achievable top-1% selection performance in the reported analysis.

Biological-AI benchmarks may misstate model headroom when they ignore disagreement among the experiments used as ground truth.

The record is a preprint and has not completed peer review.
This summary is based on the abstract; methods and supplementary analyses were not independently rechecked.

Study type: Model-free benchmark analysis
Biological system: Variant-effect, protein, drug-screen, and CRISPR assays
Organism: Multiple systems represented in public assay registries
Dataset: 2,822 MaveDB score sets, 217 ProteinGym assays, two drug screens, 1,150 CRISPR cell lines, and repeated protease assays
Sample size: Multiple public registries and assay collections
Model architecture: Support-function identity and concordance estimator
Training objective: Estimate exact metric-dependent prediction ceilings
Code: The authors state that an estimator is provided; location was not verified from the abstract.
Data: Uses named public assay registries

Tasks: Correlation prediction; Top-1% candidate selection

Metrics: Assay concordance; Fraction of achievable performance

Baselines: Published biological predictors

Assay agreement fixed substantial, metric-dependent limits on reported predictor performance.

External validation: Measured across several independent public registries

The record is a preprint and has not completed peer review.
This summary is based on the abstract; methods and supplementary analyses were not independently rechecked.