A blinded, prospective benchmark of in silico antibody discovery anchored to experimental affinity and developability

In plain English

The AIntibody challenge compared 511 AI-designed or predicted antibodies from 29 organizations under shared experimental affinity and developability assays. Several teams produced developable antibodies below 100 pM affinity, yet those wins were exceptions: performance often failed to transfer to other tasks, and nearly every model performed worse than random clone picking when ranking high-affinity antibodies within HCDR3 clusters.

What they found

• Several groups produced developable antibodies with affinities below 100 pM, especially in the affinity-maturation setting.
• Cross-task generalization was weak, and all but one model underperformed random clone picking on the within-cluster ranking task.

The catch

• The first challenge iteration evaluated one antigen, SARS-CoV-2 receptor-binding domain.
• The organizing consortium also reports the results, and blinding relied on organizer integrity rather than informatic safeguards.
• The supplied datasets were unusually rich, so the authors describe the results as an upper bound for this target class rather than evidence of general-purpose performance.

Methods at a glance

Study type

Prospective, blinded, experimentally validated benchmark

Biological system

Full-length IgG antibodies against SARS-CoV-2 receptor-binding domain

Organism

Not applicable; recombinant antibodies were evaluated in vitro

Dataset

AIntibody challenge sequencing inputs, model submissions, and experimental assay outputs

Sample size

511 antibodies submitted by 29 organizations across three tasks

Model architecture

Multiple participant methods; the benchmark does not evaluate a single shared architecture

Training objective

Task-specific antibody affinity optimization, ranking, or CDR design

Code

Participant repositories are linked from the article's code-availability statement.

Data

Challenge submissions, affinity measurements, developability results, and processed sequencing inputs are provided with the article.

Tasks and metrics

Tasks: in silico affinity maturation; within-HCDR3-cluster affinity ranking; out-of-library CDR design

Metrics: experimentally measured binding affinity; rank performance; five-assay developability score

Baselines: random clone picking; standard experimental selections

Principal findings

• Several groups generated developable antibodies with measured affinities below 100 pM.
• Successes did not consistently transfer across tasks.
• All but one method performed worse than random clone picking for within-cluster high-affinity selection.

What remains unresolved

External validation: Submissions were synthesized and evaluated with shared affinity and developability assays under the challenge protocol.

• The benchmark used a single, extensively characterized antigen.
• The organizers also authored the report, and blinding did not use informatic safeguards.
• Unusually rich input data may overstate performance in less controlled discovery settings.