A mechanism-annotated benchmark reveals limited fidelity to drug-response signatures in single-cell perturbation models

In plain English

A single-cell model can reconstruct a plausible-looking expression profile while getting the direction of important drug-response genes wrong.

A plain-language walk through the work behind the result.

Linked control and drug-treated single-cell profiles to literature-curated directional gene evidence.
Built a Mechanism Fidelity Score spanning direction, effect size, coherence, specificity, and pathway polarity.
Evaluated 12 perturbation models, three baselines, and ten data splits.

Expression-similarity metrics were weakly aligned with mechanism fidelity.
Frozen foundation-model embeddings produced local gains rather than consistent improvements.
Mechanism-aware selection improved early drug retrieval in the reported transcriptome-based evaluation.

Drug-perturbation models need tests that reward biological mechanism, not merely reconstruction of average expression patterns.

The record is a preprint and has not completed peer review.
This summary is based on the abstract; methods and supplementary analyses were not independently rechecked.

Study type: Mechanism-annotated benchmark study
Biological system: Single-cell drug perturbation responses
Organism: Multiple cell-line and source-integrated datasets
Dataset: 181 datasets, 423 response cases, 717 key genes, and 2.5 million cells
Sample size: 2.5 million cells across 181 datasets
Model architecture: Benchmark covering 12 perturbation-prediction models and frozen foundation-model embeddings
Training objective: Predict transcriptional response to drug perturbation
Code: Not reported in the abstract
Data: The abstract describes a mechanism-annotated benchmark; release details were not verified.

Tasks: Response prediction; Mechanism fidelity; Drug retrieval

Metrics: Mechanism Fidelity Score; Expression similarity; Early drug retrieval

Baselines: Three reported baselines; Expression-similarity model selection

Expression reconstruction was a weak proxy for preservation of directional drug-response mechanisms.

External validation: Ten data splits and source-integrated settings were reported

The record is a preprint and has not completed peer review.
This summary is based on the abstract; methods and supplementary analyses were not independently rechecked.