Insilico Medicine has launched an open consortium and benchmark index designed to improve transparency around how artificial intelligence drug discovery tools are evaluated.

AI drug discovery benchmarks

Insilico Medicine has launched the Open Drug Discovery and Development Consortium (O3DC), an open initiative focused on how computational drug discovery methods are evaluated.

The consortium’s first resource, the O3DC Benchmark Index, brings together hundreds of benchmarks across areas including molecular property prediction, docking and pose prediction, binding affinity, generative design, retrosynthesis, target identification, clinical prediction and AI agent evaluation.

The index records what each benchmark measures, who maintains it, where its code is hosted and whether the associated repository remains active. It also includes a section identifying known limitations, biases or published criticisms associated with individual benchmarks.

O3DC is open to researchers from academia, industry and independent research, with no membership fee or exclusivity requirement. The initiative allows contributors to suggest new benchmarks, update existing entries and challenge information recorded in the index.

The resource is intended to address concerns that benchmark scores can provide an incomplete picture of how well AI systems perform in real-world drug discovery.

Benchmark limitations

The consortium highlights several examples where established benchmarks can have limitations that affect how results should be interpreted.

DUD-E, which has been widely used to assess virtual screening methods, has been criticised because models can distinguish active compounds from decoys using ligand-level features without necessarily learning meaningful protein-target interactions.

The index also highlights concerns around PDBbind and CASF, including potential overlap between datasets used for training and evaluation that could affect reported model performance.

Another example is PoseBusters, which has been used to assess whether predicted protein-ligand poses meet basic chemical and physical requirements. Its work has highlighted cases where models can perform well on conventional structural metrics while generating poses with physically implausible characteristics.

Alex Zhavoronkov, founder and co-CEO of Insilico Medicine, said: “Benchmarks define what our field counts as progress.”

He added: “O3DC gives that knowledge a home-right alongside the benchmark itself.”

The consortium said each benchmark entry includes a public discussion thread where maintainers and users can question or challenge recorded information.

Open approach to AI evaluation

Insilico has also included its own benchmarking tools in the index. The company said this is intended to apply the same scrutiny to its own resources as to benchmarks developed by other organisations.

The entry for Insilico’s DDD Benchmark, for example, records that its reference baselines are proprietary and therefore cannot be independently inspected.

The approach is intended to make limitations visible alongside benchmark results rather than leaving researchers to identify potential weaknesses independently.

The launch comes as pharmaceutical companies and technology developers increasingly use generative AI and foundation models for tasks across drug discovery, including target identification, molecular design, property prediction and clinical development.

As these systems become more capable, benchmark selection and data quality can influence how their performance is assessed. A high score on a benchmark does not necessarily demonstrate that an AI system can deliver equivalent performance in a real drug discovery programme.

O3DC therefore aims to provide a central resource where researchers can compare benchmarks and assess their limitations before using them to evaluate AI systems.

The consortium’s benchmark index is available publicly and is designed to be updated as repositories, datasets and published evidence change.