Modeling error in experimental assays using the bootstrap principle: Understanding discrepancies between assays using different dispensing technologies

Sonya M. Hanson, Sean Ekins, and John D. Chodera.
Journal of Computer Aided Molecular Design 29:1073, 2015. [DOI] [PDF] // IPython notebook [GitHub] // preprint: [bioRxiv]
Inspired by this In the Pipeline blog post

The drug development community faced a puzzling challenge when a disturbing paper published in PLoS One demonstrated results from the same assay performed with different dispensing technologies both varied wildly and significantly different in magnitude of reported potencies. Inspired by a talk given at the 2014 CADD GRC by Cosma Shalizi on bootstrapping to model error, we show how this simple idea can help explain a large amount of the discrepancy in this assay, and provide simple mathematical tools and an IPython notebook illustrating how easy it is to model the error and bias in experimental assays even when other information about assay reliability is unavailable.