WCC's intensive training program starts with the selection of the review team and lead reviewer. Criteria are specified to measure the statistical proficiency of all readers prior to assignment and to set goals for performance.
We look at adjudication rates, apply our statistical approach and use tests such as kappa analysis with confidence intervals to test the performance of the team. To help maintain focus, we provide this data to the lead reviewer and the sponsor so potential issues can be recognized and corrected.
Our approach to retraining is adaptive with retraining being done periodically and on an "as needed" basis. When retraining is required, we select tough to interpret cases rather than randomly selected cases that allow us to better understand reader performance and hone reader skills.
Blinded Independent Central Reviews (BICRs) have the potential to minimize bias and increase the precision of trial data by utilizing multiple independent readers (in a multi-reader BICR). Currently, most multi-reader BICRs are performed primarily for radiological assessments. This is in part attributed to both an understanding of the variability associated with image interpretations and the growth of teleradiology, which allows multiple readers to make rapid assessments of imaging data with software that can automatically send discrepant reads to an adjudicator.
A carefully conducted multi-reader BICR may result in a decrease in inter-reader variability and thus increase the accuracy of the assessment. The inter-reader variability of a small team of independent reviewers can be honed through an iterative process of team training after testing with a focus on difficult cases (i.e., adjudicated cases).
Performance of readers can be explained by their precision and accuracy. When a BICR includes a cohesive team of board-certified, subspecialty-trained practicing physicians, you have the best chance of maximizing the accuracy of an assessment. When coupled with WCC's adaptive training and testing methodology, differences between reviewers are minimized to such a degree that the smallest discrepancies between treatments can be detected.
The BICR process allows you to add precision to different components of the assessment. For example, we usually recommend multi-reader BICR paradigms for primary endpoints or for complex assessments, whereas a single reader BICR paradigm should be sufficient for secondary endpoints. Regardless of the read paradigm, WCC can help titrate the cost value proposition for your BICR.
Assuming both local evaluation (LE) and BICR readers have the same precision and accuracy, an increase in sample size (multi-reader BICR) will decrease variability and result in an increase in precision over the single LE. In a simulation model using a cancer study, in collaboration with Dr. Duan from Brown University, demonstrated that compared to LEs, BICR paradigms decreased overall trial costs and sample sizes. The effect was even greater if the precision and/or accuracy of BICR reviews were better than the site reviewers.
If data validity is not being actively monitored as it is collected, submissions are at risk. Discrepancies build on one another and the data drifts further and further off course if uncorrected. We proactively assess imaging data so potential issues can be identified and corrected as early as possible. Our regular status updates on every trial and intra- and inter- reader reliability testing allows us to report on the performance of readers and even help reduce the rate that errors occur.