Dual Screening in Your Systematic Review Process: Is it Worth the Time?

Birds eye view of two laptops

Dual independent screening in your systematic review process is undoubtedly considered a best practice. But for some research teams, a lack of resources precludes this from being a standard operating procedure. 

New evidence reinforces the value in dual screening, and why it's essential to have at least some kind of process in place to mitigate the risk of using only one screener. Let's take a closer look.

The Evidence

The major risk that research teams take when only using one screener is that relevant studies will be missed, potentially invalidating the review. 

Siw Waffenschmidt, Marco Knelangen, Wiebke Sieben, Stefanie Bühn, and Dawid Pieper published a methodological systematic review, which included 4 studies comparing single versus dual independent screening. They found that single screeners missed a median of 5% of studies that would have been included by two independent reviewers reaching consensus. Although reviewer experience may affect this error rate, they also report that even reviewers with more experience missed a median of 3% of relevant studies.  

A major question, of course, is: does this matter? Are the added resource needs required to conduct the screening process fully in duplicate going to affect the results or conclusions of your review? The answer is not yet clear. Not only did the error rate vary widely between comparisons (0-58%), the authors report that, of 7 meta-analyses examining the effect of missed studies, the findings of 3 would have been “substantially” changed, while missed studies had little or no impact on the remaining four.

Another recent study by Gartlehner G, Affengruber L, Titscher V, Noel-Storr A, Dooley G, Ballarini N, König F suggests a similar pattern and reports that single reviewer screening does not "fulfill the high methodological standards that decision-makers expect from systematic reviews."

A study published by Zhen Wang, Tarek Nayfeh, Jennifer Tetzlaff, Peter O’Blenis, and Mohammad Hassan Murad suggests that both single and double reviewer methodologies produce errors. They state that it's important to note that the gold standard (independent dual screening) is not perfect, and therefore automating systematic review processes to the degree of human accuracy is sufficient when it comes to taking measures to save time and resources in a review. 

Balancing Your Resources and Risk Tolerance

What does this mean for your systematic review process? Dual independent screening is still supported as the gold standard. Individual reviewers make mistakes and these mistakes may affect your ability to draw valid conclusions. However, time and resources may still make this a challenge.

The good news is that additional resource requirements for dual screening can be minimized and efficiencies gained by using systematic review software to facilitate this process.

How Systematic Review Software Makes Screening Easier

For research teams looking to make their systematic review process more efficient without sacrificing validity, systematic review software offers many advantages. Using technology frees up more resources and makes the review process faster and more cost-effective.

Some of the main ways systematic review tools help research teams are:

  • Automatic tracking of inclusions/exclusions for improved transparency and simplified management of records throughout your workflow
  • Editing, time stamping, and version control to maintain a clear audit trail
  • Instant generation of exclusion lists and PRISMA Diagrams
  • Rapid title screening, deduplication, and artificial intelligence 

Dual Screening Alternatives

Still limited to one screener? Does your particular topic or process have a higher risk-threshold for missed studies? Most of the above options help minimize the risk and increase efficiencies. Teams will also occasionally employ alternate options such as the following:

One Screener with Verification: This method consists of one person doing the screening, and a second person double-checking the first person's work. 

Quality control: Teams will occasionally conduct dual review for a sub-sample of records (for instance a random sample) to estimate if error rates are within acceptable bounds for their specific review.

Dual Screen Exclusions Only: In this case, preliminary screening levels (e.g. title/abstract only) require dual review only for exclusions. A record labelled for inclusion by either reviewer would be passed on to the next level of screening without a second review as would records with conflicting screening decisions. This method helps avoid and potential lengthy processes of conflict resolution based on limited information, however, the relative time and financial costs of such methods are unclear.

Does your systematic review process employ one screener or two? If you’re already employing dual screening, is your process as efficient as possible? Or does it need work? Systematic review software could be the answer. Find out how it works here.

Divider2-100

 

Chip Couch
About the Author

Chip Couch

Chip Couch is the Technical Training Specialist at Evidence Partners, makers of DistillerSR. As a part of the Customer Success team at Evidence Partners, Chip educates learners on how to use DistillerSR to improve and automate their literature reviews. Chip is an experienced learning and instructional designer who has worked with adult learners in both the public and private sectors, and in higher education through instruction, coaching, and eLearning.

Stay in Touch With Our Quarterly Newsletter

Related Posts

 
Join us on October 19th, 2021 for
Evidence Matters 21 Wordmark
A virtual one-day summit for the literature review community