Evidence Partners' COVID-19 Initiatives

We are committed to supporting COVID-19 research to help accelerate the important work being done to combat COVID-19.
Learn more.

Dual Screening in Your Systematic Review Process: Is it Worth the Time?

*This blog was updated on February 4, 2020 with links to new studies published on the topic. 

For this blog, we spoke with EP Research Product Specialist, Jennifer Tetzlaff about the challenges and benefits of independent dual screening in a systematic review.

Dual independent screening in your systematic review process is undoubtedly considered a best practice. But for some research teams, a lack of resources precludes this from being a standard operating procedure. 

New evidence reinforces the value in dual screening, and why it's essential to have at least some kind of process in place to mitigate the risk of using only one screener. Let's take a closer look.

Free eBook: Learn How to Completely Streamline Your Systematic Review Process.  Click here.

The Evidence

The major risk that research teams take when only using one screener is that relevant studies will be missed, potentially invalidating the review. 

Siw Waffenschmidt, Marco Knelangen, Wiebke Sieben, Stefanie Bühn, and Dawid Pieper recently published a methodological systematic review, which included 4 studies comparing single versus dual independent screening. They found that single screeners missed a median of 5% of studies that would have been included by two independent reviewers reaching consensus. Although reviewer experience may affect this error rate, they also report that, even reviewers with more experience missed a median of 3% of relevant studies.  

A major question, of course, is: does this matter? Are the added resource needs required to conduct the screening process fully in duplicate going to affect the results or conclusions of your review? The answer is not yet clear. Not only did the error rate vary widely between comparisons (0-58%), the authors report that, of 7 meta-analyses examining the effect of missed studies, the findings of 3 would have been “substantially” changed, while missed studies had little or no impact on the remaining four.

Another recent study by Gartlehner G, Affengruber L, Titscher V, Noel-Storr A, Dooley G, BallariniN, König F suggests a similar pattern and reports that single reviewer screening does not "fulfill the high methodological standards that decisionmakers expect from systematic reviews."

 
A second study published by Zhen Wang, Tarek Nayfeh, Jennifer Tetzlaff, Peter O’Blenis, and Mohammad Hassan Murad suggests that both single and double reviewer methodologies produce errors. They state that it's important to note that the gold standard (independent dual screening) is not perfect, and therefore automating systematic review processes to the degree of human accuracy is sufficient when it comes to taking measures to save time and resources in a review. 
 


Balancing Your Resources and Risk Tolerance

What does this mean for your systematic review process? Dual independent screening is still supported as the gold standard. Individual reviewers make mistakes and these mistakes may affect your ability to draw valid conclusions. However, time and resources may still make this a challenge.

The good news is that additional resource requirements for dual screening can be minimized and efficiencies gained by using systematic review software to facilitate this process.



How Systematic Review Software Makes Screening Easier

For research teams looking to make their systematic review process more efficient without sacrificing validity, systematic review software offers many advantages. Using technology frees up more resources and makes the review process faster and more cost-effective.

Some of the main ways systematic review tools help research teams are:

  • Automatic tracking of inclusions/exclusions for improved transparency and managing records through your workflow
  • Editing, time stamping, and version control to improve the audit trail
  • Easier conflict management and resolution 
  • Reporting features such as exclusion reports and PRISMA Diagrams available with one click
  • Bonus features like rapid title screening, deduplication, and artificial intelligence 

Dual Screening Alternatives

Still limited to one screener? Does your particular topic or process have a higher risk-threshold for missed studies? Most of the above options help minimize the risk and increase efficiencies. Teams will also occasionally employ alternate options such as the following:

One Screener with Verification: This method consists of one person doing the screening, and a second person double-checking the first person's work. 

Quality control: Teams will occasionally conduct dual review for a sub-sample of records (for instance a random sample) to estimate if error rates are within acceptable bounds for their specific review.

Dual Screen Exclusions Only: In this case, preliminary screening levels (e.g. title/abstract only) require dual review only for exclusions. A record labelled for inclusion by either reviewer would be passed on to the next level of screening without a second review as would records with conflicting screening decisions. This method helps avoid and potential lengthy processes of conflict resolution based on limited information, however, the relative time and financial costs of such methods are unclear.

Does your systematic review process employ one screener or two? If you’re already employing dual screening, is your process as efficient as possible? Or does it need work? Systematic review software could be the answer. Find out how it works here.

Related Reading

Best practices for systematic literature review

 

Author

Jennifer Baguss

As a marketing specialist, Jennifer Baguss brings years of digital content writing and marketing experience to the EP team. Her background in journalism makes her a thoughtful and concise writer with a keen interest in taking complicated concepts and making them easily digestible for those who wish to learn. When she's not writing, you can catch her on two wheels, mountain biking, road biking, and even fat biking in the winter!