Excluded, but not Forgotten: 3 Tags for Tracking Excluded References in Your Systematic Reviews

How many times have you stumbled upon an interesting article or website while going about your day, but weren't able to read it right away? Did you go looking for it later, wondering why on earth you hadn't saved the link?

The same thing often happens when conducting a systematic review. During the screening process, it's common to come across references that aren't relevant to your current review project, but could be useful in another review or are of general interest to the screener.

Rather than just tossing these references into the "excluded" pile, many groups add a question or two to their screening form to allow them to be labeled for future use. Here are three handy tags to consider using on excluded references to help you find them later:

Keep as Background Material

Even if a reference doesn't meet your inclusion criteria, it may still contain relevant background material. Use a tag to keep it handy.

Let’s say you find a paper that contains substantial referenceable material, but was published outside of the target date range for the study. You can't use the results in your findings because it does not meet the exact inclusion criteria for your review.

This is the perfect situation to use a tag to identify this paper as background material. In doing so, you can easily refer to it when preparing the study write-up even though it won't appear in the tables or meta-analysis.

Get more tips for screening and data extraction with our 5 best practices for optimizing your systematic reviews

Potentially Useful for Study X

For references that would be a good fit for another project, use a tag to easily find them later.

Many groups run multiple systematic reviews on closely related subjects. As such, it is not uncommon for searches to turn up references that don’t meet the inclusion criteria for the current review, but are likely candidates for one of the other reviews being conducted.

An informed screener can tag these articles for easy retrieval from the excluded pile for use in other systematic reviews that the group is working on.

Similarly, some groups maintain repositories of references on specific topics. If the reference currently being excluded looks like a fit for one of these repositories, it can be flagged at this time for potential addition.

I'd Like to Read This

Don't interrupt your workflow to read interesting material if it's not related to your project - tag it for your "to read" list.

As the screeners make their way through hundreds or thousands of titles and abstracts, it’s highly likely they will come across material that looks interesting, but is unrelated to the review they’re working on.

By providing screeners with a way to tag this material, they can quickly save it for later and avoid distraction during the screening process.

More Questions = Faster Screening?

It may seem counter-intuitive, but many systematic review groups report that having these optional questions on a screening form can actually speed up the screening process. By quickly tagging excluded references for later follow-up, reviewers can continue with their screening uninterrupted and without fear of losing track of the notable excluded references.

Distractions and information overload are a fact of life these days, but they don’t have to derail your systematic reviews. Try tagging your excluded references - you might just end up with a head start on your next review project.


New Call-to-action


Peter O'Blenis

Peter O’Blenis is a co-founder of Evidence Partners and has assembled a collection of best practices and methodologies for using web-based software to streamline clinical research. He believes that well written web-enabled software can solve real-world problems and has presented globally on the topic.