AI Include Strictness Function in EasySLR

AI Include Strictness Function in EasySLR

The AI Include Strictness setting allows you to control how conservatively or aggressively the AI recommends studies for inclusion during the screening process. This feature helps research teams tailor AI behaviour to match the methodology of their review, balancing comprehensiveness, accuracy, and reviewer effort.

What is AI Include Strictness?
During Title & Abstract or Full-Text screening, the AI evaluates each article against your protocol and determines whether it should be Included or Excluded. The AI Include Strictness setting controls the confidence threshold the AI uses before recommending an article for inclusion.

By adjusting this setting, you decide whether the AI should:
  • Include more potentially relevant studies (higher recall)
  • Be more selective and recommend fewer studies (higher precision)
This allows the AI to better align with your project's screening strategy.

Why is AI Include Strictness Important?
Every evidence synthesis project has different priorities. For some reviews, the primary objective is ensuring that no relevant study is missed. For others, reducing reviewer workload by limiting unnecessary inclusions may be equally important. AI Include Strictness provides the flexibility to support both approaches.

How AI Include Strictness Works
The setting changes how confident the AI must be before recommending an article for inclusion.

Lower Include Strictness
When Include Strictness is lower, the AI becomes more inclusive.

Characteristics include:
  • More articles recommended for inclusion
  • Higher likelihood of identifying borderline relevant studies
  • Higher Recall
  • Lower Precision
  • Slightly increased reviewer workload
This approach minimizes the chance of overlooking potentially relevant evidence.

Higher Include Strictness
When Include Strictness is higher, the AI becomes more selective.

Characteristics include:
  • Fewer articles recommended for inclusion
  • Only high-confidence articles are included
  • Higher Precision
  • Slightly lower Recall
  • Reduced reviewer workload
This approach prioritizes efficiency while maintaining high confidence in inclusion decisions.

Understanding the Trade-off
Changing Include Strictness affects two important performance measures: Recall and  Precision

Increasing one usually affects the other.
For example:
Higher Recall often results in more articles requiring manual review.
Higher Precision often reduces reviewer workload but may miss some borderline studies.
AI Include Strictness helps teams choose the balance that best fits their review methodology.

Choosing the Right Setting
Systematic Literature Reviews (SLRs)
For comprehensive systematic reviews, researchers generally prefer maximizing Recall.

Recommended approach:
  • Medium Include Strictness
  • More comprehensive evidence identification
  • Reduced risk of missing relevant studies

Rapid Reviews
Rapid reviews typically prioritise reviewer efficiency.

Recommended approach:
  • Higher Include Strictness
  • Fewer articles for manual review
  • Faster screening

Scoping Reviews
Scoping reviews often benefit from broader evidence identification.

Recommended approach:
  • Lower Include Strictness
  • Greater sensitivity
  • Improved coverage of available literature

Living Reviews
For continuously updated reviews, teams often balance both Recall and Precision.

Recommended approach:
  • Medium Include Strictness
  • Balanced workload
  • Consistent AI recommendations over time

Example
Imagine your project contains 10,000 citations.

Lower Include Strictness
The AI may recommend:
  • 1,800 articles for inclusion
Advantages:
  • Captures nearly every potentially relevant article
  • Maximizes Recall
Consideration:
  • Reviewers manually evaluate more articles.

Higher Include Strictness
The AI may recommend:
  • 950 articles for inclusion
Advantages:
  • Reduces reviewer workload
  • Improves Precision
Consideration:
  • Some borderline studies may require additional manual checking.

Benefits of AI Include Strictness
Greater Control
Configure AI behaviour according to your review methodology.

Flexible Screening Strategy
Support:
  • Comprehensive reviews
  • Rapid reviews
  • Scoping reviews
  • Hybrid AI-assisted workflows

Better AI Performance Optimisation
Adjust AI behaviour while monitoring project statistics such as:
  • Recall
  • Precision
  • F1 Score
  • Decision Match Rate
This helps teams continually optimise AI performance across projects.

Best Practices
  • Start with the Medium Strictness  setting if you are evaluating AI for the first time.
  • Lower the strictness for projects where identifying every relevant study is critical.
  • Increase the strictness for projects focused on reviewer efficiency.
  • Review AI performance metrics after pilot screening and adjust the setting if required.
  • Document any changes to AI configuration as part of your review methodology to maintain transparency and reproducibility.

Frequently Asked Questions

Does AI Include Strictness change my protocol?
No. It only changes how confidently the AI recommends studies for inclusion. Your protocol, eligibility criteria, and screening workflow remain unchanged.

Does changing Include Strictness affect human reviewer decisions?
No. It only affects AI-generated recommendations. Human reviewers always retain full control over the final screening decisions.

Can I modify Include Strictness after screening has started?
Yes. You can update the setting during the project. However, previously generated AI decisions will not change automatically. To apply the updated setting, rerun AI on the relevant articles.

Will changing Include Strictness improve AI accuracy?
Not necessarily. It changes the balance between Recall and Precision rather than making the AI inherently "better." The ideal setting depends on your project's objectives and review methodology.

Summary
The AI Include Strictness feature gives research teams greater control over how the AI recommends studies during screening. By allowing you to balance Recall, Precision, and reviewer workload, it supports a wide range of evidence synthesis methodologies from comprehensive systematic reviews to rapid reviews while ensuring AI behaviour aligns with your project's scientific and operational requirements.