Protocol Text Review & AI Performance Analysis

Protocol Text Review & AI Performance Analysis

EasySLR is designed to help reviewers streamline their workflows and make evidence-based decisions faster. Two powerful features — Protocol Text Review and AI Performance Analysis —ensure your review protocol is aligned, AI-ready, and easy to evaluate.
​
1. Protocol Text Review
The Protocol Review is a smart tool that analyses your protocol and helps you improve it for AI screening. It focuses on strengthening two critical areas:
​
A. PICOS Refinement
The optimiser reviews your defined Population, Intervention, Comparator, Outcomes, and Study Design and provides suggestions to:
  • Clarify vague or ambiguous terms (e.g., what constitutes “elderly” or “high PD-L1 expression”)
  • Resolve conflicting inclusion/exclusion rules
  • Structure criteria more precisely so the AI can apply them accurately
  • Recommend standardised language and examples to improve reproducibility
B. Hierarchy of Selection Criteria
This section ensures that your inclusion and exclusion reasons are logically ordered and clearly described, which is crucial for:
  • Ensuring transparent reviewer decisions
  • Helping the AI apply criteria consistently
  • Avoiding confusion when multiple exclusion criteria overlap
The tool will:
  • Highlight missing or overlapping decision rules
  • Suggest reordering criteria (e.g., excluding animal studies before design-based exclusions)
  • Recommend more specific exclusion labels for edge cases (e.g., single-arm studies, subpopulation-only data)
How to use it:
  1. Go to the Tools section
  2. Select 'Protocol Text Review'
  3. Click on 'New Text Review'
  4. Select the Stage: Title/Abstract or Full Text
  5. Click Analyze Protocol
  6. Review suggestions for:
    Refining PICOS definitions
    Adjusting or reordering hierarchy of selection criteria


  7. Apply recommended changes manually.
Tip: Use the Protocol Optimiser before launching large-scale screening or rerunning AI to ensure optimal AI performance.
​
2. AI Performance Analysis 
The AI Performance Analysis provides a transparent way to evaluate how well the AI is performing on your dataset. It helps build confidence in the AI by letting you validate its decisions before scaling.
​
What it does:
  • Compares AI suggestions with reviewer decisions on a screened sample
  • Generates recall, and agreement metrics
How to use it:
  1. Go to the Tools section
  2. Select ‘AI Performance Analysis’
  3. Click ‘New Performance Analysis’


  4. Select the Stage and Scope
  5. Click 'Run Analysis'
  6. Review key metrics like:
    • Recall (AI’s ability to identify relevant studies)
    • Reviewer Agreement (how often AI and humans agree)
​
Note:
  • For best results, run the analysis on a representative sample of articles
  • If AI is disabled, performance analysis cannot be initiated
Use both together for optimal AI-guided screening:
  • Start with the Protocol Review to refine your protocol
  • Then run the AI Performance Analysis to evaluate how well AI applies your refined criteria
Need Help?
Reach out via the Live Chat option in EasySLR or email us at support@easyslr.com.


    • Related Articles

    • Conflict Analysis in EasySLR

      The Conflict Analysis feature in EasySLR helps identify patterns in reviewer disagreements during the screening process. It provides data-driven suggestions to refine your protocol and improve consistency across decisions, ultimately reducing ...

    • How to Enhance AI Performance for Optimal Results?

      EasySLR’s AI can streamline screening by providing inclusion/exclusion suggestions based on your protocol. However, to get the best results, it's important to first calibrate the AI to ensure it aligns with your decision-making. The steps below ...

    • AI as the Only Reviewer in EasySLR

      What is “AI as the Only Reviewer”? The AI as the Only Reviewer feature in EasySLR allows the AI to independently perform screening — making inclusion/exclusion decisions across your dataset without requiring input from a human reviewer. How to Enable ...

    • How to track AI credits usage?

      The Usage feature offers a comprehensive solution for managing your AI credits. It enables you to track and Analyse your credit usage in real-time, providing valuable insights throughout the lifecycle of your projects. Understanding the AI Credit ...

    • How to set up a Protocol in easySLR?

      Setting up a protocol in EasySLR is simple and user-friendly. A pre-set template is available to guide you, so you don’t have to create everything from scratch. Your protocol defines the study criteria, guiding both human reviewers and AI in ...