How to Compare AI vs Human Reviewer Decisions in EasySLR?

How to Compare AI vs Human Reviewer Decisions in EasySLR?

EasySLR allows you to assess the accuracy and reliability of its AI-driven screening by directly comparing AI decisions with those made by human reviewers. This validation step helps you understand how closely the AI aligns with your decision-making and ensures reliability before scaling the process to a larger dataset.

You can compare AI and human reviewer decisions in two ways:

A. Within the Platform
  1. Navigate to the "Statistics" section and click on the “Quality”.
  2. Evaluate AI Performance: Analyse how AI decisions align with human decisions, aiding in assessing the efficiency of AI.
    • Decision Match Rate: Percentage of citations where AI’s decision matched the human’s.
    • Recall (Sensitivity): Out of all articles that were finally included, how many the AI correctly identified.
    • Conflict %: How often your decision differed from the AI’s, giving insight into potential misalignments or misinterpretations.
These metrics help assess how closely the AI understands the protocol and where conflicts occur.

B. Using the Excel Export
  1. Navigate to the "Articles" section> Click “Downloads”> Download All to Excel.
  2. The Excel file contains the following information along with additional details:
    • Column R – AI Decision (Include/Exclude)
    • Column S – AI Reasoning
    • Column T – AI Notes (Rationale)
    • If you are using AI as one of the reviewers then you can view the conflicts between AI and human reviewer in Column U.
By reviewing this file, you can:
  • Perform a side-by-side comparison.
  • Identify articles where AI and human decisions differ.
  • Understand the reasoning behind AI decisions to evaluate whether it aligns with your interpretation of the protocol.
By following these steps, you can effectively compare AI and human decisions in EasySLR, enhancing the accuracy and efficiency of your systematic review process.​



    • Related Articles

    • How to run AI on the articles that are already screened by Human Reviewers?

      EasySLR allows you to run AI on articles after they’ve been screened by human reviewers. Just follow the steps below to initiate the AI run and compare with the human decisions. If any differences arise, the Conflict Analysis tool helps you ...

    • List of AI Models Used Across EasySLR

      EasySLR uses purpose-specific GPT models across different review stages to optimise accuracy, performance, and cost efficiency. Each model has been selected based on the complexity and nature of the task it supports, ensuring reliable AI assistance ...

    • AI as the Only Reviewer in EasySLR

      What is “AI as the Only Reviewer”? The AI as the Only Reviewer feature in EasySLR allows the AI to independently perform screening — making inclusion/exclusion decisions across your dataset without requiring input from a human reviewer. How to Enable ...

    • How to Download AI Models Used Across Different Processes in EasySLR?

      EasySLR provides full transparency into how AI is applied across different stages of the review workflow. Users can now download details of the AI models used at each stage, helping teams meet documentation, audit, and reporting requirements. ...

    • Using AI as a Second Reviewer in EasySLR

      EasySLR offers an advanced AI-powered second reviewer feature to work alongside human reviewers, revolutionizing the way literature reviews are conducted. This powerful combination ensures a comprehensive and thorough review process, blending the ...