How to Enhance AI Performance for Optimal Results?

EasySLR’s AI can streamline screening by providing inclusion/exclusion suggestions based on your protocol. However, to get the best results, it's important to first calibrate the AI to ensure it aligns with your decision-making. The steps below outline how to do this effectively.

Step 1: Upload a Sample Set for Evaluation

Start by uploading a sample of 100–200 citations to your project. This smaller, controlled dataset is ideal for testing how well the AI performs in comparison to human reviewers. We recommend using a representative sample that includes both clear includes and excludes for balanced evaluation.

Step 2: Enable and Run AI Screening

Once your sample set is uploaded:

• Enable Title & Abstract AI in the Settings. 
• Allow the AI to generate inclusion/exclusion suggestions based on the protocol.
• The AI will provide suggestions along with rationale.
  

Step 3: Manually Screen the Same Sample

Review and screen the same sample manually:

• Include or exclude articles based on your judgment.
• This creates a reference dataset for comparing the AI’s decisions.
  
  

Step 4: Compare AI and Human Decisions

Refer to the help article for a step by step guide on how to Compare AI and Human Reviewer Decisions.

Step 5: Refine and Rerun AI

After reviewing the conflicts:

• Update your protocol or inclusion/exclusion criteria if needed (e.g. descriptions, PICOS).
• Rerun the AI on the sample to see if performance improves.
• This calibration process may take a few iterations, but it’s critical to ensure accuracy in large-scale reviews.

Once you're confident in the AI’s decisions:

• Upload the remaining citations.
• Run the AI on the remaining citations and complete the screening.
  
  

Pro Tips:

• Try to include a mix of relevant/irrelevant articles in the sample set.
• Avoid editing the protocol repeatedly without testing small changes first.
• Use the AI notes and rationale to understand how the AI is interpreting protocol.

Checklist before moving to full dataset:

• Protocol finalised and saved
• Sample set reviewed manually
• Conflicts between AI and human resolved
• AI performance metrics (recall, conflict %) within acceptable range