Calibration Session Overview
Calibration sessions in Voxjar are used to assess evaluator consistency. They allow for side-by-side comparison of scores to highlight areas where evaluators may need additional training.
Creating and Running Calibration Sessions
To create a calibration session:
Open an interaction
Click “Calibration Sessions” in the panel on the right
Select a scorecard for the session
Choose the evaluators you want to participate. You can include both human evaluators and AI.
Click “Initiate Calibration Session”
Once created, the calibration session will appear in the assigned users' work queues.
Calibration Session Interface
When participating in a calibration session:
Access your work queue to find assigned calibration sessions.
You'll see a distraction-free zone with the call details and evaluation form.
Answer the questions as you would in a normal evaluation.
You can add notes to questions if needed.
Submit your evaluation when complete.
Calibration Session Reporting
To view calibration session results:
Go to the Calibration Sessions page.
Select the completed session you want to review.
You'll see a side-by-side comparison of evaluator responses.
Look for "disconnects" - areas where evaluators gave different answers.
You can drill down to see individual evaluations and notes.
This view allows you to identify inconsistencies between evaluators, which can be used to guide further training or clarification of evaluation criteria.
Using Calibration Sessions for Training
Calibration sessions can be used as a training tool:
Review the side-by-side comparisons with your team.
Discuss areas of disconnect to understand why evaluators scored differently.
Use these discussions to refine your evaluation criteria or provide additional guidance to evaluators.
You can also include AI evaluations in calibration sessions to compare how well the AI performs against human evaluators.
Calibration sessions are not included in typical evaluation reporting or scores to prevent noise on the interactions. They can be reviewed separately in their designated areas.
By periodically running calibration sessions, you can ensure that all your evaluators - both human and AI - are aligned in their understanding and application of your evaluation criteria.