Language

Jev confidence calibration

567 ThaiExam questions answered by jev-1.13.0. Does stated confidence predict correctness?

Split
Bins
–accuracy
–expected calibration error
–questions

Reliability diagram

Each point is one confidence bin; the dashed line is perfect calibration. Point size is the number of questions. Click a point — or a table row — to read its questions.

Accuracy rises with confidence, closely tracking the diagonal.

Click a point or a row above to list the questions in that confidence bin.

Original data and results: ai-vs-thai-exams.pages.dev. Exam questions from the ThaiExam dataset (SCB 10X) and the OpenThaiGPT evaluation dataset.