Ordered Partition Model for confidence marking modeling

Abstract

Confidence marking is increasingly used in multiple choice testing situations, but when the Rasch measurement model is applied to the data, only the binary data is used, discarding the information given by the con dence mark- ing. This study shows how Wilson’s ordered partition model (OPM), a member of the Rasch family of models, can be used to model the con dence information. The result is a model which is in strict relation to the binary Rasch model, since the Rasch ICC’s are “split” into a set of curves each representing a con dence level. The new model provides a set of item parameters that map the probability of being in each con dence level in relation to the test-taker’s ability. The study provides a powerful diagnostic tool to assess item dif culty, overcon dence or misuse of con dence levels but also the fact that a question is particularly tricky or creates a lot of doubt.