Stochastic programming for automated test assembly with uncertainty in the item parameters or in the responses

Items can be described by many parameters. They can be related to, for example, the content of the item, the psychometric properties, or the process of solving the item. Nowadays, response time parameters receive a lot of attention. Early research on response time modeling assumed that a test taker would show consistent response time behavior, often referred to as working speed, over the course of a test. Such models were unrealistic for various reasons — a warm-up effect may cause a test taker to respond more slowly than expected to the early items, fatigue may cause a test taker to respond more slowly than expected toward the end of a test, or as time runs out the test taker may quickly guess the answers to the last items on a test. To take these variations in working speed into account, mixture response time models have recently been investigated. When these models are applied in automated test assembly, probabilistic response time constraints have to be imposed. Stochastic programming has been applied to deal with this kind of probabilistic constraints. In the current paper, the application of stochastic programming will be generalized to uncertainties in the model, for example coming from automated item generation or open answer questions.