The current version of our randomization framework classifies randomization into six levels ranging from 0 to 5. Levels in this framework are not necessarily on a quality hierarchy from worst to best. In fact, a question is allowed to belong to multiple levels. Questions of relatively higher complexity, such as multi-part questions with many question parameters, may have the potential for a mix of level 1-4 randomization. Further note that level 5 randomization (highlighted in italics) mainly applies to assessments since it requires entirely independent variants.
As an example, this can be achieved by implementing a large question bank from which the assessment questions for each student are randomly chosen.
| Level | Label | Description | Purpose | Example |
|---|---|---|---|---|
| 0 | Unrandomized | Every student receives precisely the same question. | Classify questions that are not randomized at all. | Abdallah uses a perfect binary tree to implement a small database system with 63 nodes. How many leaves will this tree have? |
| 1 | Surface Features | Surface level features (e.g., names, colours, phrases) change for each variant. | Increase cognitive load for students when they are pattern-matching. | {{ Abdallah }} uses a perfect binary tree with 63 nodes to implement a {{ small database system }}. How many leaves will this tree have? |
| 2 | Conditions | Within a single problem scenario, conditions and values change for each variant. | Offer opportunities for repeated retrieval practice and more productive group work. | Abdallah uses a perfect binary tree to implement a small database system with {{ 63 }} nodes. How many leaves will this tree have? |
| 3 | Scenarios | Problem scenarios change for each variant, assessing a single concept. | Enable students to develop strategies, algorithms, and procedures to solve multiple problem scenarios. | Abdallah uses an {{ unbalanced }} binary tree to implement a small database system with 63 nodes. What {{ max height }} will this tree have? |
| 4 | Concepts | Randomized variations lead to assessment of different concepts. | Diversify question style so students need to synthesize knowledge across concepts and cannot easily pattern-match. | Abdallah uses an {{ M-ary tree with |
| 5 | Different Questions | Each question variant is entirely independent. | Provide students with authentic re-assessment opportunities to demonstrate proficiency. | {{ Abdallah uses a perfect binary tree to implement a small database system with 63 nodes. How many leaves will this tree have? }} |