# Question randomization: working checklist

Companion to https://lunaprompts.com/blog/assessment-question-randomization-guide

LunaPrompts Editorial. Original illustrative exercise.

## Define the workflow

- Role or campaign:
- Owner:
- Decision this workflow should support:
- Candidate instructions and support route:

## Worked example: Vary a graduate assessment without losing the blueprint

Build a graduate assessment with three slots: programming concepts, a debugging exercise, and a short explanation. For each slot, prepare alternatives that target the same capability at a similar intended level of effort.

Consider two debugging alternatives. One asks candidates to repair a boundary condition in a provided function. The other asks them to install several dependencies before finding a similar defect. Both may involve the same concept, but they do not create the same experience. Reduce setup differences or account for them explicitly before treating the forms as comparable.

Inspect several sample forms before rollout. Check that no form accidentally concentrates effort in one skill, includes overlapping questions, or provides a clue to another answer. Where the platform randomizes order or answer options rather than question selection, document that distinction.

After the campaign, review completion patterns, candidate feedback, and reviewer observations by question. A surprising result may indicate an unclear prompt or uneven effort, rather than a difference in the candidate pool. Randomization makes a well-maintained question bank more useful; it does not remove the need to maintain it.

## Review checklist

- [ ] Coverage: The required skill slots
  Evidence or open question:
- [ ] Effort: Setup and solution time
  Evidence or open question:
- [ ] Evidence: The behavior each question reveals
  Evidence or open question:
- [ ] Review: Sample forms and post-campaign feedback
  Evidence or open question:

## Working session

Use the blueprint sampler below, then compare two forms from your own bank. Identify one difference in setup effort that a difficulty label misses.

## Avoid this mistake

Shuffling answer options and selecting different questions solve different problems. Confirm the configured mechanism before describing the assessment to reviewers.

## Record the next step

- Observed evidence:
- What remains uncertain:
- Person responsible:
- Next action and date:

Keep candidate examples anonymous when sharing this worksheet.