What to collect
- Runnable implementation and setup instructions
- Ten evaluation cases, including unsupported questions and a failed dependency
- A short explanation of cost, latency, and quality tradeoffs
AI ENGINEERING
Assess AI engineers through work they can explain
A working demo is the start of the conversation. See how candidates evaluate it, handle failure, and defend their decisions.
01 / THE TASK
A practical starting point.
Build a support-answer assistant using a small, supplied product handbook. It must cite its sources, decline questions the handbook cannot answer, and recover from a failed model call. Submit the implementation and a short evaluation set.
Adapt this example to the role, agree allowed tools, and give candidates the same instructions. It is a starting template, not a validated selection test.
02 / THE RUBRIC
Agree what good looks like.
Calibrate these suggested weights with your interviewers before the first candidate. Record the evidence behind each rating.
| Criterion | Weight | Stronger evidence | Needs investigation |
|---|---|---|---|
| Correctness and grounding | 30% | Answers match the supplied handbook and cite the relevant passage. | Plausible answers cannot be traced to the source. |
| Evaluation | 25% | Tests include ambiguous requests, missing evidence, and failure cases. | Only a successful demonstration is submitted. |
| Reliability | 25% | Timeouts, retries, and unavailable dependencies have explicit behavior. | The workflow assumes every call succeeds. |
| Explanation and tradeoffs | 20% | The candidate explains a concrete change and the evidence behind it. | They describe concepts without connecting them to their code. |
03 / ILLUSTRATIVE REPORT
From observation to follow-up.
Example only · not a real candidate resultObserved
The sample solution grounds answers correctly, but retries every failure without a limit. Its evaluation set covers only answerable questions.
Reviewer’s next step
Explore failure handling and evaluation design in the interview before advancing. Do not infer overall ability from the total score alone.
Ask about their own work.
- Show a question your system should refuse to answer. What evidence supports the refusal?
- If latency doubled, what would you inspect first?
- Which result changed your implementation, and why?
04 / SEE THE PRODUCT
Bring the brief to your demo.
This LunaPrompts product walkthrough introduces the assessment workflow. Ask the team to demonstrate your exact task, chosen tools, and report requirements in the environment you plan to use.
The walkthrough shows the product. It is not a recording of the illustrative exercise above or an independent performance benchmark.
Read the detailed workflow guide ↗PUT IT TO WORK
Start with the role you’re hiring for.
Bring a job description, a few must-have skills, and your hiring timeline. We’ll help you shape the evaluation.
Talk to us ↗