# Assess AI engineers through work they can explain

https://lunaprompts.com/solutions/ai-engineer-assessment

Illustrative assessment template. Adapt and calibrate for the role before use.

A working demo is the start of the conversation. See how candidates evaluate it, handle failure, and defend their decisions.

## Task

Build a support-answer assistant using a small, supplied product handbook. It must cite its sources, decline questions the handbook cannot answer, and recover from a failed model call. Submit the implementation and a short evaluation set.

## Deliverables

- Runnable implementation and setup instructions
- Ten evaluation cases, including unsupported questions and a failed dependency
- A short explanation of cost, latency, and quality tradeoffs

## Suggested rubric

### Correctness and grounding — 30%
Stronger evidence: Answers match the supplied handbook and cite the relevant passage.
Investigate: Plausible answers cannot be traced to the source.

### Evaluation — 25%
Stronger evidence: Tests include ambiguous requests, missing evidence, and failure cases.
Investigate: Only a successful demonstration is submitted.

### Reliability — 25%
Stronger evidence: Timeouts, retries, and unavailable dependencies have explicit behavior.
Investigate: The workflow assumes every call succeeds.

### Explanation and tradeoffs — 20%
Stronger evidence: The candidate explains a concrete change and the evidence behind it.
Investigate: They describe concepts without connecting them to their code.

## Illustrative report (not a real candidate result)

The sample solution grounds answers correctly, but retries every failure without a limit. Its evaluation set covers only answerable questions.

Explore failure handling and evaluation design in the interview before advancing. Do not infer overall ability from the total score alone.

## Follow-up questions

- Show a question your system should refuse to answer. What evidence supports the refusal?
- If latency doubled, what would you inspect first?
- Which result changed your implementation, and why?

Discuss your requirements: https://lunaprompts.com/pricing