# AI agent assessment: test tools, state, and recovery

https://lunaprompts.com/solutions/ai-agent-assessment

Illustrative assessment template. Adapt and calibrate for the role before use.

An agent that completes the happy path is only part of the job. Test what it does when a tool fails, context changes, or an action needs approval.

## Task

Build an agent that triages support tickets using a supplied policy and a mock order API. It may look up orders and draft responses, but refunds require approval. Include a ticket containing instructions to ignore the policy, a tool timeout, and a repeated request. Use synthetic data and mock actions throughout.

## Deliverables

- A runnable agent with explicit tool contracts and approval boundaries
- A trace for normal, injected-instruction, timeout, and duplicate-request scenarios
- A small evaluation set and a written explanation of state, retries, and stopping rules

## Suggested rubric

### Tool use and boundaries — 30%
Stronger evidence: Validates tool arguments and requires approval before a mock refund.
Investigate: Treats a retrieved instruction as permission to perform an action.

### State and recovery — 30%
Stronger evidence: Handles timeouts and repeated requests without duplicate actions or unbounded retries.
Investigate: Loses task state or repeats an action after a retry.

### Evaluation and traces — 25%
Stronger evidence: Uses repeatable cases and traces to identify where behavior diverges from policy.
Investigate: Shows one successful run without checking failure scenarios.

### Engineering judgment — 15%
Stronger evidence: Explains when a deterministic step is preferable and defines stopping conditions.
Investigate: Adds more agent steps without explaining their cost or purpose.

## Illustrative report (not a real candidate result)

The illustrative agent requests approval correctly, but issues the same mock action twice when the first response times out.

Ask the candidate to add idempotency handling and demonstrate the retry case. A successful final message does not prove that intermediate actions were correct.

## Follow-up questions

- Where does trusted policy end and untrusted ticket content begin?
- How do you distinguish a failed action from a successful action whose response was lost?
- What would make this workflow stop and ask a human for help?

Discuss your requirements: https://lunaprompts.com/pricing