What problem is worth solving?
Teaching scenario: two devices share error code E07 but require different handling, with old manuals mixed into the knowledge base. Build model-aware, cited support with clarification when needed.
Reduce wrong answers and repeated handoff by providing device-appropriate next steps.
- WORKING ROLES
- Service specialists, document owners, retrieval engineers and FDE
How does an FDE turn symptoms into a solvable problem?
The following discovery and implementation exercise uses a synthetic scenario.
Start with difficult questions
Use unknown-model, split-table, reused-code and conflicting-manual fixtures; separate retrieval from interpretation failures.
Preserve table context
Retain page, heading, headers, units and captions; OCR text alone does not preserve applicability.
How does the solution enter the workflow?
- 01
Register
Register version, supported models and retirement status.
- 02
Filter
Determine model and version before scoped retrieval.
- 03
Cite
Cite supporting passages or tables and next-step conditions.
- 04
Clarify
Ask for missing model information; hand missing evidence or hazardous work to specialists.
Which decisions make the solution reliable?
Applicability before relevance
An identical code on another model is still invalid evidence; filter before generation.
Score retrieval and answers separately
Repair retrieval when the right page is absent; repair generation when the right evidence is present but misused.
E07: ask the model first
Synthetic manuals define A/E07 as a sensor connection issue and B/E07 as a filter timer reminder.
- Only E07 is supplied
- Ask for the model rather than selecting an arbitrary manual.
- Model B confirmed
- Retrieve the valid B version and cite its page.
- Page missing
- State that evidence is unavailable and escalate; do not invent repair steps.
What should you do and deliver?
Model devices and documents
- OUTPUT
- Index + twelve fixtures
- PASS CONDITION
- Shared codes are disambiguated
Separate internal and external material
- OUTPUT
- Access rules
- PASS CONDITION
- External users cannot retrieve internal documents
Evaluate clarification and citations
- OUTPUT
- Retrieval/answer eval report
- PASS CONDITION
- Appropriate abstention is scored explicitly
How do you prove completion and reuse the work?
Acceptance checks
Check citations, version match, clarification and handoff context, not manual-like prose.
Reusable assets
Reuse registration, filtering and evaluation for policy and product knowledge with new applicability rules.
Further reading and original sources
- Building effective agents ↗Anthropic · 2024-12-19
- Demystifying evals for AI agents ↗Anthropic · 2026-01-09