What problem is worth solving?
Teaching scenario: channel and business reports disagree on conversions. Before recommending a budget allocation, diagnose definitions and design a reviewable experiment.
Avoid budget decisions based on inconsistent reports and connect analysis to a controlled experiment.
- WORKING ROLES
- Media operations, finance/analytics, data engineers and FDE
How does an FDE turn symptoms into a solvable problem?
The following discovery and implementation exercise uses a synthetic scenario.
Recalculate one campaign-day
Align timezone, currency, campaign ID and attribution window; inspect returns and duplicate exports before explaining the gap.
Find the decision behind the report
Identify decision cadence, learning-volume requirements, stock and margin constraints; propose data collection where evidence is weak.
How does the solution enter the workflow?
- 01
Contract
Fix spend, conversion, net revenue and cutoff definitions.
- 02
Reconcile
Deduplicate with stable keys and explain each discrepancy.
- 03
Compare
Compare baseline, conservative and test scenarios with explicit assumptions.
- 04
Review
Have an authorized operator execute; review net outcomes and stop rules.
Which decisions make the solution reliable?
Historical ROAS is not incremental return
High ROAS may capture existing demand or retargeting. Additional spend may diminish; design a test.
Verify anomalies before narrating
Late-arriving attribution can look like a spike; compare event and ingestion times.
Why spend was counted twice
Synthetic CSV: two exports contain campaign C-3 spending 1,000 CNY on the same day.
- Key
- Use campaign_id + date + currency + report_scope.
- Handling
- Keep the final export version; do not sum cumulative snapshots.
- Result
- Deduplicated spend is 1,000, with the 1,000 discrepancy explained.
What should you do and deliver?
Align cross-report definitions
- OUTPUT
- Metric contract
- PASS CONDITION
- Windows, returns and currency are explicit
Handle duplicates and late data
- OUTPUT
- Cleaning + reconciliation
- PASS CONDITION
- Totals are reproducible
Design a budget experiment
- OUTPUT
- Experiment + stop rules
- PASS CONDITION
- Explain separation from organic change
How do you prove completion and reuse the work?
Acceptance checks
Review reconciliation, reasoning, executable experiments and cost; predictions are not realized returns.
Reusable assets
Reuse contracts, reconciliation and experiment patterns; adapt platform attribution and margins.
Further reading and original sources
- Building effective agents ↗Anthropic · 2024-12-19
- Demystifying evals for AI agents ↗Anthropic · 2026-01-09