W04 · Data profiling and tenant adaptation

FIELD JOURNAL / 02Business and agent modeling

See the structure, then implement and verify with the notes.

This page retains the English overview and workshop. The expanded practitioner lecture notes are currently available in Chinese. Read the complete Chinese lecture →

04.1

Account for every source row

Synthetic kit: 14 source rows = 11 accepted + 1 exact duplicate + 2 quarantined. Reconcile amounts and mappings too.

THIS WEEK IN THE FIELD

What situation are you walking into?

The connector works, but tags contain free text, timestamps disagree and orders include duplicates and refunds. Connectivity is green while business meaning is broken.

DELIVERABLE / Data diagnosis + adapter + first replayable slice

01 / Profile defects that change decisions

Check nulls, uniqueness, ranges, timestamps, freshness and cross-table consistency. Prioritize defects by decision impact. Missing channel can invalidate attribution; missing phone may not block read-only segmentation. Preserve raw snapshots and record transformation versions and quarantine reasons.

Sample normal, extreme, migrated and manually edited records.

02 / Treat the adapter as a contract boundary

Normalize names, types, timezones, units and errors. Quarantine invalid records rather than silently converting them to zero. Keep source IDs and lineage. Use contract fixtures and an idempotency test proving re-import does not inflate orders. Keep source systems read-only.

Reconcile raw records to accepted plus quarantined records, with a separate deduplication map.

03 / Prove one data chain before a full platform

Build one chain: order samples, qualifying transactions, member objects, metrics, one recommendation and source evidence. Use a minimal architecture with observable model, tool, data and logging boundaries. Integration plans need availability, permissions and reconciliation criteria.

MVA here is a teaching shorthand, not an industry certification.

04 / Make the demo reveal useful disagreement

Prepare happy, insufficient-evidence and connector-failure paths. Let a practitioner inspect outputs with their inputs and limitations visible. A proof of concept tests feasibility, a pilot tests controlled use and value, and production requires operations and governance.

Retain rejected recommendations and reasons for week-seven evaluation.

WORKED EXAMPLE · SYNTHETIC

Put the method to work

Synthetic logistics exercise: loaded on time means warehouse scan in one system and truck departure in another. Equal booleans do not establish identical events. Model timestamps, deadlines and responsible roles before defining delay. This is not a verified customer case.

Practice this week

Run profiling and rank defects by impact.
Implement an adapter with quarantine and reconciliation counts.
Demonstrate normal, missing-evidence and failed-connector paths.

Acceptance checks

Raw data stays unchanged.
Failures are not silently swallowed.
Outputs trace to data and rule versions.

Download editable worksheet Open the labs

FIELD WORKSHOP / W04

90 minutes: build a replayable data slice

Map source → validation → quarantine → normalized objects → metric → evidence. Demonstrate one accepted and one rejected record. An HTTP 200 is not the acceptance criterion.

0–20 min: read the importer in engine.py. Annotate types, null handling, date and amount rules; keep the source immutable.
20–55 min: run identical inputs twice. Record input identity, contract version, accepted/rejected counts and output identity.
55–90 min: introduce a renamed column and unknown state. Decide whether to map, quarantine or fail the run; show that data is not silently lost.

Artifact excerpt · synthetic example

Run evidence (example fields)
run_id / input digest / adapter_version / metric_version
raw = accepted + duplicate + quarantine
14 = 11 + 1 + 2
Quarantine: row identity, reason_code, review owner
Replay: identical version and input → identical metric
Change: new mapping → new contract version and regression report

Instructor review

Separate transport retries from business duplicates. Retry safety depends on action semantics; an idempotent import does not make downstream sending idempotent.

An unseen field challenge ↗

When refunds arrive late, should you replace yesterday’s report, issue a correction or recompute history? Preserve both provenance and explainability.

Download offline practice kit ↓

Case studies for this module

OpenAI Deployment Company

Who connects intelligence to actual work? ↗

Adapt and quarantine two CRM schemas · Adapter + conflict fixtures

Paid media

Reconcile reports before recommending budget ↗

Handle duplicates and late data · Cleaning + reconciliation

04.2

Map fields, then validate their meaning

Adapters normalize fields and types; contracts define meaning. Retain source IDs and versions for reconciliation.

ONE-MINUTE CHECK

HTTP 200, but tax treatment in revenue is unknown. Accept?

I have read this module (not an assessment result)

My study notes · stored in this browser only

Local storage is not encrypted; do not store sensitive content on shared devices.

References and further reading

Airbus ↗ · Airbus launches Skywise2017-06-19 · Airbus Palantir ↗ · Ontology overview查阅 / Accessed 2026-09-07 · Palantir

Read alongside the assignment and record design constraints and tests. Verify specifications against your deployment version.