AI Analytics: The Warehouse Contract
A self-learning guide to Anthropic's self-service analytics ideas, taught through Hooray's Snowflake, Product 360, Ops 360, and PIM reality.
1. The Real Thesis: Data Is Not Software
MechanismCoding agents can often test their own work: code compiles, tests pass, UI renders, an API returns the expected shape. Analytics agents do not get that same proof. A SQL query can run perfectly and still answer the wrong business question.
The hard part is not the SQL syntax. The hard part is mapping human language to the correct business entity, metric definition, grain, freshness rule, and source-of-truth path.
When a business user asks, "What was revenue for product X?", they are not asking for a table. They are asking for a trusted business answer. They usually cannot tell whether the agent used the right source, whether the product was resolved at the right identity level, or whether the metric definition was current. That is why Anthropic's article keeps coming back to one idea: the system must make correctness structural, not dependent on the user catching a hidden mistake.
Hooray caseQuestion: "What was profit for Livatro LVHD7001 humidifier on Amazon US on June 6, 2026?"
Important: Livatro LVHD7001 humidifier is the user's search phrase, not the source of truth. Human-readable names are useful for lookup, but they are too vague, similar, and changeable to be the canonical product key.
A weak analytics agent asks the business user to resolve warehouse ambiguity. A strong agent first resolves what the system already knows, then asks only for the few things that truly require human judgment.
| Resolution Step | What The Agent Should Infer Or Block | Concrete Failure If Missed |
|---|---|---|
| Market scope | Infer: Amazon US only. Do not ask whether to include all Amazon markets. | The agent blends Amazon US with MX/CA/JP/DE and reports one mixed number with wrong listings, fees, currencies, and totals. |
| Date scope | Infer: June 6, 2026 only. Do not silently turn it into "yesterday," a week, or a trailing window. | The agent answers a weekly or trailing-window question even though the user asked for one exact day. |
| Product identity | Block until resolved: map the phrase to Product 360 SKU, SPU if family-level, and exact Amazon US listing identities if listing-level finance is needed. | The agent calculates a similar SKU, the whole SPU family, or only one marketplace listing while claiming it answered the product. |
| Metric definition | Block until resolved: use the finance-approved profit rollup or semantic metric. | The agent reports gross sales, payout profit, contribution margin, or ad-adjusted profit as if they were the same thing. |
| Finance source route | Infer: after Product/Ops identity is resolved, start from the Finance-owned CORE metric route: fct_finance_item_metric_daily for daily item/metric analysis, drill to fct_finance_ledger_lines when ledger-line traceability is needed, and apply the Finance-owned profit rollup. Do not start from old profit-looking tables such as DWM_FIN_PROFIT_REPORT or DWS_FIN_PROFIT_SUBJECT_MSKU_DAY. |
The agent picks the table with "profit" in the name and returns a plausible but unofficial number. |
| Currency rule | Block until resolved if missing: use the approved currency rule from the semantic metric. | The agent mixes native currency and USD amounts, or invents a conversion rule that Finance did not approve. |
| Case Status | Answer |
|---|---|
| What this illustrates | A readable product phrase is useful for search, but it is not warehouse truth. Product identity and finance metric definition are two separate resolution steps. |
| Raw evidence | User phrase Livatro LVHD7001 humidifier, platform/market Amazon US, date 2026-06-06, requested metric profit. |
| Canonical status | Market and date are resolved from the question. Product identity is not resolved until Product/Ops returns the Product 360 SKU and exact Amazon US listing identities. Profit is not resolved until finance returns the approved rollup or semantic metric. |
| What is unresolved | Which SKU/listings/seller affiliations belong to this phrase, and which metric-code components plus currency rule make up final profit. |
| Agent next step | Do not ask whether to include all Amazon markets. Resolve product/listing identity first, then use fct_finance_item_metric_daily plus the Finance-owned profit rollup. If either resolver fails, stop with a specific blocker. |
What is good todayFinance has a comparatively strong official route: the CORE metric-code route. For daily item-level analysis, the preferred model is fct_finance_item_metric_daily. For ledger-line traceability, the drilldown source is fct_finance_ledger_lines. After the user phrase resolves to official Product 360 SKU and exact Amazon US listing identities, the answer should come from finance rows shaped like this:
platform = amazon,country_code = US,report_date = 2026-06-06metric_code = 0.1,metric_name = total_sales_quantitymetric_code = 1.1,metric_name = total_sales_amountmetric_code = 3.1,metric_name = platform_feemetric_code = 3.2.1,metric_name = fba_delivery_fee
The metric code is doing real semantic work. It tells the agent what each number means, instead of forcing the agent to choose from fields named sales, revenue, payout, fee, or profit.
Important clarification: this does not mean fct_finance_item_metric_daily is only keyed by seller SKU. The finance model includes item-grain and product-mapping fields such as item grain key, local SKU, ASIN, and parent ASIN. SELLER_SKU is a platform merchant SKU field, not final Product 360 identity.
What is bad or missingOld finance-looking alternatives such as DWM_FIN_PROFIT_REPORT and DWS_FIN_PROFIT_SUBJECT_MSKU_DAY still sound like the answer. If an agent can see them as peers, it may choose the most obvious name instead of the governed definition.
There is a second problem: some sample outputs and analyst queries show SELLER_SKU as the most visible product-like label. That visible label can be useful evidence, but it is not canonical Product 360 identity. Human-readable names have the same problem: they are useful lookup clues, but too vague, similar, and changeable to be source of truth. Both must resolve through canonical Product/Ops identity before calculation.
Correct behaviorThe agent should say: "The requested scope is Amazon US on June 6, 2026. The phrase Livatro LVHD7001 humidifier is a lookup clue, not source of truth. I will first resolve it to canonical Product 360 identity and exact Amazon US listing identities, then use the Finance-owned CORE metric route: fct_finance_item_metric_daily for daily item/metric analysis, with fct_finance_ledger_lines only when ledger-line traceability is needed. I will not use legacy profit report tables, raw SELLER_SKU display values, or human-readable names as final product identity. To produce final profit, I need the approved profit rollup that defines which metric codes and currency rules belong in profit."
Self-checkIf the user already said Amazon US, should the agent ask whether to include all Amazon markets?
Answer: No. Amazon US and June 6, 2026 are already resolved. The unresolved parts are product/listing identity and the approved profit definition.
2. Failure Mode One: Concept To Entity Ambiguity
MechanismA concept is the word the user says. An entity is the specific warehouse thing the system must use. "Revenue," "active user," "product," "listing," and "profit" are concepts. Tables, metric codes, columns, identities, filters, and grains are entities.
Analytics agents fail when one concept maps to many plausible entities. The fix is not telling the model to be more careful. The fix is reducing the number of plausible choices and forcing a resolver step before calculation.
Hooray caseWhat this illustrates: raw marketplace rows are evidence, not product truth.
Raw evidence: in the guide sample extract current_walkthrough_samples, the row where SAMPLE_TYPE = amazon_listing_mapping shows source account DINGYAO_SC_01, marketplace/domain value A1AM78C64UM0Y8, seller-SKU-style value FP-B2P, ASIN B0DDTK4PKV, and country MX.
Reader context: this sample extract uses generic column names, so read the business meaning column below. The point is not that DATE_KEY literally means a date here. The point is that the sample row is a marketplace-listing mapping row, not a Product 360 product-master row.
| Sample Extract Field | Sample Value | Business Meaning In This Example |
|---|---|---|
SAMPLE_TYPE |
amazon_listing_mapping |
This row comes from the listing-mapping evidence set, not from the canonical Product 360 product master. |
PLATFORM |
amazon |
Marketplace platform. |
SOURCE_SYSTEM |
DINGYAO_SC_01 |
Source account / seller account context. |
DATE_KEY |
A1AM78C64UM0Y8 |
Marketplace/domain value in this normalized sample extract. |
ENTITY_KEY |
FP-B2P |
Seller-SKU/MSKU-style clue. |
ATTRIBUTION_WINDOW |
B0DDTK4PKV |
ASIN value in this normalized sample extract. |
MEASURE_1 / MEASURE_2 |
FP-B2P / MX |
Seller SKU repeated, plus country. |
Nearby rows in the same sample extract use the same account, marketplace, and country with different seller-SKU-style values such as FP-K4P-new, FP-K6P, FP-MP-F, and FP-MP-K. That is another clue that this is mapping evidence, not the final product layer.
Meaning: this does not safely fit SPU, SKU, or exact listing identity yet. It needs more resolution work before any product-performance answer is safe.
| Signal | What It Tells Us | What It Does Not Prove |
|---|---|---|
DINGYAO_SC_01 |
Seller/account context. | It is not the product, SKU, or listing identity by itself. |
A1AM78C64UM0Y8 + MX |
Marketplace/domain context. | It does not prove the answer should include all Amazon markets. |
FP-B2P |
Seller-SKU/MSKU-style signal. | It is not automatically the Product 360 SKU or the SPU. |
B0DDTK4PKV |
Marketplace item evidence. | It does not replace exact platform + domain + MSKU / listing_id resolution. |
| Canonical Layer | Status In This Case | Why |
|---|---|---|
| SPU | Not resolved. | No product-family identifier has been confirmed from this evidence alone. |
| Product 360 SKU | Not resolved. | FP-B2P is a seller-SKU/MSKU-style clue, not automatically the official Product 360 SKU. |
| Listing item / exact listing | Not resolved. | ASIN and marketplace are listing evidence, but the system still needs exact listing identity such as platform, domain, MSKU, and/or listing ID. |
| Seller affiliation | Partially evidenced, not enough for product analytics. | DINGYAO_SC_01 points to seller/account context, but seller context is not the same as product identity. |
| Identity Layer | Plain Meaning | Who Owns It | What Goes Wrong If The Agent Guesses |
|---|---|---|---|
SPU |
Product family. | Product 360. | The agent may aggregate several real products and call it one SKU. |
SKU |
Product identity and Product 360 operating context. | Product 360. | The agent may miss marketplace-specific listing or seller differences. |
| Exact listing identity | Exact platform + domain + MSKU, preferably resolved to listing_id. |
Ops 360 / Listing 360. | The agent may blend different marketplaces, domains, or listings. |
| Seller affiliation | Seller/store/account attached to a shared listing. | Ops 360 / Listing 360. | The agent may treat one seller row as the whole listing or product. |
What is good todayThe Product 360 / Ops 360 / PIM design already gives us the right conceptual model. Product 360 owns SKU truth, Product-owned assets, manuals, and readiness. Ops 360 opens into Listing 360 by default and owns exact listing identity, seller affiliation, lifecycle, publish state, and listing-specific exceptions. PIM is the internal data/service boundary, not the user-facing concept.
What is bad or missingThe warehouse-facing agent contract does not yet make that identity resolver mandatory for every product-linked answer. If the agent sees FP-B2P, it may choose one seller row, one marketplace row, all MSKU matches, all SKU matches, or the whole SPU family.
All of those can produce a number. Only one matches the user's real question.
Correct implementationBefore calculating, the agent calls a Product/Ops identity resolver. The resolver returns a status, not just rows:
SKU_RESOLVED: safe for SKU-level Product 360 reporting.EXACT_LISTING_RESOLVED: safe for listing-scoped sales, ads, inventory, images, publish state, and exceptions.SELLER_AFFILIATION_RESOLVED: safe only when seller/store/account scope matters.AMBIGUOUS_LISTING_TO_SKU: stop and route mapping repair.DOMAIN_MSKU_PLATFORM_AMBIGUOUS: stop and ask for platform/domain/listing clarification.NO_MAPPING: stop; do not invent a product relationship.
Case answer: from the sample alone, FP-B2P is not safely resolved as SPU, SKU, or exact listing identity. The agent should say: "I found marketplace and seller/account evidence, but I have not resolved this to a canonical product or listing layer. I cannot calculate product performance yet. Next step is Product/Ops identity resolution; if it returns ambiguity, the mapping needs repair or a narrow listing-scope clarification."
Self-checkCan the sample value FP-B2P be treated as the official Product 360 SKU?
Answer: No. In the sample row, it is seller-SKU/MSKU-style marketplace evidence. It must be resolved before product performance can be calculated.
3. Failure Mode Two: Staleness
MechanismStaleness is not just old data. It is any situation where the source, schema, mapping, documentation, or business rule changed but the agent still acts as if the old world is true.
This is why Anthropic emphasizes maintenance, validation, colocation, and freshness. The warehouse can be correct on Monday and misleading on Friday if the business changed and the agent guidance did not.
Hooray caseThe sample profile table orders_periodic_profile shows a Lazada order_lines row with impossible future dates. The row has MIN_REPORT_DATE = 2024-10-26, MAX_REPORT_DATE = 58395-06-15, ROW_COUNT = 17427, and FUTURE_DATE_ROWS = 17017. One query client even failed trying to convert that abnormal date into a normal date.
| Profile Field | Sample Value | Reader Meaning |
|---|---|---|
PLATFORM |
lazada |
The affected sales/order platform. |
SOURCE_TYPE |
order_lines |
The source shape being profiled. |
MAX_REPORT_DATE |
58395-06-15 |
An impossible future date, so "latest" cannot be trusted. |
ROW_COUNT / FUTURE_DATE_ROWS |
17427 / 17017 |
The issue affects most of this profiled Lazada order-line set. |
Case answer: if the user asks "Show latest Lazada sales," the agent should not return a latest-sales chart. It should stop and report that the source contains invalid future dates, because "latest" is not meaningful until the date-quality issue is fixed or excluded by a governed rule.
| Case Status | Answer |
|---|---|
| What this illustrates | Sometimes the entity is clear enough, but the source shape makes the question unsafe. Bad dates break any "latest" answer. |
| Raw evidence | Lazada rows include impossible future dates such as 58395-06-15 and a known affected count of 17,017 rows. |
| Canonical status | This is not mainly an SPU/SKU/listing problem. It is a freshness and data-quality gate on the orders/sales source. |
| What is unresolved | Whether the rows should be repaired, excluded by a governed rule, or quarantined from agent-facing latest-period answers. |
| Agent next step | Stop the latest-sales answer, report the affected platform and count, and route the date-quality issue to the source owner. |
What is bad or missingFreshness and anomaly checks are not yet strong enough as agent-facing gates. It is not enough for a human analyst to know "watch out for weird dates." The agent path itself must block or warn before producing a trend, latest-period result, or business recommendation.
Another staleness pattern: old finance tables still exist. Even if the new canonical metric path is better, old sources remain stale traps unless they are deprecated, hidden, or clearly labeled as non-primary.
Correct implementationA self-service agent should perform freshness and shape checks before answering:
- maximum and minimum date by platform
- future-date count
- last complete reporting period
- expected platform/domain scope
- mapping completeness
- deprecated-source visibility
If "latest Lazada sales" includes year 58395, the correct answer is not a chart. The correct answer is: "I cannot safely answer latest sales because the source has invalid future dates. Here is the affected platform, count, and owner route."
Self-checkWhy is the right answer to "latest Lazada sales" not a chart?
Answer: Because the profiled source has impossible future dates, including MAX_REPORT_DATE = 58395-06-15. The agent must block or route the data-quality issue before summarizing latest sales.
4. Failure Mode Three: Retrieval Failure
MechanismRetrieval failure means the correct answer exists, but the agent does not find or use it. Anthropic's important lesson is that simply giving the agent more search access does not solve this. If the search space is huge, the agent can read the right thing and still choose the wrong thing.
The fix is structured retrieval: a small router, curated reference docs, semantic definitions, and domain skills that narrow the space before SQL is written.
Hooray caseAds search-term analysis is a good example. In the guide sample extract current_walkthrough_samples, rows where SAMPLE_TYPE = ads_search_term include platform amazon, source amazon_ads, date 2026-06-05, and attribution window 7d.
| Search Term | Platform / Source / Date | Attribution Window | Sample Measures |
|---|---|---|---|
バランスディスク |
amazon / amazon_ads / 2026-06-05 |
7d |
406 impressions, 11 clicks, 659.92 spend, 0 attributed sales. |
チタン まな板 |
amazon / amazon_ads / 2026-06-05 |
7d |
139 impressions, 5 clicks, 190.28 spend, 7090 attributed sales. |
The attribution window is not decoration. Without 7d, attributed sales is not a complete metric definition.
Case answer: under the shown Amazon ads source and 7d attribution window, チタン まな板 has 7090 attributed sales in the sample row, while バランスディスク has 0. The answer must say "7d attributed sales," not just "sales," and it must not imply this is profit, organic sales, or all attribution windows.
| Case Status | Answer |
|---|---|
| What this illustrates | A metric can be incomplete even when the row is real. "Sales" in ads is not fully defined unless attribution window and source are explicit. |
| Raw evidence | Search terms, impressions, clicks, spend, attributed sales, source amazon_ads, date 2026-06-05, attribution window 7d. |
| Canonical status | The ads metric is scoped to search term, platform, date, and 7d attribution. It is not profit, organic sales, or all-window sales. |
| What is unresolved | If the user wants product profitability, SKU/listing identity and finance metrics still need separate resolution. |
| Agent next step | Answer the ads question using the approved ads path and state the attribution window. Do not silently join it into profit or organic revenue. |
What is bad or missingOlder or high-row-count search-term tables can look more complete than the canonical path. A model searching the warehouse may prefer the largest, most familiar, or best-named table, especially if domain guidance is not synced into the runtime.
Raw historical SQL has the same problem. Old queries are useful raw material, but they are not truth. If an agent copies an old dashboard query, it may also copy old filters, old attribution logic, and old identity assumptions.
Correct implementationThe agent should first enter an ads domain router. The router should say: for search-term sales, use the approved ads source, require attribution window, state platform and date, and reject sources that do not expose the window. Query history should be distilled into patterns and evals, not exposed as a giant pile of examples.
Self-checkWhat word must appear next to the ads sales number in this example?
Answer: 7d attributed sales. Without the attribution window, "sales" is too vague and can be confused with organic sales, all-window sales, or profit.
5. Data Foundations: Collapse The Answer Space
MechanismData foundations are the warehouse models, transforms, tests, tables, and metadata. For agents, their main job is to shrink ambiguity. If "profit" resolves to one governed route instead of forty plausible tables, most of the problem disappears before the agent has to reason.
What is good todayHooray has a serious modeled ladder: source/ODS into STG, INT, CORE, MART, and RPT. CORE is the strongest candidate for canonical enterprise truth. There is also an initial agent catalog, table catalog, model headers, and finance metric governance.
What is bad or missingThe live warehouse still exposes substantial old/app-facing surfaces such as DWD, DWM, DWS, DIM, APP, and Tableau-oriented schemas. Those are not harmless if agents can find them. They are exactly the "five reasonable-looking answers" problem.
The issue is not that every old table is wrong. The issue is that an agent cannot know which one is canonical, which one is a speed copy, which one is an app output, which one is legacy, and which one is unsafe unless the system says so structurally.
| Surface Type | Agent Meaning | Correct Treatment |
|---|---|---|
| Canonical model | Official truth for a business concept. | Visible by default; owned, documented, tested. |
| Semantic metric | Official calculation and allowed dimensions. | Use before custom SQL for top KPIs. |
| Mechanical rollup/cache | Fast copy of canonical truth. | Visible only with parent lineage and no private logic. |
| App/report output | Presentation or workflow-specific surface. | Not primary truth unless explicitly contracted. |
| Legacy/deprecated | Old answer path. | Hidden from normal agent roles or clearly blocked. |
Correct implementationCreate an agent-safe warehouse contract. The default analytics agent role should see semantic definitions, approved canonical datasets, approved mechanical rollups, and narrow diagnostics. It should not browse old finance reports, app exports, ingestion/admin surfaces, or deprecated schemas as if they are equal options.
Case answer: old/app-facing surfaces are not automatically wrong, but they are not automatically safe. The correct implementation classifies each visible surface as canonical, semantic, mechanical rollup/cache, app/report output, diagnostic, or deprecated. If a surface has no classification, owner, lineage, or allowed-use rule, the agent should treat it as unsafe for first-answer analytics.
Article concept we missedAnthropic is explicit that governance only holds when it is enforced by tooling, CI, and mandate. Claude can help draft descriptions and spot undocumented models, but the system must structurally route agents to governed definitions first.
| Missing Foundation Concept | Plain Meaning | Hooray Learning Example |
|---|---|---|
| Metadata as product | Table descriptions, grain, valid values, lineage, ownership, and source tier are not side notes. They are the product interface agents use. | A finance row should not only expose metric_code = 3.1; it should also say who owns the metric, what grain it is at, what values are valid, and whether it is safe for agent answers. |
| Colocated artifacts | Model code, semantic metrics, reference docs, and dashboard definitions should change together. | If Product/Ops identity mapping changes, the warehouse model and the agent reference doc should update in the same change path, not weeks apart. |
| Enforced standards | A rule is not real unless tools make bypassing it difficult. | A pull request that changes an agent-facing finance model without updating metric docs, owner, or eval coverage should fail review or require an explicit exception. |
Self-checkIf an old table has a convincing name but no owner, lineage, or allowed-use rule, should an agent use it as the first answer?
Answer: No. A good name is not governance. The agent should use a classified canonical source or stop.
6. Rollups And Caches: Speed Copies, Not New Truth
MechanismA rollup is a precomputed summary. A cache is a fast copy. Anthropic is not saying to eliminate them. The point is that they must derive mechanically from canonical models. They are allowed to make the answer faster, not different.
Hooray caseA daily or weekly profit summary can be useful. But it should trace back to the finance-approved metric-code fact. If it calculates profit with hidden filters, private currency handling, or old fee logic, it has become a shadow source of truth.
Case answer: a weekly profit rollup is safe only if it is a speed copy generated from the official finance metric definition. If it has its own profit formula, it should be labeled deprecated, diagnostic, or unsafe for agents rather than treated as another official answer.
| Case Status | Answer |
|---|---|
| What this illustrates | Performance optimization is allowed, but private business logic is not. A rollup can speed up truth; it cannot become a second truth. |
| Raw evidence | A daily or weekly profit summary that looks easier to query than metric-code finance rows. |
| Canonical status | Safe only if mechanically generated from the canonical finance model at a declared grain and metric version. |
| What is unresolved | Whether the rollup has parent lineage, refresh cadence, metric version, and no hidden filters or private formulas. |
| Agent next step | Use the rollup only when lineage proves it is a cache of the canonical model. Otherwise, use the canonical finance path or stop. |
What is bad or missingLarge old rollups such as old profit subject/day tables may be useful for performance or reconciliation. But if they sit beside the canonical finance model without a hard label, agents and humans will treat them as alternate answers.
Correct implementationEvery rollup needs a parent pointer: "I am generated from this canonical model, using this metric version, at this grain, refreshed at this cadence." If the canonical revenue definition changes, the rollup changes mechanically. If it cannot do that, it is not a safe agent-facing rollup.
Self-checkWhen is a weekly profit rollup safe for an agent to use?
Answer: Only when it is a mechanical speed copy of the canonical finance model, with declared parent lineage, metric version, grain, and refresh cadence.
7. Sources Of Truth: The Trust Ladder
MechanismSources of truth are the reference surfaces the agent uses to navigate the warehouse. Anthropic's ladder is: semantic layer first, lineage if the metric does not exist, curated query patterns as raw material, and business context for meaning around the question.
| Trust Layer | What It Does | Hooray Example |
|---|---|---|
| Semantic layer | Turns a KPI into an official compiled definition. | "Finance-approved profit by SKU" defines metric codes, currency, date, and allowed dimensions. |
| Lineage graph | If no metric exists, finds the governed model path. | Orders or ads fallback should land on the canonical fact, not a random app/report table. |
| Curated query patterns | Converts old SQL into reusable guidance. | Repeated finance and ads dashboard logic becomes a small domain guide and evals. |
| Business context | Resolves names, launches, owners, and team language. | Feishu can help interpret a product nickname or launch phrase, but it is not product truth and it does not calculate the number. Snowflake canonical sources and Product/Ops identity do that. |
What is bad or missingHooray has pieces of this ladder, but not yet the full enforced ladder. The agent catalog is useful, but it is not the same as a compiled semantic layer. Historical queries and app outputs are not yet systematically distilled into curated patterns. Business context is not yet a governed analytics context layer.
Article concept we missedAnthropic tried using an LLM to bootstrap semantic metric definitions from raw tables and query logs. It produced plausible definitions that encoded the same ambiguity they were trying to remove. Their recommendation is subtle but important: use Claude to generate documentation drafts, but have a human own the definition.
| Task | Claude Can Help | Human Must Own |
|---|---|---|
| Draft metric docs | Summarize existing metric rows, propose descriptions, flag missing owner/grain/freshness fields. | Approve the actual business definition of profit, revenue, contribution margin, active listing, or sellable inventory. |
| Find ambiguity | Compare old finance queries, dashboard logic, and table descriptions to surface conflicting formulas. | Choose the one official definition and deprecate or label the near-duplicates. |
| Maintain docs | Open suggested doc updates when a model changes or a user correction appears. | Merge the change only after the owning finance/Product/Ops person confirms the definition is correct. |
Hooray example: Claude can draft a definition for "Amazon US profit for Livatro LVHD7001 humidifier" from finance metric-code rows. But a finance owner must decide which fees, currency rule, return treatment, ad costs, and listing scope are officially part of profit. If Claude owns that definition, it may create a very convincing but unofficial metric.
Correct implementationFor a top KPI, the agent should not ask "which table has revenue?" It should ask the semantic layer for the revenue metric. Only if that fails should it use lineage to choose one governed model. Only if that fails should it stop and request a human-owned definition.
Case answer: for the Amazon US profit example, business context may help the agent understand that Livatro LVHD7001 humidifier is a product phrase, but that phrase is not the source of truth. The trust ladder should resolve identity through Product/Ops, resolve profit through the semantic layer, and only then calculate. If the semantic layer does not know official profit, the agent may show component rows from the canonical finance path, but it should not invent final profit from ad hoc SQL.
Self-checkCan Feishu context or a product nickname calculate profit?
Answer: No. Business context can help interpret words, but Product/Ops identity and Snowflake canonical sources are what resolve identity and calculate metrics.
Second check: Can Claude draft the profit documentation? Yes. Can Claude be the owner of the profit definition? No. A human finance owner must own the definition.
8. Skills: The Agent's Operating Manual
MechanismIf sources of truth are declarative knowledge, skills are procedural knowledge. A source says what a metric means. A skill says what order to check sources, what traps to avoid, when to ask a question, when to stop, how to review the answer, and what proof to show.
Hooray warehouse skill should teach this workflow
- Check red flags: PII, leadership-bound answers, customer/vendor-facing use, irreversible actions.
- Extract stated facts first: date window, platform, market, product/listing term, and business decision.
- Ask only for missing blockers. Do not ask the user to restate facts already present in the question.
- Resolve Product/Ops identity: SPU, SKU, exact listing identity, seller affiliation.
- Try semantic metrics first for finance, ads, orders, inventory, and mapping completeness.
- If no semantic metric exists, use the domain router and lineage to choose one canonical model.
- Reject deprecated DWD/DWM/DWS/app/report tables as first answers.
- Run freshness, anomaly, and required-filter checks before summarizing.
- Adversarially review the source, grain, filters, and identity level.
- Answer with provenance, freshness, owner, confidence, limitations, and approval boundary.
What pairwise skills meansThe article is not describing one giant warehouse manual. It is describing a pair: a thin knowledge skill that routes the question, plus a process skill that teaches the agent how to conduct the analysis. The router answers "where should I look?" The process skill answers "how should I work once I know where to look?"
| Skill Layer | What It Does | Hooray Example |
|---|---|---|
| Top-level knowledge skill | Loads first for warehouse questions. It tries the semantic layer first, then routes to the relevant domain reference files only if more detail is needed. | Question mentions Amazon profit. Router opens Finance/profit and Product/Ops identity references, not the whole warehouse. |
| Domain reference files | Small LLM-readable files for each domain: finance, ads, orders, inventory, Product/Ops identity. They describe tables, columns, joins, grain, exclusions, and gotchas. | Finance reference says to use fct_finance_item_metric_daily, drill to fct_finance_ledger_lines only for traceability, and reject old profit-looking tables as first-answer sources. |
| Analysis process skill | Teaches senior-analyst behavior: clarify only true blockers, find sources, run query, inspect results, adversarially review, and answer with provenance. | For Livatro LVHD7001 humidifier, the agent extracts Amazon US and June 6, resolves identity, checks the finance route, reviews source/grain/filter risk, then answers or blocks. |
| Reusable analysis patterns | Stores patterns that repeat across work, so agents do not rediscover the same method each time. | Finance bridge analysis, ads attribution-window analysis, inventory freshness checks, and mapping-completeness checks each get a reusable pattern. |
Article coverage summaryAnthropic's skill docs are written for retrieval by an LLM, not as generic internal wiki pages. They describe grain, scope, exclusions, usage, when not to use a table, gotchas, common patterns, and cross-references. They also warn that skill docs go stale quickly unless maintenance is treated like engineering work.
| Skill Design Concept | Why It Matters | Hooray Learning Example |
|---|---|---|
| Reference docs written for LLM retrieval | The agent needs routing triggers and gotchas, not long narrative docs. | A finance reference should say: "IF the question asks for Amazon profit, route through fct_finance_item_metric_daily, drill to fct_finance_ledger_lines only for traceability, and apply the Finance-owned profit rollup. DO NOT use old profit subject/day tables as first answer." |
| Semantic layer first; do not bail early | The article warns against skipping the semantic layer just because a question looks custom. The skill should force the agent to try governed metrics before raw SQL. | If "profit" is missing as a final metric, the agent may use canonical component rows, but it must not invent final profit from ad hoc SQL. |
| Reference-file skeleton | Each domain file should have predictable sections: when to use, grain, required filters, allowed joins, when not to use, freshness, gotchas, examples, and cross-references. | The Product/Ops identity reference should explicitly explain SPU vs SKU vs exact listing identity vs seller affiliation, with examples of when each is required. |
| Process skill and adversarial review | The skill is not just a source list. It teaches the agent to challenge its own source, grain, filters, identity mapping, date handling, and metric definition before presenting an answer. | Before answering profit, the agent should ask: Did I use the correct market, date, identity grain, Finance-owned rollup, currency rule, and deprecated-table exclusion? |
| Maintenance as first-class engineering | Data models change daily. Skill docs decay unless model changes force doc changes. | If a Product/Ops listing resolver changes identity behavior, the reference doc and identity evals should change with it. |
| Colocation and review hooks | Skills should live close enough to the dbt/semantic model changes that they are reviewed together, not updated later from memory. | A PR that changes an agent-facing finance model should update the finance reference and evals, or explicitly explain why no skill change is needed. |
| Same answer across every surface | A skill is only canonical if Codex, CyberFlo, BI-adjacent tools, and scheduled reports all use the same routing rules. | "Amazon US profit for Livatro LVHD7001" should route to the same finance metric definition everywhere. |
| Portability across tools | A warehouse skill should not depend on one local path or one agent surface. The same reference package should be syncable into Codex, CyberFlo, MCP tools, and scheduled analysis jobs. | Finance/profit routing should live in the shared data contract, not only in one Codex prompt or one dashboard note. |
What is bad or missingCurrent guidance exists in pieces, but it is not yet a required, versioned, synced warehouse skill used by every analytics surface. Without that, Codex, CyberFlo, BI-adjacent tools, and ad hoc analysis can follow different habits.
The larger gap is that the dbt codebase is still mostly source material for a skill, not the full skill system. A real Anthropic-style setup would extract the dbt lineage, owners, grains, tests, and model descriptions into domain reference files, then tie those files to model-change review and evals.
Correct implementationCreate a thin top-level warehouse knowledge skill plus a separate analysis process skill. The top-level skill routes the question. Domain files teach finance, ads, orders, inventory, and Product/Ops identity gotchas. The process skill teaches the agent to clarify only blockers, use semantic metrics first, run the query, adversarially review the result, and answer with provenance. A model or metric change should require a skill/reference-doc/eval update or explicit exemption.
Case answer: a skill is successful when two agents given the same finance or product-identity question follow the same sequence: extract stated facts, resolve identity, select canonical metric/source, run validation, then answer or block. If one agent jumps straight to a plausible old table while another uses the CORE finance metric route, the skill is not enforced enough.
Self-checkWhat proves a warehouse skill is working?
Answer: Different agents follow the same safe sequence on the same question: stated facts, identity resolution, canonical metric/source, validation, then answer or block.
Second check: What should trigger a skill doc update? A model, metric, grain, resolver, or gotcha change that affects how agents should answer.
9. Validation: Practice Tests And Live Seatbelts
MechanismValidation answers the question: "Do we know the agent is choosing the right path?" Anthropic separates offline evals, ablations, online validation, provenance, monitoring, and correction harvesting. This is the part that turns principles into an operating system.
Evals are not discussion prompts. Each eval needs an expected source path, expected identity grain, expected stop conditions, and pass/fail criteria.
ELI5An eval is a practice stakeholder question with a known grading rule. It is not only checking whether the final number matches. It checks whether the agent used the right source, the right identity level, the right filters, and stopped when the business definition was missing.
| Eval Source | What The Article Means | Hooray Example |
|---|---|---|
| Dashboard-based evals | Claude looks at common dashboards and stakeholder workflows, proposes common business questions, then humans validate the expected behavior. | A finance dashboard suggests: "What was Amazon US profit for Livatro LVHD7001 on June 6, 2026?" Finance validates that the agent must resolve Product/Ops identity, use the CORE finance metric route, and block if final profit rollup is not approved. |
| Long-tail evals | Claude reads business context, roadmaps, table docs, and domain references to invent plausible but less common questions across the rest of the domain. | Product/Ops docs lead Claude to propose: "How is FP-B2P performing?" The expected behavior is not to calculate immediately; it is to block until SPU, SKU, or exact listing identity is resolved. |
| Correction-harvested evals | Every stakeholder correction is treated as a candidate eval because it reveals a real failure mode. | If a teammate says "wrong table; this should exclude seller X," that becomes a new eval, a reference-doc update, or a deprecated-source rule. |
| Eval | What It Tests | Expected Agent Behavior |
|---|---|---|
Amazon US profit for Livatro LVHD7001 humidifier on June 6, 2026. |
Canonical finance route and metric-code logic. | Pass: treat the phrase as lookup only, keep scope Amazon US and June 6 only, resolve Product/Ops identity, use governed finance metrics, and block if final profit definition is missing. Fail: ask whether to include all Amazon markets or use old profit report tables as first answer. |
How is FP-B2P performing? |
Product/Ops identity ambiguity. | Pass: say the sample is marketplace/seller evidence, not resolved SPU, SKU, or exact listing identity; route to Product/Ops resolver before calculation. Fail: calculate product performance from the seller-SKU-style clue alone. |
| Show latest Lazada sales. | Staleness and data-quality stop condition. | Pass: detect impossible future dates like 58395-06-15, stop before trend output, and report affected platform/count/owner route. Fail: chart latest sales using the bad future dates. |
| Which search terms drove sales on June 5, 2026? | Retrieval and required ads filters. | Pass: use approved ads path and state 7d attributed sales. Fail: describe it as generic sales, profit, organic sales, or all-window sales. |
How much stock do we have for LVCUCM001-DE-KY-SC? |
Inventory snapshot and identity grain. | Pass: state snapshot date, countries included, identity level, and whether item ID B0FH4QL9GN is only listing evidence or a resolved listing identity. Fail: blend inventory across SKU/listing/country because the item ID looked familiar. |
What is bad or missingWarehouse tests and validators are useful, but they are not the same as agent evals. A table can pass dbt tests while an agent still chooses the wrong table, wrong identity level, or wrong attribution window. Hooray does not yet have a fixed business-question eval suite, accuracy telemetry, domain launch gates, or correction-harvesting loop.
Correct implementationStart with a few dozen evals per high-risk domain. Store every run with skill version, git SHA, model ID, source path, per-assertion pass/fail, latency, wall-clock time, and token cost. When a user corrects an answer, turn that correction into a reference-doc fix and a new eval. Use ablations: run the same evals before and after a guidance change to prove whether it helped.
Article concepts we missedAnthropic treats validation as more than "write some tests." Their stack includes human-validated evals, grounded snapshots, telemetry-style result storage, launch gates, ablations, online provenance, passive monitoring, and correction harvesting.
| Validation Concept | Plain Meaning | Hooray Learning Example |
|---|---|---|
| Claude can generate eval candidates; humans validate them | Claude can draft dashboard-based and long-tail questions, but humans confirm expected behavior. | Claude can propose "How is FP-B2P performing?" as an eval. Product/Ops must validate that the expected answer is "block until identity resolution." |
| Anchor ground truth | Live-number evals drift. Pin snapshot dates or grade the query/source path instead of only the final number. | The Lazada eval should pin the profile row with MAX_REPORT_DATE = 58395-06-15, not depend on whatever today's live max date becomes. |
| Grade the path when the number moves | If the underlying business number changes daily, grade whether the agent chose the right model, filters, grain, and blocker instead of grading a stale number. | For profit, assert that the agent used Product/Ops identity resolution and fct_finance_item_metric_daily, avoided old profit tables, and blocked if the Finance-owned profit rollup was missing. |
| Dependency-aware CI | A pull request touching a model, metric, resolver, or skill should rerun the affected evals instead of waiting for a manual audit later. | A change to fct_finance_item_metric_daily should rerun finance/profit evals and any Product/Ops identity evals that depend on finance item grain. |
| Ablate changes | Hold the eval set fixed, change one thing, and compare pass rates. | If adding old dashboard SQL retrieval does not improve FP-B2P or profit answers, the bottleneck is structure, not access to more examples. |
| Store eval results like telemetry | Each run lands in a warehouse table with skill version, git SHA, model ID, eval ID, assertion results, source path, token count, and wall-clock time. | Then "did the new finance skill help?" becomes a query over eval history, not a debate over one test run. |
| Provenance footer | Every answer should show source tier, freshness, and owner so users can judge trust. | "Source: fct_finance_item_metric_daily plus Finance-owned profit rollup. Freshness: report date 2026-06-06. Owner: Finance. Identity: Product/Ops resolved listing." |
| Correction harvesting | User corrections become reference-doc fixes and new evals. | If a teammate says "wrong table; this should exclude seller X," that correction should create a doc update and an eval, not disappear in chat history. |
| Silent failure mitigation | The hardest failures are plausible wrong answers nobody corrects. | Leadership-bound finance answers should require provenance, stronger review, and human sign-off before being forwarded. |
Self-checkWhat makes an eval useful instead of just a discussion question?
Answer: It has an expected source path, identity grain, stop condition, and pass/fail criteria. The agent either followed the contract or it did not.
Second check: If a user corrects an agent answer, what should happen? Turn the correction into a reference-doc fix and a new eval tied to the owning domain.
10. What A Trustworthy Agent Answer Looks Like
Bad answer"Profit for Livatro LVHD7001 was $52.50 yesterday."
Why bad: no source, no metric definition, no identity level, no date freshness, no currency rule, no mention of legacy tables avoided, and no confidence boundary. It sounds useful, but the user cannot inspect correctness.
Better answer shape"The requested scope is Amazon US on June 6, 2026. The phrase Livatro LVHD7001 humidifier is only a lookup clue. I will not treat the human-readable name as source of truth. Before claiming SKU-level or listing-level profit, I need to resolve it to canonical Product 360 SKU and exact Amazon US listing identities. Once resolved, I will use fct_finance_item_metric_daily and report components such as total_sales_quantity, total_sales_amount, platform_fee, and fba_delivery_fee. If ledger-line proof is needed, I will drill down to fct_finance_ledger_lines. Source tier: CORE finance metric route plus Finance-owned profit rollup. Confidence: product phrase understood, final-profit value not safe until canonical identity and metric rollup are resolved."
Better blocker shape"In the current_walkthrough_samples extract, the amazon_listing_mapping row for FP-B2P shows account DINGYAO_SC_01, marketplace/domain A1AM78C64UM0Y8, ASIN B0DDTK4PKV, seller SKU FP-B2P, and country MX. That is marketplace-listing evidence. From this sample alone, I have not resolved it to SPU, Product 360 SKU, or exact listing identity. I should not calculate performance yet. Next step: run Product/Ops identity resolution; if it returns multiple possible listings or SKU mappings, route mapping repair or ask only for the missing listing scope."
The difference is not style. The better answer exposes the contract. It lets a non-data-expert understand what was resolved, what was calculated, what was avoided, and what still needs a governed definition.
11. The First Practical Package
Anthropic's practical starting point is intentionally small: a handful of canonical datasets, a few dozen offline evals, and a thin knowledge skill. For Hooray, the first package should be narrow and excellent, but it must include human ownership and maintenance loops from day one.
| Domain | First Canonical Contract | First Stop Condition |
|---|---|---|
| Finance/profit | Metric-code definitions, official profit rollups, currency rules, Product/Ops identity grain. | Old profit table selected as first answer. |
| Product/Ops identity | SPU, SKU, exact listing identity, seller affiliation, ambiguous mapping statuses. | One term maps to multiple identity levels or listings. |
| Orders/sales | Canonical order fact, platform/domain/date conventions, freshness checks. | Invalid future dates or catalog/live scope mismatch. |
| Ads | Approved ads facts, attribution window, spend/sales definitions. | Attributed sales without attribution window. |
| Inventory | Snapshot date, included locations/countries, identity grain. | Snapshot missing or product/listing identity unresolved. |
| Operating Piece | Minimum Version | Why It Belongs In The First Package |
|---|---|---|
| Human definition owner | Every top metric has a named Finance/Product/Ops owner. | Claude can draft docs, but a human must own the business definition. |
| Metadata contract | Source tier, grain, owner, freshness, valid dimensions, deprecated status. | Agents need legible warehouse products, not just tables. |
| Skill maintenance trigger | Model or metric changes require reference-doc/eval review. | Skills decay unless maintenance is tied to engineering workflow. |
| Provenance footer | Every answer reports source tier, freshness, owner, identity grain, and confidence boundary. | It helps non-data-experts judge whether to trust or escalate the answer. |
| Correction loop | Corrections become doc updates and new evals. | Without this loop, the same wrong answer returns later. |
Self-learning takeawayThe data team's job is not to make every warehouse table easier for AI to browse. The job is to make the right answer path impossible to miss, make wrong paths hard to reach, and make every answer carry enough provenance for a business user to trust or challenge it.
Done means: every launch domain has at least one answered example, one blocked example, one human definition owner, one canonical source path, one identity-grain rule, one metadata contract, one maintenance trigger, one provenance footer shape, and one eval that catches the most tempting wrong answer.
12. Takeaways For The Data Team
If the data team remembers only one thing from the Anthropic article, it should be this: self-service AI analytics is not solved by giving agents more tables. It is solved by making the correct business path owned, documented, enforced, and hard to confuse with near-duplicates.
The two quoted ideas are connected but not identical. The canonical-datasets quote is the model-design rule: reduce the number of plausible truth paths. The documentation quote is the operating rule: keep those truth paths legible, current, and human-owned even when Claude helps draft the docs.
| Takeaway | What It Means | What Hooray Should Build |
|---|---|---|
| Fewer governed models beat more accessible tables. | The agent should not browse dozens of plausible revenue, profit, order, ads, or inventory tables. It should find one governed answer path. | Curate a small set of canonical datasets for finance, Product/Ops identity, orders, ads, and inventory. Label old report tables as deprecated, diagnostic, or unsafe for first-answer analytics. |
| Claude can draft docs; humans own definitions. | Claude is useful for drafting column descriptions, summarizing query patterns, proposing metric docs, and flagging missing documentation. But Claude should not decide what profit, revenue, contribution margin, active listing, or sellable inventory officially means. | Assign human owners for top metrics and identity rules. Require owner approval when metric definitions, grains, currency rules, inclusion rules, or Product/Ops identity behavior change. |
| Documentation only works when enforced by tooling. | A wiki page saying "use the official model" is not enough. Agents and humans will drift back to familiar old tables unless the system routes them correctly. | Create an agent-safe routing layer: semantic metrics first, canonical models second, approved mechanical rollups third, diagnostics only with explicit purpose. CI should flag undocumented, ownerless, or conflicting models. |
| Lineage turns "I do not know the metric" into "I know the governed model." | If a semantic metric is missing, the agent still should not search randomly. dbt lineage and model reference counts can show which upstream model feeds the concept, which models share grain, and which paths are deprecated. | Expose dbt lineage, model tier, ownership, deprecation status, grain, and reference count to the agent. Use reference count as a signal, not truth, because old legacy tables can also be highly referenced. |
| Rollups and caches are allowed only as mechanical copies. | A fast summary is safe only if it derives from the canonical model using the same metric version, grain, and refresh rule. Otherwise it becomes a shadow source of truth. | Every rollup should declare its parent canonical model, metric version, grain, refresh cadence, and owner. If it cannot trace back, hide it from first-answer analytics. |
| Every answer needs provenance because users cannot validate hidden correctness. | The business user should not have to know whether the agent picked the right warehouse path. The answer must expose source, freshness, owner, identity grain, confidence, and blockers. | Standardize answer footers for finance, product identity, orders, ads, and inventory. A high-impact answer without provenance should be treated as incomplete. |
| Corrections must become system improvements. | If a user says "wrong table" or "wrong definition," that cannot stay as a one-off chat correction. It must update docs, routing, evals, or ownership metadata. | Build a correction loop: every meaningful correction creates a reference-doc change, a new eval, or a deprecation/ownership fix. |
Bottom-line standardA model is not agent-ready because it exists, has a good name, or is often queried. It is agent-ready only when it has an owner, grain, source tier, lineage, freshness rule, allowed-use rule, deprecation status for alternatives, and eval coverage for the most tempting wrong answer.