PART 05 · FLAGSHIP

AI × Analytics & BI

A controlled analytics assistant can translate approved questions into reviewed queries, draft explanations, and surface anomalies. It can also expose restricted data, generate invalid SQL, invent causes, amplify bias, or misstate uncertainty. Start read-only and evaluate each task against a reconciled baseline.

Source review · data contracts · causal humility · human interpretation

Natural Language BI

Turn a precise, authorized question into a reviewable query, result, and chart—with definitions and provenance attached.

BASELINE RISK

Metric definitions, joins, row-level security, timezone, currency, returns, and late events are implicit → two analysts can answer different questions with the same label.

CONTROLLED PILOT

Use a semantic layer and read-only identity → allowlisted schemas → show generated SQL and source freshness → enforce row-level access → qualified reviewer verifies totals and chart before use.

Evaluation evidence

Query validity
Representative set
Compare generated SQL and results with reviewed reference queries across ordinary, edge, permission-denied, and adversarial questions.
Access control
Must not regress
Test row, column, tenant, purpose, export, and aggregation restrictions; treat unauthorized disclosure as critical.
Total effort
Measure the baseline
Include clarification, review, correction, maintenance, and incident time—not generation time alone.
No vendor percentage from the legacy draft is retained as an expected result. Record tool/model version, test set, baseline, sample, failures, and observed pilot results.

Attribution Modeling

Attribution models answer different questions under different assumptions. None alone proves incremental effect.

BASELINE RISK

Platform and last-touch reports use different identities, windows, modeled events, and incentives → reported credit is mistaken for incrementality.

CONTROLLED PILOT

Reconcile spend and outcomes → document model assumptions and uncertainty → use geo/holdout or other appropriate experiments where feasible → have a qualified owner approve a capped reversible decision.

Evaluation evidence

Reconciliation
Before modeling
Verify currency, taxes, refunds, timezones, late conversions, channel naming, and total spend/outcomes.
Uncertainty
Never one magic number
Report intervals, sensitivity, priors, saturation, lag, collinearity, and out-of-sample checks.
Incrementality
Experiment where appropriate
Pre-register eligible population, assignment, metric, duration logic, contamination checks, and decision rule.
The old case numbers linked broad product pages that did not directly substantiate each claim. They have been removed rather than presented as evidence.
MYTH
"AI attribution gives you the definitive answer about what drove each sale."
Tap to see reality →
REALITY
No single model gives “the truth.” Platform attribution, multi-touch models, media-mix models, and experiments answer different questions and carry different assumptions. Report ranges and sensitivity, and treat a well-designed experiment as evidence within its population, period, execution, and validity limits—not universal ground truth.

Predictive Analytics

Estimate a defined outcome for a defined use—then test calibration, harm, drift, and whether acting on the score helps.

BASELINE RISK

Labels leak future information, historic policy becomes bias, scores are uncalibrated, and a high-risk customer is targeted without lawful purpose or explanation.

CONTROLLED PILOT

Define outcome and horizon → minimize authorized features → split by time → test calibration and subgroup error → run a consented capped intervention with holdout → monitor complaints and opt-outs.

Evaluation evidence

Calibration
By cohort and time
Measure discrimination, calibration, false positives/negatives, drift, and subgroup error with documented limits.
Decision value
Score ≠ outcome
Compare the approved intervention with a fair baseline; include incentives, cannibalization, contact cost, and customer harm.
Do not use inferred sensitive traits or prediction scores for consequential eligibility, pricing, employment, credit, health, or legal decisions without the applicable qualified and legal review.

Anomaly Detection

Detect a calibrated deviation, preserve context, and route it for investigation; an alert does not identify cause.

BASELINE RISK

Unversioned thresholds create noise; seasonality, outages, tracking changes, and late data are mistaken for business changes.

CONTROLLED PILOT

Calibrate by metric and cohort → enforce minimum volume and freshness → attach source and change log → alert only → named owner investigates alternatives before a reversible action.

Evaluation evidence

Precision
Alert quality
Replay known incidents and normal periods; measure useful alerts, noise, missed critical events, detection delay, and operator burden.
Triage
Hypotheses, not causes
Require evidence links, affected scope, uncertainty, alternative explanations, owner, severity, and runbook.
Legacy dollar and percentage claims were removed because the broad vendor links did not substantiate the exact result in context.

Data Quality & Governance

Data quality is an operating responsibility. Models can assist discovery; deterministic controls and accountable owners enforce it.

BEFORE

Duplicate customer records everywhere → PII scattered across systems without classification → schema changes break pipelines silently → "data cleanup" is an annual project that never finishes.

CONTROLLED PILOT

Inventory authorized systems → apply deterministic contracts and access policy → use a model only to propose classifications or matches → sample and review errors → route deletion, merge, or policy changes through accountable owners.

Evaluation evidence

Coverage by class
Not one accuracy score
Measure precision and recall for each sensitive-data class, language, format, and location; sample false negatives because missed PII matters.
Merge safety
Human-approved
Use explainable candidate matches, preserve originals, test collisions, log approvals, and support reversal.
A classifier percentage does not prove coverage across your languages, schemas, images, documents, or jurisdictions. Validate locally and layer controls.

Automated Reporting

A system can assemble a draft; a named analyst reconciles the data, checks claims, adds context, and approves recipients.

BASELINE RISK

Manual exports, inconsistent definitions, copy errors, stale snapshots, and undocumented commentary can make a report slow or unreliable. Measure the actual baseline before redesign.

CONTROLLED PILOT

Approved read-only extracts → deterministic reconciliations → cited narrative draft that separates facts, inference, and options → analyst verifies definitions and uncertainty → recipient and export approval → audit log.

Evaluation evidence

Factual agreement
Against reconciled tables
Verify every number, comparison period, segment, chart label, and citation; treat an invented cause or recommendation as a failure.
End-to-end time
Include review and repair
Measure total elapsed and human time, correction burden, incidents, and maintenance against the existing report.
The baseline worksheet below intentionally does not estimate savings. Enter observed pilot evidence into a separately reviewed business case.

Reference architecture: read-only analyst assistant

Use a dedicated least-privilege identity, allowlisted views and queries, tenant isolation, row-level security, result limits, sensitive-data controls, versioned metric definitions, source citations, evaluation traces, human approval, and an immediate kill switch. The model must not receive warehouse credentials or execute account changes.

GA4BigQueryLooker StudioMeta Ads APIGoogle Ads APITikTok API
Production readiness depends on the exact task, data contract, evaluation, owner acceptance, and ongoing monitoring.

Legacy discovery links — verify current primary evidence before citation

  1. ThoughtSpot Sage / Haleon — thoughtspot.com
  2. Power BI Copilot / EY — microsoft.com
  3. Tableau AI / Lenovo — tableau.com
  4. Looker + Gemini / Woolworths — cloud.google.com
  5. Google Meridian / Reckitt — blog.google
  6. Meta Robyn / HelloFresh — opensource.fb.com
  7. Northbeam / Ridge — northbeam.io
  8. Pecan AI / Payoneer — pecan.ai
  9. Obviously AI / Schneider — obviously.ai
  10. Google Vertex AI / Carrefour MENA — cloud.google.com
  11. Anodot / Lyft — anodot.com
  12. Monte Carlo / JetBlue — montecarlodata.com
  13. Soda / Bacardi — soda.io
  14. Collibra / ING Bank — collibra.com
  15. Ataccama / Allianz — ataccama.com
  16. Narrative BI / Wix — narrativebi.com
  17. Polymer AI / Jellyfish — polymersearch.com
  18. Copilot / L'Oréal MENA — microsoft.com