8 Interview Cases and Decision Frameworks

Strong interview answers make the reasoning traceable. Do not start with a favorite algorithm and do not invent results. State assumptions and ask for missing business context.

8.1 A reusable structure

  1. Clarify the decision. What action, audience, geography, and horizon are in scope?
  2. Define success. Choose a primary outcome, guardrails, and unit economics.
  3. Map the mechanism. Explain how the action could affect the outcome and where bias may enter.
  4. Assess data. Identify sources, grain, eligibility, logging, privacy, and missing populations.
  5. Choose the design. Prefer an experiment; otherwise state the assumptions of the observational method.
  6. Validate. Check balance, leakage, uncertainty, robustness, and business plausibility.
  7. Recommend and monitor. Connect evidence to an action, risk, rollout, and follow-up measurement.

8.2 Case 1: Upper-funnel campaign

Prompt: How would you evaluate a video campaign intended to increase brand awareness and later sales?

A concise answer:

  • define the eligible audience, markets, campaign period, and outcomes;
  • use reach and frequency as delivery diagnostics, not proof of impact;
  • randomize users, households, or geographies when feasible;
  • measure a brand outcome and downstream sales with appropriate follow-up;
  • check spillovers, contamination, sample-ratio mismatch, and heterogeneous effects specified before analysis; and
  • translate the effect interval into incremental value and a rollout decision.

If randomization is unavailable, propose a credible comparison design and state its identifying assumptions. Do not claim that view-through attribution proves incrementality.

8.3 Case 2: Conversion rate fell

Prompt: Conversion rate dropped 15%. What do you investigate?

First clarify whether 15% is absolute or relative and whether the denominator changed. Then:

  1. validate event definitions, releases, consent changes, and missing data;
  2. segment by device, geography, channel, landing page, and new versus returning users;
  3. decompose the rate into traffic mix and within-segment performance;
  4. check price, promotion, inventory, site latency, and competitive events;
  5. compare against seasonality and a suitable baseline; and
  6. propose the smallest test that distinguishes leading explanations.

8.4 Case 3: Reallocate media budget

Prompt: Leadership wants to move 20% of spend to the channel with the highest ROAS.

Explain that average attributed ROAS is not marginal causal return. Review the attribution specification, contribution margin, saturation, capacity, and channel interactions. Combine incrementality experiments with a calibrated MMM when possible. Recommend a constrained reallocation followed by a prospective test rather than a full immediate shift.

8.5 Case 4: Retention offer

Prompt: Which customers should receive a discount to prevent churn?

A churn model ranks risk but not treatment response. Define the eligible population and economic outcome, exclude customers who cannot or should not be contacted, and randomize the offer within policy constraints. Estimate incremental retention and margin, including discount cost and pull-forward. Use uplift modeling only when treatment and outcome data support it, and validate policies prospectively.

8.6 Case 5: Build a look-alike audience

Clarify the seed: recent purchasers, profitable customers, or incremental responders imply different targets. Establish a feature cutoff, prevent outcome leakage, compare against a simple baseline, and evaluate ranking plus calibration. Audit proxy and privacy risks. The final test is incremental value from targeting, not offline classification accuracy alone.

8.7 Communicate uncertainty

Prefer:

The estimated lift is positive, but the interval includes effects too small to cover campaign cost. I recommend extending the planned sample or running a limited follow-up rather than scaling now.

Avoid:

The p-value is above 0.05, so the campaign had no effect.

Failure to reject is not proof of zero effect. Discuss the effect range that remains compatible with the data.

8.8 Personal experience answers

Use the STAR structure only with verified experience:

  • Situation: necessary context;
  • Task: your actual responsibility;
  • Action: decisions and work you personally performed; and
  • Result: measured outcome with an honest source and scope.

If exact numbers are confidential or uncertain, describe direction, decision, or validated process without inventing precision. Label hypothetical cases as hypothetical.

8.9 Final interview checklist

  • Did I answer the business decision?
  • Did I define the population, outcome, comparison, and horizon?
  • Did I separate prediction, attribution, and causality?
  • Did I mention major assumptions and failure modes?
  • Did I connect uncertainty to an action?
  • Did I avoid claiming experience or results I cannot support?