Defend a Proxy Label Choice Under VP Cross-Examination

You are presenting your ML system design for a video learning platform to the VP of Product. The model's optimization target is 30-day completion rate as a proxy for "educational value." The VP listens, then asks three questions in sequence. Answer each one as you would in the room — directly, honestly, and without losing the room.

1. "How do you know that completion predicts learning, and not just that users prefer short videos?"
2. "If we optimize this proxy for 12 months, what does the product look like at the end of that period?"
3. "What specific signal would make you change the proxy in production?"

The hidden test: the VP is not trying to kill the proxy choice. She is testing whether you understand its limitations and have a plan to monitor and correct for them. A candidate who defends the proxy as perfect fails. A candidate who abandons it under pressure also fails.

Follow-up ladder

1. Rung 1: The VP asks: "Our competitor uses explicit quiz scores as their proxy. Why are you not doing that?" How do you respond?
2. Rung 2: Six months in, you notice that average content length has dropped from 18 minutes to 11 minutes per piece, and creator churn is up — shorter formats pay less per piece. Is this a proxy problem? Who owns this issue?
3. Rung 3: A/B test results: the model optimized for completion rate shows a 12% lift in completion, but subscription renewal rates are flat. The VP asks if the proxy is working. How do you answer?