Handle Multi-Week Label Lag in a Search Ranking Reframe

A home-rental marketplace's search ranking team reframed their primary objective from "booking conversion rate" to "successful trip completion" — a composite signal combining booking, no cancellation, no serious disputes, and a positive review. This is a more aligned objective, but it creates a significant label lag: the signal only arrives several weeks after a booking, because you must wait for the trip to complete and the review to be submitted.

Walk through the architectural tradeoff this creates and how you would handle it.

• What does a multi-week label lag mean for the model's view of the world?
• How would you train on lagged labels while maintaining reasonable ranking quality?
• Where else in ML systems does objective lag create model risk, and how do you decide on the maximum acceptable lag?

Follow-up ladder

1. Rung 1: You use fast proxy features (listing quality score, host response time) as model inputs rather than as the optimization target. A platform policy change recomputes listing quality scores overnight. What breaks, and how would you catch it?
2. Rung 2: The market has a seasonal spike — summer bookings surge 3x. The model was trained mostly on off-peak data. How does the label lag interact with the seasonal distribution shift?
3. Rung 3: A stakeholder proposes reverting the primary objective back to booking conversion (no lag) and using successful trip completion only as a monitoring guardrail. Is this a reasonable compromise? What are the risks?