Uplift Modeling with Delayed Feedback: Identifiability and Algorithms

Uplift modeling has obtained significant attention, with broad applications in medicine, economics, and marketing. For example, in a push notification scenario, accurately estimating the uplift of different push frequencies on user activation and notification switch close rate is critical for balancing user experience and business goals. Existing methods only use binary labels, i.e., convert or not within the observational window. However, they ignore time information (e.g., users who convert on day 1 vs. day 14 reflect different sensitivities) and fail to model potential closures outside the window, i.e., due to treatments always taking time to manifest causal impacts on outcomes, the potential outcomes of interest cannot be observed promptly and accurately. Failing to account for these issues can result in skewed uplift modeling. To address this gap, this work examines how observation timing influences the assessment of uplift by explicitly modeling the potential response time. Theoretical analysis establishes the conditions for identifiability under delayed feedback scenarios. We introduce CFR-DF (Counterfactual Regression with Delayed Feedback), a systematic framework that jointly learns both the latent response times and the underlying potential outcomes. Empirical evaluations on synthetic and real-world datasets, including an A/B test with over 1 billion users for 14 days, validate the approach, demonstrating its ability to handle temporal delays and improve estimation accuracy compared to previous uplift modeling methods.

Authors

• Chunyuan Zheng Peking University
• Anpeng Wu Department of Computer Science and Technology, Zhejiang University Mohamed bin Zayed University of Artificial Intelligence
• Chuan Zhou University of Michigan
• Taojun Hu ByteDance, Beijing, China
• Qingying Chen ByteDance, Beijing, China
• Hongyi Liu
• Chenxi Li
• Huiyou Jiang ByteDance, Beijing, China
• Haoxuan Li Peking University
• Zhouchen Lin National Key Lab of General AI, School of Intelligence Science and Technology, Peking University Peng Cheng Laboratory

Keywords

No keywords are indexed for this paper.