Uplift Modelling via Gradient Boosting
Pages 1177 - 1187
Abstract
The Gradient Boosting machine learning ensemble algorithm, well-known for its proficiency and superior performance in intricate machine learning tasks, has encountered limited success in the realm of uplift modeling. Uplift modeling is a challenging task that necessitates a known target for the precise computation of the training gradient. The prevailing two-model strategies, which separately model treatment and control outcomes, are encumbered with limitations as they fail to directly tackle the uplift problem.
This paper presents an innovative approach to uplift modeling that employs Gradient Boosting. Unlike previous works, our algorithm utilizes multioutput boosting model and calculates the uplift gradient based on intermediate surrogate predictions and directly models the concealed target. This method circumvents the requirement for a known target and addresses the uplift problem more effectively than existing solutions.
Moreover, we broaden the scope of this solution to encompass multitreatment settings, thereby enhancing its applicability. This novel approach not only overcomes the limitations of the traditional two-model strategies but also paves the way for more effective and efficient uplift modeling using Gradient Boosting.
Supplemental Material
MP4 File - Uplift Modelling via Gradient Boosting: Promo
The video will explain how we solve the uplift modeling problem using Gradient Boosting and how we overcome the issues with existing methods.
- Download
- 33.79 MB
References
[1]
Susan Athey and Guido Imbens. 2016. Recursive partitioning for heterogeneous causal effects. Proceedings of the National Academy of Sciences 113, 27 (2016), 7353--7360.
[2]
Susan Athey and Guido W Imbens. 2015. Machine learning methods for estimating heterogeneous causal effects. stat 1050, 5 (2015), 1--26.
[3]
Ismail Babajide Mustapha and Faisal Saeed. 2016. Bioactive molecule prediction using extreme gradient boosting. Molecules 21, 8 (2016), 983.
[4]
Olivier Chapelle and Yi Chang. 2011. Yahoo! learning to rank challenge overview. In Proceedings of the learning to rank challenge. PMLR, 1--24.
[5]
Tianqi Chen and Carlos Guestrin. 2016. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining. 785--794.
[6]
Chen Cheng, Fen Xia, Tong Zhang, Irwin King, and Michael R Lyu. 2014. Gradient boosting factorization machines. In Proceedings of the 8th ACM Conference on Recommender systems. 265--272.
[7]
Lei Deng, Juan Pan, Xiaojie Xu, Wenyi Yang, Chuyao Liu, and Hui Liu. 2018. PDRLGB: precise DNA-binding residue prediction using a light gradient boosting machine. BMC bioinformatics 19, 19 (2018), 135--145.
[8]
Floris Devriendt, Jente Van Belle, Tias Guns, and Wouter Verbeke. 2020. Learning to rank for uplift modeling. IEEE Transactions on Knowledge and Data Engineering 34, 10 (2020), 4888--4904.
[9]
Benoît Frénay and Michel Verleysen. 2013. Classification in the presence of label noise: a survey. IEEE transactions on neural networks and learning systems 25, 5 (2013), 845--869.
[10]
Jerome H Friedman. 2001. Greedy function approximation: a gradient boosting machine. Annals of statistics (2001), 1189--1232.
[11]
Donald P Green and Holger L Kern. 2012. Modeling heterogeneous treatment effects in survey experiments with Bayesian additive regression trees. Public opinion quarterly 76, 3 (2012), 491--511.
[12]
Pierre Gutierrez and Jean-Yves Gérardy. 2017. Causal inference and uplift modelling: A review of the literature. In International conference on predictive applications and APIs. PMLR, 1--13.
[13]
Behram Hansotia and Brad Rukstales. 2002. Incremental value modeling. Journal of Interactive Marketing - J INTERACT MARK 16 (06 2002), 35--46. https://doi. org/10.1002/dir.10035
[14]
Jennifer L Hill. 2011. Bayesian nonparametric modeling for causal inference. Journal of Computational and Graphical Statistics 20, 1 (2011), 217--240.
[15]
Leonid Iosipoi and Anton Vakhrushev. 2022. SketchBoost: Fast Gradient Boosted Decision Tree for Multioutput Problems. Advances in Neural Information Processing Systems 35 (2022), 25422--25435.
[16]
Shahin Jabbari, Robert C Holte, and Sandra Zilles. 2012. Pac-learning with general class noise models. In KI 2012: Advances in Artificial Intelligence: 35th Annual German Conference on AI, Saarbrücken, Germany, September 24--27, 2012. Proceedings 35. Springer, 73--84.
[17]
Edward H Kennedy. 2020. Towards optimal doubly robust estimation of heterogeneous causal effects. arXiv preprint arXiv:2004.14497 (2020).
[18]
Sören R Künzel, Jasjeet S Sekhon, Peter J Bickel, and Bin Yu. 2019. Metalearners for estimating heterogeneous treatment effects using machine learning. Proceedings of the national academy of sciences 116, 10 (2019), 4156--4165.
[19]
Erin LeDell and Sebastien Poirier. 2020. H2o automl: Scalable automatic machine learning. In Proceedings of the AutoML Workshop at ICML, Vol. 2020.
[20]
Ping Li, QiangWu, and Christopher Burges. 2007. Mcrank: Learning to rank using multiple classification and gradient boosting. Advances in neural information processing systems 20 (2007), 897--904.
[21]
Xiaoliang Ling, Weiwei Deng, Chen Gu, Hucheng Zhou, Cui Li, and Feng Sun. 2017. Model ensemble for click prediction in bing search ads. In Proceedings of the 26th International Conference on World Wide Web Companion. 689--698.
[22]
Xinkun Nie and Stefan Wager. 2021. Quasi-oracle estimation of heterogeneous treatment effects. Biometrika 108, 2 (2021), 299--319.
[23]
Yoshihiko Ozaki, Yuki Tanigaki, Shuhei Watanabe, and Masaki Onishi. 2020. Multiobjective tree-structured parzen estimator for computationally expensive optimization problems. In Proceedings of the 2020 genetic and evolutionary computation conference. 533--541.
[24]
Nicholas J Radcliffe and Patrick D Surry. 2011. Real-world uplift modelling with significance-based uplift trees. White Paper TR-2011--1, Stochastic Solutions (2011), 1--33.
[25]
Claudia Shi, David Blei, and Victor Veitch. 2019. Adapting neural networks for the estimation of treatment effects. Advances in neural information processing systems 32 (2019).
[26]
Xiaogang Su, Joseph Kang, Juanjuan Fan, Richard Levine, and Xin Yan. 2012. Facilitating Score and Causal Inference Trees for Large Observational Data. Journal of Machine Learning Research 13 (10 2012), 2955--2994.
[27]
Xiaogang Su, Chih-Ling Tsai, Hansheng Wang, David M. Nickerson, and Bogong Li. 2009. Subgroup Analysis via Recursive Partitioning. Econometrics: Single Equation Models eJournal (2009). https://api.semanticscholar.org/CorpusID: 12755477
[28]
Samir Touzani, Jessica Granderson, and Samuel Fernandes. 2018. Gradient boosting machine for modeling the energy consumption of commercial buildings. Energy and Buildings 158 (2018), 1533--1543.
[29]
Ilya Trofimov, Anna Kornetova, and Valery Topinskiy. 2012. Using boosted trees for click-through rate prediction for sponsored search. In In Proceedings of the Sixth International Workshop on Data Mining for Online Advertising and Internet Economy. 1--6.
[30]
Stefan Wager and Susan Athey. 2018. Estimation and inference of heterogeneous treatment effects using random forests. J. Amer. Statist. Assoc. 113, 523 (2018), 1228--1242.
[31]
Yanru Zhang and Ali Haghani. 2015. A gradient boosting method to improve travel time prediction. Transportation Research Part C: Emerging Technologies 58 (2015), 308--324.
[32]
Kailiang Zhong, Fengtong Xiao, Yan Ren, Yaorong Liang, Wenqing Yao, Xiaofeng Yang, and Ling Cen. 2022. Descn: Deep entire space cross networks for individual treatment effect estimation. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 4612--4620.
Index Terms
- Uplift Modelling via Gradient Boosting
Recommendations
Numerical and centrifuge modelling of the uplift behaviour of piles in cohesionless soil
ICECT'03: Proceedings of the third international conference on Engineering computational technologyAxisymmetric finite element analyses incorporating hyperbolic soil and soil/pile interface relationships investigating the uplift behaviour of straight-shafted piles in cohesionless soil are described. Theoretical results were strongly influenced by ...
Boosting recombined weak classifiers
Boosting is a set of methods for the construction of classifier ensembles. The differential feature of these methods is that they allow to obtain a strong classifier from the combination of weak classifiers. Therefore, it is possible to use boosting ...
Using boosting to prune bagging ensembles
Boosting is used to determine the order in which classifiers are aggregated in a bagging ensemble. Early stopping in the aggregation of the classifiers in the ordered bagging ensemble allows the identification of subensembles that require less memory ...
Comments
Information & Contributors
Information
Published In
August 2024
6901 pages
Copyright © 2024 Owner/Author.
This work is licensed under a Creative Commons Attribution International 4.0 License.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 24 August 2024
Check for updates
Author Tags
Qualifiers
- Research-article
Conference
KDD '24: The 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining
August 25 - 29, 2024
Barcelona, Spain
Acceptance Rates
Overall Acceptance Rate 1,133 of 8,635 submissions, 13%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 133Total Downloads
- Downloads (Last 12 months)133
- Downloads (Last 6 weeks)133
Reflects downloads up to 23 Sep 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in