Abstracts Category : Other

A Machine Learning Ensemble Approach to Churn Prediction : Developing and Comparing Local Explanation Models on Top of a Black-Box Classifier

by Nina Olofsson

Churn prediction methods are widely used in Customer Relationship Management and have proven to be valuable for retaining customers. To obtain a high predictive performance, recent studies rely on increasingly complex machine learning methods, such as ensemble or hybrid models. However, the more complex a model is, the more difficult it becomes to understand how decisions are actually made. Previous studies on machine learning interpretability have used a global perspective for understanding black-box models. This study explores the use of local explanation models for explaining the individual predictions of a Random Forest ensemble model. The churn prediction was studied on the users of Tink a finance app. This thesis aims to take local explanations one step further by making comparisons between churn indicators of different user groups. Three sets of groups were created based on differences in three user features. The importance scores of all globally found churn indicators were then computed for each group with the help of local explanation models. The results showed that the groups did not have any significant differences regarding the globally most important churn indicators. Instead, differences were found for globally less important churn indicators, concerning the type of information that users stored in the app. In addition to comparing churn indicators between user groups, the result of this study was a well-performing Random Forest ensemble model with the ability of explaining the reason behind churn predictions for individual users. The model proved to be significantly better than a number of simpler models, with an average AUC of 0.93. Metoder fr att prediktera uttrde r vanliga inom Customer Relationship Management och har visat sig vara vrdefulla nr det kommer till att behlla kunder. Fr att kunna prediktera uttrde med s hg skerhet som mjligt har den senasteforskningen fokuserat p alltmer komplexa maskininlrningsmodeller, ssom ensembler och hybridmodeller. En konsekvens av att ha alltmer komplexa modellerr dock att det blir svrare och svrare att frst hur en viss modell har kommitfram till ett visst beslut. Tidigare studier inom maskininlrningsinterpretering har haft ett globalt perspektiv fr att frklara svrfrsteliga modeller. Denna studieutforskar lokala frklaringsmodeller fr att frklara individuella beslut av en ensemblemodell knd som 'Random Forest'. Prediktionen av uttrde studeras panvndarna av Tink en finansapp. Syftet med denna studie r att ta lokala frklaringsmodeller ett steg lngre genomatt gra jmfrelser av indikatorer fr uttrde mellan olika anvndargrupper. Totalt undersktes tre par av grupper som pvisade skillnader i tre olika variabler. Sedan anvndes lokala frklaringsmodeller till att berkna hur viktiga alla globaltfunna indikatorer fr uttrde var fr respektive grupp. Resultaten visade att detinte fanns ngra signifikanta skillnader mellan grupperna gllande huvudindikatorerna fr uttrde.