406 Weil Hall
1949 Stadium Road
Gainesville, FL 32611
Carnegie Mellon University
Abstract: Pattern Mining for Interpretable Data-driven Sequential Decision Making
This paper studies data-driven sequential decision making with an emphasize on interpretability and sequential structure. We first develop a novel data tree model of the database that is able to fit in memory orders of magnitude larger datasets compared to traditional tabular encodings. This enables us to perform pattern mining on large databases and find novel sequential patterns of events. We then assess sequential patterns of events on the likelihood they are associated to an outcome of interest, and use patterns with a high likelihood as interpretable explanations on why those outcomes occurred. Using statistical hypothesis tests, we provide reliability measures for such explanations, and analyze the trade-off between association likelihood of individual patterns and the overall explainability of outcomes provided by the set of mined patterns. Furthermore, we show how patterns can be aggregated into a knowledge tree to provide a structural view of the decision making process as a decision support tool for sequential decision making.
To show the benefits of our approach, we consider two applications in marketing and finance. In our marketing application, we investigate reducing the skip rate of users in an online music streaming platform. We find that almost all one billion user skips in the database can be explained using an average of 6,400 sequential patterns, with an average association likelihood of 83%. In our finance application, we assess technical analysis for investment decision making in the stock market. We find that 80% of nine million price change events can be explained using approximately 7,000 sequential patterns, however, with a lower association likelihood of 53%.
Department of Industrial & Systems Engineering