GSDMM model evaluation techniques with application to British telecom data

Abstract

Statistical topic modelling has become one of the most important tools in the text processing field, as more applications are using it to handle the increasing amount of available text data, e.g. from social media platforms. The aim of topic modelling is to discover the main themes or topics from a collection of text documents. While several models have been developed, there is no consensus on evaluating the models, and how to determine the best hyper-parameters of the model. In this research, we develop a method for evaluating topic models for short text that employs word embedding and measuring within-topic variability and separation between topics. We focus on the Dirichlet Mixture Model and tuning its hyper-parameters. In empirical experiments, we present a case study on short text datasets related to the British telecommunication industry. In particular, we find that the optimal values of hyper-parameters, obtained from our evaluation method, do not agree with the fixed values typically used in the literature.