ORCID

Abstract

Telecom services are at the core of today’s societies’ everyday needs. The availability of numerous online forums and discussion platforms enables telecom providers to improve their services by exploring the views of their customers to learn about common problems that customers face. Natural Language Processing (NLP) tools can be used to process the free text collected.One way of working with such data is to represent text as numerical vectors using one of many word embedding models based on neural networks. This research uses a novel dataset of telecom customers’ reviews to perform an extensive comparative study showing how different word embedding algorithms can affect the text classification process. A variety of state-of-the-art word embedding techniques are considered, including BERT, Word2Vec, FastText, and Doc2Vec. Several PCA-based approaches are explored for feature engineering. Moreover, the energy consumption used by the different word embeddings is investigated. The findings show that BERT combined with PCA lead to consistently better text classifiers in terms of precision, recall and F1-Score, particularly for more challenging classification tasks. Moreover, our proposed PCA approach of combining word vectors using the first principal component shows clear advantages in performance over the traditional approach of taking the average.

Publication Date

2025-07-29

Publication Title

Expert Systems with Applications

Volume

297

Issue

Part A

ISSN

0957-4174

Acceptance Date

2025-05-21

Deposit Date

2025-08-14

Funding

This document is the results of the research project funded by the University of Plymouth.

Keywords

Feature extraction, Short texts, Telecom data, Text classification, Word embeddings

Creative Commons License

Creative Commons Attribution 4.0 International License
This work is licensed under a Creative Commons Attribution 4.0 International License.

Download

Share

COinS