Service Based Opinion Mining Application For Analyzing Customer Feedback

Abstract

Huduma Centres are fast becoming the government front office where citizens are guaranteed access to a majority of citizen services under the same roof. It was awarded and recognised for offering the best customer service in the public sector in the year 2014. To maintain such high quality service, there is need to have a way of capturing an analyse customer feedback to enable management make decisions that are geared towards continuous improvement and maintenance of excellent services. Customer satisfaction is a critical factor that dictates whether an organization will be a success or not. Huduma centres need to implement strategies that can help them predict customer issues which should enable them identify and solve any slight change in customer feedback in real time. This study focuses on enabling Huduma Centres analyse customer feedback to understand the customer‟s opinion about their service being offered. The previous strategies used to capture feedback include the use of notes and the “One click feedback” strategy that requires a customer to rank or rate the service. In this research, real time analysis of customer feedback is achieved through the utilisation of a hybrid model that incorporates both the supervised and the unsupervised machine learning techniques. This hybrid model relies on the Lexicons and the use of Naïve Bayes Machine learning that assumes that every feature and each word in a review being classified is independent of any other feature. The data is first extracted from Twitter, then subjected to a SOCAL algorithm that generates a semantic orientation of either positive, negative or neutral of a given opinion or feedback. Once the semantic orientation is done, the data is divided into two sets; a training set and a test set. The training set is used train the Naïve Bayes model to classify texts and it was composed positive and negative reviews. A gold standard dataset was then used to evaluate and measure the accuracy, precise, recall and F1 score of the hybrid model. This research also explored the performance of the lexicons and the Naïve Bayes Models separately to ascertain the performance of Hybrid model in comparison to the two models. The results show that in all the evaluation tests done on the hybrid model scored higher. On the accuracy, the hybrid model scored 67.27% which showed a higher degree of accuracy than the Lexicons and the Naïve Bayes models by 4% points. The same also applied to the precision and recall where the hybrid model scored 66.99% for precision and 66.57% for recall which was higher by 3.53 and 4.6 percentage points respectively. The F1 score on the other hand gave a score of 66.78%. Based on the above results, it is concluded that the hybrid iv model is best fit to be used for sentiment analysis due to its higher accuracy, precision, recall and F1 scores than the Lexicons and the Naïve Bayes when implemented separately. However further improvements on the above scores can be explored by use of ensembles where several models are combined through the use of boosting or bagging methods to smooth out predictions and combine them into one hybrid model with a best fit. Keywords: Opinion Mining, Sentiment Analysis, Supervised Machine Learning, Text classification, Customer feedback