Text Mining: Insights Beyond Numbers

by Raymond Chan
Associate Consultant, Raymond Chan give us an introduction to Text Mining, and its uses in providing deeper insights for businesses wanting to gauge customer sentiment, and improve their products and services.

Data is available everywhere. It resides in websites, blogs, emails, social media…the list goes on. The pervasiveness of data is a good thing for businesses, but the challenge that confronts them is how to make sense of it all.

Traditional analytics tend to concentrate on the quantitative aspects of the data, and often neglect the qualitative side of it. Structured data stored in databases are convenient and manageable, but what about unstructured data such as emails, text documents, blog comments and posts on Facebook, Twitter, or LinkedIn? This is where Text Mining comes in.

Text Mining, also referred as Text Analytics, covers this knowledge gap by helping organisations derive value from unstructured (textual) data. Text Analytics software analyses this data using Natural Language Processing (NLP) algorithms and identifies the “why” and “how” concepts, and facts behind the “what” aspects; all this while making the data accessible to Data Mining algorithms for predictions.

An example of Text Mining

You open Twitter and search for tweets (posts) related to Apple using the #apple hashtag. The first post that appears is the following:

Raychan social media1

What can this post tell us about Apple and the tweet’s author?
  • The author is posting about the necessity to improve the (iOS) Control Centre by adding a Cellular Data switch button.
  • The author has not used any negative words. The author has a neutral attitude regarding Apple and its products. (This is related to Sentiment Analysis which we will discuss in a future blog post).
  • Other (iPhone) users also agree with the author’s ideas, as the post has been retweeted 3000 times. (Social Network Analysis, also a future topic for discussion)

It can then be concluded that updating the Control Centre is a popular request from consumers, and Apple will certainly be able to satisfy them by adding this feature.

What we have done is similar to how Text Analytics software performs its analysis. It seems fast and easy to do manually when only analysing a single tweet, but how about 1000 tweets? Or 10,000 tweets?

Every minute, there are 300,000 posts “tweeted” worldwide (that’s more than 1 billion per day!). While not all of them might be useful, many of them contain valuable business insights…Could you really afford to miss any?

Text Mining

How do you make use of mined, unstructured data?

The above example is simplified, and only covers one use of Text Mining. Other than analysing customer satisfaction on social media, Text Mining is also used in numerous other ways.

It is common to find open-ended questions in surveys, and they are designed to allow respondents to freely express their opinions. Text Mining can be used to analyse those responses and discover insights that cannot be found due to the limitation of multiple-choice questions.

Take the example of a Telco. It can combine results obtained from social media posts and data mining tools to predict the likelihood of customer churning.

Understanding how customers feel during their subscription is an important consideration, as it increases the chance of retaining them, while also attracting new customers by improving the company’s offers and services.

In Summary

Numbers don’t always paint the full picture. The rise of social media platforms and the abundance of textual information, means that many new opportunities for meaningful analysis are now available. The next step is about utilising this otherwise unused source and start discovering new valuable business insights among other less important ones.

Recent Articles

Blog Categories

See all