Tweet Sentiment Visualization Using Maltego
By Paul Richards, Developer at Paterva
Recently AlchemyAPI, one of the primary resources that Paterva uses to analyze sentiment, asked us to share how and why we use AlchemyAPI within our tool and why we are so excited about it. As the developer of these transforms (taking one piece of information to another with a small piece of code), I will briefly describe our use and how we got to where we are now.
For those unfamiliar with Maltego, it is a data visualization and mining tool that allows you to quickly and easily mine for data as well as see the correlation between different pieces of information to visually gain intelligence. In Maltego, pieces of information, known as ‘entities’, are used to mine for additional pieces of information that link to the original.
A practical example of how Maltego is used to find links between groups of information is below. Imagine you want to find a common link between three popular brands such as Nike, Puma and Adidas. You could start with Twitter Affiliation entities of each of these brands and run a transform that returns Twitter users that have tweeted about one of the brands. The resulting graph, shown below, makes it easy to identify Twitter users that have tweeted about one or more of these brands (the nodes located in between two or more clusters). As you can quickly tell from the graph, there is one user in the middle of the graph who tweeted to all three brands making them a possible person of interest.
This chart displays all Twitter users who have tweeted about the three brands.
Sentiment analysis is the use of natural language processing (NLP) to extract the attitude/opinion of a writer towards a specific topic. With the overwhelming amount of data being posted on the Internet every day with no way for a human to read it all, sentiment analysis is valuable for extracting and aggregating opinions from many sources on a specific topic.
There are many sentiment analysis APIs out there to choose from and it was difficult to decide which one would work best within Maltego. After much experimentation, I found that Alchemy’s sentiment analysis API was one of the most accurate out of all the APIs tested.
Combining Alchemy’s sentiment analysis API with Maltego’s visualization capabilities gives an analyst a powerful tool for graphically depicting opinions on specific topics. The transform that we built takes a Tweet as its input and returns either a positive, neutral or negative entity. In this way, a large amount of Tweets can be quickly and accurately categorized according to their sentiment. There is a wide range of potential uses for this transform ranging from brand reputation monitoring, market research, customer reactions to product launches and stock market monitoring to gauging opinions towards political parties, governments or countries.
This shows a visualization of positive and negative sentiment.
Maltego also allows you to build machines which automate the process of running multiple transforms – essentially allowing you to create a macro of tasks that are commonly run sequentially. This allows continuous monitoring of a topic by running a group of transforms at a set time interval and automatically updating your graph every time it runs. We built a new machine named Twitter Analyser to use with the new sentiment analysis transform. This machine takes a specific phrase in as its input and searches Twitter for Tweets with this phrase. From these Tweets, hashtags, links, sentiment and uncommon words are extracted as children of the originals. Maltego has multiple ways of visually representing the data. In this case, I used the ‘bubble view’, which sizes the entities according to the number of incoming tweets. This makes it much easier to see commonalities across Tweets.
The Maltego graph above shows an example of using Twitter Analyser on the phrase ‘YesScotland’.
This graph allows you to easily identify groups of Tweets with the same sentiment, common URLs, hashtags and interesting words. It automatically updates the graph every five minutes by getting new Tweets posted by users.
This is just one example of the many cases in which sentiment analysis is being used to monitor social networks. The vast amount of information being posted on the internet every hour makes sentiment analysis a vital tool to monitoring public opinions on specific topics.
As always, enjoy responsibly!