Sentiment Analysis of Tweets – Connecting to Twitter API +Fetching Tweets using R

Sentiment Analysis of Tweets:

Twitter is a popular source to extract text data related to any product, company, individual or event. Let us consider an example of the Cricket World Cup which just ended. Twitter has been a hot platform for discussion. Thousands of comments were posted from viewers and cricket fans across the world over the past few weeks. Several hashtags were used for the same viz. #CWC, #CWC19, #CWC2019.

It would be interesting to do a Sentiment Analysis of Tweets related to a hashtag by pulling and working on a collection of tweets.

In this article, we will learn how to Connect to a Twitter API and fetch tweets using R.

A snapshot below shows some of the tweets for #CWC19

Things you will learn from this topic:

• Creating a Twitter App
• Code to establish a connection with the app using R
• Fetching tweets

You will have to create an account on twitter (if you are a new twitter user) and then create an individual or a team developer account to create an app. So, let us begin.

Follow the below steps:

• Create a twitter account
• Open the web page https://developer.twitter.com/en/apps
• Hit the Create an app tab highlighted in the below screenshot

• Next a popup will show up as below. Hit the ‘Apply’ button and go to the next page. We are applying for a developer account with these steps

• Twitter requires us to fill in some details for using its developer tools. There are several options out of which we have selected ‘Teaching’ (showing in green color) and clicked on the ‘Next’ button on the bottom left of the screenshot to proceed. You can check whichever options suit you

• The next page confirms us our correct account identity. It also asks us if we are going to need a ‘Team developer account’ or an ‘individual developer account’. Based on our requirement we will proceed with the individual developer account. Look at the below screenshot, and the highlights marked

• After we click the yellow highlighted link as shown in the above screenshot, the next page takes us to the individual developer account page which looks like below. We need to fill some details as an individual developer here and proceed further
• This new page is to describe our intended usage of twitter data. Fill in the details as required and move to the next page
• Here we will be asked to review our answers, account, and contact details. Proceed to the next page by hitting the “Looks good” button at the bottom right

• Finally, you agree to the terms and then make a Final Submission of the application to create an individual developer account
• Simultaneously, an email is also sent to the email id for verification. A popup tells the same information as well.
• After the email is verified, we have successfully created a developer account. It takes us to the developer page where we need to hit the ‘Create an app’ button again. We must fill some details about the app. Please fill the website URL to your website or blog URL. Do enable sign in with Twitter and give the callback URL as shown in the screenshot below
• We can leave these below sections blank. Answer other required questions and then proceed to create the app
• A popup highlights some developer terms before finally hitting the “create” button.
• Phew!!! We have successfully created our app. Next comes the connection between Twitter and R. Our app contained information related to Keys and Tokens. This API Key, API Secret Key information will be required while connecting with Twitter from R

Let us now see how to establish a connection between our app and R with the information of API Key and API Secret Key. The code for that is as follows:

Now let’s extract tweets from Twitter. If you would have noticed, we installed a package named rtweet while setting up the connection. That is the package we will use for this task. There is another package named twitteR which can be used as well. We will stick to the prior one. (We are assuming here that the reader has got some prior understanding of R.)

search_tweets is a function in rtweet package allowing us to search and extract tweets on a search string. You can learn all the features and options about this package here https://cran.r-project.org/web/packages/rtweet/rtweet.pdf

Code to extract tweets for hast tag #wimbledon2019

There are several ways to measure the sentiment of these tweets.

• We can identify sentiments based on lexicons
• In continuation to the above, we can train a machine learning model on sentiments identified and then use it to predict sentiments of test tweets
• We can use deep learning methods to identify sentiments

We will take on these topics in the next article. Hoping till then you will surely experiment with twitter API and find out the various ways in which you would want to put the use of these interesting tweets.