You are here

TCAT

What is TCAT?

The Twitter Capture and Analysis Toolset (TCAT), developed by the University of Amsterdam’s Digital Methods Initiative, is an open-source application for the capture and analysis of tweets that has been developed for researchers. The TCAT Website has more information about how to use and maintain your DMI-TCAT installation: http://github.com/digitalmethodsinitiative/dmi-tcat/wiki.  A scholarly paper by the TCAT developers that outlines how the tool works is available from the ASLIB Journal of Information Management.

We don’t provide any warranties or post-installation support for any instances of TCAT that you create using Launchpod. However, you may be able to find additional help and advice from the TCAT developers and user community. To begin with, check the TCAT FAQ, ask the @dmitools account on Twitter, or if necessary create a new issue on the TCAT Github repository.

Also note that while TCAT has proven to be very stable, NeCTAR instances are sometimes affected by network issues and other disruptions. In order to guarantee that your TCAT instance is running reliably, check in with your TCAT site on a regular basis, and reboot your instance if there appear to be any issues.

In particular, please be aware that NeCTAR only makes a limited amount of disk space available. If you are tracking Twitter keywords or hashtags that are likely to generate a high volume of tweets, you will need to download your data regularly, and delete and re-create your tracking bins in TCAT. For a high volume query your disk space may be full after only 58 hours of collection.

Obtaining Twitter access credentials

TCAT uses your Twitter account to monitor the constant global stream of tweets and find those matching your specified search terms. Before you can use TCAT, you need to create an ‘application’ within Twitter, and then obtain the Consumer Key and the Consumer Secret for this application, as well as your Access Token and Access Token Secret. The following sections describe how to obtain these. You will need ‘application’ keys and tokens for each TCAT instance you want to create.

First, navigate to the Twitter Applications Development site and log in with your Twitter credentials. From here you will create the ‘application’. This page is typically used by developers to register apps that they have created and which people can download. Most of the information is actually irrelevant for TCAT, but it needs to be entered in order to create the access credentials that TCAT needs. As of mid-2015, you also need to have a mobile phone number associated with your Twitter account to register an application. If you don't have a mobile number, go to your Twitter profile and enter one.

Click Create New App and provide a name (such as ‘TCAT’) and a brief description (such as ‘capturing tweets for research purposes’). You also need to provide a URL. It does not have to be a real website, but it does have to begin with ‘http://’. A dummy URL like ‘http://www.example.com/’ is sufficient, but you may wish to use the URL for your research project or home department instead. Scroll to the end of the page and select Yes, I agree to the Developer Agreement. Finally, click Create your Twitter Application.

Once your app has been created, navigate to the Keys and Access Tokens tab of the app settings. This page will display two of the four settings you need, the Consumer Key (API Key) and the Consumer Secret (API Secret). You can copy these straight out of Twitter and into a note or document for use later, or copy them directly into the relevant fields in Launchpod as you work through the TCAT setup process.

To generate the other two credentials, scroll to the bottom of the same page and under Your Access Token click the button Create my access token. The page will reload, and the Access Token and the Access Token Secret will be displayed at the end. Make sure when copying the Access Token, that you copy the entire line, including the 9-digit number at the front, which is separated from the rest of the secret by a dash.

If you don’t take note of your access tokens at this point, you can always log into the Twitter Applications Development site again and retrieve them.

Once you have created these keys and tokens, begin the TCAT launch process in Launchpod. Along the way you’ll be asked to enter these keys; make sure you copy them from the Twitter Applications Development site or your notes exactly as they appear there.

Exporting Tweets from TCAT

When using the "Export all tweets from selection" function, ensure that the current selection doesn't contain more than 500,000 tweets; otherwise you run the risk of running out of memory with this action and the number of tweets may be truncated in the downloaded file. It is recommended that you confirm that the number of tweets in the downloaded CSV matches the number of tweets selected for export.