Skip to Content

2.1. Twitter / X

Last modified by Matti Nelimarkka on 2025/06/10 13:49

2.1. Twitter / X

Twitter is a micro-blogging service and one of the most popular social media services in the world. The user base, however, is often strongly inclined to societal elites, journalists and politicians, which is important to keep in mind when studying Twitter (see e.g. Vainikka & Huhtamäki 2015 for Finland, Blank 2017 for the US).

Gathering Twitter data was open and free for a long time using their application programming interface, and there were several easy-to-use-tools and coding-based tools to collect the data. There used to be a  specific API for academic use. However, after Elon Musk bought the platform, heavy price tags were placed for using the APIs for retrieving tweet data, and the academic access API was closed. Current approaches need either post-api tools or budget allocated for buying data.

Regardless of the data collection approach, querying for Twitter data requires advance planning regarding the strategy for filtering the data,, i.e. searching with a keyword, hashtag or perhaps a user name. A good way to test the results is to experiment with the Twitter search on a browser.

  • Here is a blog post summarizing Twitter's previous Developer Policies (2019) from the point of view of research.
  • An overview blog post of social media research tools for Twitter with example studies can be accessed here.
  • A detailed explanation of the limitations of Twitter data and former API's offered by prof. Cornelius Puschmann here.

Recommended articles discussing research strategies in the "Post-API era":

Data extraction with API access

Currently, X offers various priced options to access the API, with quota limitations. Following the Digital Service Act legislation in the European Union, X has also opened an application form for research access for projects that meet the criteria listed in the DSA.

With API access, several existing tools can be adopted for data collection:

  1. TAGS
    • Easy Twitter monitoring from within Google sheets. Still requires an API key.
    • Added: 18.11.2021. Verified: ?
  2. Postman
    • A desktop and web application that allows you to make requests to an API from a graphical user interface, offered by Twitter.
    • Added: 18.11.2021. Verified: ?
  3. Atlas.TI
    • The popular qualitative analysis software also offers options to make queries to Twitter Search API.
    • Added: 18.11.2021. Verified: ?
  4. DMI-TCAT 
    • Retrieve and collect tweets from Twitter and analyse them in various ways. The software requires API access and a server where to install it.
    • Added: 18.11.2021. Verified: ?
  5. Mecodify
    • Similar to T-CAT: tweet collection, simple analyses and visualization. Own server and API access required, also instructions for Amazon AWS free service provided.
    • Added: 18.11.2021. Verified: 14.12.2022.

Tools that require coding skills

  1. twarc
    • Python-based command line tool for data collecting and processing
    • twarc2 supported Academic API
    • Added: 16.11.2023. Verified: early 2023
  2. Twurl
  3. AcademictwitteR
    • R package for extracting and wrangling with Twitter data
  4. Poll.emic
    • Data collecting, especially egocentric networks, visualization
    • Added: 18.11.2021. Verified: ?
  5. Python-twitter
    • This library provides a pure Python interface for the Twitter API.
    • Added: 18.11.2021. Verified: ?
  6. Pattern
    • Data mining: Google, Twitter, Wikipedia
    • Tools for data mining (Google, Twitter and Wikipedia API, a web crawler, a HTML DOM parser), natural language processing (part-of-speech taggers, n-gram search, sentiment analysis, WordNet), machine learning (vector space model, clustering, SVM), network analysis and canvas visualization
    • Added: 18.11.2021. Verified: ?
  7. Finnish language Twitter is being continuously collected since April 2020 by the Human Sciences – Computing Interaction (HSCI) research group at the University of Helsinki (kudos to Eetu Mäkelä!). The tweet id's provided can be used to download the full tweets from the Twitter API. Added: 18.11.2021. Verified: ?

  8. The Tweets Sampling Toolkit is a Python toolkit containing a set of tools for 1) creating a random sample from massive (100M+) Tweet ID datasets and for 2) performing set operations with Tweet ID datasets .

Data extraction without API access

However, not all research approaches require the API. Consider also in what ways you wish to analyze the data. In many cases analyzing the textual content from a reasonable number of tweets can also be done by querying the data manually with the Twitter Advanced Search and exporting it from the browser by printing, saving or copying (for Chrome, check the GoFullPage extension). Also, if you are interested in for example images, videos, gifs or profile pictures, they are a bit more difficult to analyze from the json/csv.

Tools:

  • Zeeschuimer app is a Firefox extension that can be used to collect different kinds of social media data, including Twitter/X tweets. Works well in tandem with 4CAT but also offers a json export option.
  • NTS Nordic Tweet Stream is a web service that offers a sample of Twitter data from the Nordic region. The NTS dataset contains nearly 74 million messages from over 888 thousand user accounts from January 2013 to May 2023. Developed in the University of Eastern Finland.

Other easy-to-use tools for collecting Twitter data (not updated frequently - some of these might work post-api, some not)

  1. Netlytic
    • Netlytic is especially made for pedagogical and learning purposes, but some research papers have been published utilizing Netlytic.
    • Capture publicly available posts from social media sites or other sources (Twitter, YouTube, RSS Feed or text/csv file)
    • Discover popular topics
    • Find and explore emerging themes of discussions
    • Build, visualize and analyse online communication networks using social network analysis
    • Map geo-coded social media data
    • Added: 18.11.2021. Verified: ?
  2. Socioviz
    • Scrape and collect
    • Analyse any topic, term or hashtag
    • Identify key influencers, opinions and contents
    • Export data to your favorite Social Network Analysis tool
    • Tutorials and instructions for Socioviz are found on their webpage.
    • Added: 18.11.2021. Verified: ?
  3. NodeXL
    • Excel-based tool for data collection and scraping, network visualizations and key statistics. Basic version is free.
    • Only for Windows.
    • See also Social Media Research Foundation student license
    • Added: 18.11.2021. Verified: ?
  4. Chorus
    • Chorus is a free, evolving, data harvesting and visual analytics suite designed to facilitate and enable social science research using Twitter data.
    • Made for social scientists for research purposes
    • Added: 18.11.2021. Verified: ?
  5. Twitonomy
    • Twitonomy is an app you can use to get information from your own account and from the accounts of others.
    • Added: 18.11.2021. Verified: ?
  6. Osome – Observatory on Social Media Overview
    • Allows you to submit queries to a massive social media database
    • Given a set of Twitter hashtags and a time period, you can count the number of matching tweets, generate a time series, count the number of tweets from each user or just list matching Tweet IDs.
    • You can subsequently utilize Twitter's REST API to retrieve more information about tweets or users of interest.
    • This API is provided on behalf of an academic institution (Indiana University) for the sole purpose of non-commercial research (https://rapidapi.com/truthy/api/osome-observatory-on-social-media-1/details).

    • Added: 18.11.2021. Verified: ?
  2. NCapture extension for NVIVO
    • Requires a NVIVO license
  3. Discovertext
    • Subject to a fee
  4. Tweetarchivist
    • Subject to a fee