Twitter’s Timeline Algorithm Buries External Links
New research on Twitter’s timeline curation algorithm sheds light on how it shapes what we’re exposed to.
How does Twitter’s algorithm change what users see in their timelines? In a new research study from the Computational Journalism Lab, we present evidence of several shifts that result from Twitter’s timeline algorithm. Specifically, compared to the old-fashioned chronological timeline, Twitter’s algorithm:
- ↘️ Showed fewer external links,
- ✨ Elevated lots of “suggested” tweets (from non-followed accounts),
- ↗️ Showed a greater diversity of sources,
- 📊 Slightly shifted exposure to different topics, and
- 🔊 Had a slight partisan “echo chamber” effect
Background
Twitter’s timeline curation algorithm now directs the attention of more than 150 million daily active users. According to Twitter, the algorithm has helped attract millions of new users by elevating interesting content and making the platform more engaging.
But longtime Twitter users will remember the complicated history behind the timeline algorithm, which includes hashtags like #RIPTwitter and phrases like “f*** your algorithms” and “algorithms ruin everything.”
Years later, the #RIPTwitter uproar has settled, but we still know fairly little about Twitter’s highly-influential timeline algorithm: what does it elevate? To what extent? And what are the potential implications?
Twitter has disclosed that they use a deep learning system, internally called “DeepBird,” to predict which tweets users will find interesting and engaging. But Twitter’s explanations provide few details, and deep learning systems are notorious for being “black boxes.” So, over the past year, I worked with my advisor Dr. Nicholas Diakopoulos to understand more about the black box that is Twitter’s timeline algorithm. Here is what we found.
Main Findings
We tested Twitter’s timeline algorithm using a group of automated “puppet accounts,” comparing the puppets’ chronological timelines (“latest tweets”) to their algorithmic timelines (“top tweets”). For those curious, I wrote a separate piece with all the technical details of the audit.
↘️ Fewer External Links
One of the largest effects we observed is that Twitter’s curation algorithm greatly reduced exposure to external links. On average, 51% of tweets in chronological timelines contained an external link, compared to just 18% in the algorithmic timelines (orange in the figure below). The exposure rate to internal Twitter links increased slightly from 12% to 13%, and the exposure rate to internal pictures increased from 19% to 30%.
✨ Lots of “Suggested” Tweets
On average, “suggested” tweets (from non-followed accounts) made up 55% of the algorithmic timeline. That means that overall, less than half of all tweets in the algorithmic timeline came from accounts that the puppets actually followed.
As detailed in the methods piece, we collected timelines twice per day, and only analyzed the first 50 tweets that appeared, so take this with a grain of salt. For example, suggested tweets would probably be less prevalent if we looked at the first 200 tweets that appeared instead of just the first 50.
↗️ Increased Source Diversity
Twitter’s curation algorithmic significantly increased source diversity. On average, the algorithm almost doubled the number of unique accounts in the timeline, from 663 in the average chronological timeline to 1,169 in the average algorithmic timeline.
Twitter’s algorithm also reined in accounts that tweeted frequently: on average, the ten most-tweeting accounts made up 52% of tweets in the chronological timeline, but just 24% of tweets in the algorithmic timeline. We also found a lower Gini coefficient for the algorithmic timelines (0.59 versus 0.72), indicating more inequality in chronological timelines.
This evidence deviates from the alleged “filter bubble” effect that many people associate with social media algorithms. Rather than trapping users in a bubble of sources, our evidence suggests that Twitter’s algorithm diversifies the timeline with accounts that would not appear in a chronological timeline, and also reins in accounts that would dominate chronological timelines. The effect was more nuanced for topic exposure and partisanship.
📊 Slight Shift in Topics
To glimpse how the timeline algorithm may shift the topical makeup of user timelines, we analyzed four tweet clusters related to the COVID-19 pandemic:
- A cluster of tweets containing political information (e.g. about the president’s response to the pandemic)
- A cluster containing health information (e.g. about risk factors)
- A cluster containing economic information (e.g. about GDP or job loss)
- A cluster about fatalities (e.g. death toll reports)
Overall, Twitter’s algorithm reduced exposure to each of these topics except for the political cluster:
In this case, our evidence suggests that social media algorithms might sometimes reduce exposure to important information (e.g. health and fatality information about COVID-19), while elevating other topics. Still, the effect did not constitute a lock-tight “filter bubble,” but more of an “echo chamber” where some topics became louder while others were drowned out. We observed a similar effect in our analysis of partisanship.
🔊 Slight Partisan Echo Chamber Effect
In analyzing partisanship, we measured how Twitter’s algorithm changed exposure to accounts with different political leanings. Importantly, this was not a test of whether the algorithm had a “political bias,” but rather how the algorithm affected exposure to partisan accounts compared to users’ chronological timelines.
Our findings somewhat echo a 2015 study by Facebook, which suggested that most partisan exposure stems from users’ choices (such as who they follow and what they click on), and is not solely the fault of curation algorithms.
Still, we did see a slight echo chamber effect across the puppets. For example, the algorithmic timeline decreased exposure to accounts that were classified as bipartisan. For left-leaning puppets, 43% of their chronological timelines came from bipartisan accounts (purple in the figure below), decreasing to 22% in their algorithmic timelines:
Right-leaning puppets also saw a drop. 20% of their chronological timelines were from bipartisan accounts, but only 14% of their algorithmic timelines:
The echo chamber effect is difficult to measure, so check out the technical details or the full paper if you want to know more. Notably, our scoring system for partisanship was pretty generous to Twitter. For example, it labeled Barack Obama and Fox News as “bipartisan” (because these accounts were commonly followed both in right-leaning communities and left-leaning communities). The echo chamber effect may have been different under an alternative scoring system.
So What?
This study is one of the first to provide an empirical, data-backed characterization of how Twitter’s timeline algorithm changes what we see on the platform. Despite some important limitations (including a small sample size), we surfaced several important patterns that warrant further attention and, ideally, explanations from Twitter.
For example, the reduced exposure to external links may have significant implications for the news ecosystem. Based on a 2018 Pew survey, around seven-in-ten adult Twitter users in the U.S. get news on the site. If Twitter’s algorithm reduces exposure to external news links, users may have less exposure to high-quality journalistic media. Also, reduced exposure in feeds and timelines has financial and existential ramifications for news outlets, especially for those that need web traffic to generate advertising revenue.
At the same time, it is helpful to recognize that Twitter’s timeline algorithm is not the Bogeyman, but rather one part of a broad and complex media ecosystem. We can definitely tweak the algorithm(s), but we should also consider other changes that might improve how we exchange news and information online.
To name some examples: media literacy, user interfaces, structures of governance, and regulatory measures all present ways to improve the ecosystem, alongside potential tweaks to the algorithm.
As Anne Applebaum and Peter Pomerantsev wrote recently, “the internet doesn’t have to be awful.” We hope this research supports broader efforts to understand and improve the public’s relationship with algorithmic media platforms like Twitter.
