Many divisive X accounts are foreign-based. What does that tell us?
Workers install lighting on an "X" sign atop the company headquarters in downtown San Francisco, July 28, 2023.
Noah Berger/AP/File
On X, . The account, which comes with a blue check mark and describes itself as 鈥淐onstitutionalist, Patriot, Ethnically American,鈥� has been sharing a steady stream of pro-Trump, U.S.-centric content since March of 2016.
But The General may not be what their followers envision: The account is actually based in Turkey.
It鈥檚 just one of many highly active and relentlessly partisan accounts on X, the social media platform owned by Elon Musk, that have been affected by a new feature allowing users to see posters鈥� country of origin. This geolocation data has unmasked scores of accounts that generate millions of impressions posting about U.S. politics and other hot-button issues despite being located outside the U.S. The transparency feature, released on Nov. 21, has itself become a viral topic on X and other platforms.
Why We Wrote This
A new feature on the social media platform X is revealing that many popular accounts featuring inflammatory content about U.S. politics are located in Asia, Africa, and Eastern Europe. While it鈥檚 hard to know who is behind them, experts say many are just trying to profit from outrage.
While the geolocation data is incomplete and, in some cases, disputed by account holders, users have seized on revelations that some prominent X accounts appear to be frauds. Pro-Trump posters who extol the virtues of America First . who purport to be U.S. voters.
Who is behind these accounts isn鈥檛 always clear. Foreign governments have been known to to spread disinformation and influence U.S. elections. was Russia鈥檚 attempt to stoke tensions over police shootings and Black Lives Matter protests in 2016, using fake personas on both sides.
But researchers who study misinformation say much of the divisive content is motivated by something more basic: money. Posters on X can profit from clicks. And what reliably engages many U.S. users is outrage-inducing political content that can be wildly inaccurate and still deemed credible by partisans.
Still, as more and more Americans derive their news from social media rather than mainstream journalism sources, the impact of that grift can be far-reaching.
鈥淭he core issue here is the inherent unreliability of the information ecosystem,鈥� says Mor Naaman, a professor of information science at Cornell Tech in New York City. 鈥淥ur information ecosystem is built on incentives that very much invite anyone to post divisive or other engagement-based content, because sometimes they make money.鈥�
In the Middle East, who purport to be in Gaza are located elsewhere, about their authenticity. Some accounts that relay images of life under duress in Gaza and ask for donations have been exposed as potential scams. In the United Kingdom, some accounts that advocate for Scotland鈥檚 independence and the Netherlands.
In the U.S., this 鈥済reat unmasking鈥� has shone a spotlight on a that are located in Asia, Africa, and Eastern Europe. Many of these accounts feature AI-generated images of blond-haired American women and of President Donald Trump and his family. Surprising locations have also turned up for left-leaning accounts 鈥� such as one called , with nearly half a million followers, which was shown to be based in Austria before changing its location to the U.S.
Analysts say incentives for 鈥渞age baiters鈥� are baked into X鈥檚 business model under Mr. Musk鈥檚 ownership. Take the blue check mark, which was originally meant to signal that an account holder was who they claimed to be. It has now become a pay-to-play badge that helps content makers, authentic or not, build audiences that can be monetized via revenue sharing with the platform.
鈥淭hat makes it easier, not harder, for fake or proxy accounts to appear credible,鈥� says Emilio Ferrara, a professor of computer science at the University of Southern California, via email.
At the same time, X has cut its moderation and trust-and-safety teams and instead asked users to provide context for misleading posts in the form of community notes. Crowd-sourced tools can be helpful, but it鈥檚 hard to assess what X鈥檚 features mean for fake and manipulative content since X no longer allows independent researchers to access its data, says Professor Ferrara.
Some accounts have responded to X鈥檚 geolocation data by claiming to be Americans living or working abroad. and claimed to be in Turkey on a business trip.
When , Nikita Bier, the company鈥檚 head of product, called it 鈥渁n important first step鈥� and promised to introduce more ways for users to verify the authenticity of content. , he said it still had 鈥渁 few rough edges鈥� to be resolved. Some account owners have complained about inaccuracies and said the use of VPNs and other proxy connections could lead to mislabeling of legitimate accounts.
There are technical challenges in identifying what is and isn鈥檛 an authentic 鈥済rassroots鈥� voice in online debates, since foreign actors can be sophisticated at hiding their tracks, says Professor Ferrara, who studies inauthentic behavior on digital platforms. 鈥淧eople may put too much trust in a signal that is still inaccurate,鈥� he says of X鈥檚 new feature.
, Zach Schapira, a former X executive, and Sean Rad, founder of the dating app Tinder, call for X and other platforms to 鈥渆xpand geographical transparency鈥� by showing users where viral posts are being generated and spread, and giving them options to filter from specific regions. They also recommend that algorithms prioritize authentic domestic content. 鈥淎mericans debating issues that affect their communities deserve a feed that reflects real local voices and the real local popularity of those voices,鈥� they write.
Still, some worry that any whack-a-mole approach will fail, as duplicitous posters rely on VPNs and other tools to mask their origin while
The targeting of Americans, in particular, with provocative and divisive political content is, in part, a reflection of U.S. consumers鈥� spending power. Popular MAGA content frequently attracts ads for gold and male health supplements, among other products. 鈥淚t could be that there are just more immediate products to sell to a right-leaning鈥� X user, says Professor Naaman. (The platform also skews right; Bluesky, another social-media platform, has become a liberal hub.)
Mr. Musk, who has 229 million followers on X with whom he shares , hasn鈥檛 yet commented on the geolocation revelations. It鈥檚 something of a double-edged sword for his stewardship of a platform that he frames as the world鈥檚 best news source, says Ren茅e DiResta, an associate research professor at Georgetown University. The added transparency 鈥渉elps people be more aware that these kinds of rage baiters exist in their information ecosystem,鈥� she says. At the same time, it鈥檚 highlighted 鈥渢hat the platform does have these perverse incentives鈥� for inauthentic posters to make money.
Even if account holders aren鈥檛 disguising their country of origin, 鈥渂eing confrontational and provocative and divisive鈥� reaps rewards, says Professor DiResta. 鈥淓verybody in tech knows this is happening.鈥�