Elon Musk, Donald Trump.

Elizabeth Culliford, Reuters

In January, just weeks after Twitter Inc permanently banned former President Donald Trump following the storming of the US Capitol, the social media company started asking US users to help identify and fact-check misleading tweets in a new pilot programme.

But ‘Birdwatch,’ which has about 2,000 participants and is currently cordoned off in its own section of the site, is already facing many of the same challenges as Twitter itself -discerning facts from partisan opinion and dealing with the potential for harassment or people trying to manipulate the system.

“There is a lot to do to get there, to the point where we’re comfortable putting these things on tweets,” Keith Coleman, Twitter’s vice president of product, told Reuters.

“Birdwatchers” can flag misleading tweets and annotate them with “notes” to give more information, which other participants can rate as helpful.

Under pressure to clean up its site, Twitter started labelling misleading tweets for the first time last year, a move that intensified debates about the role major social media platforms play in public discourse. It also fueled allegations from Republican lawmakers that tech companies are censoring conservatives.

In asking users to contribute their own checks, Twitter will have to balance curating Birdwatch to make it useful without losing the legitimacy it wanted from relying on its community.

Public Birdwatch data shows notes ranging from balanced fact-checks to partisan criticism. For example, some marked the baseless claim of widespread voter fraud in November’s US presidential election “not misleading.” Many simply gave opinions — a tweet from SpaceX and Tesla CEO Elon Musk should “go to Mars. And stay there” — while others added notes to opinions.

People are “fact-checking things that professional fact checkers never would,” said Alex Mahadevan, a reporter with the Poynter Institute’s MediaWise project, who analyzed Birdwatch’s data.

Coleman said the Twitter team’s next move would be updating the rating algorithm that determines which notes to highlight to make sure Birdwatchers with different views agree the information is helpful.

Crowd-sourced knowledge and community moderation are not new models: they underpin platforms like social network Reddit Inc, and Facebook Inc also runs a “community review” program in which users are paid to identify suspect content for vetting by professional fact-checkers. Thomson Reuters Corp-owned Reuters is one of Facebook’s paid third-party fact-checkers.

One of the most prominent examples of a crowd-based approach is Wikipedia, where volunteers write and edit millions of articles.

Katherine Maher, CEO of the Wikimedia Foundation that runs Wikipedia, said in an interview that the community’s mission to build an encyclopedia — making it what she called a “purpose platform” rather than “an expression platform” — defines how contributors behave and that Twitter, which has a more diffuse purpose, could be more difficult to wrangle. Borrowing methods from how Wikipedia promotes and rewards credible contributions could help, Maher said. Public editors on Wikipedia are granted greater controls by other users, based on their work.

Maher also said Twitter would need to develop standards and their enforcement for Birdwatch and decide how people could appeal annotation. It needs to solve the issue, she said, of “Who watches the watchers?”

To help guide Birdwatch’s development, Twitter said it is creating an advisory council of outside experts with backgrounds ranging from crowdsourcing to political science.  The company has also acknowledged it will have to work out how to prevent its unpaid Birdwatchers from being harassed for their notes. Coleman said it was considering options like removing people’s Twitter handles from their annotations and working out whether there would be extra rules for Birdwatch content. Contributors also will be allowed to use pseudonym accounts to protect their identities.

“We don’t know what will happen and whether people will feel safe,” said Coleman. “It’s really critical they do.”