Google is developing a free moderation tool that smaller websites can use to identify and remove terrorist material, as new laws in the UK and EU force internet companies to do more to tackle illegal content.
The software was developed in partnership with the research and development unit of the giant Jigsaw and Tech Against Terrorism, a UN-backed initiative that helps technology companies police online terrorism.
“There are a lot of websites that don’t have people doing enforcement. It’s something that requires manpower to create algorithms [and] then you need all the human reviews,” said Yasmin Green, Jigsaw’s chief executive.
“[Smaller websites] don’t want the content of Isis there, but there are tons all over [them],” he added.
The move comes as internet companies will be forced to remove extremist content from their platforms or face fines and other penalties under laws such as the European Union’s Digital Services Act, which comes into force in November, and the UK’s Online Safety bill, which is expected to become law this year. .
The law has been pushed by politicians and regulators across Europe who argue that Big Tech groups have not gone far enough to police content online.
But the new regulatory regime has raised concerns that smaller startups are ill-equipped and lack resources will limit their ability to compete with larger tech companies.
“I have noticed a big change in [leading] platforms are becoming more effective at moderating, and pushing terrorist content and Covid hoax claims [other sites],” added Green.
A report from the Global Internet Forum for Counter Terrorism in 2021 estimates that for every 10,000 posts on Facebook, six will contain terrorist or extremist content. On smaller platforms, this number can be as high as 5,000, or 50 percent of the content.
GIFCT, a non-governmental organization established by Facebook, Microsoft, Twitter and YouTube in 2017 to develop partnerships between many technology platforms, supports the project by Jigsaw. Non-governmental organizations have a database of terrorist content shared with members of tech companies, which their moderation system can use to detect existing material.
On December 13, the owner of Facebook and Instagram Meta launched an open-source software that other platforms can distribute to match terrorist content with images or videos in the database and highlight them for important human reviews.
The Jigsaw tool aims to address the next step of the process and help human moderators make decisions about content deemed harmful and illegal. It will begin testing with two unnamed sites earlier this year.
“In our experience, we have found that terrorists seek to exploit smaller platforms where content moderation is challenging due to limited resources,” said Adam Hadley, director of Tech Against Terrorism.
Jigsaw has about 70 staff, primarily based in Google’s New York office. Green, who became chief executive in July, said the lossmaking division was not expected to become profitable.
“There is an understanding that there is a long-term business return. . . Google needs a healthier internet,” added Green. “We’re helping Google and helping the internet in a way that gives value even if it’s not money.”