Vittoria Elliott at Wired has a new article. I’m in it!
It’s called: Big Tech Ditched Trust and Safety. Now Startups Are Selling It Back As a Service
Here’s the link: https://www.wired.com/story/trust-and-safety-startups-big-tech/.
I’m quoted as saying:
Sahar Massachi, a former member of Meta’s civic integrity team and cofounder and executive director of the Integrity Institute think tank, worries that by outsourcing key functions, platforms may be undermining their ability to improve products. Trust and safety issues can sometimes be more about product design than active moderation—should a user be able to reshare content? How much weight should different metrics be given within a recommendation algorithm? “The vendors could be great, but they won’t be able to have insight into that because of the ways that companies work,” Massachi says.
If you think of the work as “take in a stream of content, tag it, and then emit that altered stream of content” — then you’ve already lost. That’s a component of the work, maybe, but not the core of it. The core of it includes looking at behavior over time. It involves looking at data from a variety of different sources in the product. It involves changing the product, the metrics workers are held to, and company decisions. T&S vendors do good work, and I’m glad that many of them are run by (or hired) my friends. But they’re at best a large component of a bigger strategy — hiring them cannot be the strategy, if you want to do things right.
And to put it maybe a bit more succinctly:
The most important levers to doing integrity work right — design, ranking changes, setting the right metrics — are explicitly out of the control of vendors. If they handle whack a mole while product teams do the rest — that’s great. If they’re the only line of defense? Bad news.