The Future of the Internet Depends on Trust and Safety Investment
Risks are higher than ever for both companies and users. T&S solutions keep everyone safe and set investors up for a profit.
Sorry if this is late and/or if you are getting this twice. The email didn’t seem to go out this morning at 9 a.m. Eastern, and I don’t know if that’s a bug on Substack’s end or if I accidentally unchecked a box. Regardless, thanks for understanding.
I am back in DC after twelve days in Austin, the last four of which were spent attending South by Southwest (SXSW). I’m exhausted but exhilarated by the many conversations I had. It was amazing to see so many of you. Keep a look out for tomorrow’s podcast, which will contain the audio from the discussion Sasha Issenberg, and I had on Friday about his new book Lie Detectives: How Political Campaigns Fight Misinformation.
Not surprisingly, AI was the topic of the day, along with the creator economy, the news industry's future, elections, and more. Government regulators had a considerable presence, from the U.S. executive branch to the European Union and the United Kingdom. The UK won for the best AI and elections panel name from me, “The Great British Fake Off,” and I was also lucky to speak at events with SeedAI and attend a breakfast with Executive Vice-President of the European Commission Margrethe Vestager about the future of tech regulation.
Trust and safety were sprinkled throughout all of this. AI-powered disinformation was a big topic, but companies also promoted their tools to help label and detect AI content. A lot of conversation about how the need to keep users safe is expanding rapidly - it’s more than just about content or elections, but about how models are trained, prompts are written, and outputs are designed. Every industry is thinking about how to incorporate AI - for instance, I met someone working on an AI-powered guitar amp called Lune that helps people create music and learn guitar.
He also told me that there are many experts in using generative AI for film who don’t even know what trust and safety is, yet it is something they too will need to think about.
I also got to try the Meta Ray-Ban smart glasses for the first time. Not only can you use them to take photos or audio, but Zuckerberg posted on Instagram a multimodal beta feature this week where you can look at an object, monument, or whatever and ask it to tell you more about it. AI powers a lot of that. I was smitten enough that I bought a pair to play around with, but I also recognize the many trust and safety issues that come with a device like this.
That’s why I’m excited about a new report that the company I work for - Duco - released today with funding from the Future of Online Trust and Safety Fund. This first-of-its-kind Trust & Safety Market Research Report outlines how T&S solutions keep users and platforms safe and provide investors with a significant market opportunity. This report analyzes the total and serviceable addressable market, T&S vendor landscape, and investment activity to illustrate T&S as a distinct industry that manages the sociotechnical risks affiliated with user– and AI–generated content and services. It builds upon the DFR Lab’s Scaling Trust on the Web report that Kat Duffy spearheaded, which recommended building an empirically informed case for increased T&S investment with support from private sector influencers and investors.
To accurately assess the size of the industry and the opportunity, researchers conducted expert interviews, desk research, and quantitative market analysis to define the T&S solutions market scope and identify platforms that could consume these T&S solutions. Researchers found:
While outsourced T&S services currently represent the most significant portion of the $24.8B T&S total addressable market (TAM), T&S software solutions are poised to rapidly capture outsourced services market share as available LLMs now provide a strong baseline to aid T&S efforts at a lower cost. By 2028, T&S software TAM is projected to grow from $7.4B to $15.4B at a compound annual growth rate (CAGR) of 15.7%, while T&S services TAM is projected to only grow from $17.3B to $17.5B at a CAGR of 0.2%.
Over the past decade, governments worldwide have been increasingly introducing legislation on T&S issues. As platforms increase their global scope, companies must equip their T&S workflows with the appropriate localization. The E.U. and U.K.’s significant presence in the T&S market can be attributed to its leadership in T&S-related regulation, and the T&S market is poised for growth in other regions as similar legislation gets passed worldwide.
A combination of T&S professionals noticing gaps in the T&S market coupled with technology industry layoffs has led to a significant outflux of talent building new T&S software startups, which start to fill the gaps in a $14.7B serviceable addressable market.
Media scrutiny, brand safety concerns, and user expectations have led to an era of heightened platform accountability from the public. News events such as elections and global conflicts will have all eyes on platforms for the foreseeable future.
Such factors have contributed to an inflection point in the growing T&S software market over the past few years. From 2019-2023, the total venture capital investment in the T&S software market ($7.78B) was 4.7x the capital invested from 2014-2018 and 40x the amount of capital invested from 2009-2013. As a result, the number of T&S software companies on the market has grown 138% within the past 13 years.
Watching so many talented T&S professionals see better ways to do this work and leave big tech companies to start their own is amazing. This includes folks like Cinder, ActiveFence, TrustLab, and Cove. I also met women working at companies like Open Policy Group and Creedo.ai at SXSW who are building new ways to make technology safer. Another company, Click, has an app to help authenticate photos and videos on blockchain. Duco is a different model, but it is also built to quickly help teams across tech access experts - including those in trust and safety.
T&S are not cost centers. They can keep users safe while also making a profit for investors. The market opportunity is only going to grow. I encourage you to read the report to learn more, and let me know if you have any questions!
A huge thank you to the authors and contributors to the report.
Authors: Joe Ammirato, Vivian Chong, Josh Lawson, Maura O’Sullivan, Sarah Oh, Lauren Wagner
Contributors: Sofia Arimany, Connie Chung, Kat Duffy, Michael Dworsky, Brian Fishman, Kelly Graziadei, Lucia Stacey Harris, Sara Ittelson, Vaishnavi J, Agne Kaarlep, Christine Keung, John Redgrave, Aaron Rodericks, Alex Rosenblatt, Yoel Roth, Micah Schaffer, Tom Siegel, Derek Slater, Dave Willner, Glen Wise, Shu Dar Yao, JooHo Yeo.