Look, I made you some content
What are the lessons we should be taking from the Facebook Papers reports, and what does content moderation look like in 2021?
 |
A few days ago, I saw a post from Substack co-founders Chris Best and Hamish McKenzie about the lessons of the Facebook Papers reporting. I think it’s worth a quick read:
In it, Best and McKenzie argue that there aren’t any top-down solutions to the problems illustrated in the leaks (which prove once and for all that Facebook gives right-wing media outlets preferential treatment for fear of upsetting conservatives). They’re probably right on that point.
There will be no perfect internet, but there can be a better one. A healthier internet requires overhauling incentives by putting people back in charge. While tech companies can’t filter only the good parts of human nature, they can stop amplifying the bad and the ugly. Through business models that let people choose to pay with money instead of attention, we can foster discourse that is more thoughtful, civil, and intellectually diverse. This kind of model respects people’s agency and judgment by inviting them to mindfully seek out valuable ideas rather than mindlessly scroll through a feed of promoted content.
People will hate-read and doom-scroll, but they won’t hate-pay or doom-subscribe. While people pay attention to content that makes them agitated, they’ll only pay money for content they trust and value. With this kind of model, free content can still exist, but it will be truly free — not masquerading as such while quietly extracting costs in the form of personal data or manipulated behavior. The profit driver is no longer exploiting emotions, but fostering trusting relationships between the people who create content and the people who consume it.
I completely agree that the internet’s incentive structure, which prioritizes rage above just about anything else, needs an overhaul. And I completely agree that “while tech companies can’t filter only the good parts of human nature, they can stop amplifying the bad and the ugly.” What this boils down to is basically a call for an end (or at least an extreme overhaul) of the way content on social media platforms is ranked and how internal algorithms decide what’s relevant. Great. Good luck getting Facebook on board with that, but great. Totally agreed.
The Present Age is a reader-supported newsletter about communication, media, culture, and politics in a time of hyperconnectedness. There are paid and free subscriptions available.
But the second half of that excerpt, which essentially argues that Substack’s model, which doesn’t rely on discovery tools or built-in reading suggestions, is the answer. It may be an answer, but I just don’t know if it’s the best answer (but it might be). Even without algorithms geared toward encouraging people to embrace rage, I wouldn’t go so far as to suggest that Substack’s model doesn’t encourage the exploitation of audience emotions.
Our content-consuming decisions, generally speaking, will always tie into emotion. People may not “hate-pay” for subscriptions, but there are certainly people who specifically subscribe to publications because those places hate on the same people they do. Add in the fact that there’s always the concern that if people only consume content from others they agree with (as many people do), we still have bubbles and alternate realities that will only become more difficult to escape over time.
But this is not a Substack-specific problem. Or a Facebook-specific problem. Or even an internet-specific problem. This is a human problem that may not actually have an answer.
That said, it got me thinking about the various platforms and their guidelines for acceptable content.
There would seem to be two separate ways for platforms to set their content rules:
Create a set of rules so permissive that advertisers (or investors) would (rightly) worry about what, exactly, their ads are displayed alongside.
Create a set of rules restrictive enough to assuage advertisers’ concerns, but then end up forced to grapple with a way to actually enforce those rules.
Both have their own set of unique challenges. In the first case, there’s the worry of losing both advertisers and users. As much as some people claim to want a total free-for-all, setting the bar for allowed content at “Well, if it’s legal, it should be allowed” will inevitably drive some users away. And on platforms that rely on advertising (Facebook, Twitter, YouTube, and well, most places), that’s obviously something that stays front of mind from a business sense. In 2020, more than 80% of Google parent company Alphabet’s $183 billion in revenue came from its ad business. And understandably, advertisers don’t want their ads running alongside controversial content that could hurt their brand image.
In 2017, in response to brand concerns about where their ads were showing up , Google announced plans to improve the topic and site-level exclusion processes:
We know advertisers don't want their ads next to content that doesn’t align with their values. So starting today, we’re taking a tougher stance on hateful, offensive and derogatory content. This includes removing ads more effectively from content that is attacking or harassing people based on their race, religion, gender or similar categories. This change will enable us to take action, where appropriate, on a larger set of ads and sites.
We’ll also tighten safeguards to ensure that ads show up only against legitimate creators in our YouTube Partner Program—as opposed to those who impersonate other channels or violate our community guidelines. Finally, we won’t stop at taking down ads. The YouTube team is taking a hard look at our existing community guidelines to determine what content is allowed on the platform—not just what content can be monetized.
And in the other example of possible social network rules, in which companies do create rules that make their sites both a safe bet for advertisers and something less than a cesspool for users, these platforms are then tasked with actually enforcing those rules. This, as these companies have learned, is easier said than done. Moderating gigantic platforms like Facebook, for instance, might actually be impossible, as Lance Ulanoff argued at One Zero earlier this year.
(On this topic, I highly recommend checking out Casey Newton’s excellent 2019 articles about the lives of Facebook moderators, which often involve hours of watching videos of everything ranging from graphic pornography to actual beheadings.)
When it comes to creating and enforcing policies, platforms are damned if they do and damned if they don’t — so most simply say they do… and then… don’t.
Social media platforms tend to create big, sweeping policies aimed at making them attractive to users and advertisers alike. For instance, here’s a portion of what Twitter’s “hateful conduct policy” states:
We prohibit targeting others with repeated slurs, tropes or other content that intends to dehumanize, degrade or reinforce negative or harmful stereotypes about a protected category. This includes targeted misgendering or deadnaming of transgender individuals. We also prohibit the dehumanization of a group of people based on their religion, caste, age, disability, serious disease, national origin, race, or ethnicity. In some cases, such as (but not limited to) severe, repetitive usage of slurs, epithets, or racist/sexist tropes where the primary intent is to harass or intimidate others, we may require Tweet removal. In other cases, such as (but not limited to) moderate, isolated usage where the primary intent is to harass or intimidate others, we may limit Tweet visibility as further described below.
And whether or not you believe those to be good policies, it doesn’t really matter since Twitter isn’t exactly enforcing it anyway. As a trans person who spends a lot of time on Twitter, whew, I can say with some certainty that the company really doesn’t seem to actually care about “targeted misgendering” unless it’s persistent to an absolutely absurd point.
And it’s not just Twitter. YouTube, for instance, had a harassment policy that said that “content that makes hurtful and negative personal comments/videos about another person” would be removed from the platform. So when Carlos Maza, who is gay, called out right-wing YouTuber Steven Crowder after years of calling Maza things like “a lispy sprite,” "a “little queer,” “Mr. Lispy queer from Vox,” “an angry little queer,” “gay Mexcican,” and more, it should have been a fairly open and shut case, a clear breach of YouTube’s rules as they were written. Instead, YouTube twisted itself in knots to try to argue that Crowder hadn’t actually violated its policies.
Maza really went out on a limb to call that out, and in the end, all it resulted in was Vox letting him go. (We discussed this a bit when I had Maza on my podcast a few months back:)
Facebook, for its part, has its own set of rules in place, but as I mentioned earlier, doesn’t seem to have the ability or the will to actually enforce them. In September, The Wall Street Journal reported on XCheck, a program that essentially held a number of high-profile right-wing Facebook users to more relaxed standards when it came to enforcement of policy violations.
All of this shows just how pointless these policies are. Scattershot, inconsistent enforcement with exceptions carved out for high-profile users make these documents worthless. And that’s why I think companies should start from scratch.
Platforms should start fresh with policies they can actually enforce.
In 2018, I wrote an article for The Verge about Twitter and Alex Jones — sort of. The truth is that I’d had that article bouncing around in my head for some time, and Twitter’s indecisiveness around Jones and whether he had a place on the platform (despite regularly breaking its rules) made for a great jumping off point.
I wrote about Twitter’s identity crisis and its tendency to create advertiser-friendly standards of conduct for the platform while not actually enforcing them.
The company’s “hateful conduct policy” prohibits users from engaging in targeted harassment, making unwanted sexual advances, or harassing others on the basis of “race, ethnicity, national origin, sexual orientation, gender, gender identity, religious affiliation, age, disability, or serious disease.” But take a quick glance at Twitter, and you’ll find no shortage of accounts promoting racist, homophobic, transphobic, or Islamophobic content, which are all seemingly clear violations of the site’s own policies. But should you actually report the tweets and accounts that are promoting those views, you’ll no doubt receive your fair share of notices informing you that, actually, none of the company’s policies were violated.
It’s no wonder that there’s so much confusion about whether Alex Jones and Infowars belong on the platform given how opaque the company is about its own policies and haphazard enforcement of them. Reading through the rules as they’re laid out on the company’s website, it feels as though those documents are less actual guidelines of governance and more just a collection of vague ideas held together with masking tape and chewing gum. Perhaps it’s time for the company to call something of a corporate constitutional convention, delete the entire document, and start fresh with a clear purpose — and the will to enforce it.
As it stands, Twitter acts the way you might expect an external litigator in search of a legal loophole would, despite having written the laws itself. Dorsey tweets that he’s simply holding Jones to the same standards as every other user, but the company very clearly doesn’t apply its rules evenly. When people pointed to the way Trump would flout the platform’s rules with personal attacks, the company issued a statement explaining that the rules the rest of us are expected to follow don’t concern him because of his status as a world leader. While many companies took the opportunity to ban Jones, using each other for cover — perhaps giving way to a bit of insight into what they’d have done if not for fear of political fallout — Twitter’s inclination seems to lean in the opposite direction: away from a good-faith reading of its own rules.
In the years since, Trump and Jones have both been booted from the platform, but it took repeated, flagrant disregard of Twitter’s rules for that to happen. If that’s the standard, then it’s likely that these policies aren’t so great to begin with. Twitter, like Facebook and YouTube and even Substack1, should only set rules it can and will actually enforce. If that means paring back some of the more advertiser-friendly terms of service, then that’s what needs to happen.
These platforms — Twitter, Facebook, YouTube, etc., — should each convene something of a constitutional convention for themselves, ditching the old documents around rules and regulations, and replace them with clear, easy to understand guidelines that fall within platforms’ ability to actually moderate them.
What are your thoughts?
Currently, Substack’s own terms of service include this line: “Substack cannot be used to publish content or fund initiatives that call for violence, exclusion, or segregation based on protected classes. Offending behavior includes serious attacks on people based on their race, ethnicity, national origin, religion, sex, gender, sexual orientation, age, disability or medical condition.”
There are certainly a handful of accounts that violate these rules regularly that the company is almost certainly aware of. As is my view with Facebook, Twitter, etc., I think it should pare back its rules to whatever it feels comfortable actually enforcing. That way people know exactly what they’re in for when they sign up.
I've been on both Twitter and Facebook for a long time and I've taken numerous "breaks" from Twitter (just avoiding visiting it for weeks at a time) and I've also taken a couple of "complete breaks" from Facebook: deactivating my account for up to two months at a time. Both platforms seem to be increasing drowning in advertising these days and neither platform seems to show any effectiveness in stemming hate speech or misinformation. Several of my Facebook friends have found themselves in "Facebook Jail" for actually challenging misinformation -- while the perpetrators' accounts stay active.
I finally got to the post this week where I'd just had enough of it all and deleted my accounts on both platforms. Too much noise, too much political grandstanding, and far too much hatefulness. And, so far, I'm not missing it -- and my stress levels are already much better!
I'd love that.
What I would really like, though, is proper explanations of decisions, and the ability to get them to be consistent. I appreciate that they need to have a policy they are actually prepared to enforce, but they also need to be prepared to get into discussion over enforcement decisions. Create a right to take them to court for not enforcing their actual policy, perhaps? (they could do that by contract if they want to bind themselves)
As an aside, I can't think of a successfully-moderated space that is viewpoint-neutral. I can think of lots of spaces - including many inside larger sites (subreddits, Facebook groups, substack comments sections) that have good moderation and valuable discussions. There's even the occasional substack where the comments are better than the articles. But all of those have a clear viewpoint. If you want to ban things that are offensive to both, say, Tom Cotton and Alexandria Ocasio-Cortez, then you're going to have a very bland space and drive everyone away. But you could definitely adopt a policy that either one of those two people would be relatively content with, and build a community around that.
Twitter, which doesn't segregate the communities to anything like the same extent as other platforms, is the hardest platform to do this on. You can't have a terf bit and a trans people bit of Twitter; you can on Facebook or reddit. You can't have a landlord bit and a renters bit, to pick something which is every bit as contentious, but where many fewer would want to ban the other side from having their own space.