Last week, Gawker received a curious document. Turned over by an aggrieved worker from the online freelance employment site oDesk, the document iterated, over the course of several pages and in unsettling detail, exactly what kinds of content should be deleted from the social networking site that had outsourced its content moderation to oDesk’s team. The social networking site, as it turned out, was Facebook.

The document, antiseptically titled “Abuse Standards 6.1: Operation Manual for Live Content Moderators” (along with an updated version 6.2 subsequently shared with Gawker, presumably by Facebook) is still available from Gawker. It represents the implementation of the Facebook’s Community Standards, which present Facebook’s priorities around acceptable content, but stay miles back from actually spelling them out. In the Community Standards, Facebook reminds users that “We have a strict ‘no nudity or pornography’ policy. Any content that is inappropriately sexual will be removed. Before posting questionable content, be mindful of the consequences for you and your environment.” But, an oDesk freelancer looking at hundreds of pieces of content every hour needs more specific instructions on what exactly is “inappropriately sexual” — such as removing “Any OBVIOUS sexual activity, even if naked parts are hidden from view by hands, clothes or other objects. Cartoons / art included. Foreplay allowed (Kissing, groping, etc.). even for same sex (man-man / woman-woman”. The document offers a tantalizing look into a process that Facebook and other content platforms generally want to keep under wraps, and a mundane look at what actually doing this work must require.

It’s tempting, and a little easy, to focus on the more bizarre edicts that Facebook offers here (“blatant depictions of camel toes” as well as “images of drunk or unconscious people, or sleeping people with things drawn on their faces” must be removed; pictures of marijuana are OK, as long as it’s not being offered for sale). But the absurdity here is really an artifact of having to draw this many lines in this much sand. Any time we play the game of determining what is and is not appropriate for public view, in advance and across an enormous and wide-ranging amount of content, the specifics are always going to sound sillier than the general guidelines. (It was not so long ago that “American Pie’s” filmmakers got their NC-17 rating knocked down to an R after cutting the scene in which the protagonist has sex with a pie from four thrusts to two.)

Lines in the sand are like that. But there are other ways to understand this document: for what it reveals about the kind of content being posted to Facebook, the position in which Facebook and other content platforms find themselves, and the system they’ve put into place for enforcing the content moderation they now promise.

Facebook or otherwise, it’s hard not to be struck by the depravity of some of the stuff that content moderators are reviewing. It’s a bit disingenuous of me to start with camel toes and man-man foreplay, when what most of this document deals with is so, so much more reprehensible: child pornography, rape, bestiality, graphic obscenities, animal torture, racial and ethnic hatred, self-mutilation, suicide. There is something deeply unsettling about this document in the way it must, with all the delicacy of a badly written training manual, explain and sometimes show the kinds of things that fall into these categories. In 2010, the New York Times reported on the psychological toll that content moderators, having to look at this “sewer channel” of content reported to them by users, often experience. It’s a moment when Supreme Court Justice Potter Stewart’s old saw about pornography, “I know it when I see it,” though so problematic as a legal standard, does feel viscerally true. It’s a disheartening glimpse into the darker side of the “participatory web”: no worse or no better than the depths that humankind has always been capable of sinking to, though perhaps boosted by the ability to put these coarse images and violent words in front of the gleeful eyes of co-conspirators, the unsuspecting eyes of others, and sometimes the fearful eyes of victims.

This outpouring of obscenity is by no means caused by Facebook, and it is certainly reasonable for Facebook to take a position on the kinds of content it believes many of its users will find reprehensible. But, that does not let Facebook off the hook for the kind of position it takes: not just where it draws the lines, but the fact that it draws lines at all, the kind of custodial role it takes on for itself, and the manner in which it goes about performing that role. We may not find it difficult to abhor child pornography or ethnic hatred, but we should not let that abhorrence obscure the fact that sites like Facebook are taking on this custodial role — and that while goofy frat pranks and cartoon poop may seem irrelevant, this is still public discourse. Facebook is now in the position of determining, or helping to determine, what is acceptable as public speech — on a site in which 800 million people across the globe talk to each other every day, about all manner of subjects.

This is not a new concern. The most prominent controversy has been about the removal of images of women breastfeeding, which has been a perennial thorn in Facebook’s side; but similar dustups have occurred around artistic nudity on Facebook, political caricature on Apple’s iPhone, gay themed books on Amazon, and fundamentalist Islamic videos on YouTube. The leaked document, while listing all the things that should be removed, is marked with the residue of these past controversies, if you know how to look for them. The document clarifies the breastfeeding rule, a bit, by prohibiting “Breastfeeding photos showing other nudity, or nipple clearly exposed.” Any commentary that denies the existence of the Holocaust must be escalated for further review, not surprising after years of criticism. Concerns for cyber-bullying, which have been taken up so vehemently over the last two years, appear repeatedly in the manual. And under the heading “international compliance” are a number of decidedly specific prohibitions, most involving Turkey’s objection to their Kurdish separatist movement, including prohibitions on maps of Kurdistan, images of the Turkish flag being burned, and any support for PKK (The Kurdistan Workers’ Party) or their imprisoned founder Abdullah Ocalan.

Facebook and its removal policies, and other major content platforms and their policies, are the new terrain for longstanding debates about the content and character of public discourse. That images of women breastfeeding have proven a controversial policy for Facebook should not be surprising, since the issue of women breastfeeding in public remains a contested cultural sore spot. That our dilemmas about terrorism and Islamic fundamentalism, so heightened over the last decade, should erupt here too is also not surprising. The dilemmas these sites face can be seen as a barometer of our society’s pressing concerns about public discourse more broadly: how much is too much; where are the lines drawn and who has the right to draw them; how do we balance freedom of speech with the values of the community, with the safety of individuals, with the aspirations of art and the wants of commerce.

But a barometer simply measures where there is pressure. When Facebook steps into these controversial issues, decides to authorize itself as custodian of content that some of its users find egregious, establishes both general guidelines and precise instructions for removing that content, and then does so, it is not merely responding to cultural pressures, it is intervening in them, reifying the very distinctions it applies. Whether breastfeeding is made more visible or less, whether Holocaust deniers can use this social network to make their case or not, whether sexual fetishes can or cannot be depicted, matters for the acceptability or marginalization of these topics. If, as is the case here, there are “no exceptions for news or awareness-related content” to the rules against graphic imagery and speech, well, that’s a very different decision, with different public ramifications, than if news and public service did enjoy such an exception.

But the most intriguing revelation here may not be the rules, but how the process of moderating content is handled. Sites like Facebook have been relatively circumspect about how they manage this task: they generally do not want to draw attention to the presence of so much obscene content on their sites, or that they regularly engage in “censorship” to deal with it. So the process by which content is assessed and moderated is also opaque. This little document brings into focus a complex chain of people and activities required for Facebook to play custodian.

The moderator using this leaked manual would be looking at content already reported or ‘flagged” by a Facebook user. The moderator would either “confirm” the report (thereby deleting the content), “unconfirm” it (the content stays) or “escalate” it, which moves it to Facebook for further or heightened review. Facebook has dozens of its own employees playing much the same role; contracting out to oDesk freelancers, and to companies like Caleris and Telecommunications On Demand, serves as merely a first pass. Facebook also acknowledges that it looks proactively at content that has not yet been reported by users (unlike sites like YouTube that claim to wait for their users to flag before they weigh in). Within Facebook, there is not only a layer of employees looking at content much as the oDesk workers do, but also a team charged with discussing truly gray area cases, empowered both to remove content and to revise the rules themselves.

At each level, we might want to ask: What kind of content gets reported, confirmed, and escalated? How are the criteria for judging determined? Who is empowered to rethink these criteria? How are general guidelines translated into specific rules, and how well do these rules fit the content being uploaded day in and day out? How do those involved, from the policy setter down to the freelance clickworker, manage the tension between the rules handed to them and their own moral compass? What kind of contextual and background knowledge is necessary to make informed decisions, and how is the context retained or lost as the reported content passes from point to point along the chain? What kind of valuable speech gets caught in this net? What never gets posted at all, that perhaps should?

Keeping our Facebook streets clean is a monumental task, involving multiple teams of people, flipping through countless photos and comments, making quick judgments, based on regularly changing proscriptions translated from vague guidelines, in the face of an ever-changing, global, highly contested, and relentless flood of public expression. And this happens at every site, though implemented in different ways. Content moderation is one of those undertakings that, from one vantage point, we might say it’s amazing that it works at all, and as well as it does. But from another vantage point, we should see that we are playing a dangerous game: the private determination of the appropriate boundaries of public speech. That’s a whole lot of cultural power, in the hands of a select few who have a lot of skin in the game, and it’s being done in an oblique way that makes it difficult for anyone else to inspect or challenge. As users, we certainly cannot allow ourselves to remain naive, believing that the search engine shows all relevant results, the social networking site welcomes all posts, the video platform merely hosts what users generate. Our information landscape is a curated one. What is important, then, is that we understand the ways in which it is curated, by whom and to what ends, and engage in a sober, public conversation about the kind of public discourse we want and need, and how we’re willing to get it.

This article first appeared on Salon.com, and is cross-posted at Culture Digitally and Social Media Collective.

The mistake the entertainment industry continues to make is that they want to stop both kinds of piracy, and they seem unwilling to admit that they are different and start dealing with them as separate problems, with different tools, and with a different “threat level” in their rhetoric. SOPA was problematic in so may respects, but in particular because it tried to address both kinds of piracy at once, and failed to handle either appropriately. The kinds of measures it was suggesting for “rogue, foreign websites” (let’s assume they meant the hardcore piracy networks) wouldn’t be enough: if the DoJ got a court order to remove these sites from Google’s search and the major ISPs, you or I might not be able to access these sites. But determined file traders don’t find them through Google. And SOPA got so much blowback because it also tried to include the second kind of piracy at the same time – which, in fact, is handled relatively well with the “notice-and-takedown” rules that already apply to content platforms like YouTube.

It’s not only that these two kinds of piracy are so different that they require distinctly different approaches: it’s that the entertainment industry needs to let go of trying to squelch them both in the same breath. If they could start distinguishing the two, and make clear that they don’t want to catch up YouTube and Facebook in their net in the process, I think the technology industries will be more willing to develop and uphold gentle norms and procedures for the kinds of infringement that may happen on their networks.

[Cross posted on Culture Digitally and MSR Social Media Collective]

While it’s reasonable for Congress to look for progressive, legislative ways to enforce copyrights and discourage flagrant piracy, SOPA (the Stop Online Piracy Act) and PIPA (the Protect IP Act) now under consideration are a fundamentally dangerous way to go about it. Their many critics have raised many compelling reasons for why [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16]. But in my eyes, they are most dangerous because of their underlying logic: policing infringement by rendering sites invisible.

Under SOPA and PIPA, if a website is even accused of hosting or enabling infringing materials, the Attorney General can order search engines to delete that site from their listings, require ISPs to block users’  access to it, and demand payment services (like PayPal) and advertising networks to cancel their accounts with it. (This last step can even be taken by copyright holders themselves, with only a good faith assertion that the site in question is infringing.) What a tempting approach to policing the Internet: rather than pursuing and prosecuting this site and that site, in an endless game of whack-a-mole, just turn the large-scale intermediaries, and use their power to make websites available, in order to make them unavailable. It shows all too plainly that the Internet is not some wide open, decentralized, unregulatable space, as some have believed. But, it undercuts the longstanding American tradition of how to govern information, which has always erred on the side of letting information, even abhorrent or criminal information, be accessible to citizens, so we can judge for ourselves. Making it illegal to post something is one thing, but wiping the entire site clean off the board as if it never existed is another.

Expunging an infringing site from being found is problematic in itself, a clear form of “prior restraint.” But it is exacerbated by the fact that whole sites might be rendered invisible on the basis of just bits of infringing content they may host. This is a particular troubling to sites that host user-generated content, where one infringing thread, post, or community might co-exist amidst a trove of other legitimate content. Under SOPA and PIPA, a court order  could remove not just the offending thread, but the entire site from Google’s search engine, from ISPs, and from ad networks, all in a blink.

These are the same strategies, not only that China, Iran, and Vietnam currently use to restrict political speech (as prominent critics have charged), but that were recently used against Wikileaks right here at home. When Amazon kicked Wikileaks off its cloud computing servers, when Wikileaks was de-listed by one DNS operator, when Mastercard and Paypal refused to take donations for the organization, they were attempting to render Wikileaks invisible before a court ever determined, or even alleged, that Wikileaks had broken any laws. So it is not a hypothetical that this tactic of rendering invisible will not only be dangerous for commercial speech, or the expressive rights of individual users, but for vital, contested, political speech. SOPA and PIPA would simply organize these tactics into a concerted, legally-enforced effort to erase, to which all search engines and ISP would be obligated to impose.

A lighthearted aside: In the film Office Space, the soulless software company chose not to fire the hapless Milton. Instead, they took away his precious stapler, moved him to the basement, and simply stopped sending him paychecks. We laughed at the blank-faced cruelty, because we recognized how tempting this solution would be, a deft way to avoid having to someone to their face. Congress is considering the same “Bobs” strategy here. But while it may be fine for comedy, this is hardly the way to address complex legal challenges around the distribution of information that should be dealt with in the clear light of a court room. And it risks rendering invisible elements of the web that might deserve to remain.

We are at a point of temptation. The Internet is both so powerful and so unruly because anyone can add their site to it (be it noble or criminal, informative or infringing) and it will be found. It depends on, and presumes, a principle of visibility. Post the content, and it is available. Request it, from anywhere in the world, and the DNS servers will find it. Search for it in Google, and it will appear. But, as those who find this network most threatening come calling, with legitimate (at least in the abstract) calls to protect children / revenue / secrets / civility, we will be sorely tempted to address these challenges simply by wiping them clean off the network.

This is why the response to SOPA and PIPA, most prominently in the January 18 blackouts by Reddit, Wikipedia, and countless blogs, are so important. Removing their content, even for a day, is meant to show how dangerous this forced invisibility could be. It should come as no surprise that, while many other Internet companies have voiced their concerns about SOPA, it is Wikipedia and Reddit that have gone the farthest in challenging the law. Not only do they host, i.e. make visible, an enormous amount of user-generated content. But they are themselves governed in important ways by their users. Their decisions to support a blackout were themselves networked affairs, that benefited from all of their users having an ability to participate — and recognized that commitment to openness as part of their fundamental mission.

Whether you care about the longstanding U.S. legal tradition of information freedoms, or the newly emergent structural logic of the Internet as a robust space of public expression, both require a new and firm commitment in our laws: to ensure that the Internet remains navigable, that sites remain visible, that pointers point and search engines list, regardless of the content. Sites hosting or benefitting from illegal or infringing content should be addressed directly by courts and law enforcement, armed with a legal scalpel that’s delicate enough to avoid carving off huge swaths of legitimate expression. We might be able to build a coalition of content providers and technology companies willing to partner on anti-piracy legislation, if copyright holders could admit that they need to go after the determined, underground piracy networks bent on evading regulation, and not in the same gesture put YouTube at risk for a video of a kid dancing to a Prince tune — there is a whole lot of middle ground there. But a policy premised on rendering parts of the web invisible is not going to accomplish that. And embracing this strategy of forced invisibility is too damaging to what the Internet is and could be as a public resource.

(Cross-posted at Culture Digitally and MSR’s Social Media Collective.)

Moveon.org began circulating this infographic yesterday; The (much more detailed) original is from OWNI.eu. It tells a now-familiar-but-still-important story about the increasing consolidation of commercial media (and by implication, a concern about its impact on public discourse). Despite the times, the attention here is not on online media or new forms of information distribution, though that attention would shift the image only slightly — where might we add Hulu as a “notable property”… under News Corp, GE, and Disney? We might also have to add Google, Apple, and Facebook. But would that change the basic concern? Do they shift the “staggering” percentage listed at the top? And what does “control” mean when we talk about not just content providers but distributors, platforms, and networks as well?

I’m going back to read some scholarship on joirnalistic objectivity; this quote was cited in Michael Schudson’s essay “The Objectivity Norm in American Journalism.” This is the best articulation I’ve come across of the idea of the “marketplace of ideas” and, with it, the call for editorial neutrality. Unfortunately, I can only agree with the first half of the statement. Still, well said.

This conference call looked particularly interesting.

The Nonhuman Turn in 21st Century Studies
May 3-5, 2012
Center for 21st Century Studies, University of Wisconsin, Milwaukee

abstracts due, Dec 19, 2011 (CFP)

This conference takes up the “nonhuman turn” that has been emerging in the arts, humanities, and social sciences over the past few decades. Intensifying in the 21st century, this nonhuman turn can be traced to a variety of different intellectual and theoretical developments from the last decades of the 20th century:

- actor-network theory, particularly Bruno Latour’s career-long project to articulate technical mediation, nonhuman agency, and the politics of things

- affect theory, both in its philosophical and psychological manifestations and as it has been mobilized by queer theory

- animal studies, as developed in the work of Donna Haraway, projects for animal rights, and a more general critique of speciesism

- the assemblage theory of Gilles Deleuze, Manuel DeLanda, Latour, and others

- new brain sciences like neuroscience, cognitive science, and artificial intelligence

- new media theory, especially as it has paid close attention to technical networks, material interfaces, and computational analysis

- the new materialism in feminism, philosophy, and marxism

- varieties of speculative realism like object-oriented philosophy, vitalism, and panpsychism

- and systems theory in its social, technical, and ecological manifestations

Such varied analytical and theoretical formations obviously diverge and disagree in many of their aims, objects, and methodologies. But they are all of a piece in taking up aspects of the nonhuman as critical to the future of 21st century studies in the arts, humanities, and social sciences.

I just came across a notice for two new books, both of which seemed relevant to the ideas that get discussed on this blog. Thought I would pass them along.

Art Platforms and Cultural Production on the Internet
Olga Goriunova
http://www.routledge.com/books/details/9780415893107/
Routledge Research in Cultural and Media Studies

In this book, Goriunova offers a critical analysis of the processes that produce digital culture. Digital cultures thrive on creativity, developing new forces of organization to overcome repetition and reach brilliance. In order to understand the processes that produce culture, the author introduces the concept of the art platform, a specific configuration of creative passions, codes, events, individuals and works that are propelled by cultural currents and maintained through digitally native means. Art platforms can occur in numerous contexts bringing about genuinely new cultural production, that, given enough force, come together to sustain an open mechanism while negotiating social, technical and political modes of power.

Software art, digital forms of literature, 8-bit music, 3D art forms, pro-surfers, and networks of geeks are test beds for enquiry into what brings and holds art platforms together. Goriunova provides a new means of understanding the development of cultural forms on the Internet, placing the phenomenon of participatory and social networks in a conceptual and historical perspective, and offering powerful tools for researching cultural phenomena overlooked by other approaches.

Olga Goriunova is Senior Lecturer in Media Practices at London Metropolitan University, curator of the recent show Funware (Arnolfini, Mu, Baltan) and an editor of Computational Culture.

Wirelessness, Radical Empiricism in Network Cultures
Adrian Mackenzie
http://mitpress.mit.edu/catalog/item/default.asp?ttype=2&tid=12285
The MIT Press

How has wirelessness—being connected to objects and infrastructures without knowing exactly how or where—become a key form of contemporary experience? Stretching across routers, smart phones, netbooks, cities, towers, Guangzhou workshops, service agreements, toys, and states, wireless technologies have brought with them sensations of change, proximity, movement, and divergence. In Wirelessness, Adrian Mackenzie draws on philosophical techniques from a century ago to make sense of this most contemporary postnetwork condition. The radical empiricism associated with the pragmatist philosopher William James, Mackenzie argues, offers fresh ways for matching the disordered flow of wireless networks, meshes, patches, and connections with felt sensations.

For Mackenzie, entanglements with things, gadgets, infrastructures, and services—tendencies, fleeting nuances, and peripheral shades of often barely registered feeling that cannot be easily codified, symbolized, or quantified—mark the experience of wirelessness, and this links directly to James’s expanded conception of experience. “Wirelessness” designates a tendency to make network connections in different times and places using these devices and services. Equally, it embodies a sensibility attuned to the proliferation of devices and services that carry information through radio signals. Above all, it means heightened awareness of ongoing change and movement associated with networks, infrastructures, location, and information.

The experience of wirelessness spans several strands of media-technological change, and Mackenzie moves from wireless cities through signals, devices, networks, maps, and products, to the global belief in the expansion of wireless worlds.

Adrian Mackenzie is Reader and Codirector at the Centre for Science Studies at Lancaster University, U.K, author of Cutting Code, software and society and Transductions, bodies and machines at speed and an editor of Computational Culture.

I keep running into this question, such that it feels like it is making a resurgence: concerns about the sexualization of avatars, in both comics, graphic novels, and video games. Here is a pointed discussion on Racalicious about a video game design conference where one set of panelists made a little too plain what the criteria are for designing female video game heroes:

“After making a semi-disparaging remark about female characters drawn in a North American style, he concludes “I’d rather have female characters from Final Fantasy or Soul Caliber to sleep with.” This draws chuckles from the crowd. And there it was, the truth about character design that so many players know but most designers wouldn’t usually articulate: most of the egregiously sexist character designs are based on fuckability, rather than playability.

Drawing attractive characters isn’t a crime. But it starts to become grating when characters are not only attractive, but hypersexualized and mostly defined by their appearance. Even when characters aren’t hypersexualized, they can still be boring and flat in execution if there is more attention paid to animating her curves than the character herself.” (excerpt)

This follows on the heels of some discussion I’ve run across about the way female comic book heroes are being re-booted, in a way that over-emphasizes their sexuality. Ire and rebuke have arisen over the reboot of Starfire character from Teen Titans in DC Comics’ Red Hood and the Outlaws, including a post at Comics Alliance in which the author’s seven-year old daughter expresses her troubled ambivalence on the highly sexualized version of her once favorite character. There has also been recent discussion of the both sexualized and degraded version of Catwoman in her graphic novel reboot and in the new Batman video game, Arkham City. Laura Hudson puts it this way:

“In Catwoman, this is what DC Comics tells me a male hero looks like, and what a female hero looks like:

This is not an anomaly. This is the primary message that I hear. And it is one that I only hear about the people who are like me — the women — and not the men…

Female characters are only insatiable, barely-dressed aliens and strippers because someone decided to make them that way. It isn’t a fact. It isn’t an inviolable reality, especially in a comic book universe that has just been rebooted. In the end, what matters is what you choose to show people and how you show them, not the reasons you make up to justify it. Because this is comics, everybody. You can make up anything.”

In a number of these comments, though the criticism is reserved for the publishers, there is often a comment that these sexualized portrayals have emerged from fanfiction. Hudson highlights “the aggressively fanfictiony on-panel sex between Batman and Catwoman” as indicative of the problem; Peterson’s quote from the Racalicious post continues, ”But the model for art in our fandom communities is often sex appeal first, to the detriment of characters.”

The question of sexualized womens’ bodies in media is certainly not a new one. But this reminds me of a conversation we have been having, about what questions of “content” look like in new media schoalrship: I keep having this itchy feeling that, though we’ve nominally charged ourselves with talking about “digital cultural production,” we’ve had little substantive discussion of digital cultural productions. We seem to have a lot of strength in examining technologies, distribution, producers, and user practices, but the dimension of what in the end gets made seems absent. Perhaps we are too quick to drop persistent questions about the character of the content, and the complexity of how these images emerge both from professional media organizations and from fan communities, and are appealing and troubling among both.