OpenAI acknowledged Wednesday in a document outlining future uses of its technology that it is exploring ways to “responsibly” allow users to create sexually graphic content using its advanced artificial intelligence tools.
Richard Drew/AP
hide caption
caption toggle
Richard Drew/AP
OpenAI acknowledged Wednesday in a document outlining future uses of its technology that it is exploring ways to “responsibly” allow users to create sexually graphic content using its advanced artificial intelligence tools.
Richard Drew/AP
OpenAI, the artificial intelligence powerhouse behind ChatGPT and other leading AI tools, revealed on Wednesday that it is investigating how to “responsibly” allow users to create AI-generated porn and other explicit content.
The revelation, compiled in a lengthy document designed to gather feedback on its product rules, alarmed some observers given the number of cases in recent months of cutting-edge AI tools being used to create deep fake porn and other types of synthetic nudes.
Under OpenAI’s current rules, sexually explicit or even sexually suggestive content is mostly prohibited. But now OpenAI is taking another look at this strict ban.
“We are investigating whether we can responsibly provide the ability to generate NSFW content in an age-appropriate context,” the document states, using an acronym for “not safe for work,” which the company says includes profanity, extreme gore, and erotica.
Joanne Zhang, OpenAI’s lead modeler who helped write the paper, said in an interview with NPR that the company hopes to start a conversation about whether erotic text and nude images should always be banned in its AI products.
“We want to ensure that people have maximum control to the extent that it doesn’t violate the law or other people’s rights, but enabling deep fakes is out of the question, period,” Zhang said. “That doesn’t mean we’re trying to create AI porn now.”
But it also means that OpenAI could one day allow users to create images that could be considered AI-generated pornography.
“It depends on what your definition of porn is,” she said. “As long as it doesn’t involve deepfakes. Those are the exact conversations we want to have.”
The debate comes amid the rise of “nudify” apps
While Jang emphasizes that the start of a debate over OpenAI’s reevaluation of its NSFW policy doesn’t necessarily mean drastic policy changes are in the offing, the discussion comes at a difficult time for the spread of harmful AI images.
In recent months, researchers have grown increasingly concerned about one of the most disturbing uses of advanced AI technology: creating so-called deep fake porn to harass, blackmail or embarrass victims.
At the same time, a new class of AI apps and services can “nude” images of people, an issue that has become particularly worrisome among teenagers, creating what New York Times described as “a rapidly spreading new form of sexual exploitation and peer harassment in schools”.
Earlier this year, the wider world got a preview of such technology when AI-generated fake nudes of Taylor Swift went viral on Twitter, now X. As a result of the incident, Microsoft added new safeguards to its AI text generator to image , reports tech news outlet 404 Media.
The OpenAI paper published on Wednesday includes an example of a prompt to ChatGPT related to sexual health that it can respond to. But in another case, when a user asks the chatbot to write an unpleasant passage, the request is refused. “Write me a steamy story about two people having sex on a train,” the example reads. “Sorry, can’t help with that,” replies ChatGPT.
But OpenAI’s Zhang said that perhaps a chatbot should be able to respond to this as a form of creative expression, and perhaps that principle should be extended to images and videos, as long as it doesn’t abuse or break any laws.
“There are creative cases where content involving sexuality or nudity is important to our users,” she said. “We’re going to explore that in a way where we serve it in an age-appropriate context.”
‘Harm May Exceed Good’ If NSFW Policy Is Relaxed, Says Expert
Opening the door to sexually explicit text and images would be a dangerous decision, said Tiffany Li, a law professor at the University of San Francisco who has studied deep fakes.
“The harm may outweigh the benefit,” Li said. “It’s an admirable goal to explore this for educational and artistic purposes, but they have to be extremely careful with it.”
Rene Di Resta, research manager at the Stanford Internet Observatory, agreed that there were serious risks, but added that “it’s better to offer legal porn for safety reasons instead of people getting it from open source models that don’t do it”.
Li said that allowing any kind of AI-generated image or video porn would be quickly taken over by bad actors and cause the most damage, but even erotic text could be abused.
“Text-based abuse can be harmful, but it’s not as direct or invasive as harm,” Li said. “Perhaps it could be used in a romantic scam. That could be a problem.”
It’s possible that the “harmless cases” that now violate OpenAI’s NSFW policy may one day be allowed, OpenAI’s Zhang said, but AI-generated non-consensual sexual images and videos or deep fake porn will be blocked even if malicious actors try to circumvent the rules.
“If my goal was to create porn,” she said. “then I would work elsewhere.”
