Lensa’s viral AI art creations were bound to hypersexualize users

This year, it feels like artificial intelligence-generated art has been everywhere.

Many of us used the summer to enter silly prompts into DALLE Mini (now Craiyon), which generated a collection of nine hilariously janky AI images. But more recently, there’s been a boom of AI-powered apps that can create cool avatars. MyHeritage AI Time Machine creates photos of users in historic settings and settings. AI TikTok filters are popular for creating animated versions of people. This past week, “magic avatars” from Lensa AI flooded social media platforms like Twitter with illustrative and painterly renderings of people’s headshots, as if truly made by magic.

These avatars, created using Stable Diffusion — which allows the AI to “learn” someone’s features based off of submitted images — also opened an ethical can of worms about AI’s application. People discovered that the “magic avatars” tended to sexualize women and appeared to have fake artist signatures on the bottom corner, prompting questions about the images that had been used to train the AI and where they came from. Here’s what you need to know.

What’s Lensa AI?

It’s an app created by Prisma Labs that recently topped the iOS app store’s free chart. Though it was created in 2018, the app became popular after introducing a “magic avatar” feature earlier this month. Users can submit 10 to 20 selfies, pay a fee ($3.99 for 50 images, $5.99 for 100, and $7.99 for 200), and then receive a bundle of AI-generated images in a range of styles like “kawaii” or “fantasy.”

The app’s “magic avatars” are somewhat uncanny in style, refracting likenesses as if through a funhouse mirror. In a packet of 100, at least a few of the results will likely capture the user’s photo well enough in the style of a painting or an anime character. The images are now a common sight on Twitter, TikTok and Facebook. Prisma Labs declined to give a number for the estimate. Megan Fox, Sam Asghari and Chance the Rapper shared Lensa-created looks.

Lensa makes these magical avatars.

Lensa makes use of Stable Diffusion which is an open-source AI deep-learning model. It draws on a collection of internet art. The database, called LAION-5B contains 5.85 billion images-text pairs. It is filter by CLIP, an open-source neural network. Stable Diffusion released on Aug. 22 and Lensa was not the only app to use its text/to-image capabilities. Canva has recently introduced a new feature with the open source AI.

An independent analysis of 12 million images from the data set — a small percentage, even though it sounds massive — traced images’ origins to platforms like Blogspot, Flickr, DeviantArt, Wikimedia, and Pinterest, the last of which is the source of roughly half of the collection.

More concerningly, this “large-scale dataset is uncurated,” says the disclaimer section of the LAION-5B FAQ blog page. This means that this AI was trained using a torrent of unadulterated, pure internet images. Stability AI only removed “illegal content” from Stable Diffusion’s training data, including child sexual abuse material, The Verge reported. Stability AI implemented some changes in November to make it difficult for NSFW images to be made. This week, Prisma Labs told Polygon it too “launched a new safety layer” that’s “aimed at tackling unwanted NSFW content.”

Stable Diffusion’s license says users can’t use it for violating the law, “exploiting, harming or attempting to exploit or harm minors,” or for generating false information or disparaging and harassing others (among other restrictions). However, the technology can generate images that are incompatible with these terms. As The Verge put it, “once someone has downloaded Stable Diffusion to their computer, there are no technical constraints to what they can use the software for.”

This year, AI art generators are so in-demand.

This technology has been around for many years. However, this year saw a few AI art creators go public or become publicly accessible.

This type of generative AI allows users to enter a series of words to generate impressive images. Many of the examples are whimsical and charming, including the one that puts a Shiba Inu inside a beret. You can also see how this technology could easily be used for creating deepfakes and pornography..

Image: DALLE-2

There’s also a degree of finesse that AI art just can’t seem to get — at least, not yet. It tends to struggle with fingers — did you want 12? — and has produced downright nightmarish creations like multiple broken heads and faces.

Stable Diffusion, unlike DALL-E, Midjourney, and Google’s Imagen, is open-source and has thus proliferated widely. Midjourney, which was built by an independent group, went open beta in the summer. If you are a member of its Discord, you will be able to generate 25 free images. OpenAI created DALLE in April. It was removed from its waitlist in September and made beta available in September. Users generated around 2 million images each day. DALL-E provides monthly credit that users can use to generate images. Users can also purchase additional credits. Stable Diffusion is available to anyone who has sufficient processing power. It is also, compared to its competitors, much more unfiltered — and thus able to be used to make more offensive images.

Stability AI, the company behind Stable Diffusion, acknowledged in a release that “the model may reproduce some societal biases and produce unsafe content.” Polygon reached Stability AI for comment and will bring this up with their response.

Image: DALL-E 2

Prisma Labs acknowledges Stable Diffusion’s biases in its FAQ as well. When Polygon asked Prisma Labs about the existence of bias in generative AI, we got this response: “It’s crucial to note that creators of Stable Diffusion Model trained it on a sizable set of unfiltered data from across the internet. We, as well as Stability AI (creator) of Stable Diffusion Model could not consciously use any representation biases. More precisely, AI was created using unfiltered online data. This model is able to reflect the biases that exist in humankind. Essentially, AI is holding a mirror to our society.”

Lensa AI identifies what types of biases it detects.

A number of reporters have pointed out Lensa AI’s “magic avatars” tend to sexualize women and anglicize minorities. Lensa has added large breasts and cartoonish cleavage to images of women — along with generating nudes — when such images weren’t requested.

Olivia Snow, a research fellow at UCLA’s Center for Critical Internet Inquiry, pointed out in Wired that Lensa produced sexualized images even when she submitted photos of herself as a child. A Jezebel reporter asked Prisma Labs about Snow’s findings; Prisma Labs said that Snow had “explicitly and intentionally violated [its] Terms of Use.” Lensa’s terms of service forbid submitting nudes and direct users to submit images of adults, “no kids.” The app also prompts users to click a box indicating that they are 18 years old or over before creating these “magic avatars.”

Lensa perpetuates racism stereotypes like the fetishization and exploitation of Asian women. An Asian journalist writing for MIT Technology Review detailed her experience with Lensa’s app giving her a number of avatars that were “nude” or “showed a lot of skin,” while her white female colleagues “got significantly fewer sexualized images.”

TechCrunch has also noted that it’s fairly easy to create NSFW images of celebrities simply by feeding the AI photoshopped images. For example, this has alarming implications regarding the possibility that such software can be used for creating revenge porn. It is particularly concerning because the COVID-19 pandemic saw a dramatic increase in the number of victims.

PrismaLabs released new features that tackle NSFW images on Dec. 13. A communications representative pointed Polygon to a press release: “This was achievable by a thorough investigation to update and tweak several parameters of the Stable Diffusion model leveraged by the app. The app’s designers at Prisma Labs made it less likely that such Avatars would be created in order to address safety issues and enhance user experience. On rare occasions, when the new NSFW algorithm fails to perform and deliver desired results, the next security layer kicks in to blur any inappropriate visual elements and nudity in the end results.”

It is possible to see it live.

Yes! Lensa received 19 photos on December 13th. They were able to create 100 avatars for $5.99. These images showed my face from different angles with various lighting. One of these images showed my body — I was wearing a loose dress, and taking a photo in the mirror.

In return for 100 photos, Lensa provided a variety of images that showed the features on many different faces. They ultimately didn’t look like me. Lensa seemed to have no idea what to do with my face — I am Taiwanese and white — creating some images that looked East Asian but otherwise like complete strangers, save for one particular quirk or other of mine, like my jawline or eye shape. Some images simply looked like white women, with Lensa even giving me blue eyes — though I have brown eyes, and none of the images I submitted showed me with blue eyes.

These modifications clustered around particular “categories” delineated by the app. The images for “kawaii” looked more East Asian, with a few generating a body that was slim. Under “light” and “fantasy,” the results looked more white. Some of the images in the “iridescent” pack made me look like an android — I’d be interested in comparing my results to others’, as it reflects a trope where Asian women in science fiction tend to be depicted as robots more than they exist as human people. One image in the “cosmic” set gave me random cleavage. None of these images had nudes.

These AI image generators are racist and misogynistic.

It comes down to how these AIs are “trained.” AI will reflect what it has “learned” through the data set it was fed, whether that be a gorgeous art style or grotesque societal bias.

A study conducted in June 2022 by researchers at Georgia Institute of Technology and Johns Hopkins University, among others, found that robots trained by the neural network CLIP “definitively show robots acting out toxic stereotypes” regarding gender and race. They were also “less likely to recognize women and people of color.” The robot more frequently chose a Black man’s face when prompted with “criminal,” for example, and selected Black women and Latina women when prompted “homemaker.”

Racism in AI isn’t new, but for years it’s felt more science fiction than reality; it’s only becoming more relevant as AI-generated art has “arrived” to the extent that you can pay a fee to enjoy it yourself. And it’s not just Stable Diffusion. Images generated by DALL-E reinforce racist and misogynistic stereotypes. Inputting “nurse” yields images of women, while “CEO” yields mostly images of white men. OpenAI recognizes this. An OpenAI blog post, published in July, detailed a “new technique” to “reflect the diversity of the world’s population.” OpenAI also blocks certain words that would yield hateful responses, like the word “shooting.”

Image: Lensa AI

The “Risks and Limitations” section of OpenAI’s Github, updated April 2022, gives a bit of insight into the hurdles that came with training the AI. “Graphic sexual and violent content” were filtered from the training data set, but this also reduced the number of “generated images of women in general.” Put simply, getting rid of sexual violence meant the AI created fewer images of women.

“Bias is a huge industry-wide problem that no one has a great, foolproof answer to,” Miles Brundage, the head of policy research at OpenAI, told Vox in April.

Even Craiyon (née DALL-E Mini) has a limitations and biases section in its FAQ noting that it might “reinforce or exacerbate societal biases.” It further notes “because the model was trained on unfiltered data from the Internet, it may generate images that contain harmful stereotypes.”

How do they feel about Lensa, artists?

Artists have expressed concern about Stable Diffusion training its AI model with art on the internet — some of which is almost certainly copyrighted, given the breadth of what was scraped — without asking those artists for their permission. There isn’t really a way for artists to opt out currently.

Some of Lensa’s “magic avatars” appear to have an artist’s signature on the bottom corner, which sparked debate on Twitter. The letters look inconsistent, but closer examination reveals that they were trained using images. DoHave artist signatures. (PrismaLabs recognizes the phantom signs in its Lensa FAQs.

Using the site “Have I Been Trained” allows people to search whether an image has been scraped into the LAION-5B data set. Some individuals have discovered images of their own in the data sets, but don’t know how. This is why they added an ethical Gordian knot.

People have also argued that AI could replace many artists. In September, art created by Midjourney won first place at the Colorado State Fair’s fine arts competition. DALL-E was featured on the cover of Cosmopolitan’s June magazine. Others have suggested that the process of creating an input query to create AI art can take a lot of time and be very iterative.

Already, examples of AI resembling artistic aesthetics have been appearing on the Internet. Polish digital artist Greg Rutkowski’s artwork has become a dominant style that many of these AI-generated images appear to be based on. Twitter users have fed the prompt “in the style of Wes Anderson” to create frames of other films in the director’s signature twee style. Directors like Guillermo del Toro and Hayao Miyazaki (the latter in 2016, when the technology was far more emergent) have spoken against the use of AI in filmmaking, both calling it “an insult to life itself.”

Meanwhile, some artists have already cited Midjourney as crucial to their creative process, particularly designers who might otherwise not be able to afford early mock-ups — or professionals like interior designers who used it to render what a newly decorated room might look like.

There’s already one prominent example of AI art in video games. It’s High On LifeCreated by Rick and Morty creator Justin Roiland, uses Midjourney-created AI art for “finishing touches,” Roiland confirmed to Sky News. Though he didn’t state what it was used for, Redditors have pointed out that in-game posters appear to be AI-generated. Zoom in and you’ll see that the text is a bit sloppy. It’s not hard to imagine how AI art might displace environment and texture artists, for example, when it is free — or, at least, cheap — and human labor is not.

For its part, Prisma Labs provided this extraordinarily optimistic quote to Polygon about the future of AI and AI-generated art: “‘Democratization of access’ to cutting-edge technology like Stable Diffusion, which is now packaged in the shape and form of an app feature – is quite an incredible milestone. This technology was previously only available to technically-skilled people. You don’t need any technical skills. AI technology is becoming more accessible and sophisticated. It’s likely we will start to see AI-powered features and tools integrated into many consumer-facing apps. This will make each app more customizable, powerful and more user-friendly. We’d like to imagine that AI may also become more integrated into our daily lives with more consumers opting to use AI-powered services to enhance their experiences and ultimately make life a little easier and less stressful. Overall, we believe that the future of AI-powered apps looks bright and full of potential.”

It’s hard enough, on the internet, to detect fact from fiction. It’s also already difficult to make a living in a creative field, as competition and inflation wreak havoc across the industry. The future of AI-generated creativity will make it even more challenging. Regardless of how this technology is applied — and the degree to which artists are impacted — this much seems certain.