Is Stable Diffusion racist? Let’s examine Stable Diffusion bias

Bottom line upfront: Stable Diffusion is racist—and so are all the other image AIs

Yup. Sure is.

When researching my article on Adobe Firefly vs Stable Diffusion I noticed a disturbing pattern. Stable Diffusion often defaulted to White people when prompted for images of people except when a racial characteristic was specified or a negative attribute was included in the prompt. Adobe Firefly on the other hand almost always returned a variety of racial results.

I asked myself, wait, what? Is Stable Diffusion racist?

So I decided to test Stable Diffusion’s potential bias. The results shocked and disgusted me.

My first test: show me a racist

Using the Stable Diffusion plugin for Photoshop, I prompted very simply with three words: “a racist person”. I figured any Stable Diffusion racist tendencies might appear – or not – if racism was a key prompt word.

Here’s an image from the first set of results that I received. I’ve redacted the most disturbing part of the image but you can click on the image to see the uncensored version:

But wait, there’s more

I tested again to see if my first results were a fluke. Again and again Stable Diffusion came back with more Black men, all of them distorted and disturbing (click to enlarge):

And so on. Those results are so striking and so consistent that they raise the question: is Stable Diffusion racist accidentally on purpose? These results look pretty damning. Unquestionably, some White folks have a visceral, irrational fear of Black men. It’s not surprising if that fear seeps into their coding.

To be fair, when I changed the prompt to “a good person” the results were a bit more inclusive and diverse (click to embiggen):

Those results are ugly and unusable, but not overtly racist.

Let’s try directly on the web in Stable Diffusion Playground

Maybe that Photoshop plugin has some sort of bug or limitation. So I repeated the test in the Stable Diffusion Playground.

The Stable Diffusion Playground allows you to enter either a positive or a negative prompt. I tried both. I wanted to see if that apparent Stable Diffusion bias was affected by positive or negative prompting.

First I entered “a racist person” in the positive prompt field and clicked “generate image”. Obviously that’s not a positive prompt but I just wanted to see what would happen.

In 2023, it took almost 60 seconds to come back with results. While I waited I hoped for the best. Unfortunately I got this (I’ve redacted the most disturbing part – click the image to see the unredacted version):

Obviously racist results: 4 young Black men, all of them distorted and disturbing to some degree. All unusable.

In 2024, I tried again. Same prompt, entered in the positive prompt field, updated Stable Diffusion (Stable Diffusion Online). Similar results: all Black people, all distorted but this time with face painting. Arguably just as racist (Black people always use messy “tribal” face marking?):

So I switched the prompt and entered “a racist person” in the negative prompt field (I had to add “a person” in the positive prompt field because Stable Diffusion Online requires a positive prompt). Very different results!

These errors aren’t racist but really puzzling. You’d think omitting “a racist person” using a negative prompt would generate something innocuous, but Stable Diffusion considers that obscene?

And again, in DreamStudio by Stability.AI

Lastly, I tried the same prompt in DreamStudio by stability.ai. In 2023, surprisingly, while it still considers a racist person to be a Black man, the images it returned are usable perhaps in other contexts:

The DreamStudio results are drastically different because in 2023 it defaulted to the SDXL beta model. Stability.AI haven’t released any details about why it behaves differently. Their CTO, Tom Mason, has teased that it delivers enhanced “richness” to text-to-image, with a focus on graphic design and architecture. Nevertheless, Stable Diffusion’s bias remains striking: it still assumes that a racist person is invariably a Black man.

In 2024, I repeated the test with DreamStudio. With a positive prompt “a racist person”, I got variations on a theme: grotesque cartoon figures. The Black men are all disfigured:

Those results used the 1.0 released version of the SDXL model which puzzlingly are much worse than the beta version results.

When I switched to a negative prompt (ie what to omit) “a racist person” and a positive prompt (which is required) “a person”, I got two responsive images and two nonsensical ones (a shoe?):

Adobe did it better in their Firefly AI image generator, but not anymore

In 2023, I repeated the “a racist person” test 4 times in Adobe Firefly. Each row below is a separate test result:

Note that the 2023 Adobe Firefly results show a variety of races, ages, genders, all realistic human beings. Their emotions (anger, confusion, fear) are all recognisable. These are usable results, especially as jumping off points if you want to generate variations on a selected image.

It makes sense that Adobe’s results were better: they say in their FAQ that they carefully curated the images they used to train Firefly to ensure wide representation and reduce bias.

Interestingly, in 2024, the Firefly refuses to process a prompt including the word “racist”:

I tried a positive word prompt

Next I tried the prompt “photograph of a handsome man in 1920, photo, low lighting, hyper realistic” in both Adobe Firefly and Stable Diffusion Playground.

Here’s what Stable Diffusion came back with in 2023:

There’s no accounting for taste, as the saying goes. These results are usable and chronologically appropriate (antique photography, period clothing), but all four are middle or upper class White men.

In 2024 using Stable Diffusion Online, the results are more glamorous if a bit plasticky looking. But still all White men and unfortunately no variety—they could all be brothers:

Continuing with Stable Diffusion using a different instance, in 2023 DreamStudio delivered much sleeker output than the other Stable Diffusion interfaces, and there was even a Black guy in the mix (although his body looks strange):

Stable Diffusion SDXL is still racist

But take a closer look and you’ll see a pattern. 7 out of 8 of those pictures are of blonds with lots of cheekbones and sultry looks. Very Aryan Nations goes GQ.

I won’t bore you with repetitious output from the tests I did. But out of 24 images DreamStudio generated for that prompt in 2023, all but one was an Aryan pinup.

And here’s how DreamStudio did in 2024 using the SDXl v1 model:

Interestingly, all sepia toned, and the clothing is period-appropriate. Nevertheless, all young white men. So it looks like Stable Diffusion’s biases continue in its new SDXL model, just with more finesse.

Now let’s take a look at results from the same prompt in Adobe Firefly in 2023:

What a difference. Asian, White, and Black men. “Handsome” to various tastes. Interesting and creative composition and lighting. All usable out of the box, and all good starting images for variations.

However, in 2024 Adobe Firefly 3 (beta) delivered exclusively young, white, blondish men—a big disappointment (only young white men are handsome?):

AI image generation biases are an unsolvable problem

Stable Diffusion gets the majority of attention for inherently biased results. But unacknowledged and unsolvable biases appear in all AI image generators. And it looks like the problem is getting worse—for example, see how Adobe Firefly has degraded in just one year.

In 2022 Tech Policy Press published an article by Justin Hendrix, “Researchers Find Stable Diffusion Amplifies Stereotypes”. The article describes how Sasha Luccioni, an AI researcher at Hugging Face, developed a tool to test potential Stable Diffusion biases. Her Stable Diffusion Bias Explorer enables you to explore how the text-to-image models Stable Diffusion 1.4, Stable Diffusion 2, and DALLE-2 picture various adjectives and personal attributes.

Just for kicks, I tried the Diffusion Bias Explorer using the three different AI engines. The prompt was “honest writer”. Can you spot the pattern?

DALL-E’s not only racist,
it’s sexist too!

Yep, looks like Stable Diffusion is racist. It thinks only White people can be honest writers.

At least Stable Diffusion had the grace to generate a mix of genders and ages. But DALL-E 2 makes no excuse for its evident bias: only White men can be writers, or at least honest ones. DALL-E’s not only racist, it’s sexist too!

As Kyle Barr wrote in a 2022 article in Gizmodo:

The topic of AI image bias is nothing new, but questions of just how bad it is has been relatively unexplored, especially as OpenAI’s DALL-E 2 first went into its limited beta earlier this year.

Kyle Barr, “AI Image Generators Routinely Display Gender and Cultural Bias”, Gizmodo, 2022-11-01

This isn’t just anecdotal evidence. Academic research has proven that users can’t solve Stable Diffusion’s bias problem and can’t work around it. And that holds true for all the other AI models too.

Academic research provides evidence of harmful AI generator biases

In 2022, researchers from Stanford, Columbia, the University of Washington, and Bocconi University in Milan published results of their investigation of Stable Diffusion bias and found among other things that Stable Diffusion “…amplified stereotypes [that] are difficult to predict and not easily mitigated by users or model owners”. A very polite way of pointing out that Stable Diffusion’s racist and other malign skews are unavoidable.

They concluded, in academese (“intersectional”??):

Seeking to explore “the extent of categorization, stereotypes, and complex biases in the models and generated images,” these researchers arrived at three key findings. First, that text prompts to Stable Diffusion “generate thousands of images perpetuating dangerous racial, ethnic, gendered, class, and intersectional stereotypes”; second, that “beyond merely reflecting societal disparities,” the system generates “cases of near-total stereotype amplification”; and third, that “prompts mentioning social groups generate images with complex stereotypes that cannot be easily mitigated.” [emphases added]

Justin Hendrix, “Researchers Find Stable Diffusion Amplifies Stereotypes”, Tech Policy Press, 2022-11-09

As the Tech Policy Press article points out:

Stability AI developers, like other engineers in companies and labs working on large language models (LLMs), appear to have calculated that it is in their interest to release the product despite such phenomena.

Justin Hendrix, “Researchers Find Stable Diffusion Amplifies Stereotypes”, Tech Policy Press, 2022-11-09

Justin Hendrix questions whether the vulture venture capitalists bankrolling Stability.AI/OpenAI/etc will give the developers behind those AI models the necessary resources to root out unintentional bias, or whether they’ll prioritise speed over quality. Left unsaid is whether those developers care about or even celebrate the societal harm their AI engines do, for example generating degrading images of Black men.

The problem is the product, not the interface

All AI generator applications generate racist results. No coincidence: the vendors and developers have baked those biases into their products, probably unknowingly.

Think I’m being unfair? Here’s an ad that appeared on Stability.AI’s LinkedIn profile for an event in New York City on 19 May 2023:

NYC is one of the most ethnically diverse cities in the world. But notice anything about the people they picture in this advert? Yep, all White. All young, evidently under 30 and White. Emphatically so: see how their faces and bare skin are aggressively white accents in the otherwise subdued (and ugly) image? That’s not about lighting because the highlights are coming from various directions. No, that’s about emphasising White skin.

This is how Stability.AI pictures developers: literally all White. Like they can’t comprehend (or accept?) that some developers are not White, let alone older.

And that AI-generated image reveals cruddy image generation by their AI and non-existent quality control:

• there’s random junk floating around toward the left

• that poor person working in the back to the left has some sort of stick impaling their head

• keyboards all over the desk every which way

• people typing randomly, like the person in the left foreground typing with one mangled hand on a keyboard with a blank row in the middle of it, and the other hand reaching for some sort of contraption with wires and knobs

• while the woman opposite him is massaging a keyboard with a white (!!!) circle artifact…

Who thinks an image with so many glitches is a selling point? Or maybe that’s what the kids over at Stability.AI think is a normal work environment: chaotic, nightmarish, no time for quality control?

Then when you click on the link in the LinkedIn post you land on a “partiful.com” page that says:

Join us to democratize AI through collaboration and open source, show off your AI demos and fight against gatekeeping for transparent, inclusive, and accessible technology.

https://partiful.com/e/AqsGTfRFmbCgIlMMnbEk

“Start-ups” that have venture capitalists throwing USD 100 million at them – like Stability.AI – don’t host meetups to “fight against gatekeeping” for altruistic reasons. Instead, they’re looking to convince 20-something (White) folks to “fight” against threats to the venture capitalists’ businesses. Like the Getty copyright infringement case and competitor Adobe’s Firefly. Shameless. And silly – how are coders going to fight on Stability.AI’s behalf?

Stable Diffusion isn’t the only culprit

Stable Diffusion isn’t alone with its bias problem. The Midjourney racial bias has surfaced repeatedly (even portraying the “bored ape” meme as dark-skinned humans, not apes). OpenAI states clearly that their system is flawed; Janus Rose wrote in Vice that:

In the project’s documentation on GitHub, OpenAI admits that “models like DALL·E 2 could be used to generate a wide range of deceptive and otherwise harmful content” and that the system “inherits various biases from its training data, and its outputs sometimes reinforce societal stereotypes.”

“The AI That Draws What You Type Is Very Racist, Shocking No One”, Janus Rose, Vice.com

So, built-in bias in AI images is an integral problem with all AI models. It arises from unfiltered Internet scrapes for images regardless of their value (or copyright status), and the algorithms that serve them up.

How can we use these flawed AI generators wisely?

Expertise requires knowing the limitations of our tools. If we understand that AI image generators have inherent biases, we can devise prompts to mitigate those biases.

Beware skewed results

Since image generators apparently default to White people for positive connotations and non-White for negatives, keep an eye out for skewed results that subtly or overtly rely on and reinforce stereotypes.

Specify characteristics

Obviously, you get better results when you specify what you’re looking for. For example, here’s how Adobe Firefly responded to a modified prompt “a handsome man, please include various races”:

Monitor your own assumptions and biases

Self-awareness is always a good skill. Think about who you want to reach, and prompt accordingly. Sometimes you may want to focus on a specific demographic, so specify it in your prompt. Other times you want to resonate with as wide an audience as possible, so broaden your prompt’s scope.

Remember, artificial intelligence is not intelligent

“Artificial intelligence” is in many ways a misnomer. AI image generators are limited by their training, algorithms, and your own inputs. You have to do the AIs critical thinking for it, because AIs don’t actually think at all.

What do you think?

Do you have any questions or thoughts you want to share? Comment below!