Randomize Prompts in Stable Diffusion Notebook

Currently there are number of notebooks that are available that let you create AI Art using Google Colab Notebooks and of course the most popular is the Stable Diffusion at the moment. You can also consider Disco Diffusion, VQGAN or JAX Diffusion which all work on the premise of a text based prompt to create your desired image.

However, it can be cumbersome to create prompts and try variations of a prompt quickly until you find the right combination. So I’ve always been keen and curious on ways to automate it. I used randomization to create an AI art collection called The Symms which worked really well for the purpose at hand. However, I’m always keeping an eye out of easier and faster methods.

I discovered this GitHub source code through Twitter when one of the people I follow shared some awesome looking images created using some randomization of their initial prompt.

Let’s use an example:

a robot standing in front of a epic abandoned spaceship

You can take this prompt and create many variations by replacing the highlighted words. eg. robot becomes, boy, girl, or alien. abandoned becomes rusted, damaged or crashed etc.

Of course you could write these variations but its easier to write a dynamic prompt so you don’t have to and the code does the rest.

The code I use is not mine and I have extracted the useful bits and modified as needed to make it portable to use in any notebook that uses prompts variable to define prompts.

Usually in this format:

prompts =     [
               "prompt one is here",
               "prompt two is here",
               "prompt three is here",
              ]

There is a more advanced option where you can point to a .TXT file that contains the varying attributes of the random values you want to use. This is another quick and easy way to create many random variations of the prompt. See advanced prompt section below.

So here is the code that you should insert before the RUN section of the notebook. Most of these notebooks have integration with Google Drive so they save the images there. I am assuming you WILL use this and therefore point to the path below if its diffeernt

import random
# pick a random item from the corresponding text file
def randomizer(category):
    random.seed()
    randomizers = []
    with open(f'/content/drive/MyDrive/AI/{category}.txt', encoding="utf-8") as f:
        for line in f:
            randomizers.append(line.strip())
    random_item = random.choice(randomizers)
    return(random_item)

# replace anything surrounded by underscores with a random entry from the matching text file
def randomize_prompt(prompt):
    while "_" in prompt:
        start = prompt.index('_')
        end = prompt.index('_', start+1)
        swap = prompt[(start + 1):end]
        swapped = randomizer(swap)
        prompt = prompt.replace(f'_{swap}_', swapped, 1)
    return prompt

# Dynamic value - takes ready-made possible options within a string and returns the string with an option randomly selected
# Format is "I will return <Value1|Value2|Value3> in this string"
# Which would come back as "I will return Value2 in this string" (for example)
# Optionally if a value of ^^# is first, it means to return that many dynamic values,
# so <^^2|Value1|Value2|Value3> in the above example would become:
# "I will return Value3 Value2 in this string"
# note: for now assumes a string for return. TODO return a desired type
def dynamic_value(incoming):
    if type(incoming) == str:  # we only need to do something if it's a string...
        if incoming == "auto" or incoming == "random":
            return incoming
        elif "<" in incoming:   # ...and if < is in the string...
            text = incoming
            while "<" in text:
                start = text.find('<')
                end = text.find('>')
                swap = text[(start + 1):end]
                value = ""
                count = 1
                values = swap.split('|')
                if "^^" in values[0]:
                    count = values[0]
                    values.pop(0)
                    count = int(count[2:])
                random.shuffle(values)
                for i in range(count):
                    value = value + values[i] + " "
                value = value[:-1]  # remove final space
                text = text.replace(f'<{swap}>', value, 1)
            return text
        else:
            return incoming
    else:
        return incoming

prompt = "" #@param {type:"string"}
p_batch =  20#@param {type:"number"}
# process the prompt for randomizers and dynamic values 

newprompts = []
prompt_count = 0
#for prompt in prompts:
while prompt_count < p_batch:
    randomprompt = randomize_prompt(prompt)
    randomprompt = dynamic_value(randomprompt)
    print(f'\nPrompt: {randomprompt}')
    newprompts.append(randomprompt)
    prompt_count+=1
prompts = newprompts

Using the Code

Now you will have two fields show up in the notebook: prompt which is where we define the prompt variations and p_batch is the number of random prompts you want the code to create.

Taking our above example we will extend the original prompt so we can randomize it.

The < > brackets define the list of values you want the code to use from for that position in the prompt.

The | is the OR, eg. use robot or boy or girl or alien

You can further expand by using ^^number at the start of the list of values, that is just after the < bracket. Here is an example:

In this case I am telling the code that I want to use two random values from the list of values that follows. So you will end up with “…epic abandoned crashed spaceship” or “…epic rusted crashed spaceship” or “…epic abandoned damaged spaceship” etc.

Hopefully can start to see the possibilities of this kind of randomization of the prompts.

Advanced Prompts

You can add your random values to a text file, one value per line and upload it to the notebook or save them on Google Drive. Most of these notebooks will have code already added to mount Google Drive as this is where they will save the final image. I am assuming you know how this works and are able to modify one line of code to point to the location where you save/uploaded your txt files.

Modify the location /content/drive/MyDrive/AI/{category}.txt as required to /your/drive/folder/location/{category}.txt, while leaving /{category}.txt intact. Now to use this in your prompt your attribute and filename should match eg. if my file is called adjectives.txt then my attribute in the prompt will be _adjectives_ (without the .txt). Hence the above prompt could be adapted like this:

a <robot|boy|girl|alien> standing in front of a epic _adjectives_ spaceship

You don’t have to create these text files, I found a bunch of them here in this GitHub repository

Thanks to the wonderful open source community that keeps sharing, I hope you found this post useful and have fun using and creating random prompts with this code. I am doing my bit to help others who may want to do this but are intimidated by code, so let me know if you found it useful in the comments below.