A few days ago there was all this excitement about a reddit post talking about QR codes embedded in AI generated images using Stable Diffusion and ControlNet. Unfortunately the thread was not very detailed on how to achieve this. All it hinted was the fact that there is a ControlNet model that allows you to create AI Generated art which can be scanned as QR code using your phone.
So I tried a few different methods without much success. I tried using Txt2Img as well as Img2Img in order to get some decent results. The closest I got to something artistic was this maze style QR codes that could be scanned with your phone.
As I scroll through my social media I saw several people talking about this process and some of the hypothesis was that the person who originally made this was going to release the model for public use but instead he got offered hefty amount of money to keep it private and perhaps the investor would have the idea to create online app that would do this for general public with ease.
Which is not a bad idea, with all the advancements if you can make something that will create a market niche and make you money, then why not do it. I discovered this awesome tutorial and embarked on experimenting the process shared in this video.
What you need
In order to get started you need to have Stable Diffusion running with ControlNet installed already. Next make sure you have these two models downloaded.
- SD15_tile – https://huggingface.co/lllyasviel/ControlNet-v1-1/tree/main
- SD15_brightness – https://huggingface.co/ioclab/ioc-controlnet/tree/main/models
Download and install these models in your <stable diffusion webui>\models\ControlNet folder. Then run up Stable Diffusion WebUI. Have your QR code generated, I’m not going to detail how to do this – you should know this much if you are reading this. I used the site QR.io
As per the video I also used the rev_animated model in my case as well and Sampling method as DPM++ 2M Karras (I prefer these over Euler a).
You need to use 2x units of ControlNet, where in ControlNet Unit 0 you want to set:
- Upload the QR image
- Enable ControlNet Unit 0
- Preprocessor: inpaint_global_harmonious
- Model: control_v1p_sd15_brightness
- Control Weight: 0.4
- Start and Stop step: 0 and 1
ControlNet Unit 1 needs to be setup with:
- Upload the same QR image
- Enable ControlNet Unit 1
- Preprocessor: inpaint_global_harmonious
- Model: control_v11f1e_sd15_tile
- Control Weight: 0.5
- Start and Stop step: 0.35 and 0.75
After this setup your pompt and negative prompt and generate One image and test scanning the result. On my setup I was easily able to run a batch of 4 images per generation so it was faster to test the QR code.
You should note that not all images generated will be scanned accurately as QR, some won’t work. You will need to experiment this yourself.
prompt: ancient village, (intricate details), hdr, hyperdetailed, cinematic shot,
Negative prompt: ((double image)), ((double head)), plastic, Deformed, blurry, bad anatomy, bad eyes, crossed eyes, different color eyes, disfigured, poorly drawn face, mutation, mutated, ((extra limb)), ugly, poorly drawn hands, missing limb, blurry, floating limbs, disconnected limbs, malformed hands, blur, out of focus, long neck, long body, ((((mutated hands and fingers)))), (((out of frame))), blender, doll, cropped, low-res, close-up, poorly-drawn face, out of frame double, two heads, blurred, ugly, disfigured, too many fingers, deformed, repetitive, black and white, grainy, extra limbs, bad anatomy,High pass filter, airbrush, portrait, zoomed, soft light, smooth skin, deformed, extra limbs, extra fingers, mutated hands, bad anatomy, bad proportions , blind, bad eyes, ugly eyes, dead eyes, blur, vignette, out of shot, out of focus, gaussian, monochrome, grainy, noisy, text, writing, watermark, logo, oversaturation , over saturation, over shadow, Asian, ((nude)),((naked)), ((watermark)), ((text))The results are pretty cool and 20-30% of the images generated work as QR code as well. I tried a few different things but then got hung up on motorcycles as you can see below, its one of my other passions when I can get away from the GPUs and AI Generation.
Hope you found this useful, make sure you follow the YouTuber above and show your support.
If you'd like to support our site please consider buying us a Ko-fi, grab a product or subscribe. Need a faster GPU, get access to fastest GPUs for less than $1 per hour with RunPod.io