Cheap Solution for running Flux Dev using Runpod
2024
Harmeet
Flux has two models Dev and Schnell both of which are rated to require upwards of 32GB VRAM, there are other solutions available that let you run Flux under 12GB but if you are truly wanting to explore the power of this model and how good it can be a higher resolution then you need more VRAM.
My solution for this VRAM limitation (as I run RTX4080 16GB on my local PC) is to use Runpod and use their A40 GPU which only costs $0.35 per hour which is half a cent per minute. I don’t even know if the electricity used by GPU is costing them ore than that or not. It is ridiculously cheap and readily available on their platform.
I am using the ComfyUI Image for Runpod, which provides latest ComfyUI version with standard SD and SDXL models.
Download Workflow
You can open the various applications via the links provided.
To download the models I created a JupyterLab Notebook with all the links so you can download all the models, for Flux Dev you do need to use Huggingface access Token which is free so just need to enter it in the notebook before you run it. The way I have built the notebook is that it will download the Models, Clip and Realism LoRA in the required ComfyUI directories.
Simply upload the notebook I provided (IPYNB) in the Workspace folder on Runpod and double click to open. Once launched you can click on the play button for each of the cells with the links (after updating the access token).
Once this is done you can simply import the provided Workflow in ComfyUI and you are good to go. By default the workflow I use utilises all the higher settings of Flux Dev because I built it for RunPod where you will be using higher VRAM GPU. The workflow is available in the Download provided, simply drag the PNG file into ComfyUI and the workflow will load. You can modify them as you like.
The workflow (PNG file) I provided uses a few custom nodes so if you end up with red boxes upon loading the workflow just use the ComfyUI Manager to install missing nodes.
Download Workflow
I also have a detailed video walkthrough of my setup of Runpod on my YouTube which is below so feel free to browse through it if you like video format better.
I hope you find this information useful and helpful. If you have any queries or suggestions leave them in the comment section.
Error occurred when executing UNETLoader:
Error while deserializing header: HeaderTooSmall
File “/workspace/ComfyUI/execution.py”, line 313, in execute
output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/workspace/ComfyUI/execution.py”, line 188, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/workspace/ComfyUI/execution.py”, line 165, in map_node_over_list
process_inputs(input_dict, i)
File “/workspace/ComfyUI/execution.py”, line 154, in process_inputs
results.append(getattr(obj, func)(**inputs))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/workspace/ComfyUI/nodes.py”, line 863, in load_unet
model = comfy.sd.load_diffusion_model(unet_path, model_options=model_options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/workspace/ComfyUI/comfy/sd.py”, line 647, in load_diffusion_model
sd = comfy.utils.load_torch_file(unet_path)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/workspace/ComfyUI/comfy/utils.py”, line 34, in load_torch_file
sd = safetensors.torch.load_file(ckpt, device=device.type)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/workspace/ComfyUI/venv/lib/python3.11/site-packages/safetensors/torch.py”, line 313, in load_file
with safe_open(filename, framework=”pt”, device=device) as f:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
UNET Loader is erroring – make sure your selected model is correct one and file is completely downloaded. Also make sure your ComfyUI instance is up to date via ComfyUI manager.
See similar issue here https://huggingface.co/stabilityai/stable-diffusion-3-medium/discussions/69
Where is the notebook for download?
The notebook you mention is missing from the workspace
It’s available in the blog post via “Download” button. You then need to upload it to the Workspace.
where is the workflow? im confused how to run the workflow after running jupyter notebook.
Hi Joanna, Workflow is embedded in the PNG file in the download. Simply drag and drop it onto your ComfyUI.
The notebook is the IPYNB file which you should upload and then double click it to launch it. Pls watch the YT video where I go through all this in lot of details.