Allow for using CPU if no CUDA device is detected

[Urcra]

Allows for running on the CPU if no CUDA device is detected instead of just giving a runtime error.

This should allow for more people to experiment even without owning an nvidia GPU

Solves:

• Errors when attempting to generate image using torch cpu() #118
• Update txt2img.py for colab crowsonkb/latent-diffusion#1 (comment)
• Run inference on CPU #81

[leszekhanusz]

Thanks, it works!

It is my understanding that by default, pytorch puts everything on the cpu and you have to specify .cuda() or .to(torch.device("cuda") to move things to the gpu.

So I think that maybe all the model.cpu() calls are not needed. Same for .to(torch.device("cpu"))

[leszekhanusz]

Currently I am using export CUDA_VISIBLE_DEVICES="" to test using the only the cpu with your code.

It would be neat if we could use a --disable-cuda flag as described in pytorch device-agnostic example to use the cpu explicitely without having to mess with environment variables.

This would be useful for example when you have a GPU but not not enough VRAM to put the model on it.

[Urcra]

Thanks, it works!

It is my understanding that by default, pytorch puts everything on the cpu and you have to specify .cuda() or .to(torch.device("cuda") to move things to the gpu.

So I think that maybe all the model.cpu() calls are not needed. Same for .to(torch.device("cpu"))

👍 Yeah you are right, just didn't think about that when I first made the PR, but I fixed it now, so it's a bit cleaner

[Urcra]

Currently I am using export CUDA_VISIBLE_DEVICES="" to test using the only the cpu with your code.

It would be neat if we could use a --disable-cuda flag as described in pytorch device-agnostic example to use the cpu explicitely without having to mess with environment variables.

This would be useful for example when you have a GPU but not not enough VRAM to put the model on it.

Wanted to also do this, but it's pretty annoying to pass new arguments into the classes from encoder/modules.py and I wasn't really sure if I wanted to create a global variable for it or how it could best be solved, open to any ideas though