Bloom Setup
I am having a hard time finding information on how to setup bloom on my local machine, does big science have any official documentation for setup/config. Stuff i've found online seems to be rather specific to cloud or virtual machine setups. If not does anyone have any good resources for setting up the 178b bloom model on a local machine. (bearing in mind I have four gtx 1080s in a rig to run it on).
Yeah, i've come to the same conclusion. there no tutorial what so ever...it's free but no one knows how to set it up...
Hi.
I've followed all instructions for installing on: https://towardsdatascience.com/run-bloom-the-largest-open-access-ai-model-on-your-desktop-computer-f48e1e2a9a32
It downloaded almost a day a 350 gigs, with the .git +/- 745 gigs in dir.
Then used the python code from this web page, set the path in the python script to the bloom directory.
Then it keeps hanging for ever on this line:
input_ids = tokenizer.encode(input_sentence, return_tensors='pt').to(device)
Consuming around 4 to 25 percent of CPU, and using up between 1 and 10 gigs of ram.
When setting: device = 'cuda' i get error because i have 6 gig GPU and it needs more.
#gettingusedtoit