Unable to run `Ollama` with 4 NVIDIA GPUs

Nusic Rusko

Cadet
Joined
Aug 18, 2023
Messages
4
Hello World!

Im trying to run a OLLAMA instance and It does not start properly.
I am able to start this OLLAMA instance but only when there is no gpus selected.

1708478382100.png



1708478425310.png



I also keep seeing this error/event show up on TrueNAS

```
2024-02-20 17:10:22
Allocate failed due to rpc error: code = Unknown desc = request for 'nvidia.com/gpu: 2' too large: maximum request size for shared resources is 1, which is unexpected
```

Does anyone have this issue as well?
 
Top