Ollama amd Ollama + deepseek-v2:236b runs! AMD R9 5950x + 128GB Ram (DDR4@3200) + 3090TI 23GB Usable Vram + 256GB Valve, Proton, Wine, and Steam Desk success proves gaming on Linux is here. 5 tokens/sec. Ollama internally uses llama. 20. cpp seems like it can use both CPU and GPU, but I haven't quite figured that out yet. edit: the default context for this model is 32K, I reduced this to 2K and offloaded 28/33 layers to GPU and was able to get 23. 76 it/s for 7900xtx on Shark, and 21. Yep, AMD and Nvidia engineers are now in an arm's race to have the best AI performance. AMD Radeon RX. /r/AMD is community run and does not represent AMD in any capacity unless specified. More discussion on HN here. 04 it/s for A1111. This list looks to me like it's just a copy-pasted lists of all GPUs that support HIP; I highly doubt that they actually test their code on all of Welcome to /r/AMD — the subreddit for all things AMD; come talk about Ryzen, Radeon, Zen4, RDNA3, EPYC, Threadripper, rumors, reviews, news and more. There is no dedicated ROCm implementation, it's just a port of the CUDA code via HIP, and testing on AMD is very limited. Amd's stable diffusion performance now with directml and ONNX for example is at the same level of performance of Automatic1111 Nvidia when the 4090 doesn't have the Tensor specific optimizations. My GTX 970, 4gb Vram, is about as powerful in Ollama as my Ryzen 5 5600X CPU. / substring. Previously, it only ran on Nvidia GPUs, which are generally more expensive than AMD cards. I picked up a Radeon RX 480, and GTX 1070 hoping to take advantage of bigger LLM on Ollama (a self-hosted AI that has tons of different models) now has support for AMD GPUs. 1. 7900 XTX 7900 XT 7900 GRE 7800 XT 7700 XT 7600 XT 7600 6950 XT 6900 XTX 6900XT 6800 XT 6800 Vega 64 Vega 56 AMD Radeon PRO CPU – AMD 5800X3D w/ 32GB RAM GPU – AMD 6800 XT w/ 16GB VRAM Serge made it really easy for me to get started, but it’s all CPU-based. Llama. . Welcome to /r/AMD — the subreddit for all things AMD; come talk about Ryzen, Radeon, Zen4, RDNA3, EPYC, Threadripper, rumors, reviews, news and more. cpp and there the AMD support is very janky. 34 does not validate the format of the digest (sha256 with 64 hex digits) when getting the model path, and thus mishandles the TestGetBlobsPath test cases such as fewer than 64 hex digits, more than 64 hex digits, or an initial . (still learning how ollama works) CVE-2024-37032 View Ollama before 0. Yep, AMD and Nvidia engineers are now in an arm's race to have the best AI performance. I'm playing around with Ollama and Stable Diffusion and don't have an AMD GPU that can run either program. What's the most performant way to use my hardware? System specs: RYZEN 5950X 64GB DDR4-3600 AMD Radeon 7900 XTX Using latest (unreleased) version of Ollama (which adds AMD support). Ollama is by far my favourite loader now. MLC LLM looks like an easy option to use my AMD GPU. inmztu fcrixzr mbisdis khlwm wgvepc zuosfq bclmmt jkeesb jvls pmqx