What runs Gemma 4 31B? 24–32 GB GPUs and mini PCs

Install and run Gemma 4 31B

Steps and commands come from the official docs listed at the end of this section, checked Oct 27, 2026, 2:30 PM CT. We don't add flags the docs don't show. Commands can change, so check the linked page if one fails.

1. Before you start: GPU drivers

2. Install Ollama

Windows 10 22H2 or newer: download and run OllamaSetup.exe from ollama.com/download. It installs in your user account without admin rights and runs in the background. Then the ollama command works in PowerShell or cmd. You'll need at least 4 GB for the app, plus room for models.

macOS: open ollama.dmg and drag Ollama into Applications. On first start it offers to link the ollama command into /usr/local/bin.

Linux (one command):

curl -fsSL https://ollama.com/install.sh | sh

Linux, AMD GPU, manual install: extract the main package, then the extra ROCm package:

curl -fsSL https://ollama.com/download/ollama-linux-amd64.tar.zst \
    | sudo tar x -C /usr
curl -fsSL https://ollama.com/download/ollama-linux-amd64-rocm.tar.zst \
    | sudo tar x -C /usr

Linux, AMD ROCm driver (Ubuntu 24.04 example from AMD's quick start):

wget https://repo.radeon.com/amdgpu-install/7.2.4/ubuntu/noble/amdgpu-install_7.2.4.70204-1_all.deb
sudo apt install ./amdgpu-install_7.2.4.70204-1_all.deb
sudo apt update
sudo apt install "linux-headers-$(uname -r)" "linux-modules-extra-$(uname -r)"
sudo apt install amdgpu-dkms
sudo apt install python3-setuptools python3-wheel
sudo usermod -a -G render,video $LOGNAME
sudo apt install rocm

Then reboot. Other distros have their own commands on AMD's quick start page.

Intel Arc with Intel's Ollama build: download the IPEX-LLM Ollama portable zip (Windows) or tgz (Linux) and extract it. On Windows, open cmd in that folder and run start-ollama.bat, then run the ollama run command below from the same cmd window.

Check that Ollama is installed:

ollama -v

3. Download and run Gemma 4 31B

Gemma 4 31B

ollama run gemma4:31b

Smaller option: Gemma 4 26B-A4B

ollama run gemma4:26b-a4b

The first run downloads the model, then opens a chat prompt. Type /bye to exit. Ollama also serves an API at http://localhost:11434.

4. Check that it's running on the GPU

ollama ps

The PROCESSOR column should read 100% GPU. A CPU/GPU split means the model didn't fit in GPU memory. You can also watch GPU memory with:

nvidia-smi   # NVIDIA
rocm-smi     # AMD on Linux (ROCm)

Logs: on Linux, journalctl -u ollama --no-pager --follow --pager-end. On macOS, ~/.ollama/logs/server.log. On Windows, run explorer %LOCALAPPDATA%\Ollama and open server.log.

5. Common fixes

• Out of memory, or part of the model runs on the CPU: pick a smaller tag, or shorten the context. Ollama's default context is 4096 tokens, and longer contexts use more memory. Set it for the server or inside a chat:

  OLLAMA_CONTEXT_LENGTH=8192 ollama serve
  /set parameter num_ctx 4096

  Flash Attention reduces memory use as context grows. Ollama turns it on automatically where supported; to force it on, set OLLAMA_FLASH_ATTENTION=1 when starting the server.
• Strix Halo mini PC, GPU memory: on Windows, AMD's Variable Graphics Memory turns part of system RAM into dedicated graphics memory. Open AMD Software: Adrenalin Edition, then Performance, then Tuning, and pick a size; this needs a restart. On a 128 GB system up to 96 GB can be converted. On Linux, AMD's ROCm docs say GPU memory comes from GTT, which defaults to about 50% of RAM. You can raise what the GPU gets by increasing VRAM in the BIOS, or by reducing the GTT size so it's below the reserved amount.
• NVIDIA on Linux falls back to CPU after sleep: reload the UVM driver: sudo rmmod nvidia_uvm && sudo modprobe nvidia_uvm.
• AMD card not supported by ROCm on Linux: Ollama documents HSA_OVERRIDE_GFX_VERSION to try a close supported target (their example: HSA_OVERRIDE_GFX_VERSION="10.3.0" for an RX 5400). On some distros, add the ollama user to the render group for Vulkan.
• Windows with both an iGPU and a dGPU picks the wrong one on Vulkan: set GGML_VK_VISIBLE_DEVICES to the discrete GPU's index.
• Several GPUs: limit Ollama with CUDA_VISIBLE_DEVICES (NVIDIA) or ROCR_VISIBLE_DEVICES (AMD). nvidia-smi -L and rocminfo list the devices.
• Not enough disk space: set OLLAMA_MODELS to store models on another drive. On Windows, set it in your account's environment variables, then quit and restart Ollama.

Other apps that run the same models

LM Studio is a desktop app. It needs an Apple silicon Mac on macOS 14+, or Windows (x64 with AVX2, or ARM), or Linux as an AppImage (Ubuntu 20.04+). LM Studio recommends 16 GB+ of RAM and at least 4 GB of dedicated VRAM on Windows. llama.cpp is the engine underneath many of these tools. Its README gives this install and usage (swap in a GGUF repo for the model you want):

# Linux/macOS
curl -LsSf https://llama.app/install.sh | sh
# Windows PowerShell
irm https://llama.app/install.ps1 | iex

llama cli -hf ggml-org/Qwen3.5-0.8B-GGUF
llama serve -hf ggml-org/Qwen3.5-0.8B-GGUF

Install sources (checked Oct 27, 2026, 2:30 PM CT)

• Ollama download page
• Ollama docs: Windows
• Ollama docs: macOS
• Ollama docs: Linux install
• Ollama docs: Hardware support (NVIDIA, AMD, Apple Metal, Vulkan)
• Ollama docs: FAQ (context length, ollama ps, Flash Attention)
• Ollama docs: Troubleshooting (logs)
• NVIDIA CUDA Toolkit downloads
• NVIDIA: CUDA GPU compute capability list
• AMD ROCm docs: Quick start installation (Linux)
• AMD Radeon drivers for Linux
• AMD ROCm docs: Strix Halo system optimization (memory settings)
• AMD: FAQs on Variable Graphics Memory
• Intel Arc & Iris Xe graphics driver (Windows)
• Intel client GPU driver installation (Linux)
• IPEX-LLM: Run Ollama Portable Zip on Intel GPU
• LM Studio docs: System requirements
• llama.cpp README: Quick start