Abstract.
I document my complete workflow for installing AMD ROCm drivers on Ubuntu Desktop 24.04 and building the llama.cpp library with HIP (GPU) backend support for my AMD Strix Halo APU with its Radeon 8060S iGPU. I cover every step — adding the ROCm repository, installing the drivers, cloning and configuring the llama.cpp build with the correct GPU target, and verifying the final GPU-accelerated binary.
Attributions:
None for this post.
An Introduction.
I run llama.cpp on my system to serve large language models locally. By default, the Nix package manager installs a CPU-only build that ignores the -ngl GPU offload flag — meaning my Radeon 8060S iGPU sits idle while the CPU churns through inference. That is a waste of my hardware. I write this post to capture exactly how I enable GPU acceleration: installing AMD's ROCm drivers and rebuilding llama.cpp with the HIP backend so my iGPU does the heavy lifting.
The Big Picture.
My workflow follows three broad phases. First, I install the AMD ROCm driver stack on my Ubuntu system — adding the official repository, pinning the package priority, and installing the rocm metapackage. Second, I clone the llama.cpp repository from GitHub and configure the CMake build with the HIP backend enabled, targeting my Strix Halo's gfx1151 architecture. Third, I build the project, install the binary to a local prefix, verify the HIP support is active, and add the accelerated binary to my PATH so every llama-server invocation uses my iGPU.
Prerequisites.
Updating my Base System.
- From the terminal, I update my system:
sudo apt clean && \
sudo apt update && \
sudo apt dist-upgrade -y && \
sudo apt --fix-broken install && \
sudo apt autoclean && \
sudo apt autoremove -y
Installing OpenSSL.
💡
OpenSSL is an open-source command line tool used for implementing cryptography, including generating private keys, creating certificate signing requests (CSRs), and managing SSL/TLS certificates. It is widely used in IT security for various cryptographic tasks. CMake is used later in this post and OpenSSL is employed (if available) during the build.
sudo apt update && sudo apt install openssl
Installing ROCm.
💡
AMD `ROCm` is an open-source software stack for programming that supports AI, HPC, and general-purpose computing on AMD GPUs and iGPUs. It includes drivers, tools, compilers, and libraries like HIP and MIOpen to enable development across frameworks such as `PyTorch` and `TensorFlow`.
Step 1: Adding the ROCm Repository.
I open a terminal.
I add the ROCm GPG key:
wget https://repo.radeon.com/rocm/rocm.gpg.key -qO - | gpg --dearmor | sudo tee /etc/apt/keyrings/rocm.gpg > /dev/null
- I add the
ROCm repository to my sources list:
echo "deb [arch=amd64 signed-by=/etc/apt/keyrings/rocm.gpg] https://repo.radeon.com/rocm/apt/latest $(lsb_release -cs 2>/dev/null) main" | sudo tee --append /etc/apt/sources.list.d/rocm.list
- I set the appropriate apt pin priority for
ROCm:
echo -e 'Package: *\nPin: release o=repo.radeon.com\nPin-Priority: 600' | sudo tee /etc/apt/preferences.d/rocm-pin-600
Step 2: Updating Package Lists.
- I update my system to make the new packages available for installation:
sudo apt update
Step 3: Installing the ROCm Package.
- I Install the
ROCm package:
sudo apt install -y rocm
💡
This step will take some time to complete. I do NOT assume that the install process has hung simply because the percentage values have not changed in a very long while. I step outside, I have a break, and I walk barefoot on the grass (unless there are prickles). At the very least, I make myself a hot beverage during the installation, i.e. "There is no dismay when I just walk away."
Step 4: Adding My Account to the ROCm Groups.
- I (-a)ppend my ($USER) account to the
ROCm (-G)roups:
sudo usermod -aG render,video $USER
- I refresh the terminal:
source ~/.bashrc
- I test the
ROCm installation:
sudo rocminfo
- I reboot my system (1/3 times):
reboot
Building the llama.cpp Library for ROCm.
💡
The default `Nix` build of `llama.cpp` does not include GPU support (no `ROCm`, no `HIP`, no `Vulkan`). Installing `nix-env --install --attr llama-cpp` results in a CPU-only binary that ignores the `-ngl` GPU offload flag, i.e. I just installed a `ROCm` driver that will not be used. So, I build the `llama.cpp` library with the `HIP` backend enabled.
Setting up the Environment.
- I set up the environment variables so the build tools can find the
ROCm installation:
export PATH=/opt/rocm/bin:$PATH
export ROCM_PATH=/opt/rocm
Cloning the Repo from GitHub.
💡
`GitHub` is a proprietary developer platform that allows developers to create, store, manage, and share their code using `Git` for version control. It is widely used for hosting open source software projects. `GitHub` has features like access control, bug tracking, and task management.
- I change to my
Home directory, clone the llama.cpp repository, and change to the llama.cpp directory:
cd ~
git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
Configuring the Build for ROCm.
- I create the
build directory and change to it:
mkdir -p build && cd build
💡
The (`-p`)arent flag is used with the `mkdir` command and allows me to create parent directories as needed. This means it will create the entire directory path if it doesn't already exist and will not return an error if the directory does exist. The `-p` flag ensures `mkdir` is useful because a single command that creates nested directory structures is utile.
sudo apt install -y cmake
💡
`CMake` is a free, cross-platform software development tool that helps manage the build process of applications using compiler-independent instructions. It can automate tasks like testing, packaging, and installation, making it easier to build projects across different platforms.
- I configure
CMake with the HIP backend enabled, static libraries, and server-only mode:
cmake .. \
-DGGML_HIP=ON \
-DCMAKE_BUILD_TYPE=Release \
-DAMDGPU_TARGETS="gfx1151" \
-DCMAKE_HIP_COMPILER_ROCM_ROOT=/opt/rocm \
-DCMAKE_PREFIX_PATH=/opt/rocm \
-DBUILD_SHARED_LIBS=OFF \
-DCMAKE_POSITION_INDEPENDENT_CODE=ON \
-DCMAKE_INSTALL_PREFIX=./install
💡
The `BUILD_SHARED_LIBS=OFF` flag links ggml statically, avoiding the `.so` symlink creation that fails on the CIFS network mount. The `LLAMA_BUILD_SERVER_ONLY=ON` flag skips the UI build, which also requires symlinks for npm. The `CMAKE_HIP_COMPILER_ROCM_ROOT` flag tells the `HIP` compiler where to find `ROCm` and passes the `--rocm-path` argument to clang++.
- I reboot my system (2/3 times):
reboot
- To confirm the correct target for my hardware, I run:
rocminfo | grep -E 'Name:|gfx'
💡
The `AMDGPU_TARGETS` value of `gfx1151` ensures the Radeon 8060S iGPU in the Strix Halo APU is selected. Also, this `ROCm` information command will display results only after a reboot. If the output shows a different architecture (e.g. gfx1100 or gfx1030), I can adjust the AMDGPU_TARGETS accordingly.
Building llama.cpp with ROCm Support.
- I build the project using all available
CPU or APU cores:
cmake --build . --config Release -j$(nproc)
💡
This step will take some time to complete. The build compiles the `GPU` or `iGPU` kernels and the server binary together. I do NOT assume that the build has hung simply because the terminal output has not changed in a very long while. I step outside, I have a break, and I walk barefoot on the grass (unless there are prickles).
Installing the Binary.
- I install the resulting binaries to a local prefix:
cmake --install . --prefix ./install
💡
The `llama-server` binary is now at `~/llama.cpp/build/install/bin/llama-server`.
Verifying the ROCm Build.
- I confirm the build includes
HIP support by running a quick smoke test:
./install/bin/llama-server --help 2>&1 | grep -i hip
💡
The `ROCm` backend is active if `hip` appears in the help output.
Adding ROCm to my PATH.
💡
With `ROCm` drivers installed and a `GPU`-enabled/`iGPU`-enabled build of `llama.cpp` available, I can now serve AI models using the `-ngl` offload flag for `llama.cpp`. It can accelerate matrix operations, improving inference throughput across any of the AI models. I can set the `-nlg` flag to `-1` to offload all of the AI model layers from the `CPU`/`APU` to the `GPU`/`iGPU`.
I add the ROCm-enabled build to my PATH so the GPU/iGPU-accelerated binary is used:
echo 'export PATH=$HOME/llama.cpp/build/install/bin:$PATH' >> ~/.bashrc
source ~/.bashrc
- I reboot my system (3/3 times):
reboot
The Results.
I end up with a working ROCm installation and a GPU-accelerated llama-server binary. Running rocminfo confirms the ROCm driver stack is active and sees my Radeon 8060S iGPU. Running llama-server --help | grep hip confirms the build includes HIP backend support. I can now serve models with the -ngl -1 flag, offloading every layer to my iGPU for dramatically faster inference than the CPU-only build I used before.
In Conclusion.
I have successfully installed AMD ROCm drivers and built a GPU-accelerated llama.cpp binary on my Ubuntu 24.04 system. My Radeon 8060S iGPU now handles inference through the HIP backend, and I serve models with full layer offload using the -ngl -1 flag. This workflow eliminates the wasted compute I had with the default CPU-only Nix build and puts my hardware to proper use.
Until next time: Be safe, be kind, be awesome, kia kaha!!
Document Details.
The following information is the metadata for this post.
#ROCm #llama-cpp #AMD #GPU #iGPU #HIP #Ubuntu #Linux #Strix-Halo #Radeon8060S #AI #LLM #Machine-Learning #Open-Source
SEO Title (60 Characters).
Installing AMD ROCm Drivers & Building llama.cpp with GPU Acceleration
SEO Description (150 Characters).
I install AMD ROCm drivers on Ubuntu 24.04 and build llama.cpp with HIP backend GPU acceleration for my AMD Strix Halo APU with its Radeon 8060S iGPU, covering the complete workflow.
Permalink
https://solodev.app/installing-rocm-llama-cpp-for-cpus-apus
Version
v0.5.2
