# Installing ROCm & Llama.cpp for CPUs & APUs.

## Abstract.

I document my complete workflow for installing AMD ROCm drivers on Ubuntu Desktop 24.04 and building the llama.cpp library with HIP (GPU) backend support for my AMD Strix Halo APU with its Radeon 8060S iGPU. I cover every step — adding the ROCm repository, installing the drivers, cloning and configuring the llama.cpp build with the correct GPU target, and verifying the final GPU-accelerated binary.

***Attributions:***

***None for this post.***

* * *

## An Introduction.

I run llama.cpp on my system to serve large language models locally. By default, the Nix package manager installs a CPU-only build that ignores the `-ngl` GPU offload flag — meaning my Radeon 8060S iGPU sits idle while the CPU churns through inference. That is a waste of my hardware. I write this post to capture exactly how I enable GPU acceleration: installing AMD's ROCm drivers and rebuilding llama.cpp with the HIP backend so my iGPU does the heavy lifting.

* * *

## The Big Picture.

My workflow follows three broad phases. First, I install the AMD ROCm driver stack on my Ubuntu system — adding the official repository, pinning the package priority, and installing the `rocm` metapackage. Second, I clone the `llama.cpp` repository from GitHub and configure the CMake build with the `HIP` backend enabled, targeting my Strix Halo's `gfx1151` architecture. Third, I build the project, install the binary to a local prefix, verify the HIP support is active, and add the accelerated binary to my `PATH` so every `llama-server` invocation uses my iGPU.

* * *

## Prerequisites.

*   [Ubuntu Desktop 22.04 LTS](https://solodev.app/system-tools-for-ubuntu-desktop#creating-an-ubuntu-installation-thumb-drive) or later,
    
*   An AMD CPU or APU (Accelerated Processing Unit).
    

* * *

## Updating my Base System.

*   From the terminal, I update my system:
    

```shell
sudo apt clean && \
sudo apt update && \
sudo apt dist-upgrade -y && \
sudo apt --fix-broken install && \
sudo apt autoclean && \
sudo apt autoremove -y
```

* * *

## Installing OpenSSL.

<div data-node-type="callout">
<div data-node-type="callout-emoji">💡</div>
<div data-node-type="callout-text">OpenSSL is an open-source command line tool used for implementing cryptography, including generating private keys, creating certificate signing requests (CSRs), and managing SSL/TLS certificates. It is widely used in IT security for various cryptographic tasks. <code>CMake</code> is used later in this post and <code>OpenSSL</code> is employed (if available) during the build.</div>
</div>

*   I install `OpenSSL`:
    

```shell
sudo apt update && sudo apt install openssl
```

* * *

## Installing `ROCm`.

<div data-node-type="callout">
<div data-node-type="callout-emoji">💡</div>
<div data-node-type="callout-text">AMD `ROCm` is an open-source software stack for programming that supports AI, HPC, and general-purpose computing on AMD GPUs and iGPUs. It includes drivers, tools, compilers, and libraries like HIP and MIOpen to enable development across frameworks such as `PyTorch` and `TensorFlow`.</div>
</div>

***Step 1: Adding the*** `ROCm` ***Repository.***

1.  I open a terminal.
    
2.  I add the `ROCm` GPG key:
    

```shell
wget https://repo.radeon.com/rocm/rocm.gpg.key -qO - | gpg --dearmor | sudo tee /etc/apt/keyrings/rocm.gpg > /dev/null
```

3.  I add the `ROCm` repository to my sources list:
    

```shell
echo "deb [arch=amd64 signed-by=/etc/apt/keyrings/rocm.gpg] https://repo.radeon.com/rocm/apt/latest $(lsb_release -cs 2>/dev/null) main" | sudo tee --append /etc/apt/sources.list.d/rocm.list
```

4.  I set the appropriate apt pin priority for `ROCm`:
    

```shell
echo -e 'Package: *\nPin: release o=repo.radeon.com\nPin-Priority: 600' | sudo tee /etc/apt/preferences.d/rocm-pin-600
```

***Step 2: Updating Package Lists.***

1.  I update my system to make the new packages available for installation:
    

```shell
sudo apt update
```

***Step 3: Installing the*** `ROCm` ***Package.***

1.  I Install the `ROCm` package:
    

```shell
sudo apt install -y rocm
```

<div data-node-type="callout">
<div data-node-type="callout-emoji">💡</div>
<div data-node-type="callout-text">This step will take some time to complete. I do NOT assume that the install process has hung simply because the percentage values have not changed in a very long while. I step outside, I have a break, and I walk barefoot on the grass (unless there are prickles). At the very least, I make myself a hot beverage during the installation, i.e. "There is no dismay when I just walk away."</div>
</div>

***Step 4: Adding My Account to the*** `ROCm` ***Groups.***

1.  I (-a)ppend my ($USER) account to the `ROCm` (-G)roups:
    

```shell
sudo usermod -aG render,video $USER
```

2.  I refresh the terminal:
    

```shell
source ~/.bashrc
```

3.  I test the `ROCm` installation:
    

```shell
sudo rocminfo
```

4.  I reboot my system (1/3 times):
    

```shell
reboot
```

* * *

## Building the `llama.cpp` Library for `ROCm`.

<div data-node-type="callout">
<div data-node-type="callout-emoji">💡</div>
<div data-node-type="callout-text">The default `Nix` build of `llama.cpp` does not include GPU support (no `ROCm`, no `HIP`, no `Vulkan`). Installing `nix-env --install --attr llama-cpp` results in a CPU-only binary that ignores the `-ngl` GPU offload flag, i.e. I just installed a `ROCm` driver that will not be used. So, I build the `llama.cpp` library with the `HIP` backend enabled.</div>
</div>

### Setting up the Environment.

*   I set up the environment variables so the build tools can find the `ROCm` installation:
    

```shell
export PATH=/opt/rocm/bin:$PATH
export ROCM_PATH=/opt/rocm
```

### Cloning the `Repo` from `GitHub`.

<div data-node-type="callout">
<div data-node-type="callout-emoji">💡</div>
<div data-node-type="callout-text">`GitHub` is a proprietary developer platform that allows developers to create, store, manage, and share their code using `Git` for version control. It is widely used for hosting open source software projects. `GitHub` has features like access control, bug tracking, and task management.</div>
</div>

*   I change to my `Home` directory, clone the `llama.cpp` repository, and change to the `llama.cpp` directory:
    

```shell
cd ~
git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
```

### Configuring the Build for `ROCm`.

*   I create the `build` directory and change to it:
    

```shell
mkdir -p build && cd build
```

<div data-node-type="callout">
<div data-node-type="callout-emoji">💡</div>
<div data-node-type="callout-text">The (`-p`)arent flag is used with the `mkdir` command and allows me to create parent directories as needed. This means it will create the entire directory path if it <em>doesn't</em> already exist and will <em>not</em> return an error if the directory <em>does</em> exist. The `-p` flag ensures `mkdir` is useful because a single command that creates nested directory structures is utile.</div>
</div>

*   I install `CMake`:
    

```shell
sudo apt install -y cmake
```

<div data-node-type="callout">
<div data-node-type="callout-emoji">💡</div>
<div data-node-type="callout-text">`CMake` is a free, cross-platform software development tool that helps manage the build process of applications using compiler-independent instructions. It can automate tasks like testing, packaging, and installation, making it easier to build projects across different platforms.</div>
</div>

*   I configure `CMake` with the `HIP` backend enabled, static libraries, and server-only mode:
    

```shell
cmake .. \
    -DGGML_HIP=ON \
    -DCMAKE_BUILD_TYPE=Release \
    -DAMDGPU_TARGETS="gfx1151" \
    -DCMAKE_HIP_COMPILER_ROCM_ROOT=/opt/rocm \
    -DCMAKE_PREFIX_PATH=/opt/rocm \
    -DBUILD_SHARED_LIBS=OFF \
    -DCMAKE_POSITION_INDEPENDENT_CODE=ON \
    -DCMAKE_INSTALL_PREFIX=./install
```

<div data-node-type="callout">
<div data-node-type="callout-emoji">💡</div>
<div data-node-type="callout-text">The `BUILD_SHARED_LIBS=OFF` flag links ggml statically, avoiding the `.so` symlink creation that fails on the CIFS network mount. The `LLAMA_BUILD_SERVER_ONLY=ON` flag skips the UI build, which also requires symlinks for npm. The `CMAKE_HIP_COMPILER_ROCM_ROOT` flag tells the `HIP` compiler where to find `ROCm` and passes the `--rocm-path` argument to clang++.</div>
</div>

*   I reboot my system (2/3 times):
    

```shell
reboot
```

*   To confirm the correct target for my hardware, I run:
    

```shell
rocminfo | grep -E 'Name:|gfx'
```

<div data-node-type="callout">
<div data-node-type="callout-emoji">💡</div>
<div data-node-type="callout-text">The `AMDGPU_TARGETS` value of `gfx1151` ensures the Radeon 8060S iGPU in the Strix Halo APU is selected. Also, this `ROCm` information command will display results only after a reboot. If the output shows a different architecture (e.g. <code>gfx1100</code> or <code>gfx1030</code>), I can adjust the <code>AMDGPU_TARGETS</code> accordingly.</div>
</div>

### Building `llama.cpp` with `ROCm` Support.

*   I build the project using all available `CPU` or `APU` cores:
    

```shell
cmake --build . --config Release -j$(nproc)
```

<div data-node-type="callout">
<div data-node-type="callout-emoji">💡</div>
<div data-node-type="callout-text">This step will take some time to complete. The build compiles the `GPU` or `iGPU` kernels and the server binary together. I do NOT assume that the build has hung simply because the terminal output has not changed in a very long while. I step outside, I have a break, and I walk barefoot on the grass (unless there are prickles).</div>
</div>

### Installing the Binary.

*   I install the resulting binaries to a local prefix:
    

```shell
cmake --install . --prefix ./install
```

<div data-node-type="callout">
<div data-node-type="callout-emoji">💡</div>
<div data-node-type="callout-text">The `llama-server` binary is now at `~/llama.cpp/build/install/bin/llama-server`.</div>
</div>

### Verifying the `ROCm` Build.

*   I confirm the build includes `HIP` support by running a quick smoke test:
    

```shell
./install/bin/llama-server --help 2>&1 | grep -i hip
```

<div data-node-type="callout">
<div data-node-type="callout-emoji">💡</div>
<div data-node-type="callout-text">The `ROCm` backend is active if `hip` appears in the help output.</div>
</div>

### Adding `ROCm` to my PATH.

<div data-node-type="callout">
<div data-node-type="callout-emoji">💡</div>
<div data-node-type="callout-text">With `ROCm` drivers installed and a `GPU`-enabled/`iGPU`-enabled build of `llama.cpp` available, I can now serve AI models using the `-ngl` offload flag for `llama.cpp`. It can accelerate matrix operations, improving inference throughput across any of the AI models. I can set the `-nlg` flag to `-1` to offload all of the AI model layers from the `CPU`/`APU` to the `GPU`/`iGPU`.</div>
</div>

I add the `ROCm`\-enabled build to my PATH so the `GPU`/`iGPU`\-accelerated binary is used:

```shell
echo 'export PATH=$HOME/llama.cpp/build/install/bin:$PATH' >> ~/.bashrc
source ~/.bashrc
```

*   I reboot my system (3/3 times):
    

```shell
reboot
```

* * *

## The Results.

I end up with a working ROCm installation and a GPU-accelerated `llama-server` binary. Running `rocminfo` confirms the ROCm driver stack is active and sees my Radeon 8060S iGPU. Running `llama-server --help | grep hip` confirms the build includes HIP backend support. I can now serve models with the `-ngl -1` flag, offloading every layer to my iGPU for dramatically faster inference than the CPU-only build I used before.

* * *

## In Conclusion.

I have successfully installed AMD ROCm drivers and built a GPU-accelerated `llama.cpp` binary on my Ubuntu 24.04 system. My Radeon 8060S iGPU now handles inference through the HIP backend, and I serve models with full layer offload using the `-ngl -1` flag. This workflow eliminates the wasted compute I had with the default CPU-only Nix build and puts my hardware to proper use.

Until next time: Be safe, be kind, be awesome, kia kaha!!

* * *

## Document Details.

The following information is the metadata for this post.

### Hash Tags.

#ROCm #llama-cpp #AMD #GPU #iGPU #HIP #Ubuntu #Linux #Strix-Halo #Radeon8060S #AI #LLM #Machine-Learning #Open-Source

### SEO Title (60 Characters).

Installing AMD ROCm Drivers & Building llama.cpp with GPU Acceleration

### SEO Description (150 Characters).

I install AMD ROCm drivers on Ubuntu 24.04 and build llama.cpp with HIP backend GPU acceleration for my AMD Strix Halo APU with its Radeon 8060S iGPU, covering the complete workflow.

### Permalink

https://solodev.app/installing-rocm-llama-cpp-for-cpus-apus

### Version

v0.5.2