Why do you need automated firmware builds for your embedded project?
More often than not, we find that customers do not trust their firmware releases because firmware builds require some tribal knowledge to make changes, such as how to get it working on your machine. At best, there’s a build machine in the corner of the room. We believe this needs to change. A Continuous Integration (CI) pipeline automatically compiles, analyzes, and tests releases, and the output files and reports can be distributed accordingly when code changes are pushed to the source control provider.
Important Concepts
YAML File
A YAML (YAML Ain’t Markup Language) file is a human-readable, parsable data-serialization language. It is optimized for configuration settings, log files, internet messaging, and filtering. Most of the existing CI tools require some kind of YAML file to kick off the CI pipeline. The YAML format is used in the CI tools because it provides a unified development experience with multiple-stage pipelines in a single file, it can be source controlled, and duplicating or sharing a pipeline can be as simple as copy and paste.
If a repository is hosted on a source control platform such as GitHub, one can kick off a CI pipeline by adding a CI configuration YAML file in the repository and committing the changes to the source control platform. Different source control platforms name their CI configuration file differently and have different workflow syntax. For example, GitHub does not enforce a name for their CI configuration file, but the file has to live in the .github/workflows directory, while GitLab CI will look for a CI configuration file named .gitlab-ci.yml. To learn more about YAML, check out the “Learn X in Y minutes” blog post.
Docker Image
A Docker image is a lightweight virtual machine that provides an encapsulated environment for building or testing applications. Each source control platform hosts runners to run their pipeline. A runner is basically a virtual machine that runs OSes such as Linux, Windows, and macOS, and it only comes with the basic tools and libraries. Based on the project, we have to install a specific compiler, library, or tool on top of the runner. One of the ways to do it is by installing them in the CI workflow every time a CI pipeline runs, but it can get really slow. By creating and using a Docker image with a specific compiler or tool pre-installed, it will speed up the CI pipeline and save some money from the build minutes. A list of public Docker images can be found on Docker Hub. To learn more about Docker, check out our “Using Docker for Embedded Development” and “Trusting the Docker Images Your Code is Built On” blogposts.
How to set up an embedded CI pipeline
In this tutorial, we will be focusing on setting up a CI pipeline on GitHub. As mentioned in the “YAML File” section, for GitHub Actions to decide whether to kick off a CI pipeline, it will look for a CI configuration YAML file in the .github/workflows directory. The following is an example of a project structure:
automated-fw-build-example/ # The top dir of the project
├── .github/workflows # The dir that GitHub Actions looks for
│ └── pipeline.yml # CI configuration file
│
├── src/
│ ├── linker.ld
│ ├── main.c
│ └── startup.c
│
├── .gitignore
│
└── README.md
The CI configuration file basically describes the workflow of how we build an application locally in our host machine. Therefore, we have to first figure out how to build the application. Let’s say this project uses a bare-metal ARM Cortex-M microcontroller (e.g., STM32F103, LPC1768, etc.) with no operating system or standard library dependencies. In order to build it, we need to install the GNU Arm Embedded Toolchain. Once we build the application, we want to put the binaries in a build directory that will be ignored by the source control. The following is an example of the CI configuration file without using a Docker image:
name: Automated Firmware Build
on: push
jobs:
build_myapp:
runs-on: ubuntu-24.04
steps:
- name: Set up build environment
run: |
sudo apt-get update -yqq
sudo apt-get install -y gcc-arm-none-eabi
arm-none-eabi-gcc --version
- name: Check out repository
uses: actions/checkout@v4
- name: Build application
run: |
mkdir build
cd build
arm-none-eabi-gcc -mcpu=cortex-m3 -mthumb -nostdlib -T ../src/linker.ld ../src/startup.c ../src/main.c -o led.elf
arm-none-eabi-objcopy -O binary led.elf led.bin
- name: Upload artifacts
uses: actions/upload-artifact@v4
with:
name: led
path: |
${{ github.workspace }}/build/led.bin
${{ github.workspace }}/build/led.elf
It looks intimidating, but don’t worry, we will explain the syntax from top to bottom. Workflows are made up of one or more jobs and can be scheduled or triggered by an event. We start off by naming the workflow, “Automated Firmware Build”. The next key-value pair defines the event that triggers the workflow. Setting it to push means the pipeline will run when a commit is pushed to any branch of the repository. It also accepts multiple events. We can achieve that by setting the value as a list e.g. [push, pull_request].
The following screenshot shows that a pipeline will show up under the Action tab when a commit is pushed to the repository. A red ‘x’ symbol means the pipeline failed while a green check symbol means the pipeline passed.
In this workflow, we only have a single job named build_myapp. If we have multiple jobs, they will be run in parallel by default unless we specify the dependencies between jobs. The build_myapp job is using an Ubuntu version 24.04 runner that is hosted on GitHub. A job contains a sequence of tasks called steps. Steps can run commands, run setup tasks, or run an action in our repository, a public repository, or an action published in a Docker registry. There are four steps in this job, and we name each of the steps. The first step runs a series of commands to install the required toolchain to build our project. This is the step that can be replaced by using a Docker image. The -y option of the apt-get is needed because everything runs automatically and we cannot interact with the installation of the packages.
The following screenshot shows the steps of the build_myapp job.
In the second step, we use a checkout action to check out the repository so that the workflow can access it. An action is a reusable unit of code. A list of public actions can be found on GitHub Marketplace. Then, we are ready to build the application. The workflow above shows a brute force way of building the application. It can be optimized and look much cleaner by using a Bash script or Makefile. Last but not least, we use another action called upload-artifact to upload the binaries as artifacts so that we can download them on GitHub and flash the binary to the hardware.
The following screenshot shows that the artifact named led can be found on the Summary page under the Actions tab.
Tips: During the initial setup or maintaining of a CI pipeline, a syntax checker such as actionlint will come in handy.
Improve CI pipeline with Docker
To get started on using Docker, download the Docker Desktop. Once the Docker Desktop is installed on your machine, we are going to create, build, and tag a Docker image by using the Docker CLI that comes with the Docker Desktop. Add the following content into a file named Dockerfile (you can place it in the top directory of your repository or in a docker/ directory):
FROM ubuntu:24.04
RUN apt-get update -yqq && \
apt-get install -y --no-install-recommends gcc-arm-none-eabi && \
apt-get -y clean && \
apt-get -y autoremove && \
rm -rf /var/lib/apt/lists/*
This simple Docker file starts off from a base Ubuntu version 24.04 Docker image that you can find on Docker Hub. Then, it installs the GNU Arm Embedded Toolchain. When installing packages from apt, we avoid installing the recommended packages and clean up the lists directory after installation to reduce space. To build and tag the Docker image, use the following command:
docker build . -t ghcr.io/<GITHUB_USERNAME>/arm-none-eabi-gcc:1:0
If you place the Dockerfile in a docker/ directory change the . to docker/. The string that comes after the -t flag is the tag of the Docker image. It is required to start with ghcr.io because we are going to push it to the GitHub Package Registry. Then, change the <GITHUB_USERNAME> to our GitHub username. The last part of the path is the name of the Docker image and its version.
In order to push the Docker image to the GitHub Package Registry, we need to create a personal access token. The following are the steps to create a personal access token:
- Click on the user logo on top right, go to Settings.
- On the left bar, scroll to the below, and go to Developer settings.
- Under the Personal access token, click on Tokens (classic), select Generate new token (classic).
- Enter a name and set the expiration date.
- Enable write:packages scope.
- Generate token.
- Copy and save the token to a safe space.
Back to the terminal and we are going to log into the GitHub Package Registry and push the Docker image to the registry with.
$ docker login ghcr.io --username github-account
[Paste your GitHub token on this prompt]
$ docker push ghcr.io/<GITHUB_USERNAME>/arm-none-eabi-gcc:1.0
Next, we want to connect our repository to the package with the following steps:
- Click on the user logo on top right, go to “Your Profile”.
- Go to the “Packages” tab.
- Click on the Docker image that you just pushed.
- Go to “Package settings” and add your repository, and set the role to “Read”
Last but not least, we can now update our CI configuration file to use the Docker container. The following is the updated version of the CI configuration file:
name: Automated Firmware Build
on: push
jobs:
build_myapp:
runs-on: ubuntu-24.04-arm
container:
image: ghcr.io/zhixuenlai/arm-none-eabi-gcc:1.0
credentials:
username: ${{ github.actor }}
password: ${{ secrets.github_token }}
steps:
- name: Check build environment
run: arm-none-eabi-gcc --version
- name: Check out repository
uses: actions/checkout@v4
- name: Build application
run: |
mkdir build
cd build
arm-none-eabi-gcc -mcpu=cortex-m3 -mthumb -nostdlib -T ../src/linker.ld ../src/startup.c ../src/main.c -o led.elf
arm-none-eabi-objcopy -O binary led.elf led.bin
- name: Upload artifacts
uses: actions/upload-artifact@v4
with:
name: led
path: |
${{ github.workspace }}/build/led.bin
${{ github.workspace }}/build/led.elf
We updated the runner machine to use an ARM version of Ubuntu because we built the Docker image on macOS. If you build it on Linux, you can remove the -arm from the runner. Instead of installing the toolchain in a step, we use the container key and define the Docker image to be used. Then, we provide the credential to pull the image from the package registry. Push the changes up and observe the speed improvement of the CI pipeline.
Conclusion
Automated builds improve the firmware development cycle by speeding up and unifying the build process and producing high-quality firmware. It requires one to put in a lot of effort up front, but the rewards that come later will be tenfold.
In the early stage of a project, the CI workflow can be simple, but as the project progresses, the workflow gets more complicated. For example, at some point, we will need to have a debug versus release build, set up a scheduled pipeline to run a Hardware-In-The-Loop (HITL) test, or tag a commit to initiate a release automatically.
Setting up a CI pipeline can be time-consuming and frustrating, but Dojo Five can help. If you are interested in bringing your project to the next level, let’s schedule a call with our team! Or, check out EmbedOps, our CI build platform.
We look forward to hearing from you! -D5
