In A NutSHELL: Uses in CI/CD and Docker

What is shell? What is bash?

Shell is a program for the user to interact with an operating system. The terminal window on Linux or macOS has the shell which lets the user interact with the operating system by using commands such as ls, cd, mkdir, etc. Bash also known as Bourne Again Shell is a command/shell interpreter, in other words, it is a type of shell. It is the default interpreter on many GNU/Linux systems. There are various other shell interpreters to choose from, such as dash, ash, Korn shell, etc.

What is shell scripting?

There are times when you are doing the same tasks every day and starting to get frustrated. What do you do? The solution is to write a python script to automate those tasks. Same with shell scripting, it is basically automating the commands on the shell.

The difference between #!/bin/bash and #!/bin/sh

sh (Shell Command Language) is a programming language described by the POSIX standard. it is a specification and bash is one of its implementations. In most GNU/Linux systems, /bin/sh is linked to bash. So, both #!/bin/bash and #!/bin/sh can be used as the shell interpreter in the shell script. /bin/bash could be a symbolic or a hard link. To figure out what does the /bin/sh link to, you can use the following commands.

# Symbolic link
% file -h /bin/sh

# Hard link
% find -L /bin -samefile /bin/sh

Functionality of shell script

CI pipelines

In most cases, a YAML file is required to trigger a CI pipeline and the script section is the main ingredient of the CI pipeline. Your code is built and tested by running commands in the script section. With the increase in complexity, the script section can be long and tedious. Therefore, bash script or python script comes into play. The script encapsulates and automates the steps to build or test your code. With the scripts in place, the code lines in the script section can be reduced to just a few lines scripts calling.

An example of .gitlab-ci.yml before using a bash script.

release_build_status:
  stage: status
  dependencies:
    - build
  script:
    - zip -r "$CI_COMMIT_REF_SLUG-$CI_COMMIT_SHORT_SHA-artifacts".zip _build
    - 'curl --user "$BITBUCKET_USERNAME:$BITBUCKET_ACCESS_TOKEN" -X POST "<https://api.bitbucket.org/2.0/repositories/$BITBUCKET_NAMESPACE/$BITBUCKET_REPOSITORY/downloads>" -F [email protected]"$CI_COMMIT_REF_SLUG-$CI_COMMIT_SHORT_SHA-artifacts.zip"'
  when: on_success
  only:
    - /^release-.*$/

An example of .gitlab-ci.yml after using a bash script

release_build_status:
  stage: status
  dependencies:
    - build
  script:
    - ./ci_files/post_artifacts_to_bitbucket.bash
  when: on_success
  only:
    - /^release-.*$/

and the bash script.

#!/bin/bash

BITBUCKET_API_ROOT="<https://api.bitbucket.org/2.0>"
BITBUCKET_DOWNLOAD_API="$BITBUCKET_API_ROOT/repositories/$BITBUCKET_USERNAME/$BITBUCKET_REPOSITORY/downloads"

zip -r "$CI_COMMIT_REF_NAME-$CI_COMMIT_SHORT_SHA".zip $*

echo "Pushing artifacts to $BITBUCKET_DOWNLOAD_API..."
curl --request POST $BITBUCKET_DOWNLOAD_API \\
--user "$BITBUCKET_USERNAME:$BITBUCKET_ACCESS_TOKEN" \\
--form [email protected]"$CI_COMMIT_REF_NAME-$CI_COMMIT_SHORT_SHA.zip"

Docker

In the CI pipeline, a Docker image creates an isolated build or test environment for your project. Given the example above, the bash script comes with the repository. If the bash script is always used in a particular Docker image, it can be copied into the image when the image is built and used in the CI pipeline.

An example of a Dockerfile.

FROM alpine:latest

RUN apk add --no-cache --upgrade bash && \\
    apk add --update curl && \\
    apk add --update zip && \\
    rm -rf /var/cache/apk/*

COPY bitbucket_post_artifact_to_download.bash .

An example of .gitlab-ci.yml using the bash script in the Docker image.

release_build_status:
  stage: status
	image: namespace/alpine-zip-curl
  dependencies:
    - build
  script:
    - ./post_artifacts_to_bitbucket.bash
  when: on_success
  only:
    - /^release-.*$/

Programming/debugging tools

A CLI tool usually comes with an IDE and it uses the commands on terminal windows. There will be a set of commands for you to interact with your code and the micro. Imaging you just need to run a bash script and your code is compiled and flashed to the micro. Yes, it is possible because bash scripting is basically gathering a set of commands and automating them. Same with the RTT console for debugging, you can create a debugging tool by writing a bash script.

In a Nutshell

Like any other programming language, bash scripting has its own syntax and logic and it is not hard to learn when you have a basic knowledge of coding. Once you get the hang of it, it can be a powerful tool in firmware development.

Dojo Five provides a number of services, including firmware development in C, C++, Python, embedded CI/CD, testing, and automation. And if you have questions about an embedded project you’re working on, Dojo Five can help you with all aspects of your devops for embedded journey! We are always happy to hear about cool projects or interesting problems to solve, so don’t hesitate to reach out and chat with us on LinkedIn or through email!