Testing Python Scripts in A Docker Container
Most Python scripts require external packages. Many require a specific version of these packages or even the Python interpreter. Instead of installing these dependencies globally on your machine, you can either create an isolated virtual environment using Python virtual environments (venv) or create a Docker container. While there is a lot to like about virtual environments, in this blog post we will focus on using a Docker container. To learn more about Docker, check out our blog post on Docker: An ideal development environment and Controlled Development Environment.
Docker Container
To begin creating a virtual runtime environment for our Python script, we need a base image with the Python environment. Dockerhub has a list of official Python Docker images that we can pull to our local machine. Instead of getting a fully-featured Python container (read: large), we recommend grabbing an image that is smaller in size such as a slim buster release. Click here to learn more about different types of Docker images.
To run the Docker image on the example above, enter the following command at the terminal. If it can’t find the image on the local machine, the program will pull the image from Dockerhub automatically.
% docker run -it --rm python:3.10.0a2-slim-buster /bin/bash
Unable to find image 'python:3.10.0a2-slim-buster' locally
3.10.0a2-slim-buster: Pulling from library/python
bb79b6b2107f: Pull complete
35e30c3f3e2b: Pull complete
3255e0f89733: Pull complete
f86ef0712550: Pull complete
fcd3f56ef084: Pull complete
Digest: sha256:eeacf8e7907a1e7a8e98f512b6b1a81f75a053f52f22ee408815852ed92d3b3f
Status: Downloaded newer image for python:3.10.0a2-slim-buster
root@6b7e7b1a7481:/#
Here is a quick explanation of what we just did: With the -it option the container will run in interactive mode – meaning you get dropped to a terminal where you can run commands. The --rm flag erases the container once it is exited to save disk space. And by adding the /bin/bash at the end of the command, the container will be put in a bash shell session instead of the Python shell.
# Run the Python image without /bin/bash
% docker run -it --rm python:3.10.0a2-slim-buster
Python 3.10.0a2 (default, Nov 4 2020, 01:36:46)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
# Run the Python image with /bin/bash
% docker run -it --rm python:3.10.0a2-slim-buster /bin/bash
root@8bb93e66a8f1:/# ls
bin boot dev etc home lib lib64 media mnt opt proc root run sbin srv sys tmp usr var
How do you exit a container?
To exit a container, enter exit on the terminal if it is at a bash shell or exit() if it is at a Python shell.
Docker Volumes
Now, we have a container running but the script we want to test is not in the container. This is where Docker volumes come into play, specifically “bind mounts”. These allow us to mount a local directory that contains the script into the container. Any modification of the script locally or in the container will be reflected on both sides. Therefore, while developing the script in the container, the local script will always be up-to-date.
% docker run -it --rm -v $(pwd)/<DIR>:/<DIR> python:3.10.0a2-slim-buster /bin/bash
Volumes are indicated by a -v flag. There are two parts after the flag which are separated by the colon :. The path to the local directory to be mounted is on the left side while the path of the directory in the container is on the right side.
# For example, there is a local directory called scriptDir and we want to
# mount it to the container with a different name, scriptDirInContainer
% docker run -it --rm -v $(pwd)/scriptDir:/scriptDirInContainer python:3.10.0a2-slim-buster /bin/bash
root@1fae03a0170d:/# ls
bin boot dev etc home lib lib64 media mnt opt proc root run sbin scriptDirInContainer srv sys tmp usr var
# We can mount more than one local directory to the container.
# Now, we have two local directories, scriptDir and helperDir
% docker run -it --rm -v $(pwd)/scriptDir:/scriptDirInContainer -v $(pwd)/helperDir:/helperDirInContainer python:3.10.0a2-slim-buster /bin/bash
root@6e50f0a73c87:/# ls
bin boot dev etc helperDirInContainer home lib lib64 media mnt opt proc root run sbin scriptDirInContainer srv sys tmp usr var
Test Environment Setup
There are two ways to set up a suitable environment for testing the Python script.
Install Packages in the Python Docker Container
Once we are in the container, we can install the required Python packages manually by using the pip3 installer. Alternatively, we can create a requirements.txt file, put it in the folder that will be mounted to the container, and run the file to install the packages. The following is a simple requirements.txt example for installing the requests package and how to run it.
requests==2.22.0
# To run a requirements.txt file
% pip3 install -r requirements.txt
Create a Docker image with packages installed
If the script will be tested more than once, it makes sense to create a customized Docker image with the required packages pre-installed. In the Dockerfile, set the official Python image as the base image using the FROM instruction and install the packages with the pip3 installer by using the RUN instruction. Then, build a customized Docker image with the Dockerfile. Click here to learn more about Dockerfile. The following is a simple Dockerfile example that installs the requests package when building the Docker image.
FROM python:3.10.0a2-slim-buster
RUN pip3 install requests
Conclusion
Script or application testing in a virtual environment like a Docker container will not only prevent developers from installing random packages on their local machine but also provide an enclosed, controlled environment for verifying its functionality. By utilizing Docker volumes, the process of testing is improved without the developer copying files back and forth between the local machine and the Docker container.
And if you have questions about an embedded project you’re working on, Dojo Five can help you with all aspects of your EmbedOps journey! We are always happy to hear about cool projects or interesting problems to solve, so don’t hesitate to reach out and chat with us on LinkedIn or through email!