Docker hero? Check yourself!

Answer

Not exactly. 

James wanted to optimize the Dockerfile’s readability, but he also slowed the developers’ productivity.

Developers divided this COPY command into two different layers purposely – one before the requirements installation, and one after that. This way they can use Docker caching and save their time on rebuilds. 

Docker stores the image as layers when each layer relies on the previous ones. The layer that was not changed can be reused from the existing image built before. However, if one of the layers is changed, Docker will rebuild it and all the following layers, although the last ones weren’t changed.

The requirements file usually changes much less frequently than Python code, so, the obvious use case is that the Python code has changed, but the requirements file has not. That’s why in the original Dockerfile, the pip install layer will be taken from the cache, without rebuilding, but in the “optimized” Dockerfile it will be rebuilt although it hasn’t changed.

However, the updated Dockerfile can be used when no Docker caching is used, for example in CI.

Answer

Not exactly. Reminder – Docker stores the images as layers. The files that are removed in Dockerfile instructions, still exist in previous layers.

Would you like to be sure?

A quick workshop for a junior hacker:

Let’s create a secret file, then let’s copy it into a Docker image with COPY command, and delete it in the next instruction:

echo MY_SECRET_CODE_12345 > secret.txt
echo -e 'FROM busybox\nCOPY secret.txt .\nRUN rm secret.txt' > Dockerfile
docker build -t myimage .

We have now a Docker image myimage with a removed secret.txt secret file. 

Now, let’s run the docker save command to convert this Docker image into an archive:

docker save myimage -o myimage.tar

Unpack myimage.tar, and you will see the folder with subfolders when each one of the subfolders represents the Docker layer. The subfolder that represents a COPY layer contains a tar archive with secret.txt inside. 

Answer

Unfortunately, no.

It looks like James didn’t ever hear about the docker history command.

This command makes build-time variables visible to any user of the image:

Here is a very short example:

echo -e 'FROM busybox\nARG TOKEN\nRUN echo 123' | docker build -t myimage --build-arg TOKEN=my_token_123 - 
docker history myimage

<Frustration>

So what should James do? There are a few best practices to avoid the injection of secrets into docker image.

• The easy way is to pull the secret at the beginning of the layer and remove it at the end of the same layer, i.e. in a single Docker instruction:

RUN TOKEN=$(<pull token from a vault>) \
         && sed -i 's/GIT_TOKEN/${TOKEN}/' requirements.txt \
         && pip install -r requirements.txt \
         && rm requirements.txt

• The official Docker recommendation is to use RUN with --mount=type=secret argument. This argument allows the creation of filesystem mounts that the build can access, mount type secret allows accessing secure files without baking them into the image.

Answer

Yes. And much more complicated to maintain. 

Now the developers will have to remember to add each new file and folder to the Dockerfile. The recommended best practice is to ask developers to put all the resources required for their app in a separate folder. However, it’s not always applicable. In our case, many resources were shared between different applications in this repo.

To avoid copying unwanted files into a docker image, James would be better off using a .dockerignore file. It allows specifying a pattern for files and folders that should be ignored by the Docker client when generating a build context. 

Besides avoiding unintended secrets exposure, it also prevents Docker build cache invalidation. It happens often with system or IDE temporary files, like IntelliJ’s .idea of or Mac’s .DS_Store.

We usually are not aware that these files were changed, but the Docker client is.

Answer

Nothing but crashed unit tests. Developers were using

docker run

with --publish-all argument in their unit tests. This argument exposes all ports mentioned in EXPOSE Dockerfile instructions. After James’ last commit, there were no ports left to expose this way.