Images

Exercise - Build your image using commit (Optional)

To recap the essentials from the Container Exercises run a container in detached mode, using port mapping, environment variables and a custom name (base_container):

docker run -d -p 8080:8080 -e PROPERTY=Stuttgart --name base_container novatec/technologyconsulting-hello-container:v0.1

Milestone: DOCKER/IMAGES/BUILD-COMMIT

and validate if the container is doing what it is supposed to do:

curl localhost:8080/hello; echo

Hello World (from 34f40588f2e0) to Stuttgart

Containers are basically isolated processes. To find out more about this execute the ps command in the following way:

docker ps --no-trunc

Taking a closer look at the command tab you can see:

[...] COMMAND                                     [...]
[...] "./application -Dquarkus.http.host=0.0.0.0" [...]

This is the core process of your container.

Another way to find out more about your running container is to invoke:

docker container inspect base_container

Most likely this command will return more information than you can handle, but if you scroll through you may find this section:

    "Config": {
        "...": "..."
        "Env": [
            "PROPERTY=Stuttgart",
            "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
            "container=oci"
        ],
        "Cmd": [
            "./application",
            "-Dquarkus.http.host=0.0.0.0"
        ]
        "...": "..."
    }

This one shows the environment properties set and the application process.

Or, for quick access, filtering the JSON output via the command-line JSON processor jq, listing just the environment settings:

docker container inspect base_container | jq '.[].Config.Env'

There also is a possibility to execute another process in the same scope by executing:

docker exec -it base_container /bin/bash

This will open a new shell process (/bin/bash) within the container and connect to it in interactive mode.

docker exec -it base_container /bin/bash
bash-4.4$

This kind of feels like being “logged into” the container. Execute a couple of things, e.g. check which user (ID, that is) you are and list your file system information:

whoami

bash-4.4$ whoami
whoami: cannot find name for user ID 1001

ls /

bash-4.4$ ls /
bin  boot  dev  etc  home  lib  lib64  lost+found  media  mnt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var  work

Display the environment variable

echo $PROPERTY

bash-4.4$ echo $PROPERTY
Stuttgart

and create a file within the container

echo "this container has been modified" >> /work/info

and eventually exit the shell and return to your host OS via

exit

To copy a file from “outside” of the container, i.e. from the host OS, into the container filesystem, execute 2 commands

echo "mysterious file from outside" >> message

docker cp message base_container:/work

Now create a new container image from the running container using commit:

docker commit base_container hello-container:v0.2

If it is being created successfully the response should look like this:

$ docker commit base_container hello-container:v0.2
sha256:34a4911c09c2d9ea4982c59eafec867370fb0ddac4c14e7aa8ce3854a7b8c685

Observe the changes in your local repository:

docker images

REPOSITORY                                     TAG       IMAGE ID       CREATED          SIZE
hello-container                                v0.2      34a4911c09c2   23 seconds ago   137MB
novatec/technologyconsulting-hello-container   v0.1      c4550e0a7743   19 hours ago     137MB

The whole sequence can be displayed as follows:

sequenceDiagram
    participant U as User
    participant D as Docker Daemon
    participant C as Container
    U->>D: "docker exec ..."
    D->>C: opens shell
    Note over C: execute commands
    C-->>D: logout
    D-->>U: release terminal
    U->>D: "docker cp ..."
    D->>C: put file
    Note over C: new file present
    D-->>U: release terminal
    U->>D: "docker commit ..."
    Note over D: store new image
    D-->>U: release terminal

Now try to do the following steps by yourself:

Run a new container instance from the newly created image called new_container
Observe the running container
Run a shell in the new container
Observe if the changes made to image have taken effect (meaning are the files still there under /work)

Solution

docker run -d -p 8081:8080 -e PROPERTY=Stuttgart --name new_container hello-container:v0.2

docker ps

$ docker ps
CONTAINER ID   IMAGE                                               COMMAND                  CREATED          STATUS         PORTS                                       NAMES
644c9b988cd3   hello-container:v0.2                                "./application -Dqua…"   12 seconds ago   Up 9 seconds   0.0.0.0:8081->8080/tcp, :::8081->8080/tcp   new_container
34f40588f2e0   novatec/technologyconsulting-hello-container:v0.1   "./application -Dqua…"   5 minutes ago    Up 5 minutes   0.0.0.0:8080->8080/tcp, :::8080->8080/tcp   base_container

docker exec -it new_container /bin/bash

and then inside the container

ls /work and tail info message which should yield

bash-4.4$ ls /work
application  info  message
bash-4.4$ tail info message
==> info <==
this container has been modified

==> message <==
mysterious file from outside

Recap: There are two images in your repo now. One was pulled from a remote repository, the second one was built locally based on a container based on the first image. The images in itself are immutable. State changes can only be applied to running containers. Unless this is persisted into a new image the changes are lost once a container terminates. Exit from your running container.

Stop and remove your recent containers again:

docker rm $(docker ps -q | xargs docker stop)

Milestone: DOCKER/IMAGES/BUILD-COMMIT-RM

Exercise - Build your images using Dockerfiles

Building containers via writing, copying files and running “docker commit” can be a cumbersome task and will always require manual steps, which can be error prone. A docker built-in mechanism to automate this is called Dockerfile.

In a Dockerfile you specify multiple steps to be done in sequence that result in a new container image.

There is a pre-built distributed application provided for you to be run in containers. Pull it from git and switch to the exercises directory:

cd && git clone https://github.com/NovatecConsulting/technologyconsulting-containerexerciseapp.git

cd /home/novatec/technologyconsulting-containerexerciseapp

Milestone: DOCKER/IMAGES/GIT-CLONE

You will find two Dockerfiles in this directory:

ls -ltr Dockerfile*

$ ls -ltr Dockerfile
-rw-rw-r-- 1 novatec novatec 121 Dec  1 10:19 Dockerfile-todoui
-rw-rw-r-- 1 novatec novatec 204 Dec  1 10:19 Dockerfile-todobackend

Have a look at the backend one first:

cat --number Dockerfile-todobackend

$ cat --number Dockerfile-todobackend
    1  FROM eclipse-temurin:17-alpine
    2  RUN mkdir -p /opt/todobackend
    3  WORKDIR /opt/todobackend
    4  COPY todobackend/target/todobackend-0.0.1-SNAPSHOT.jar /opt/todobackend
    5  CMD ["java", "-jar", "todobackend-0.0.1-SNAPSHOT.jar"]

In order to execute this, run the docker command the following way:

docker build -f Dockerfile-todobackend -t todobackend:v0.1 .

This will tell docker to use the file with the given path (via -f) and tag a new image (-t) executed from the current directory (.)

Now a lot of incredible things happen:

[+] Building 23.0s (9/9) FINISHED                                                                                          docker:default
 => [internal] load build definition from Dockerfile-todobackend                                                                     2.3s
 => => transferring dockerfile: 255B                                                                                                 0.0s
 => [internal] load .dockerignore                                                                                                    2.6s
 => => transferring context: 2B                                                                                                      0.0s
 => [internal] load metadata for docker.io/library/eclipse-temurin:17-alpine                                                         2.2s
 => [1/4] FROM docker.io/library/eclipse-temurin:17-alpine@sha256:60b102f57f99ee12190394019d4da827afd5bf6a182aa520f192993effbc4cf0  12.3s
 => => resolve docker.io/library/eclipse-temurin:17-alpine@sha256:60b102f57f99ee12190394019d4da827afd5bf6a182aa520f192993effbc4cf0   0.4s
 => => sha256:60b102f57f99ee12190394019d4da827afd5bf6a182aa520f192993effbc4cf0 1.64kB / 1.64kB                                       0.0s
 => => sha256:44b3cea369c947527e266275cee85c71a81f20fc5076f6ebb5a13f19015dce71 947B / 947B                                           0.0s
 => => sha256:a3562aa0b991a80cfe8172847c8be6dbf6e46340b759c2b782f8b8be45342717 3.40kB / 3.40kB                                       0.0s
 => => sha256:e7c96db7181be991f19a9fb6975cdbbd73c65f4a2681348e63a141a2192a5f10 2.76MB / 2.76MB                                       0.3s
 => => sha256:f910a506b6cb1dbec766725d70356f695ae2bf2bea6224dbe8c7c6ad4f3664a2 238B / 238B                                           0.4s
 => => sha256:c2274a1a0e2786ee9101b08f76111f9ab8019e368dce1e325d3c284a0ca33397 70.73MB / 70.73MB                                     3.0s
 => => extracting sha256:e7c96db7181be991f19a9fb6975cdbbd73c65f4a2681348e63a141a2192a5f10                                            0.6s
 => => extracting sha256:f910a506b6cb1dbec766725d70356f695ae2bf2bea6224dbe8c7c6ad4f3664a2                                            0.0s
 => => extracting sha256:c2274a1a0e2786ee9101b08f76111f9ab8019e368dce1e325d3c284a0ca33397                                            2.6s
 => [internal] load build context                                                                                                    1.5s
 => => transferring context: 34.45MB                                                                                                 1.1s
 => [2/4] RUN mkdir -p /opt/todobackend                                                                                              1.5s
 => [3/4] WORKDIR /opt/todobackend                                                                                                   0.8s
 => [4/4] COPY todobackend/target/todobackend-0.0.1-SNAPSHOT.jar /opt/todobackend                                                    1.2s
 => exporting to image                                                                                                               1.6s
 => => exporting layers                                                                                                              1.4s
 => => writing image sha256:0a6c8b63fb84d2f5ce8c9d97b200ba49b59581a8ed50051b95969684fa955d86                                         0.1s
 => => naming to docker.io/library/todobackend:v0.1                                                                                  0.1s

In Step 1 a base image from the Docker Hub is being pulled (eclipse-temurin:17-alpine)
Step 2 creates a new directory within the container
Step 3 sets the default working directory in the container (to the one created in Step 2)
A file from the local file system (todo…jar) is being copied into the container’s working dir in Step 4
The last step (not explicitly show in the output) sets the process/command of the container. This one will be started if an instance is being run
And finally a new image is being built and tagged with the name given in the command (todobackend:v0.1)

sequenceDiagram
    participant U as User
    participant D as Docker Daemon
    participant C as Container
    participant R as Remote Registry
    U->>D: "docker build ..."
    opt fetching base image
        D->>R: image pull
        R-->>D: image
    end
    D->>C: instantiate base container
    loop execute
        C->>C: runs in background until terminated
    end
    D->>C: execute further Steps
    Note over C: Step X
    Note over C: Step Y
    Note over C: Step ...
    C-->>D: release
    Note over D: store new image
    D->>C: terminate container
    D-->>U: release terminal

Milestone: DOCKER/IMAGES/DOCKERFILE-TODOBACKEND, requires: DOCKER/IMAGES/GIT-CLONE

Have a look at your local images and validate if it is really there (as well as some others omitted here):

docker images

REPOSITORY                                     TAG        IMAGE ID       CREATED          SIZE
todobackend                                    v0.1       fd13afea3880   5 minutes ago    369MB

Do a test run, providing an environment variable that tells the backend to just use a simple in-memory database:

docker run -d -p 8080:8080 -e SPRING_PROFILES_ACTIVE=dev --name todobackend todobackend:v0.1

Milestone: DOCKER/IMAGES/TODOBACKEND-RUN, requires: DOCKER/IMAGES/GIT-CLONE

And have a look at the logs:

docker logs todobackend

Or even better keep watching the logs (-f means follow):

docker logs -f todobackend

Step out by pressing Ctrl+C

You will see some Spring Boot output:

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::                (v3.1.1)

...
2023-12-01 09:28:43.375  INFO 1 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat initialized with port(s): 8080 (http)
...
2023-12-01 09:28:54.593  INFO 1 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port(s): 8080 (http) with context path ''
2023-12-01 09:28:54.606  INFO 1 --- [           main] i.n.todobackend.TodobackendApplication   : Started TodobackendApplication in 20.842 seconds (JVM running for 22.476)

If this output is displayed things went well.

Next step is to build the image for the UI part of the application.

Exercise - Write your own Dockerfile

Have a look at the 2nd Dockerfile in the directory:

cat Dockerfile-todoui

$ cat Dockerfile-todoui
FROM ------
RUN mkdir -p /opt/todoui
WORKDIR ------
COPY todoui/target/------ /opt/todoui
CMD ["java", "-jar", "------"]

This one has some intended gaps in it and you need to replace the “-----” with the correct content. The structure of the UI component is equivalent to the backend component except the difference in the name.

Try yourself to:

Complete the Dockerfile (using nano or vim)
Build a new image with the Dockerfile
Check the new image in the registry
Run a container using a port mapping 8090:8090

Or look at the solution below if you have problems

Solution

Identify the name of the jar file:

ls todoui/target/*.jar

$ ls todoui/target/*.jar
todoui/target/todoui-0.0.1-SNAPSHOT.jar

Edit the file and correct the missing fields

nano Dockerfile-todoui

FROM eclipse-temurin:17-alpine
RUN mkdir -p /opt/todoui
WORKDIR /opt/todoui
COPY todoui/target/todoui-0.0.1-SNAPSHOT.jar /opt/todoui
CMD ["java", "-jar", "todoui-0.0.1-SNAPSHOT.jar"]

Build the docker image

docker build -f Dockerfile-todoui -t todoui:v0.1 .

[+] Building 4.8s (9/9) FINISHED                                                              docker:default
 => [internal] load build definition from Dockerfile-todoui                                             0.3s
 => => transferring dockerfile: 220B                                                                    0.0s
 => [internal] load .dockerignore                                                                       0.2s
 => => transferring context: 2B                                                                         0.0s
 => [internal] load metadata for docker.io/library/eclipse-temurin:17-alpine                            0.0s
 => CACHED [1/4] FROM docker.io/library/eclipse-temurin:17-alpine                                       0.0s
 => [internal] load build context                                                                       1.5s
 => => transferring context: 19.15MB                                                                    1.0s
 => [2/4] RUN mkdir -p /opt/todoui                                                                      1.5s
 => [3/4] WORKDIR /opt/todoui                                                                           0.4s
 => [4/4] COPY todoui/target/todoui-0.0.1-SNAPSHOT.jar /opt/todoui                                      0.8s
 => exporting to image                                                                                  1.3s
 => => exporting layers                                                                                 1.2s
 => => writing image sha256:b2c2827e6804f59a5aef9365036c300b34075c44cb042b951296fbc475194c13            0.0s
 => => naming to docker.io/library/todoui:v0.1                                                          0.0s

List the images

docker images

$ docker images
REPOSITORY                                     TAG        IMAGE ID       CREATED          SIZE
todoui                                         v0.1       b2c2827e6804   47 seconds ago   338MB
todobackend                                    v0.1       fd13afea3880   10 minutes ago   369MB
[...]

Run an instance

docker run -d -p 8090:8090 --name todoui todoui:v0.1

Validate the running instance

docker ps -a

$ docker ps -a
CONTAINER ID   IMAGE              COMMAND                    CREATED          STATUS         PORTS                                       NAMES
d266fc71ab06   todoui:v0.1        "/__cacert_entrypoin..."   12 seconds ago   Up 6 seconds   0.0.0.0:8090->8090/tcp, :::8090->8090/tcp   todoui
265da979af36   todobackend:v0.1   "/__cacert_entrypoin..."   4 minutes ago    Up 4 minutes   0.0.0.0:8080->8080/tcp, :::8080->8080/tcp   todobackend

What is this “/__cacert_entrypoin…” COMMAND, you might ask? This is just a convenience wrapper for importing additional certificates if needed, provided as an [Docker ENTRYPOINT][docker-entrypoint]. However, for us now it is a minor distraction. Still, our predefined CMD will indeed be executed, for reference cf.

docker exec todobackend cat /__cacert_entrypoint.sh

and

docker inspect todobackend | jq '.[].Config.Entrypoint, .[].Args'

and also check in the host process list for the java process via

ps auxwwkstart_time | grep java.

Milestone: DOCKER/IMAGES/TODOUI-RUN, requires: DOCKER/IMAGES/GIT-CLONE

Exercise - Customize a standard image

So far there are two application containers running, which are based on images that are self-built via Dockerfiles. Often containers can also be useful through re-usability, when a certain functionality is already given and only needs to be customized slightly.

The sample application consists of 2 application components and a database. Many database vendors provide the databases as docker images. You can easily look them up at the docker Hub.

Open the following link in a new tab: Docker Hub Search

Here you can easily find base images for MongoDB, MySQL, Postgres and many more.

Open the link for Postgres also in a new tab: Docker Hub Postgres

If you scroll down on this page you will find instructions on how to run it, which parameters exists, examples and so on. This is pretty typical for most images you can find there.

For the application of this exercise a Postgres database is required with the following requirements:

Name: postgresdb
Port Mapping: 5432:5432
Environment Variables:
- POSTGRES_USER=matthias
- POSTGRES_PASSWORD=password
- POSTGRES_DB=mydb

Try to construct the according docker run command yourself or have a look at the solution below.

Solution

docker run --name postgresdb -p 5432:5432 -e POSTGRES_PASSWORD=password -e POSTGRES_USER=matthias -e POSTGRES_DB=mydb -d postgres:latest

(Please note that is is perfectly possible to adjust these credentials to your liking. However, doing so means we’d have to heed this adjustment in quite some places further below, so best refrain from any adjustments unless you are keen for some additional challenges.)

Milestone: DOCKER/IMAGES/POSTGRES-RUN

You can check if the container has come up well using:

docker logs postgresdb

[...]
PostgreSQL init process complete; ready for start up.

2023-12-01 09:34:53.723 UTC [1] LOG:  starting PostgreSQL 16.1 (Debian 16.1-1.pgdg120+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 12.2.0-14) 12.2.0, 64-bit
2023-12-01 09:34:53.724 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
2023-12-01 09:34:53.724 UTC [1] LOG:  listening on IPv6 address "::", port 5432
2023-12-01 09:34:53.746 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2023-12-01 09:34:53.774 UTC [62] LOG:  database system was shut down at 2023-12-01 09:34:53 UTC
2023-12-01 09:34:53.798 UTC [1] LOG:  database system is ready to accept connections

Exercise - Validate and wrap-up (Optional)

List your currently running containers:

docker ps

$ docker ps
CONTAINER ID   IMAGE              COMMAND                  CREATED          STATUS          PORTS                                       NAMES
b2a968f4f4b0   postgres:latest    "docker-entrypoint.s…"   59 seconds ago   Up 57 seconds   0.0.0.0:5432->5432/tcp, :::5432->5432/tcp   postgresdb
d266fc71ab06   todoui:v0.1        "/__cacert_entrypoin…"   3 minutes ago    Up 3 minutes    0.0.0.0:8090->8090/tcp, :::8090->8090/tcp   todoui
265da979af36   todobackend:v0.1   "/__cacert_entrypoin…"   7 minutes ago    Up 7 minutes    0.0.0.0:8080->8080/tcp, :::8080->8080/tcp   todobackend

List the images in your repo:

docker images

$ docker images
REPOSITORY                                     TAG        IMAGE ID       CREATED          SIZE
todoui                                         v0.1       b2c2827e6804   4 minutes ago    338MB
todobackend                                    v0.1       fd13afea3880   14 minutes ago   369MB
hello-container                                v0.2       34a4911c09c2   20 minutes ago   137MB
postgres                                       latest     2167863c43fd   3 weeks ago      425MB
novatec/technologyconsulting-hello-container   v0.1       c4550e0a7743   19 hours ago     137MB

You have now seen various ways to build container images, how to re-use existing ones and how to run and configure them according to your need. The problem with the current state is that the images are not able to talk to each other. Each container lives in a world of its own and is by default not connected to anyone else. With the port mappings you allow inbound traffic on a certain defined port, but not vice versa. This will be addressed in the next chapter “Network”.