Volumes

Data Persistence Without Volumes

In this chapter we are going to take a look at how to persist data with docker.

First we want to observe how data behaves in docker containers with the setup we have so far and no additional steps taken towards data persistence.

Subtask 1: Create a PostgreSQL Container without Volumes

Let’s start removing the old and creating a new database. While the previously created database is working fine, we want to illustrate the differences between starting a database with and without volume more clearly. Therefore, remove the old database first by running docker rm -f postgresdb.

Start a new PostgreSQL container without volume attachment as we did in the previous exercise but add it to the network todonet right away.

docker run -d \
  --name postgresdb \
  --network todonet \
  -e POSTGRES_PASSWORD=password \
  -e POSTGRES_USER=matthias \
  -e POSTGRES_DB=mydb \
  -p 5432:5432 \
  postgres:latest

We’re creating a PostgreSQL container without attaching any volumes. Data stored inside this container will be lost if the container is stopped or removed.

Before we can continue, we have to make sure that our backend connects to our new database and initializes the relations within.

Restart todobackend to force a reconnect to the new database via:

docker restart todobackend

Note that it will take a few moments until the todobackend catches up.

Subtask 2: Add Data

Next, we want to view existing and add new data to the database. To do so, we could access the todoui through the browser, but the database needs to be initialized first. So let’s display and add data through the Terminal first, which will also then have the database initialized and its content ready to be viewed through the todoui.

We want to take a look inside the container, where we’re connecting to the PostgreSQL container’s bash terminal and then access the PostgreSQL service through psql. These steps can be executed in a single command.

docker exec -it postgresdb psql -h localhost -U matthias mydb

Solution

To understand what the command above does, we can take a look at how to achieve the same with two individual commands. Start by connecting to the container’s shell through the ‘docker exec’ command:

docker exec -it postgresdb /bin/bash

Inside the container, we’re connecting to the PostgreSQL container’s bash terminal. Next, we will be accessing the PostgreSQL service through psql.

psql -h localhost -U matthias mydb

But note that if you use this approach, you will later have to exit the PostgreSQL shell before you can exit the container, meaning that you will execute the ’exit’ command two times in a row.

Display current data in the database

SELECT * FROM todo;

and insert data:

INSERT INTO todo VALUES ('Learn Docker volumes');
INSERT INTO todo VALUES ('Restart Database');
INSERT INTO todo VALUES ('Check if Data is still there');

Display data inside database again to verify that your data has successfully been created.

Solution

run the command SELECT * FROM todo; again. You should see something similar to the following data

mydb=# SELECT * FROM todo;
             todo
------------------------------
 Learn Docker volumes
 Restart Database
 Check if Data is still there
(3 rows)

If you are accessing the UI now you should also see that the data you have inserted via the terminal is also present after a reload (and vice versa).

Exit the container’s shell:

exit

Subtask 3: Verify Data Persistence

Stop and remove the PostgreSQL container:

docker rm -f postgresdb

Start a new PostgreSQL container with identical configuration as the previous one.

Solution

docker run -d \
  --name postgresdb \
  --network todonet \
  -e POSTGRES_PASSWORD=password \
  -e POSTGRES_USER=matthias \
  -e POSTGRES_DB=mydb \
  -p 5432:5432 \
  postgres:latest

Reconnect to the PostgreSQL container and check if the data still exists. Note that it will take a few moments until the todobackend catches up. Restart todobackend to force a reconnect to the new database docker restart todobackend and display the data directly in the new UI.

Now you can also run the terminal verification again to see if the data is still present.

After stopping and removing the container, we’re starting a new PostgreSQL container with the same name and configuration. When we reconnect and check the data, we find that it’s lost. This demonstrates the challenge of data persistence without using volumes.

Before we move to the next segment, make sure to remove the current postgres container again as we will no longer need this version of the database.

Solution

Once again run

docker rm -f postgresdb

and our postgres container will be removed and we can look into creating a more suitable container in the next segment.

Milestone: DOCKER/VOLUMES/NOPERSISTENCE

Data Persistence with Volumes

In this chapter, let’s take a look at what changes we can make to allow data to persist from one instance of the database to another. For this, we will utilize docker volumes.

Subtask 1: Create a Docker Volume and Mount It to a Container

First, let’s create a new volume. Create a named Docker volume called “postgres_data”:

docker volume create postgres_data

Now we can create a new postgresdb container and attach the volume to it on startup. This will allow the database to persist data within the volume that resides within the file system.

Start a new PostgreSQL container with the volume mount by adding -v postgres_data:/var/lib/postgresql/data to the previous docker run command for our postgresdb just before the image name.

Solution

docker run -d \
  --name postgresdb \
  --network todonet \
  -e POSTGRES_PASSWORD=password \
  -e POSTGRES_USER=matthias \
  -e POSTGRES_DB=mydb \
  -p 5432:5432 \
  -v postgres_data:/var/lib/postgresql/data \
  postgres:latest

Here, we’re creating a Docker volume named “postgres_data” using docker volume create. Then, we start a new PostgreSQL container and use the -v flag to mount the newly created volume to the container’s /var/lib/postgresql/data directory. This allows data to be stored outside the container, ensuring persistence even if the container is stopped or removed.

Subtask 2: Connect to the PostgreSQL Container and Verify Data Persistence

We once again want to connect to the bash terminal inside of the PostgreSQL container and access the PostgreSQL service using psql.

Solution

Connect to the container’s bash terminal and inside the container, connect to the PostgreSQL service through psql.
```
docker exec -it postgresdb psql -h localhost -U matthias mydb
```

and insert data:

INSERT INTO todo VALUES ('Create Volume');
INSERT INTO todo VALUES ('Kill and Restart Database');
INSERT INTO todo VALUES ('Check if Data is still there');

Display data inside database again to verify that your data has successfully been created.
```
SELECT * FROM todo;
```
Should the todo relation not exist yet, we wan solve this by restarting the todobackend once more for it to initialize this relation for us.

If you are accessing the UI now you should also see that the data you have inserted via the terminal is also present after a reload (and vice versa).
Exit the PostgreSQL shell:
```
exit
```

Subtask 3: Stop and Remove the PostgreSQL Container

Stop and remove the PostgreSQL container:

docker rm -f postgresdb

We’re stopping and removing the container, but since the data is stored in the volume, it remains intact. This highlights the data persistence achieved through Docker volumes.

Subtask 4: Start a New PostgreSQL Container Using the Same Volume

Start a new PostgreSQL container with the same volume mount:

Solution

docker run -d \
  --name postgresdb \
  --network todonet \
  -e POSTGRES_PASSWORD=password \
  -e POSTGRES_USER=matthias \
  -e POSTGRES_DB=mydb \
  -p 5432:5432 \
  -v postgres_data:/var/lib/postgresql/data \
  postgres:latest

Reconnect to the PostgreSQL container and verify that the data is still intact.

By starting a new container and mounting the same volume, we’re able to access the same data.

By querying the todos table, we confirm that the data persists even after stopping and restarting the container.

This showcases the robustness of Docker volumes in maintaining data persistence across container instances.

Milestone: DOCKER/VOLUMES/PERSISTENCE