You’ve probably already heard about containers and one of the most popular tool used to create them: Docker. You may wonder how a container stores data on its host, if those data are persistent, if you can find them on the server hosting this container and more broadly how a container is stored on this host. Those are very good questions so let’s dive into this topic by looking at some examples.

Data into one Docker container

I’ve got a Rocky Linux 8 virtual machine and I’ve installed docker in it. Let’s just create a container with an Ubuntu image on it to play with and understand its interactions with the host.

$ sudo docker container run -it --name myubuntu ubuntu:18.04
...
root@5dd708fd5afb:/#

We directly connect into our Ubuntu container and the first test we are going to do is to create a file inside this container with a name that will be easy to find on the host:

root@5dd708fd5afb:/# touch benoit

root@5dd708fd5afb:/# ls
benoit  bin  boot  dev  etc  home  lib  lib64  media  mnt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var
root@5dd708fd5afb:/#

I’ve created a file with my first name that I can now find on my Rocky host:

[rocky ~]$ sudo find /var/lib/docker/ -name benoit
/var/lib/docker/overlay2/6796ec566b5bbdbcfa99efe742a042b574c17633a504d2d31f87a604e194f0a1/diff/benoit
/var/lib/docker/overlay2/6796ec566b5bbdbcfa99efe742a042b574c17633a504d2d31f87a604e194f0a1/merged/benoit

This file is found in 2 folders. Docker container uses the storage driver overlay2 to store files. You can find all the detailed informations related to it in the Docker documentation. On the Rocky host you can see in the path name to our file the default folder where all the container data are stored: /var/lib/docker/overlay2/ It is then followed by a directory ID and one of the folder used by overlay2: diff, link, lower, merged and work. Without entering into too much details, diff is the content of the container layer and merged is the unified view of files from the image and container layer.

So from the host you have access to the content of a container but it is not recommended to interact directly with it as Docker could get confused (it is one of its tasks to manage that).

Then, I exited from this container:

root@5dd708fd5afb:/# exit
exit
[rocky ~]$

Let’s have a look at what we can see on the host:

[rocky ~]$ sudo find /var/lib/docker/ -name benoit
/var/lib/docker/overlay2/6796ec566b5bbdbcfa99efe742a042b574c17633a504d2d31f87a604e194f0a1/diff/benoit

The file only appear in the diff folder. The container has exited but has not been deleted so let’s restart it and see if we can still see our file:

[rocky ~]$ sudo docker ps -a
CONTAINER ID   IMAGE           COMMAND                  CREATED             STATUS                      PORTS     NAMES
5dd708fd5afb   ubuntu:18.04    "/bin/bash"              About an hour ago   Exited (0) 6 minutes ago              myubuntu

[rocky ~]$ sudo docker start myubuntu
myubuntu

[rocky ~]$ sudo docker ps
CONTAINER ID   IMAGE          COMMAND       CREATED             STATUS         PORTS     NAMES
5dd708fd5afb   ubuntu:18.04   "/bin/bash"   About an hour ago   Up 8 seconds             myubuntu

[rocky ~]$ sudo docker exec -it -u 0 myubuntu bash

root@5dd708fd5afb:/# ls
benoit  bin  boot  dev  etc  home  lib  lib64  media  mnt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var

So yes our file is still there! Do we still see it on our Rocky host then?

[rocky ~]$ sudo find /var/lib/docker/ -name benoit
/var/lib/docker/overlay2/6796ec566b5bbdbcfa99efe742a042b574c17633a504d2d31f87a604e194f0a1/diff/benoit
/var/lib/docker/overlay2/6796ec566b5bbdbcfa99efe742a042b574c17633a504d2d31f87a604e194f0a1/merged/benoit

Yep, as the container is started we can now see our file again in the merged folder. Until the container is deleted, the files created in it are available as they are stored on the host.

Data into two Docker containers

Now let’s do another test by deleting this container and creating two of them. In each container we’ll create a file called benoit. In one ssh session:

[rocky ~]$ sudo docker container run -it --name myubuntu ubuntu:18.04

root@68171839b27d:/# touch benoit

root@68171839b27d:/# ls
benoit  bin  boot  dev  etc  home  lib  lib64  media  mnt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var

Then in another ssh session:

[rocky ~]$ sudo docker container run -it --name myubuntu2 ubuntu:18.04

root@4ba803c23419:/# touch benoit

root@4ba803c23419:/# ls
benoit  bin  boot  dev  etc  home  lib  lib64  media  mnt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var

Let’s now find out what we see on the Rocky host:

[rocky ~]$ sudo find /var/lib/docker/ -name benoit
/var/lib/docker/overlay2/3f074ba37b28f5f2cc533b405ecab3762c092fa406511dc2637264fc0a936200/diff/benoit
/var/lib/docker/overlay2/3f074ba37b28f5f2cc533b405ecab3762c092fa406511dc2637264fc0a936200/merged/benoit
/var/lib/docker/overlay2/710466e337a994b608e0e65bdd55b803d25c5ebbec81db77f17d74eb830c852f/diff/benoit
/var/lib/docker/overlay2/710466e337a994b608e0e65bdd55b803d25c5ebbec81db77f17d74eb830c852f/merged/benoit

As expected we have 2 pairs of our file in 2 separate folders. One for each container. You may wonder if there is a link between this directory ID and the container ID. Let’s have a look:

[rocky ~]$ sudo docker ps
CONTAINER ID   IMAGE          COMMAND       CREATED         STATUS         PORTS     NAMES
4ba803c23419   ubuntu:18.04   "/bin/bash"   7 minutes ago   Up 7 minutes             myubuntu2
68171839b27d   ubuntu:18.04   "/bin/bash"   7 minutes ago   Up 7 minutes             myubuntu

There is no link so how would you know which file belong to which container? I’m glad you asked! We need to inspect our container in order to link the directory ID to a container ID as shown below:

[rocky ~]$ sudo docker inspect 4ba803c23419 |grep -i upper
                "UpperDir": "/var/lib/docker/overlay2/710466e337a994b608e0e65bdd55b803d25c5ebbec81db77f17d74eb830c852f/diff",

[rocky ~]$ sudo docker inspect 68171839b27d |grep -i upper
                "UpperDir": "/var/lib/docker/overlay2/3f074ba37b28f5f2cc533b405ecab3762c092fa406511dc2637264fc0a936200/diff",

Look at this! By inspecting our container we can see the folder used to store its data on our Rocky host.

Finally let’s delete both containers and see what happens to the data it contained:

[rocky ~]$ sudo docker ps -a
CONTAINER ID   IMAGE           COMMAND                  CREATED          STATUS                       PORTS     NAMES
4ba803c23419   ubuntu:18.04    "/bin/bash"              17 minutes ago   Exited (127) 8 seconds ago             myubuntu2
68171839b27d   ubuntu:18.04    "/bin/bash"              17 minutes ago   Exited (0) 14 seconds ago              myubuntu

[rocky ~]$ sudo docker rm 4ba803c23419 68171839b27d
4ba803c23419
68171839b27d

[rocky ~]$ sudo find /var/lib/docker/ -name benoit

[rocky ~]$ sudo ls /var/lib/docker/overlay2/3f074ba37b28f5f2cc533b405ecab3762c092fa406511dc2637264fc0a936200
ls: cannot access '/var/lib/docker/overlay2/3f074ba37b28f5f2cc533b405ecab3762c092fa406511dc2637264fc0a936200': No such file or directory

[rocky ~]$ sudo ls /var/lib/docker/overlay2/710466e337a994b608e0e65bdd55b803d25c5ebbec81db77f17d74eb830c852f
ls: cannot access '/var/lib/docker/overlay2/710466e337a994b608e0e65bdd55b803d25c5ebbec81db77f17d74eb830c852f': No such file or directory

Now our files are gone with the container and they have been deleted from the Rocky host as well. In order to keep container data, you’ll need to create volumes on which data persists even after the container has been deleted.

How is a container stored on its host

Now that we have seen where a file created inside a container is stored on the Rocky host, let’s find out where the container itself is stored on our Rocky host:

[rocky ~]$ sudo docker ps
CONTAINER ID   IMAGE          COMMAND       CREATED          STATUS          PORTS     NAMES
c2032ee57de7   ubuntu:18.04   "/bin/bash"   10 minutes ago   Up 10 minutes             myubuntu

[rocky ~]$ sudo docker inspect c2032ee57de7| grep Id
        "Id": "c2032ee57de7097d6e050bc0b2561c8044baae710f01be570cf65ad7e62c7e24",

[rocky ~]$ sudo ls /var/lib/docker/containers/c2032ee57de7097d6e050bc0b2561c8044baae710f01be570cf65ad7e62c7e24
c2032ee57de7097d6e050bc0b2561c8044baae710f01be570cf65ad7e62c7e24-json.log  checkpoints	config.v2.json	hostconfig.json  hostname  hosts  mounts  resolv.conf  resolv.conf.hash

Easy! With the container ID we can find the associated folder that is stored in the containers folder of /var/lib/docker

After exiting from the container, we can now delete it and see the following:

[rocky ~]$ sudo docker ps -a
CONTAINER ID   IMAGE           COMMAND                  CREATED          STATUS                      PORTS     NAMES
c2032ee57de7   ubuntu:18.04    "/bin/bash"              16 minutes ago   Exited (0) 10 seconds ago             myubuntu

[rocky ~]$ sudo docker rm c2032ee57de7
c2032ee57de7

[rocky ~]$ sudo ls /var/lib/docker/containers/c2032ee57de7097d6e050bc0b2561c8044baae710f01be570cf65ad7e62c7e24
ls: cannot access '/var/lib/docker/containers/c2032ee57de7097d6e050bc0b2561c8044baae710f01be570cf65ad7e62c7e24': No such file or directory

Our container folder on the Rocky host has been deleted at the same time as the container as we could expect.

Wrap up

You should now have all the responses to our questioning at the beginning of this blog. I hope you’ve learned something interesting today. If you want to learn more about Docker and Kubernetes, or if you feel you have some gap in your knowledge, have a look at our dedicated Workshop on this topic. You’ll learned everything you need about those technologies with a lot of opportunities to practice with docker container. See you there!

Post Views: 2,038
Thumbnail [60x60]
by
Benoît Entzmann