Docker for Polkadot & Decentralization
Docker is a powerful virtualization tool. While this is not the only virtualization solution available today, it remains one of the simplest to use. Let’s see if and how it plays a critical role in decentralized systems.
Not using Docker
Before going any further, it is probably worth mentioning that many people are using Docker to run nodes and validator nodes. The option is totally valid and (I may argue) even better. A good amount of node operators however prefer to run their nodes on bare metal. I am not aware of any issue since the launch of Polkadot where using or not using docker would have been the solution to any problem. So feel free to chose freely :)
I personally find docker convenient to run several versions simultaneously, even when some of the core dependencies would diverge. Docker is however NOT a requirement to run a Polkadot node.
Back around mid 2018, seeing the complexity for newcomers to install, setup and use the Rust compiler to build an executable such as Polkadot, I decided to make a docker image to streamline the process and allow users without any of the Rust toolchain to build and run their own docker container.
You can read the //www.chevdor.com/post/2018-07-polkadot-syncing-node-under-30s/[article] I wrote back then for more context.
This allowed many people (with more or less programming skills) to be able to run a node and increased the number of validator nodes when the network launched.
Docker helped making the network a tiny bit more decentralized by allowing more users running nodes.
The use of Docker images, especially from a central hub, may also a negative aspect: if an image is corrupted maliciously or by mistake, many users depending on this image will - at least temporarily - fail running their nodes. This is especially critical for validator nodes.
A faulty image may come from a faulty or malicious build process, or even the docker hosting process. Tomorrow, some CI runner may get hacked, docker hub may decide that Polkadot images are no longer 'appropriate'…
In those events, we want to make sure we will be able to generate those images ourselves.
Browsing images on Docker hub, you will likely find many images. Many are small variant of the major base images:
Parity image vs Chevdor image
First let me state that the goal in this article is not to state which of the image is 'better' but to stress out on the differences between the images and help you chose which one may be best for you.
You will likely find references to one or the other image in the various documentations and the reason for seeing one instead of the other is likely a matter of personal preference from the author of the documentation.
Most of the time, you will be able to swap the images with very little changes. These changes are described below.
Docker image: chevdor/polkadot
This image is the original image that has been written for Polkadot and later on adapted for Substrate as well. The original Polkadot repo has been later renamed into Substrate so the PR can be found there.
This image is a 2 stage images. That means that a first image is created solely for the purpose of building polkadot itself, inside a docker container. This first stage image contains all the rust tooling required. This image can be rather big but this is not a concern since it will be deleted later one.
Once the first stage container is done building the polkadot executable, it passes it to a second stage image. This second image is much smaller and does not contain the whole Rust tool chain since we no longer need it.
Altogether, this solution allows making very small docker images containing only the binaries we care about but it also allow anyone, independently from their OS and without any tooling installed, to build their own image.
Historically, the image has been even smaller and some work could be done today to improve the size a bit further. Polkadot and Rust are however evolving very quickly and optimizing too early showed being not viable in the past and builds would fail at times. This is why I reverted to images slightly bigger but more flexible and stable regarding the Rust toolchain.
chevdor/polkadot image as it currently exist has a major drawback. While allowing users to build their own version of the image, it does is rather slowly. New images are usually build manually for new versions.
Docker image: parity/polkadot
This is why the parity devops team created another version of the image that is much faster to build and works much better for all CI and QA needs. Both Dockerfile files are available in the Polkadot repo.
This image however, cannot be simply built by a user, unless having the Rust whole tool chain. For active developers, this won’t be an issue. For the casual node builder, this maybe more of a challenge.
Moreover, if you are strict about security, using this image mean you need to trust the infrastructure that was used to build the binary that is copied inside the docker image.
There is no need to doubt about Paritytech and the images they produce!
Nonetheless, in a trustless environment, you usually trust what you build yourself and NONE of
parity/polkadot are home made as far as you are concerned: they have not been compiled on your trusty own machine.
A benefit of using
chevdor/polkadot is that, if you are ok lowering your trust level at first, you can get started easily and decide at any later time to build the image yourself.
A major benefit of the parity image is that it is built automatically and is usually available very quickly after some code has been pushed into the repository.
Some highlighted differences
There is currently another small difference between the 2 images. Let’s see it in practice with a simplified example:
docker run --rm -it chevdor/polkadot polkadot --version
docker run --rm -it parity/polkadot --version
You may have noticed that the parity image does not require you to specify the name of the polkadot executable. This is convenient and shorter… as long as there is only a single executable. The
chevdor/polkadot image has been built with the plan to include more tools into the image as tooling becomes available.
To be fair, additional tooling could also be put into another image… So while you may want to be aware of the difference, this is definitely not a big deal.
Which one ?
As I mentioned earlier, the answer to that question really depends on what you are trying to do. Both images are 'official' and available in the Polkadot and Substrate repositories.
If you are a developer and want to have the cutting edge images (almost) as soon as the code is available, think no further and jump on
If you can wait a bit until it gets build, run only stable versions, and want to eventually build your own image later, you may want to prefer
Don’t bother too much, you cannot go really wrong. Just install the image that is mentioned in the documentation and you will be fine.
Keep in mind that it is rather easy to switch from an image to the other so you are not making a huge commitment anyway.
Getting some help
If you need help or assistance, with one or the other image, the best way is to stop by in the Riot Channel 'Kusama Watercooler'. If run into issues, make sure to provide the exact and full docker command you are using and someone will help you spot the issue.