IPFS and Docker
With IPFS, you can share files without the need for a centralized product such as Dropbox, Google Drive and Co. That means that you need no account, you have no size or bandwidth limits…
It makes sharing files convenient and very secure. Indeed users cannot modify a file after submitting it. This is due to the fact that IPFS references content using hashes of the content itself and not urls. This article provides a quick introduction to what IPFS is and shows how to install and use it thanks to Docker.
Keep in mind that IPFS is not encrypting any content. If you share secrets on IPFS and someone is lucky enough to guess the hash of your data, they would access your secrets. You may however, share on IPFS content that you encrypted.
Let’s take the example of Dropbox. I could share with you a file with some content as http://somedropboxlink.com/myfile.pdf. After sharing this file with you, I am free to modify the content. The link will remain valid but what you get when downloading the file will depend on WHEN you download it. If you consider some sort of contractual agreement, this is not a nice property.
Using IPFS, things are different. When sharing a file over IPFS, the content you get from a given link is guaranteed to be always exactly the same. I no longer can sneak changes into the contract after sharing it with you :)
You do not need to install anything to view and retrieve files from the IPFS network. This is possible thanks to the IPFS Gateway hosted at https://ipfs.io.
Opening this link in your Browser will show exactly the following content:
Hello and Welcome to IPFS! ██╗██████╗ ███████╗███████╗ ██║██╔══██╗██╔════╝██╔════╝ ██║██████╔╝█████╗ ███████╗ ██║██╔═══╝ ██╔══╝ ╚════██║ ██║██║ ██║ ███████║ ╚═╝╚═╝ ╚═╝ ╚══════╝ If you're seeing this, you have successfully installed IPFS and are now interfacing with the ipfs merkledag! ------------------------------------------------------- | Warning: | | This is alpha software. Use at your own discretion! | | Much is missing or lacking polish. There are bugs. | | Not yet secure. Read the security notes for more. | ------------------------------------------------------- Check out some of the other files in this directory: ./about ./help ./quick-start <-- usage examples ./readme <-- this file ./security-notes
CONGRATULATIONS! You just used IPFS!
When I say exactly, this time it will NOT depend on when you open the URL. In 10 years from now, when opening this IPFS link, the content will be either no longer available, or if it is available, exactly the same.
If someone would modify the content, even a very small change such as adding a space, it would modify the hash you see at the end of the URL. So the original content remains available at the original URL (which is mainly the IPFS hash) and the new content become available at a NEW url (= new IPFS Hash).
You may wonder why the data would then no longer be available?
IPFS is a peer to peer network. When you request a document, your peer (or the IPFS gateway when using the link above) is asking peers around whether someone has the content. If yes, your peer will fetch it, keep it for a while, and returned it. if you need this content again, your peer will already have it (for a while, depending on the configuration of your node).
If no peer has the content matching the hash, no data can be returned. It is up to the peers to decide what is 'interesting content' and how long to keep it.
There is a way to tell your IPFS node to PIN data. It means that we tell our node that some piece of data is important and we want to keep it forever so that it remains available through the network.
There is nowadays a very simple way to use IPFS. You can download the IPFS Desktop from the link below: https://github.com/ipfs-shipyard/ipfs-desktop/blob/master/README.md
This will install and IPFS node on your machine as well as a little utility allowing you to interact with the node. That makes using IPFS very straight forward.
After installing this utility, you may 'add a file to IPFS' with a simple drag&drop. This utility even allows you to add your screenshots automatically to IPFS. That can be convenient!
But keep in mind that any of the file you share will only be available as long as your machine and your IPFS nodes are available. If you turn your computer off and no other peer has your data, people will no longer be able to access your content until your node comes back online.
For this reason, it may be interesting to run IPFS on a server that is running 24/7.
Running IPFS on a server that is available 24/7 is very valuable. First of all, all the content you want to make available will always be available. But you also contribute and help the network by making some popular content available.
The number of Docker related articles on my blog is rapidly growing because Docker makes it very straight forward to run any service, especially if you want to test things out.
Installing IPFS on a new server does require a few steps as you can see here: https://ipfs.io/ipfs/QmR7tiySn6vFHcEjBeZNtYGAFh735PJHfEMdVEycj9jAPy/docs/install/
|The IPFS documentation is stored… well… on IPFS!|
In short, you will need to install some GOLANG stuff, tweak a bit your system, tell the IPFS daemon to start as service, take care of the access rights, etc… I am not sure about you but I seldom have the time or the patience to deal with all of that…
Installing Docker on a server also requires a few steps but once this is done, Docker really makes it simpler.
Here is how you can run an IPFS node on your server and access the
ipfs command running inside your Docker container.
First, you will need to connect to your server using
We will now create 2 folders. I know you hate copy/pasting so here it is as a single line:
mkdir -p $HOME/ipfs/staging; mkdir -p $HOME/ipfs/data
If you have the
tree command on your system, it will show:
$ tree -d -L 1 $HOME/ipfs /home/will/ipfs ├── data └── staging 2 directories
We are now ready for the docker part. While the line is a little longer, it is also rather straight forward and you should have problem with that as long as you installed Docker properly:
docker run -d --name ipfs -v $HOME/ipfs/staging:/export -v $HOME/ipfs/data:/data/ipfs -p 4001:4001 -p 127.0.0.1:8080:8080 -p 127.0.0.1:5001:5001 ipfs/go-ipfs:latest
That’s it! We are done :) IPFS is now running.
But how can we use it? I could give you a few docker commands to do that here but that is rather longer and boring. Let’s make an alias that will make things much easier. I am naming my alias
dipfs (for docker-ipfs) since I have an old
ipfs command on my system. If you don’t, you may replace
ipfs in the commands below:
alias dipfs='docker exec ipfs ipfs'
In order for this alias to be persistent, you need to add it to your
.bashrc. Here is another one-liner:
echo "alias dipfs='docker exec ipfs ipfs'" >> ~/.bashrc; source ~/.bashrc
We now can use our new command. Let’s check out the
help command first:
$ dipfs --help USAGE ipfs - Global p2p merkle-dag filesystem. SYNOPSIS ipfs [--config=<config> | -c] [--debug | -D] [--help] [-h] [--api=<api>] [--offline] [--cid-base=<base>] [--upgrade-cidv0-in-output] [--encoding=<encoding> | --enc] [--timeout=<timeout>] <command> ... OPTIONS -c, --config string - Path to the configuration file to use. -D, --debug bool - Operate in debug mode. --help bool - Show the full command help text. -h bool - Show a short version of the command help text. <... truncated ...> EXIT STATUS The CLI will exit with one of the following values: 0 Successful execution. 1 Failed executions. For more information about each command, use: 'ipfs <subcmd> --help'
The documentation mentions the
Now let’s fetch some content:
$ dipfs cat QmYwAPJzv5CZsnA625s3Xf2nemtYgPpHdWEz79ojWnPbdG/readme Hello and Welcome to IPFS! ██╗██████╗ ███████╗███████╗ ██║██╔══██╗██╔════╝██╔════╝ ██║██████╔╝█████╗ ███████╗ ██║██╔═══╝ ██╔══╝ ╚════██║ ██║██║ ██║ ███████║ ╚═╝╚═╝ ╚═╝ ╚══════╝ If you're seeing this, you have successfully installed IPFS and are now interfacing with the ipfs merkledag! ------------------------------------------------------- | Warning: | | This is alpha software. Use at your own discretion! | | Much is missing or lacking polish. There are bugs. | | Not yet secure. Read the security notes for more. | ------------------------------------------------------- Check out some of the other files in this directory: ./about ./help ./quick-start <-- usage examples ./readme <-- this file ./security-notes
Now let’s add one of our files to IPFS. For the example, we will generate a file. Since IPFS is effectively de-duplicating content, adding a "Hellow World" would be rather useless since someone likely did it already. So you would not add NEW content to IPFS. Instead, we will use the current date as content, that should be fair enough:
$ echo `date` > ipfs/staging/myfile.txt $ dipfs add /export/myfile.txt; rm ipfs/staging/myfile.txt added QmYbdWGwaGjDMzSHDQygAXZsQA559cqesUz19DKgwSfwkt myfile.txt 29 B / 29 B 100.00%
|In this example, we are adding a text file but you could add a PDF, a whole video, … anything…|
|Notice that we have a little extra work to put the file in a specific location, known my the container|
You will likely add files and content using the IPFS Desktop utility we saw at the beginning of this article. Now lets see how we can pin the content we added to our new server so it keeps on providing those files.
Let’s take the IPFS hash
QmYwAPJzv5CZsnA625s3Xf2nemtYgPpHdWEz79ojWnPbdG as example. It may be data from a friend or colleague and you want to help making this content available.
You may use the IPFS Desktop utility to see the content:
$ dipfs pin --help USAGE ipfs pin - Pin (and unpin) objects to local storage. SYNOPSIS ipfs pin SUBCOMMANDS ipfs pin add <ipfs-path>... - Pin objects to local storage. ipfs pin ls [<ipfs-path>]... - List objects pinned to local storage. ipfs pin rm <ipfs-path>... - Remove pinned objects from local storage. ipfs pin update <from-path> <to-path> - Update a recursive pin ipfs pin verify - Verify that recursive pins are complete. For more information about each command, use: 'ipfs pin <subcmd> --help'
$ dipfs pin add QmYwAPJzv5CZsnA625s3Xf2nemtYgPpHdWEz79ojWnPbdG pinned QmYwAPJzv5CZsnA625s3Xf2nemtYgPpHdWEz79ojWnPbdG recursively
We have seen how to use the IPFS Desktop utility, which makes using IPFS a breeze. We have also seen how to install an IPFS node, using Docker, on a server that will be running 24/7.
We have finally seen how to pin content on our server so that our content remains available even when our desktop computer or laptop goes offline.