Why Docker and Firewall don’t get along with each other!
In this article, I’m gonna get you through my own experience with interference between Docker and the Firewall on Linux (originally
iptables and its interfaces like
ufw) which took me hours to find out why my firewall rules are not working. If you are a Docker fan too, reading this article may save you a lot of time. Enough with blah blah blah, let’s dive into the story!
The story began on a beautiful rainy day in which I was deploying a Redis container. Firstly, I ran a container with the official Redis image that was using port 6379, and yes, this port was exposed to the outside world because my application service needed to be able to reach this node by its IP address. The second action was restricting access to this Redis node by defining some simple rules in
iptables (my interface was
firewalld). I didn’t want someone to just step in and make some trouble for me by attacking this node (flushing it, making it a slave of another Redis, etc). So I simply created a new
firewalld rule to block the incoming requests. Guess what? It didn’t work in a very beautiful way!
The docker documentation explains that Docker manipulates firewall rules for network isolation by default. It installs two custom chains called
DOCKER-USER . The
DOCKER chain contains all necessary docker rules. And the other one,
DOCKER-USER chain, is created for user-level rules, and these user-defined rules are applied before any rule that Docker creates automatically.
We aren’t able to add our custom rules directly (to protect Redis) to the
INPUT/FORWARD chains as usual. But if we do, they’ll be ignored. Docker in its documentation clearly says so:
Rules added to the
FORWARDchain -- either manually, or by another iptables-based firewall -- are evaluated after these chains. This means that if you expose a port through Docker, this port gets exposed no matter what rules your firewall has configured.
So I tried to add my Redis restriction rule to this
# I droped incoming traffics from all IP addresses except an authorized one (126.96.36.199 for example)
iptables -I DOCKER-USER -i eth0 --dport 6379 -s 188.8.131.52 -j ACCEPT
iptables -I DOCKER-USER -i eth0 --dport 6379 -j DROP
Unfortunately, it didn’t work too! I tried to remove and add it again multiple times but there was no difference with the final result. So I took another shot with Docker's official example:
iptables -I DOCKER-USER -i eth0 ! -s 184.108.40.206 -j DROP
Yohu, it worked!! But there were still some limitations:
- The restriction was applied only to the IP address, not the IP and Port.
- I couldn’t add more trusted IPs. (technically, it is possible with
ipset, but it brings more complexity with itself which wasn’t my option— but if you want to do it anyway, check this link).
- I had to put my rules directly to the iptables but I still wanted to use an easy-to-use interface (
Because of the mentioned reasons, I decided to find a better solution that not only works with
iptables, but also works with its interfaces.
I was sick of searching for hours and finding nothing. Yes you’re right, the problem was the Docker and
firewalld and that’s an answer to “why I couldn’t find a solution”.
I came back to the same Docker and Iptables documentation page and at the end of this page, I saw a section with the title “Prevent Docker from manipulating iptables”. It was the game-changer of this story! Let’s break down what it really means.
The Docker gives you the option to disable iptables rules manipulation with a flag; the
iptables=false. But it also mentions that you may face some networking issues if you use it. Don’t worry, I’ve covered that in this article.
Anyway, it seemed a legit solution to me. So I created a file in the following path:
And put the instructions to disable iptables manipulation at the service level. By making changes to the docker service, it won’t get back to the default settings after every restart:
ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock --iptables=false
It was time to reload the system daemon (because of service config modification) and restart the Docker service:
sudo systemctl daemon-reload
sudo systemctl restart docker
That was all! After these steps, I could configure my firewall rules!
iptablesmanipulation, make sure that:
- Your containers have access to the internet. A simple ping from inside the container would be good.
- Your containers can communicate with each other on the same network.
If this article helped you in any way, please give it a clap.
Happy DevOps! 🎉