Skip to content

Docker Network bypasses Firewall, no option to disable #22054

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
BenjamenMeyer opened this issue Apr 14, 2016 · 230 comments
Open

Docker Network bypasses Firewall, no option to disable #22054

BenjamenMeyer opened this issue Apr 14, 2016 · 230 comments

Comments

@BenjamenMeyer
Copy link

Output of docker version:

Client:
 Version:      1.10.3
 API version:  1.22
 Go version:   go1.5.3
 Git commit:   20f81dd
 Built:        Thu Mar 10 15:54:52 2016
 OS/Arch:      linux/amd64

Server:
 Version:      1.10.3
 API version:  1.22
 Go version:   go1.5.3
 Git commit:   20f81dd
 Built:        Thu Mar 10 15:54:52 2016
 OS/Arch:      linux/amd64

Output of docker info:

Containers: 14
 Running: 5
 Paused: 0
 Stopped: 9
Images: 152
Server Version: 1.10.3
Storage Driver: aufs
 Root Dir: /var/lib/docker/aufs
 Backing Filesystem: extfs
 Dirs: 204
 Dirperm1 Supported: false
Execution Driver: native-0.2
Logging Driver: json-file
Plugins: 
 Volume: local
 Network: bridge null host
Kernel Version: 3.13.0-58-generic
Operating System: Ubuntu 14.04.4 LTS
OSType: linux
Architecture: x86_64
CPUs: 8
Total Memory: 7.793 GiB
Name: brm-pheonix-dev
ID: Y6Z4:6D53:RFOL:Z3CM:P7ZK:H6HL:RLV5:JT73:LZMC:DTBD:7ILK:2RS5
Username: benjamenmeyer
Registry: https://index.docker.io/v1/

Additional environment details (AWS, VirtualBox, physical, etc.):
Rackspace Cloud Server, Ubuntu 14.04, but that shouldn't really matter

Steps to reproduce the issue:

  1. Setup the system with a locked down firewall
  2. Create a set of docker containers with exposed ports
  3. Check the firewall; docker will by use "anywhere" as the source, thereby all containers are exposed to the public.

Describe the results you received:
root@brm-pheonix-dev:~/rse# iptables --list DOCKER
Chain DOCKER (1 references)
target prot opt source destination
ACCEPT tcp -- anywhere 172.17.0.2 tcp dpt:6379

Describe the results you expected:
root@brm-pheonix-dev:~/rse# iptables --list DOCKER
Chain DOCKER (1 references)
target prot opt source destination
ACCEPT tcp -- 127.0.0.0/24 172.17.0.2 tcp dpt:6379
ACCEPT tcp -- 172.16.0.0/16 172.17.0.2 tcp dpt:6379

Additional information you deem important (e.g. issue happens only occasionally):

By default docker is munging the firewall in a way that breaks security - it allows all traffic from all network devices to access the exposed ports on containers. Consider a site that has 2 containers: Container A exposes 443 running Nginx, and Container B runs an API on port 8000. It's desirable to open Container A to the public for use, but hide Container B entirely so that it can only talk to localhost (for testing by the user) and the docker network (for talking to Container A). It might also be desirable for testing purposes to have Container C be a database used by Container B with the same kind of restrictions.

I found this because of monitoring logs on a service I had thought was not open to the public. After finding log entries from sources trying to break in, I checked the firewall rules and found there was no limit on the source addresses or interfaces. I use UFW and only allow SSH onto this particular box, and would prefer to keep it that way. This can dramatically impact using Docker containers to deploy services and lead to potential security problems if people are not careful.

The best security practice would be to by default limit the networking to work like above desired effect example, and then allow the user to add the appropriate firewall, etc rules to override such behavior, or have an option to revert to the current behavior. I know that for legacy reasons that is not likely since it would break a lot of things on up-date; so at least having an option to enable the above that can be turned on now would be a good first step, and perhaps later after much warning make it the default behavior. Assuming the default behavior is secure, having functionality to manage this (firewall->enable public port, ip) in the docker-compose yml would be a great way to visibly make it known what is going on.

I did find the --iptables=false option, however, I don't want to have to be setting all the rules myself. The only thing I am objecting to is the source setting for the rules.

While I have not verified it, I suspect all the firewalls supported by docker will have the same issue.

@BenjamenMeyer
Copy link
Author

Note: I noticed in https://github.com/docker/docker/blob/master/vendor/src/github.com/docker/libnetwork/iptables/iptables.go that there is not even an existing option to set the source, so it's just using the iptables defaults for source ip/device.

@thaJeztah
Copy link
Member

Duplicate of #14041?

@justincormack
Copy link
Contributor

It is not quite #14041 as this issue is talking about exposed ports. Exposing ports is intended to make them publicly accessible, as this is how you expose services to the outside world. If you are working in a development environment, you can either disable access to ports from outside your computer with a host firewall, or simply not expose the ports and access the services directly, or from other containers on the same network.

I would recommend you use the newer docker networking features to set up private networks for services that you do not want exposed at all, see https://docs.docker.com/engine/userguide/networking/

@thaJeztah
Copy link
Member

That's what I thought of first; but was a bit confused, because exposing a port (EXPOSE), doesn't actually do anything, but publishing a port (-p / -P) actually exposes it on the host.

If you're actually talking about publishing, then this is as designed;

In your example, container B and C should not publish their ports, and container A can communicate with them through the Docker Network, e,g.

docker network create mynet

docker run -d --net=mynet --name=api api-image
docker run -d --net=mynet --name=db database-image
docker run -d --net=mynet --name=web -p 443:443 nginx

This only publishes the "web" container to the host, The web container can access the "API" and "database" containers through their name (I.e. http://api:80/, and db:3306 (assuming MySQL))

@BenjamenMeyer
Copy link
Author

@justincormack so I don't think using a private network solves the issue. In my case I'm using a private network between the containers, and they're still publicly exposed because the host firewall isn't configured to limit the exposure to the private network.

@thaJeztah the issue still comes down to the firewall support - there's no firewall support in docker to limit it to a specific network. You can probably still access those containers from another system as the firewall will not prevent other systems from accessing the port on the host.

Now I'm running this via docker-compose; however, it's not entirely a docker-compose issue since the libnetwork functionality has no capability of limiting the network in the firewall rules - iptables rules have no source specification so regardless of how one configures the network as long as one relies on docker to create the firewall rules (which one should because it's more likely to get them right) then this becomes an issue. Consider the following in a docker-compose.yml file:

nginx:
    build: ./docker/nginx/.
    ports:
        - "127.0.0.1:8080:80"
        - "127.0.0.1:443:443"
    environment:
        DESTINATION_HOST: repose
    links:
        - repose
repose:
    build: ./docker/repose/.
    ports:
        - "127.0.0.1:80:8080"
    environment:
        DESTINATION_HOST: phoenix
        DESTINATION_PORT: 8888
    links:
        - phoenix
curryproxy:
    build: ./docker/curryproxy/.
    ports:
        - "127.0.0.1:8081:8081"
    external_links:
        - rse_rse_1
        - rse_rse_2
        - rse_rse_3
phoenix:
    build: .
    ports:
        - '127.0.0.1:88:8888'
    links:
        - curryproxy:curry
    external_links:
        - rse_rse_1:rse
        - rse_rse_2
        - rse_rse_3
        - rse_cache_1:cache
    volumes:
        - .:/home/phoenix

The above is an excerpt from one of my projects. While I want to be able to test all of them locally from my host, I don't want anyone else to be able to access anything but the nginx instance.

I'm not sure how this translates to your nomenclature...it may be that this is part of the "publishing" aspect, and the publishing capability needs to be expanded to do what I'm saying.

If this is by design, then it's a poor security model as you now expose all developers to extreme risks when on unfamiliar networks (e.g traveling).

As I said, I don't expect the default to change immediately but having the option would be a good first step.

@justincormack
Copy link
Contributor

I am a bit confused then, can you give some examples externally of what you can connect to? The backend services will be (by default) on the 172.17.0.0/16 network, which you will not be able to access externally I wouldn't think first because you will not have a route to that defined from an external host.

There is a potential issue if your external IP is also a private IP that traffic will not be dropped that is routed for the internal networks (whereas it should be from public to private) - is that the issue?

@BenjamenMeyer
Copy link
Author

@justincormack so I'm primarily setting up proper proxying so that some services can only be hit via the proxy (nginx - ssl termination), which then filters through an authentication proxy (repose), and finally off to another service (phoenix). I could care less if all of them are bound to the 0.0.0.0 interface; but I only want nginx to be externally accessible (or at the very least the repose portion if I didn't have nginx in place here). An easy solution, for example, would be to not have to set "127.0.0.1" in the configuration, but have a firewall section where it's easy to specify that to allow through the firewall with a base configuration of only the docker network and local host (loopback) interfaces enabled to talk - something like:

firewall:
    external:
        ports:
            - 80
            - 443

Now the situation can be mitigated somewhat by limited the network mapping on the host to 127.0.0.1 instead of the default 0.0.0.0 map. Note that this is what really mitigates it because otherwise the bridging will forward the host port into the docker network.

And yes, I did verify that that limiting works; however, it still leaves potential vulnerabilities in place and the firewall rules do not match what is actually being done.

As another example, there was a Linux Kernel vulnerability a little while back (having trouble finding it at the moment) that was related to ports that were marked in IPtables as being opened for use by applications, but then not actually being connected to an application - for instance, being on a local host port but not a public IP port. This potentially sets that up, and it would be better practice to limit the IPtables rules to the expected networks instead of leaving them open to connect from any where. As I said, at the very least have the option to specify. They've likely fixed that particular issue but why leave the possibility open?

IOW, it's all about security.

@thaJeztah
Copy link
Member

@BenjamenMeyer if you don't want the other services to be accessible, why do you publish their ports at all? i.e. "127.0.0.1:8081:8081" is not needed if it's only accessed through the docker network (other services connect directly through the docker network)

@seeruk
Copy link

seeruk commented Apr 23, 2016

One issue I have that's related to this one is that I would like to publish ports, but only allow certain IP addresses to access them.

For example, I run a Jenkins environment in a couple of containers. The master node is "published", but I have to make some pretty convoluted iptables rules to lock it down so that only the 2 offices we have can access it.

Is there a way around this currently built into Docker? Or at least a recommended practice? I've seen in the documentation how you might restrict access to 1 IP address; but not several. The other issue with this is that if you have a server that already has an iptables configuration, you might be resetting all of the rules before applying your rules (hence the convoluted rules I have had to set up).

@enzeart
Copy link

enzeart commented Apr 23, 2016

I have an issue similar to the one stated by @seeruk. There is a jarring violation of expectation when preexisting firewall rules don't apply to the published container ports. The desired behavior is as follows (for me at least)

  1. User connection attempt is filtered based on INPUT configurations, etc
  2. Traffic forwarding then happens as usual based on the docker-added FORWARD rules

Is there a succinct way to achieve this in iptables, or does it not easily permit such a construct. I'm particularly limited in my knowledge of iptables so bear with me. I've just recently picked up knowledge about it while trying to understand docker's interactions with it.

@seeruk
Copy link

seeruk commented Apr 23, 2016

What I've actually resorted to for the time being, since I am actually running these containers on a pretty powerful dedicated server, I've set up a KVM VM running Docker, then using some more standard iptables rules to restrict access from the host. The VM has it's own network interface that's only accessibly from the server, so I have to add rules to explicitly allow access to ports in iptables on the host. I have lost a little bit of performance, but not much.

@BenjamenMeyer
Copy link
Author

BenjamenMeyer commented Apr 25, 2016

@thaJeztah I want to be able to access it from the local system, and test against it easily. For example, setting up a RESTful HTTP API that has a Health end-point and being able to reliably run curl against it by using localhost (I have to document this for others and having IP addresses that change is not reliable). In most cases for my dev environment I only want the containers to talk to each other, but I also want to be able to access it from the host.

For @seeruk's case, being able set an IP block (5.5.0.0/16 - a valid parameter for a source address in iptables rules) would be a very good thing. IPtables already has the capability to do the limiting, but docker is not taking advantage of it.

@BenjamenMeyer
Copy link
Author

@thaJeztah I set "127.0.0.1:8081:8081" explicitly to keep it off the external network; I had found logs in my docker containers from people trying to crack into the containers via the exposed ports.

My work around right now is to turn off docker containers before I leave for the day because I can't ensure the environment I want to be external actually is external, or that the environment is properly limited for security purposes.

@thaJeztah
Copy link
Member

@BenjamenMeyer one way to do this is running those tests in a container, e.g.

docker run --net -it --rm --net=mynetwork healthchecker 

@jcheroske
Copy link

The issue that Ben is bringing to light is real and surprising (a bad combination). Many admins, like myself, are using the tried-and-true ufw firewall. Docker is doing and end-run around ufw and altering the iptables rules is in such a way that it 1) causes ufw to misreport the current status of the packet filtering rules, and 2) exposes seemingly private services to the public network. In order for docker to remain in the good graces of the sysadmin community, another approach must be devised. Right now there are many admins out there, who, like Ben and myself, inadvertently opened ports to the wider Internet. Unlike Ben and myself though, they have not figured it out yet.

@BenjamenMeyer
Copy link
Author

@thaJeztah that assumes that I am doing it via the command-line and not using another tool where I only have to set an IP address.

For example, I'm working on an API. I have a tool that I can use to work with that API in production to support it; for development of the tool and the API I want to just point the tool at the dockerized API. The tool knows nothing about docker, nor should it. And I don't necessarily want to put the tool into docker just to use it - pointing it at a port exposed only to the local host should be sufficient.

@BenjamenMeyer
Copy link
Author

@jcheroske I agree, but I don't know that there's a good solution to that aspect. For that, ufw probably needs to be made smarter to be able to lookup and report on rules that it wasn't involved in creating. There's lot of software out there that can adjust the iptables rules in ways that ufw (or AFAIK firewalld, etc) won't know about. There's not really a simple solution to fixing that either.

That said, it would be nice if Docker could integrate with those to dump out the appropriate config files to be able to enable/disable them, or integrate with those tools such that it gets hooked in and dumps out the information appropriately, however, given there are better solutions I don't think that aspect will really be solved. Here, it's more about just limiting the scope of the iptables rules that are being generated to at least minimize the potential impacts by allowing the specification of the source (lo, eth0, 127.0.0.0/24, etc).

@seeruk
Copy link

seeruk commented May 5, 2016

If you are willing to do so, using iptables does make this totally possible.

This is a trimmed-down example of how you can use it: https://gist.github.com/SeerUK/b583cc6f048270e0ddc0105e4b36e480

You can see that right at the bottom, 1.2.3.4 is explicitly given access to port 8000 (which is exposed by Docker), then anything else to that port is dropped. The PRE_DOCKER chain is inserted to be before the DOCKER chain so that it is hit first, meaning the DROP stops the blocked requests from ever reaching the DOCKER chain.

It's a bit annoying that Docker doesn't have this functionality built-in, but it is possible work around it right now.

Another alternative would be using an external firewall. Some places like AWS and Scaleway offer things like security groups where you can manage access to your boxes from outside, from there every port behaves the same way.

I never actually managed to figure out how to make this work with UFW. Though for now, I'm happy with using iptables as a solution. It seems to be working out very well for me so far.

Obviously, this isn't much of a great solution if you have already built a reasonably complicated set of firewall rules around UFW. Though, it does make using things like iptables-persistent quite easy. You can also use alternative ways of allowing access to this way that seem more "normal" in iptables.

@mavenugo
Copy link
Contributor

mavenugo commented May 6, 2016

@BenjamenMeyer have you thought of using an user-defined docker network with a subnet & ip-range option and assigning a static ip-address for containers & using them for local development so that you don't have to depend on a virtual static ip such as 127.0.0.1 ? That will avoid the need to have port-mapping all together for those containers that are private to a host.

docker network create --subnet=30.1.0.0/16 --ip-range=30.1.0.0/24 mynetwork
docker run --net=mynetwork --ip=30.1.1.1 --name=myservice1 xxxx
docker run --net=mynetwork --ip=30.1.1.2 --name=myservice2 yyyy

With this setup, myservice2 can reach myservice1 by name myservice1 and there is no need to even depend on the static ip. Also the host can reach the static-ip freely without the need to have port-mapping.

Also with compose 1.7, you can specify static ip address for containers and specify network subnets and ranges.

@jcheroske
Copy link

I did figure out a simple workaround.

  1. Edit /etc/default/docker: DOCKER_OPTS="--iptables=false"

  2. Add ufw rule: ufw allow to <private_ip> port <port>

So simple that it really makes me wonder why the --iptables=false option is not the default. Why create such a situation when all docker has to do is say, "Hey, if you're running a firewall you're going to have to punch a hole through it!" What am I missing?

https://fralef.me/docker-and-iptables.html
http://blog.viktorpetersson.com/post/101707677489/the-dangers-of-ufw-docker

@enzeart
Copy link

enzeart commented May 8, 2016

I can't get docker to stop modifying iptables to save my life. Tried updating /etc/default/docker to no avail on Ubuntu 16.04

@seeruk
Copy link

seeruk commented May 8, 2016

@enzeart Try /lib/systemd/system/docker.service.

@enzeart
Copy link

enzeart commented May 8, 2016

@seeruk Bless your soul

@thaJeztah
Copy link
Member

@enzeart to configure a daemon running on a host that uses systemd, it's best to not edit the docker.unit file itself, but to use a "drop in" file. That way, you won't run into issues when upgrading docker (in case there's a newer docker.unit file). See https://docs.docker.com/engine/admin/systemd/#custom-docker-daemon-options for more info.

You can also use a daemon.json configuration file, see https://docs.docker.com/engine/reference/commandline/daemon/#daemon-configuration-file

@BenjamenMeyer
Copy link
Author

@mavenugo There's already a docker network in place.

@jcheroske that works, but as I noted it would mean that the end-user (me) would then have to make sure that all iptables rules were correct, which is not optimal and not nearly as likely to happen as having docker do it automatically, thus this issue.

@schklom
Copy link

schklom commented Aug 8, 2024

@msimkunas If you don't want to use a VM, look into Rootless Docker.
As I wrote (even though I was condescendingly replied to then ignored for no valid reason), #42563 (comment) shows Rootless Docker does not conflict with UFW. I have done that since Rootless mode came out and UFW started doing its job without surprises.

Edit: I wonder how many downvotes I'll get this time

@msimkunas
Copy link

@schklom Actually, I prefer running multiple VMs with Docker inside for isolation reasons. In the worst case, if something doesn’t work, I can just destroy the VM from scratch.

@BenjamenMeyer
Copy link
Author

Saw https://www.linkedin.com/pulse/docker-engine-v28-hardening-container-networking-default-docker-wvrse - glad this conversation contributed to the changes for v28. Certainly looking forward to trying it to see if this issue persists.

@Tipiak99
Copy link

Tipiak99 commented May 1, 2025

I do not have a lot fo experience with Docker but I've setup a brand new Docker 28 and I do not see a lot of change related to network security ?

Docker is still bypassing UFW rule for published ports ?

@robmry
Copy link
Contributor

robmry commented May 1, 2025

Yes, this still holds true ... https://docs.docker.com/engine/network/packet-filtering-firewalls/#docker-and-ufw

@SystemParadox
Copy link

SystemParadox commented May 6, 2025

PSA for anyone else using carefully-crafted iptables restore scripts that avoid clobbering DOCKER rules - beware of this lovely iptables bug where user-defined tables are flushed even with iptables-restore -n:

So if you specify :DOCKER - [0:0] because your rules need to include -j DOCKER somewhere then it'll get flushed. This recently resulted in my mail server becoming an open relay because it removed the docker iptables rules but then fell back to docker-proxy (which it turns out is normally completely bypassed by iptables and exists only for compatibility with non-iptables systems) which can't preserve the source IP so now the mail server thinks everything is coming from the host.

@jacoscaz
Copy link

jacoscaz commented May 6, 2025

Personally I've settled onto running Docker - if I cannot use something else - behind other dedicated infrastructure (whether a VM, a separate host behind a firewall, ...) precisely because I know I am not a competent enough user of iptables. I don't work with it anywhere near enough so I don't get to learn from spaced repetition. The things I inevitably re-discover those rare times I do work with iptables do not to stick in my head for rapid recall.

I find the mental overhead in simply treating a machine running Docker as something that can't be secured on its own to be significantly lower than the overhead brought by implementing workarounds.

EDIT: to clarify, this is not meant as a critique to @SystemParadox 's PSA, which I actually really appreciate even as someone who doesn't directly benefit from it. I'm just sharing my perspective as someone who uses containers on a daily basis, both in development and production environments, but that also, like many others, is first and foremost a developer. I do have enough sysadmin chops to get by in most cases but some (most?) of the stuff I've seen in this thread goes well above my head.

@msimkunas
Copy link

@jacoscaz Exactly my thinking as well. I could hack around and come up with a solution if I wanted. Do I want this maintenance overhead which will inevitably break at some point? No.

@SystemParadox
Copy link

Personally I've settled onto running Docker - if I cannot use something else - behind other dedicated infrastructure (whether a VM, a separate host behind a firewall, ...) precisely because I know I am not a competent enough user of iptables.
I find the mental overhead in simply treating a machine running Docker as something that can't be secured on its own to be significantly lower than the overhead brought by implementing workarounds.

EDIT: to clarify, this is not meant as a critique to @SystemParadox 's PSA, which I actually really appreciate even as someone who doesn't directly benefit from it. I'm just sharing my perspective as someone who uses containers on a daily basis, both in development and production environments, but that also, like many others, is first and foremost a developer. I do have enough sysadmin chops to get by in most cases but some (most?) of the stuff I've seen in this thread goes well above my head.

As someone who has maintained their own iptables rules alongside docker for a long time I actually completely agree with you. I consider myself an iptables expert and I'm still tripping up on this. I just cannot recommend that anyone else even tries it - it's just too dangerous.

@schklom
Copy link

schklom commented May 6, 2025

Are the limitations of Rootless Docker so consequential that people would rather use custom iptables or be in a VM instead?

@msimkunas
Copy link

@schklom Personally, I've been using VMs for development for years so yes, I'd rather use a VM for simplicity so that I don't have to tinker with custom iptables rules. Custom setups increase the risk of breakage so I tend to avoid that.

@dm17
Copy link

dm17 commented May 6, 2025

Maybe we can provide a less technical summary of the state of things that will help stick in more heads. Please correct me if I'm wrong:

`If you run iptables, nftables or UFW, then you can only consider it applies to applications running on bare metal. Whatever is running in via docker (and other container systems on Linux?) should be considered by default to be running as if iptables/nftables/UFW is not enabled on the host system.

@Tipiak99
Copy link

Tipiak99 commented May 6, 2025

I've tried to add this config in /etc/ufw/after.rules but published port in docker config file are still reachable from internet.

My knowledge related to network config is limited, any idea ?
I'm using debian 12 and docker v28

Source : https://github.com/chaifeng/ufw-docker

BEGIN UFW AND DOCKER

*filter
:ufw-user-forward - [0:0]
:ufw-docker-logging-deny - [0:0]
:DOCKER-USER - [0:0]
-A DOCKER-USER -j ufw-user-forward

-A DOCKER-USER -j RETURN -s 10.0.0.0/8
-A DOCKER-USER -j RETURN -s 172.16.0.0/12
-A DOCKER-USER -j RETURN -s 192.168.0.0/16

-A DOCKER-USER -p udp -m udp --sport 53 --dport 1024:65535 -j RETURN

-A DOCKER-USER -j ufw-docker-logging-deny -p tcp -m tcp --tcp-flags FIN,SYN,RST,ACK SYN -d 192.168.0.0/16
-A DOCKER-USER -j ufw-docker-logging-deny -p tcp -m tcp --tcp-flags FIN,SYN,RST,ACK SYN -d 10.0.0.0/8
-A DOCKER-USER -j ufw-docker-logging-deny -p tcp -m tcp --tcp-flags FIN,SYN,RST,ACK SYN -d 172.16.0.0/12
-A DOCKER-USER -j ufw-docker-logging-deny -p udp -m udp --dport 0:32767 -d 192.168.0.0/16
-A DOCKER-USER -j ufw-docker-logging-deny -p udp -m udp --dport 0:32767 -d 10.0.0.0/8
-A DOCKER-USER -j ufw-docker-logging-deny -p udp -m udp --dport 0:32767 -d 172.16.0.0/12

-A DOCKER-USER -j RETURN

-A ufw-docker-logging-deny -m limit --limit 3/min --limit-burst 10 -j LOG --log-prefix "[UFW DOCKER BLOCK] "
-A ufw-docker-logging-deny -j DROP

COMMIT

@robmry
Copy link
Contributor

robmry commented May 8, 2025

I've tried to add this config in /etc/ufw/after.rules but published port in docker config file are still reachable from internet.

Hi @Tipiak99 - could you expand on that a bit? What does your port publishing rule look like, and what do you want it to do?

As I understand it, the bad interaction between docker's rules and ufw means it's not possible to filter access to a published port from specific external hosts. (So, once the port is published to an address on a host's external interface, it can be accessed from anywhere. But, only published ports can be accessed.)

If you don't want to allow any external access, "-p 127.0.0.1:8080:80" should forward from port 8080 on the host's loopback address to the container's port 80, while disabling access from outside the host.

@robmry
Copy link
Contributor

robmry commented May 9, 2025

Hoping to clarify that ...

  • Without a -p option, a container's ports are all accessible from the host.
    • There's no need to publish ports if you only want to access them from the host, and can use the container's IP address (by looking up the address, for example by using inspect, or by configuring the address explicitly).
    • This is true even for an --internal network, whose containers have no access to other networks.
    • We've blocked packets routed to container ports from outside the host, by default.
  • Publishing a port with -p makes it accessible via a host address.
    • If the address is a localhost address (for example -p 127.0.0.1:8080:80), the port is only accessible from the host.
      • Including when a packet addressed to 127.0.0.0/8 is sent from outside the host.
    • Direct routing to container ports is not allowed by default any more, published or not.
    • Ports published to a specific host address (for example -p 192.0.2.100:8080:80) are exposed to anything that can route to that address.
    • Ports published to the default 0.0.0.0 (for example -p 8080:80) are exposed to anything that can route to any host address.
      • The default address can be configured, but still has to be configured separately for the default bridge and user-defined bridge networks.

@Tipiak99
Copy link

Tipiak99 commented May 9, 2025

@robmry thanks for your explanations!

I've done the following thing to secure my docker :

  1. Upgrade to docker v28

  2. Install a Nginx docker to act as proxy between Internet and other container

  3. Disable all published ports in docker compose, eg :

    #ports:
    #  - "7000:7000"
  1. Put all docker container inside the same nginx docker network, it allows me to call container without publishing ports and also resolve container names

@msimkunas
Copy link

@robmry thanks for your explanations!

I've done the following thing to secure my docker :

  1. Install a Nginx docker to act as proxy between Internet and other container
  2. Disable all published ports in docker compose, eg :
    #ports:
    #  - "7000:7000"
  1. Put all docker container inside the same nginx docker network, it allows me to call container without publishing ports and also resolve container names

How does this secure your docker containers? Packets can still be routed to container IP addresses, as has already been demonstrated here.

@robmry
Copy link
Contributor

robmry commented May 9, 2025

How does this secure your docker containers? Packets can still be routed to container IP addresses, as has already been demonstrated here.

Hi @msimkunas - not since moby 28.0 ... https://www.docker.com/blog/docker-engine-28-hardening-container-networking-by-default/

@Tipiak99
Copy link

Tipiak99 commented May 9, 2025

@robmry thanks for your explanations!
I've done the following thing to secure my docker :

  1. Install a Nginx docker to act as proxy between Internet and other container
  2. Disable all published ports in docker compose, eg :
    #ports:
    #  - "7000:7000"
  1. Put all docker container inside the same nginx docker network, it allows me to call container without publishing ports and also resolve container names

How does this secure your docker containers? Packets can still be routed to container IP addresses, as has already been demonstrated here.

Not 100% sure but in the example, they are publishing port to localhost while me i'm not publishing port of containers at all (except port of nginx proxy)
victim@192.168.0.100# docker run -e POSTGRES_PASSWORD=password -p 127.0.0.1:5432:5432 postgres

@msimkunas
Copy link

@robmry Thanks. I don't understand what's the point in fiddling with firewall rules after v28, though.

@robmry
Copy link
Contributor

robmry commented May 9, 2025

@robmry Thanks. I don't understand what's the point in fiddling with firewall rules after v28, though.

Sorry @msimkunas, I'm not sure what you mean ... who's fiddling with what rules?

(@Tipiak99 - your setup sounds good to me!)

@msimkunas
Copy link

@robmry Sorry, I was under the impression that people here were still trying to resolve the issue fixed in v28 by manually editing firewall rules.

@robmry
Copy link
Contributor

robmry commented May 9, 2025

Oh, sure - got it! Thanks @msimkunas.

@msimkunas
Copy link

@robmry It's good to hear that things have improved in v28. Personally though, I'm still much more comfortable running such trigger happy software like Docker inside a VM because editing firewall rules in the host is a privilege I don't feel like giving out lightly. Like someone else already said a few comments ago, the cognitive overhead of treating Docker as something to be contained or deployed behind dedicated infrastructure is lower than the overhead of making sure it's not misconfigured, and doesn't perform undesired firewall changes. I'm very picky about which software gets to mess with my firewall and containing Docker inside a VM gives me more peace of mind.

@dm17
Copy link

dm17 commented May 9, 2025

@robmry It's good to hear that things have improved in v28. Personally though, I'm still much more comfortable running such trigger happy software like Docker inside a VM because editing firewall rules in the host is a privilege I don't feel like giving out lightly. Like someone else already said a few comments ago, the cognitive overhead of treating Docker as something to be contained or deployed behind dedicated infrastructure is lower than the overhead of making sure it's not misconfigured, and doesn't perform undesired firewall changes. I'm very picky about which software gets to mess with my firewall and containing Docker inside a VM gives me more peace of mind.

We're in a situation where we have to sandbox an increasing amount of bad decision making leading to an angry "turtles all the way down" situation. Most of my docker usage is on VPSs which are already VMs and if I must then run another VM inside of those VPSs just to "sandbox" docker behavior - and I'm using docker just to easily use a vast array of images (as well as my own) - then it has just gone too far imo.

@msimkunas
Copy link

@dm17 Yeah, it can get tricky once you start playing around with nested containment... My own use case is simpler, it's local development. To me it's infinitely easier to just run multipass launch docker and start working instead of installing Docker on bare metal and going through pages of documentation and trying to verify that Docker does not expose any of its resources in some unexpected manner (and the more complex your setup is, the higher the cognitive load). I understand though that not all use cases are like this and once you get to nested virtualization all of this can seem a bit too much. Which is why it's nice that at least something is improving with v28.

@BenjamenMeyer
Copy link
Author

How does this secure your docker containers? Packets can still be routed to container IP addresses, as has already been demonstrated here.

Hi @msimkunas - not since moby 28.0 ... https://www.docker.com/blog/docker-engine-28-hardening-container-networking-by-default/

@robmry I'm not sure v28 fully solves it yet - the underlying issue is still there AFAICT; it's just mitigated now since the default binding address is no longer 0.0.0.0 but 127.0.0.1 so containers should not be accessible off localhost by default. They did do a lot more documentation about the firewall (https://docs.docker.com/engine/network/packet-filtering-firewalls/). All-in-all, I'm glad it's getting noticed, mitigations are finally in place, and things are more secure by default so by default.

So this is certainly 100% better. I'd need to do some research to see how close this is to being resolved though.

@robmry
Copy link
Contributor

robmry commented May 12, 2025

Hi @BenjamenMeyer - the default binding address is still 0.0.0.0.

There are now rules in raw-PREROUTING to drop packets sent from outside the host to container addresses, and to ports mapped to the loopback address. Rules in the filter-FORWARD chain no longer depend on its default policy.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests