Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

network - error when mixing bridge networks with slirp4netns options #13109

Closed
HidingCherry opened this issue Feb 2, 2022 · 8 comments
Closed
Labels
kind/feature Categorizes issue or PR as related to a new feature. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments.

Comments

@HidingCherry
Copy link

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description

  • run container with:
    -- 2 internal networks (driver: bridge)
    -- slirp4netns:port_handler=rootlesskit

Results in error (different errors, depending on the order of the parameters)

Steps to reproduce the issue:

podman network create --internal nextcloud-pub
podman network create --internal synapse-pub

podman run --name=reverse-proxy -d --net bridge:alias=nextcloud-pub --net bridge:alias=synapse-pub --net slirp4netns:port_handler=rootlesskit -p 8080:80 -p 8443:443 docker.io/library/nginx:1-alpine
OR
podman run --name=reverse-proxy -d --net slirp4netns:port_handler=rootlesskit --net bridge:nextcloud-pub --net bridge:synapse-pub -p 8080:80 -p 8443:443 docker.io/library/nginx:1-alpine

Describe the results you received:

  1. Error: can only set extra network names, selected mode bridge conflicts with bridge: invalid argument
  2. Error: cannot set multiple networks without bridge network mode, selected mode slirp4netns: invalid argument

Describe the results you expected:

container with two networks and 2 exposed ports with rootlesskit as a port_handler

Additional information you deem important (e.g. issue happens only occasionally):

Output of podman version:

$ podman version
Client:       Podman Engine
Version:      4.0.0-dev
API Version:  4.0.0-dev
Go Version:   go1.17.6
Git Commit:   c2f4747fea508a6c6b0fdbf7a51eb6c80ba57f02
Built:        Fri Jan 28 17:38:52 2022
OS/Arch:      linux/amd64

Output of podman info --debug:

host:
  arch: amd64
  buildahVersion: 1.24.0
  cgroupControllers:
  - memory
  - pids
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: /usr/bin/conmon is owned by conmon 1:2.1.0-1
    path: /usr/bin/conmon
    version: 'conmon version 2.1.0, commit: bdb4f6e56cd193d40b75ffc9725d4b74a18cb33c'
  cpus: 4
  distribution:
    distribution: arch
    version: unknown
  eventLogger: journald
  hostname: xxx
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1002
      size: 1
    - container_id: 1
      host_id: 165536
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1002
      size: 1
    - container_id: 1
      host_id: 165536
      size: 65536
  kernel: 5.15.15-hardened1-1-hardened
  linkmode: dynamic
  logDriver: journald
  memFree: 136822784
  memTotal: 4035624960
  networkBackend: cni
  ociRuntime:
    name: crun
    package: /usr/bin/crun is owned by crun 1.4.2-1
    path: /usr/bin/crun
    version: |-
      crun version 1.4.2
      commit: f6fbc8f840df1a414f31a60953ae514fa497c748
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
  os: linux
  remoteSocket:
    exists: true
    path: /run/user/1002/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /etc/containers/seccomp.json
    selinuxEnabled: false
  serviceIsRemote: false
  slirp4netns:
    executable: /sbin/slirp4netns
    package: /usr/bin/slirp4netns is owned by slirp4netns 1.1.12-1
    version: |-
      slirp4netns version 1.1.12
      commit: 7a104a101aa3278a2152351a082a6df71f57c9a3
      libslirp: 4.6.1
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.5.3
  swapFree: 0
  swapTotal: 0
  uptime: 277h 6m 30.74s (Approximately 11.54 days)
plugins:
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  - ipvlan
  volume:
  - local
registries:
  search:
  - ghcr.io
  - quay.io
  - docker.io
store:
  configFile: /home/pods/.config/containers/storage.conf
  containerStore:
    number: 1
    paused: 0
    running: 1
    stopped: 0
  graphDriverName: btrfs
  graphOptions: {}
  graphRoot: /home/pods/.local/share/containers/storage
  graphStatus:
    Build Version: 'Btrfs v5.16 '
    Library Version: "102"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 7
  runRoot: /run/user/1002/containers
  volumePath: /home/pods/.local/share/containers/storage/volumes
version:
  APIVersion: 4.0.0-dev
  Built: 1643387932
  BuiltTime: Fri Jan 28 17:38:52 2022
  GitCommit: c2f4747fea508a6c6b0fdbf7a51eb6c80ba57f02
  GoVersion: go1.17.6
  OsArch: linux/amd64
  Version: 4.0.0-dev

Package info (e.g. output of rpm -q podman or apt list podman):

# pacman -Qs podman
local/podman-git 4.0.0_dev.r14158.gc2f4747fe-1
    Tool and library for running OCI-based containers in pods (git)

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide? (https://github.com/containers/podman/blob/main/troubleshooting.md)

Yes

Additional environment details (AWS, VirtualBox, physical, etc.):
physical

@openshift-ci openshift-ci bot added the kind/bug Categorizes issue or PR as related to a bug. label Feb 2, 2022
@HidingCherry
Copy link
Author

My goal is to create (at least) two isolated networks for reverse proxying and have two exposed ports with correct IPs.

I tried different options, but I will need to set the IP-Address static for at least one of the isolated networks, so the bridge driver is a requirement.

@mheon
Copy link
Member

mheon commented Feb 2, 2022

This won't work at present, due to the way our parsing of the --network flag works. I'm going to re-mark this as a feature request, to add the ability to set custom slirp4netns options while simultaneously joining networks.

As a workaround, in the meantime I suggest you retain --net slirp4netns:options... and manually join the other networks with podman network connect after creation.

@mheon mheon added kind/feature Categorizes issue or PR as related to a new feature. and removed kind/bug Categorizes issue or PR as related to a bug. labels Feb 2, 2022
@Luap99
Copy link
Member

Luap99 commented Feb 2, 2022

This cannot work! You cannot use both slirp4netns and networks.

First the bridge option is just an alias for the default network name podman. So when you want to join to you networks you have to use --net nextcloud-pub --net synapse-pub, the alias key word is used to set network alias names for the container which can be used to resolve this given name via the dns server from other containers.

Second, you cannot set slirp4netns option because slirp4netns is not used per container when you use rootless networks. We only spawn one slirp4netns process for the rootless network namespace. Rootless containers which use networks will always use the rootlesskit port handler since the slirp4netns one does not work for this at the moment.

@Luap99
Copy link
Member

Luap99 commented Feb 2, 2022

I will close the issue since this is not a bug or something we can fix with the way it is designed but feel free to continue the conversation.

@Luap99 Luap99 closed this as completed Feb 2, 2022
@HidingCherry
Copy link
Author

This won't work at present, due to the way our parsing of the --network flag works. I'm going to re-mark this as a feature request, to add the ability to set custom slirp4netns options while simultaneously joining networks.

As a workaround, in the meantime I suggest you retain --net slirp4netns:options... and manually join the other networks with podman network connect after creation.

This doesn't work.

$ podman run --name=reverse-proxy -d --net slirp4netns:port_handler=slirp4netns -p 8080:80 -p 8443:443 docker.io/library/nginx:1-alpine
12d40fe65da76e36971e8417725513af4f8108fac2fefe4439ca8eaa094fa5f8
$ podman network connect --ip 10.89.0.2 nextcloud-pub reverse-proxy 
Error: "slirp4netns" is not supported: invalid network mode

Maybe reopen this issue? @mheon

First the bridge option is just an alias for the default network name podman. So when you want to join to you networks you have to use --net nextcloud-pub --net synapse-pub, the alias key word is used to set network alias names for the container which can be used to resolve this given name via the dns server from other containers.

Ok - thanks for the explanation.

Second, you cannot set slirp4netns option because slirp4netns is not used per container when you use rootless networks. We only spawn one slirp4netns process for the rootless network namespace. Rootless containers which use networks will always use the rootlesskit port handler since the slirp4netns one does not work for this at the moment.

Totally my mistake - sorry.
slirp4netns port handler is the one I need, because it forwards the correct IP.
With rootlesskit port handler I just see the internal IP of the container.
Thus slirp4netns port handler does work? - still podman v4.0.0-dev
@Luap99

I also haven't worked with namespaces and pods yet - I am at the beginning of understanding podman without helping tools.

@Luap99
Copy link
Member

Luap99 commented Feb 2, 2022

You cannot use slirp4netns port_handler with the networks at the moment, see #8193

@kriansa
Copy link

kriansa commented May 22, 2023

I am facing the same problem, with a similar use case:

  • two different bridge networks with containers running on them
  • one container that has access to both networks, publishes the https port and acts as a web reverse proxy

I would like to have my reverse proxy to see the IP that originated the request, however, I see the internal IP of the container.

Is there any workaround for that? Apparently using pasta or slirp4netns is incompatible with containers operating in bridge mode.

@Loader009 have you found a working solution for that case?

@HidingCherry
Copy link
Author

I'm using a bare-installed nginx as the first reverse proxy (which I actually could put in a container) and another reverse proxy for all container, everything with default network driver. The second reverse proxy allows me to use the headers from the first reverse proxy and "correct" the IP.
This ofc doesn't work with container which are not able to process the forwarded-for headers.

For those I try to directly open a port - but since I stopped using synapse (it definitely needs a SSD for the database, with about 80GB of new data per day), I kinda don't need to resolve other issues.

Beside that I also have conntrack issues with podman, but that's a completely different issue.

@github-actions github-actions bot added the locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. label Aug 23, 2023
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Aug 23, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
kind/feature Categorizes issue or PR as related to a new feature. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments.
Projects
None yet
Development

No branches or pull requests

4 participants