Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

generate systemd from "docker-composed" container #13289

Closed
web-engineer opened this issue Feb 18, 2022 · 10 comments
Closed

generate systemd from "docker-composed" container #13289

web-engineer opened this issue Feb 18, 2022 · 10 comments
Labels
kind/bug Categorizes issue or PR as related to a bug. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments.

Comments

@web-engineer
Copy link

web-engineer commented Feb 18, 2022 •
edited

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description

Created a container using docker-compose - which goes up fine, we can see runnin "podman ps" the runnign container, however when we try to export this so we can run this at startup it fails with -

Error: container's create command is too short or invalid: [/usr/bin/podman --log-level=info system service]

Steps to reproduce the issue:

  1. Created container using docker-compose and started the container, verified its runinng.
root@kvm1:~/containers/dnsmasq# cat docker-compose.yml
version: '3'
services:
  dns:
    restart: always
    image: jpillora/dnsmasq
    # entrypoint: "webproc"
    command: "webproc --config /etc/dnsmasq.conf -- dnsmasq --no-daemon"
    volumes:
      - ./dnsmasq.conf:/etc/dnsmasq.conf
    ports:
      - "0.0.0.0:53:53/udp"
      - "0.0.0.0:5380:8080"
    cap_add:
      - NET_ADMIN
root@kvm1:~/containers/dnsmasq# export DOCKER_HOST=unix:///run/podman/podman.sock
root@kvm1:~/containers/dnsmasq# docker-compose up -d
Starting dnsmasq_dns_1 ... done
root@kvm1:~/containers/dnsmasq# podman ps
CONTAINER ID  IMAGE                              COMMAND               CREATED            STATUS            PORTS                                       NAMES
701911ca0a35  docker.io/jpillora/dnsmasq:latest  webproc --config ...  About an hour ago  Up 9 seconds ago  0.0.0.0:53->53/udp, 0.0.0.0:5380->8080/tcp  dnsmasq_dns_1

So far so good, the command line I added above, with or without the same result.

  1. Attempt to generate smstemd scripts tried -
root@kvm1:~/containers/dnsmasq# podman generate systemd --new --name --log-level=debug dnsmasq_dns_1
INFO[0000] podman filtering at log level debug
DEBU[0000] Called systemd.PersistentPreRunE(podman generate systemd --new --name --log-level=debug dnsmasq_dns_1)
DEBU[0000] Reading configuration file "/usr/share/containers/containers.conf"
DEBU[0000] Merged system config "/usr/share/containers/containers.conf": &{Containers:{Devices:[] Volumes:[] ApparmorProfile:containers-default-0.33.4 Annotations:[] CgroupNS:private Cgroups:enabled DefaultCapabilities:[CHOWN DAC_OVERRIDE FOWNER FSETID KILL NET_BIND_SERVICE SETFCAP SETGID SETPCAP SETUID SYS_CHROOT] DefaultSysctls:[net.ipv4.ping_group_range=0 0] DefaultUlimits:[nproc=4194304:4194304] DefaultMountsFile: DNSServers:[] DNSOptions:[] DNSSearches:[] EnableKeyring:true EnableLabeling:false Env:[PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin TERM=xterm] EnvHost:false HTTPProxy:true Init:false InitPath: IPCNS:private LogDriver:k8s-file LogSizeMax:-1 NetNS:bridge NoHosts:false PidsLimit:2048 PidNS:private SeccompProfile:/usr/share/containers/seccomp.json ShmSize:65536k TZ: Umask:0022 UTSNS:private UserNS:host UserNSSize:65536} Engine:{ImageBuildFormat:oci CgroupCheck:false CgroupManager:systemd ConmonEnvVars:[PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin] ConmonPath:[/usr/libexec/podman/conmon /usr/local/libexec/podman/conmon /usr/local/lib/podman/conmon /usr/bin/conmon /usr/sbin/conmon /usr/local/bin/conmon /usr/local/sbin/conmon /run/current-system/sw/bin/conmon] DetachKeys:ctrl-p,ctrl-q EnablePortReservation:true Env:[] EventsLogFilePath:/run/libpod/events/events.log EventsLogger:journald HooksDir:[/usr/share/containers/oci/hooks.d] ImageDefaultTransport:docker:// InfraCommand: InfraImage:k8s.gcr.io/pause:3.2 InitPath:/usr/libexec/podman/catatonit LockType:shm MultiImageArchive:false Namespace: NetworkCmdPath: NetworkCmdOptions:[] NoPivotRoot:false NumLocks:2048 OCIRuntime:crun OCIRuntimes:map[crun:[/usr/bin/crun /usr/sbin/crun /usr/local/bin/crun /usr/local/sbin/crun /sbin/crun /bin/crun /run/current-system/sw/bin/crun] kata:[/usr/bin/kata-runtime /usr/sbin/kata-runtime /usr/local/bin/kata-runtime /usr/local/sbin/kata-runtime /sbin/kata-runtime /bin/kata-runtime /usr/bin/kata-qemu /usr/bin/kata-fc] runc:[/usr/bin/runc /usr/sbin/runc /usr/local/bin/runc /usr/local/sbin/runc /sbin/runc /bin/runc /usr/lib/cri-o-runc/sbin/runc /run/current-system/sw/bin/runc]] PullPolicy:missing Remote:false RemoteURI: RemoteIdentity: ActiveService: ServiceDestinations:map[] RuntimePath:[] RuntimeSupportsJSON:[crun runc] RuntimeSupportsNoCgroups:[crun] RuntimeSupportsKVM:[kata kata-runtime kata-qemu kata-fc] SetOptions:{StorageConfigRunRootSet:false StorageConfigGraphRootSet:false StorageConfigGraphDriverNameSet:false StaticDirSet:false VolumePathSet:false TmpDirSet:false} SignaturePolicyPath:/etc/containers/policy.json SDNotify:false StateType:3 StaticDir:/var/lib/containers/storage/libpod StopTimeout:10 TmpDir:/run/libpod VolumePath:/var/lib/containers/storage/volumes VolumePlugins:map[]} Network:{CNIPluginDirs:[/usr/libexec/cni /usr/lib/cni /usr/local/lib/cni /opt/cni/bin] DefaultNetwork:podman NetworkConfigDir:/etc/cni/net.d/}}
DEBU[0000] Using conmon: "/usr/bin/conmon"
DEBU[0000] Initializing boltdb state at /var/lib/containers/storage/libpod/bolt_state.db
DEBU[0000] Using graph driver
DEBU[0000] Using graph root /var/lib/containers/storage
DEBU[0000] Using run root /run/containers/storage
DEBU[0000] Using static dir /var/lib/containers/storage/libpod
DEBU[0000] Using tmp dir /run/libpod
DEBU[0000] Using volume path /var/lib/containers/storage/volumes
DEBU[0000] Set libpod namespace to ""
DEBU[0000] cached value indicated that overlay is supported
DEBU[0000] cached value indicated that metacopy is not being used
DEBU[0000] cached value indicated that native-diff is usable
DEBU[0000] backingFs=extfs, projectQuotaSupported=false, useNativeDiff=true, usingMetacopy=false
INFO[0000] [graphdriver] using prior storage driver: overlay
DEBU[0000] Initializing event backend journald
DEBU[0000] using runtime "/usr/bin/runc"
INFO[0000] Error initializing configured OCI runtime kata: no valid executable found for OCI runtime kata: invalid argument
DEBU[0000] using runtime "/usr/bin/crun"
INFO[0000] Found CNI network podman (type=ptp) at /etc/cni/net.d/87-podman-ptp.conflist
INFO[0000] Found CNI network dnsmasq_default (type=bridge) at /etc/cni/net.d/dnsmasq_default.conflist
DEBU[0000] Default CNI network name podman is unchangeable
INFO[0000] Setting parallel job count to 73
Error: container's create command is too short or invalid: [/usr/bin/podman --log-level=info system service]

Describe the results you expected:

podman generate to generate the output it should.

Additional information you deem important (e.g. issue happens only occasionally):

Hppends every time. Have verified runc and crun are present also

root@kvm1:~/containers/dnsmasq# crun --version
crun version 0.17
commit: 0e9229ae34caaebcb86f1fde18de3acaf18c6d9a
spec: 1.0.0
+SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL
root@kvm1:~/containers/dnsmasq# runc --version
runc version 1.0.0~rc93+ds1
commit: 1.0.0~rc93+ds1-5+b2
spec: 1.0.2-dev
go: go1.15.9
libseccomp: 2.5.1

Output of podman version:

Version:      3.0.1
API Version:  3.0.0
Go Version:   go1.15.9
Built:        Thu Jan  1 01:00:00 1970
OS/Arch:      linux/amd64

Output of podman info --debug:

host:
  arch: amd64
  buildahVersion: 1.19.6
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: 'conmon: /usr/bin/conmon'
    path: /usr/bin/conmon
    version: 'conmon version 2.0.25, commit: unknown'
  cpus: 24
  distribution:
    distribution: debian
    version: "11"
  eventLogger: journald
  hostname: kvm1
  idMappings:
    gidmap: null
    uidmap: null
  kernel: 5.15.0-0.bpo.3-amd64
  linkmode: dynamic
  memFree: 66131468288
  memTotal: 67525332992
  ociRuntime:
    name: crun
    package: 'crun: /usr/bin/crun'
    path: /usr/bin/crun
    version: |-
      crun version 0.17
      commit: 0e9229ae34caaebcb86f1fde18de3acaf18c6d9a
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL
  os: linux
  remoteSocket:
    path: /run/podman/podman.sock
  security:
    apparmorEnabled: true
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: false
    seccompEnabled: true
    selinuxEnabled: false
  slirp4netns:
    executable: ""
    package: ""
    version: ""
  swapFree: 1027600384
  swapTotal: 1027600384
  uptime: 2h 11m 47.24s (Approximately 0.08 days)
registries:
  search:
  - docker.io
  - registry.fedoraproject.org
  - quay.io
  - registry.access.redhat.com
  - registry.centos.org
store:
  configFile: /etc/containers/storage.conf
  containerStore:
    number: 1
    paused: 0
    running: 1
    stopped: 0
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /var/lib/containers/storage
  graphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Using metacopy: "false"
  imageStore:
    number: 1
  runRoot: /run/containers/storage
  volumePath: /var/lib/containers/storage/volumes
version:
  APIVersion: 3.0.0
  Built: 0
  BuiltTime: Thu Jan  1 01:00:00 1970
  GitCommit: ""
  GoVersion: go1.15.9
  OsArch: linux/amd64
  Version: 3.0.1

Package info (e.g. output of rpm -q podman or apt list podman):

Listing... Done
podman/stable,now 3.0.1+dfsg1-3+b2 amd64 [installed]

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide? (https://github.com/containers/podman/blob/main/troubleshooting.md)

Yes - at least the latest upsrtream version available on debian 11 (backports enabled).

Additional environment details (AWS, VirtualBox, physical, etc.):

Running bare metal, from root user account - same containers run fine in docker, I'm migrating to podman but would like to use compose for the container definintions, auto-start doesnt seem to be supported which is fine, systemd feels like a good approach,, however I'm not having any joy getting running with this - apolgies if this is a noob problem but I'm struggling to find the root cause of this.
@openshift-ci openshift-ci bot added the kind/bug Categorizes issue or PR as related to a bug. label Feb 18, 2022
@Luap99
Copy link
Member

Luap99 commented Feb 19, 2022

As documented the --new option is not usable for contianers that are not created via podmanrun/create, see https://github.com/containers/podman/blob/main/docs/source/markdown/podman-generate-systemd.1.md#--new

@Luap99 Luap99 closed this as completed Feb 19, 2022
@web-engineer
Copy link
Author

Note to self RTFD... many thanks - confirmed indeed --new is the issue...

Thanks for pointing us in the right direction. It is obvious on reflection but transitioning from docker / docker-compose this wasn't particularly clear - since the concept of a create command is very much abstracted...

@mheon
Copy link
Member

mheon commented Feb 21, 2022

Theoretically this should be possible once the podman container clone work from @cdoern is merged, but it would require some additional work on our end to wire the two together.

@vrothberg
Copy link
Member

vrothberg commented Feb 21, 2022 •
edited

Theoretically this should be possible once the podman container clone work from @cdoern is merged, but it would require some additional work on our end to wire the two together.

Can you elaborate on that? How would we find a reverse mapping from a container to the CLI options/flags it has been created with.

@mheon
Copy link
Member

mheon commented Feb 21, 2022

If you look at what Charlie has done there - I think we're very close. It gets us to a Specgen, which is basically a parsed CLI.

We'd need extra plumbing (a CLI way of creating a container directly from a Specgen - podman run but with all input defined by a JSON file, not CLI flags), but this would not be hard.

@web-engineer
Copy link
Author

Theoretically this should be possible once the podman container clone work from @cdoern is merged, but it would require some additional work on our end to wire the two together.

Im just running with docker-compose hooked into podmans rest service via DOCKER_HOST variable. Essentially as docker compose must ultimately define a command to bring the container up I'd assume that this could / should be transposed somehow - this does all sound interesting and I'm happy to help where I can if needed - it would be great to be able to sync the systemctl command to the compose structure when using that as a basis for your container spec.

@cdoern
Copy link
Collaborator

cdoern commented Feb 21, 2022

i'd be really interested in looking into a way to get all the way back to CLI options... could be helpful for many scenarios.

@vrothberg
Copy link
Member

That would indeed be great. I am still a worried how robust it would be though. Would for instance run --pull=never be supported?

@mheon
Copy link
Member

mheon commented Feb 21, 2022

We'd potentially need to extend it to support this, but we have a mechanism for this (use annotations to store configuration settings like --pull, --rm that aren't in the Specgen itself- we use this for podman inspect).

@mheon
Copy link
Member

mheon commented Feb 21, 2022

Or, alternatively, we just add these fields into Specgen itself - no real reason we can't have them there.

@github-actions github-actions bot added the locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. label Sep 20, 2023
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 20, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
kind/bug Categorizes issue or PR as related to a bug. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@web-engineer @mheon @vrothberg @cdoern @Luap99