[Bug 1992485] Re: git cannot fetch: fatal: fetch-pack: unable to fork off sideband demultiplexer

[Denys Vitali]

Thanks Lukas!

Just mind the fact that the issue might not be with the Docker image per
se, or it might be a combination of multiple werid behaviours.


As per https://github.com/argoproj/argo-cd/issues/9888#issuecomment-1271726968, a weird thing I've noticed is that git is not linked anymore with libpthread starting from Ubuntu 22.04:

ArgoCD v2.11 uses ubuntu:21.04 as a base image
ArgoCD v2.12 uses ubuntu:22.04 as a base image

---

argocd at argocd-repo-server-v2.11 $ git --version
15:08:35.903602 git.c:444               trace: built-in: git version
git version 2.30.2

argocd at argocd-repo-server-v2.11 $ ldd $(which git)
	linux-vdso.so.1 (0x00007ffd4d59f000)
	libpcre2-8.so.0 => /lib/x86_64-linux-gnu/libpcre2-8.so.0 (0x00007f6918811000)
	libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f69187f5000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f69187d3000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f69185e7000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f6918c27000)

argocd at argocd-repo-server-v2.11 $ dpkg-query -l libc6 git gpg gpg-agent
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name           Version           Architecture Description
+++-==============-=================-============-=====================================================
ii  git            1:2.30.2-1ubuntu1 amd64        fast, scalable, distributed revision control system
ii  gpg            2.2.20-1ubuntu3   amd64        GNU Privacy Guard -- minimalist public key operations
ii  gpg-agent      2.2.20-1ubuntu3   amd64        GNU privacy guard - cryptographic agent
ii  libc6:amd64    2.33-0ubuntu5     amd64        GNU C Library: Shared libraries

---

argocd at argocd-repo-server-v2.12 $ git --version
15:12:14.779580 git.c:455               trace: built-in: git version
git version 2.34.1

argocd at argocd-repo-server-v2.12 $ ldd $(which git)
	linux-vdso.so.1 (0x00007ffeb5bae000)
	libpcre2-8.so.0 => /lib/x86_64-linux-gnu/libpcre2-8.so.0 (0x00007f2fbe15a000)
	libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f2fbe13e000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f2fbdf16000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f2fbe598000)

argocd at argocd-repo-server-v2.12 $ dpkg-query -l libc6 git gpg gpg-agent
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name           Version             Architecture Description
+++-==============-===================-============-=====================================================
ii  git            1:2.34.1-1ubuntu1.4 amd64        fast, scalable, distributed revision control system
ii  gpg            2.2.27-3ubuntu2.1   amd64        GNU Privacy Guard -- minimalist public key operations
ii  gpg-agent      2.2.27-3ubuntu2.1   amd64        GNU privacy guard - cryptographic agent
ii  libc6:amd64    2.35-0ubuntu3.1     amd64        GNU C Library: Shared libraries

** Bug watch added: github.com/argoproj/argo-cd/issues #9888
   https://github.com/argoproj/argo-cd/issues/9888

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to git in Ubuntu.
https://bugs.launchpad.net/bugs/1992485

Title:
  git cannot fetch: fatal: fetch-pack: unable to fork off sideband
  demultiplexer

Status in Ubuntu Docker Images:
  New
Status in Ubuntu:
  New
Status in git package in Ubuntu:
  New

Bug description:
  There is a weird bug affecting the Ubuntu 22.04 Docker Image that
  seems to only be triggered in certain environments.

  For some reason, when running certain git commands, the following happens:
  https://github.com/argoproj/argo-cd/issues/9888

  ```
  fatal: fetch-pack: unable to fork off sideband demultiplexer
  ```

  On top of that, with the same 22.04 Docker Image, some errors related to GPG are happening:
  https://github.com/argoproj/argo-cd/issues/9809

  I believe this is due to an incompatible OS + Kernel combination, or
  due to some VMware customization.

  The entirety of the analysis is available in those two GitHub issues -
  although they're initially related to ArgoCD, the issue seems to be
  with the vanilla Ubuntu 22.04 Docker Image (and maybe with the Ubuntu
  4.15.0-167-generic kernel).

  Our enviornment:

  Kubernetes cluster running on VMware Tanzu (v1.21.9+vmware.1):
  Ubuntu 16.04.7 LTS 
  Linux 4.15.0-167-generic
  Container Runtime: Docker 20.10.9

  Image having issues:

  - ubuntu:22.10

  Steps to reproduce the issue:

  - Get a VMware Tanzu Kubernetes cluster
  - Start a Pod with ubuntu:21.10
  - Try to do `git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git` (or any repo, really)
  - Notice that you've successfully cloned the repo

  Now, with Ubuntu 22.10:

  - Get a VMware Tanzu Kubernetes cluster
  - Start a Pod with ubuntu:22.10
  - Try to do `git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git` (or any repo, really)

  Result:
  Cloning into 'linux'...
  error: cannot create async thread: Operation not permitted
  fatal: fetch-pack: unable to fork off sideband demultiplexer

  
  We believe this might be an issue with the Ubuntu 22.10 Docker image (?) and VMware Tanzu together.

  We've noticed that libpthread is not linked in the git binary, and
  that might explain the fork issue, althoguh not completely.

  We've also noticed (but this should be OK?) that we don't have enough
  entropy on /dev/random (being a VM), but I don't think this is really
  the problem.

  Can someone help us out and point us in the right direction?
  We can eventually continue the conversation in one of the two above mentioned ArgoCD issues.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu-docker-images/+bug/1992485/+subscriptions