[Bug 2056570] Re: Installation failed when Subiquity installs SSH server on noble
Olivier Gayot
2056570 at bugs.launchpad.net
Mon Mar 11 13:02:38 UTC 2024
Without the --mount-proc option, calling `systemctl daemon-reload` in
the chroot prints out "Running in chroot, ignoring command 'daemon-
reload'" and then exits with status 0.
With the --mount-proc option, calling `systemctl daemon-reload` in the
chroot fails with "Failed to connect to bus: No data available" and
fails with status 100.
To determine if we are running in a chroot, systemd calls fstatat(2) on
/ and then fstatat(2) on /proc/1/root. It then compares the resulting
structures, looking specially at the inode number, inode type and
backing device. If anything looks different, systemd assumes we are in a
chroot.
Using stat(1), we can observe what happens:
Without the --mount-proc option, the backing device (i.e. "Device") is
different, therefore systemd assumes we are in a chroot:
# stat -L / /proc/1/root
File: /
Size: 4096 Blocks: 8 IO Block: 4096 directory
=> Device: 252,0 Inode: 2 Links: 20
Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2024-03-11 08:01:50.538756312 +0000
Modify: 2024-03-11 08:01:49.398777854 +0000
Change: 2024-03-11 08:01:49.398777854 +0000
Birth: 2024-03-11 08:00:36.000000000 +0000
File: /proc/1/root
Size: 260 Blocks: 0 IO Block: 4096 directory
=> Device: 0,28 Inode: 2 Links: 1
Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2024-03-11 08:06:22.017527026 +0000
Modify: 2024-03-11 08:00:26.458886048 +0000
Change: 2024-03-11 08:00:26.458886048 +0000
Birth: 2024-03-11 07:58:30.876000000 +0000
But with the --mount-proc option, the structures look identical,
therefore systemd thinks we are not running in a chroot:
File: /
Size: 4096 Blocks: 8 IO Block: 4096 directory
=> Device: 252,0 Inode: 2 Links: 20
Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2024-03-11 08:01:50.538756312 +0000
Modify: 2024-03-11 08:01:49.398777854 +0000
Change: 2024-03-11 08:01:49.398777854 +0000
Birth: 2024-03-11 08:00:36.000000000 +0000
File: /proc/1/root
Size: 4096 Blocks: 8 IO Block: 4096 directory
=> Device: 252,0 Inode: 2 Links: 20
Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2024-03-11 08:01:50.538756312 +0000
Modify: 2024-03-11 08:01:49.398777854 +0000
Change: 2024-03-11 08:01:49.398777854 +0000
Birth: 2024-03-11 08:00:36.000000000 +0000
Explanation
-----------
* When we run a command in a ChrootableTarget, we have:
** /proc bind mounted to /target/proc
** /sys bind mounted to /target/sys
** /run bind mounted to /target/run
** /dev bind mounted to /target/dev
* When we run, `unshare --pid --fork chroot /target apt-get ...`
** the content of /target/proc is inherited from outside the chroot, because of the bind-mount.
** /target/proc/1 corresponds to the process with PID 1 in the "parent" PID namespace (which is the systemd/init process)
** /target/proc/1/root is therefore the "root" of the systemd process, which is outside of the chroot
** in other words /target/proc/1/root == /
** systemd effectively compares /target/proc/1/root with /target and since they are different, it assumes we are in a chroot.
* When we run, `unshare --pid --fork --mount-proc=/target chroot /target apt-get ...`
** the content of /target/proc is fresh (the bind-mount is masked)
** /target/proc/1 corresponds to the process with PID 1 in the "child" PID namespace
** /target/proc/1/root is therefore the "root" of the chroot
** in other words /target/proc/1/root == /target
** systemd effectively compares /target/proc/1/root with /target and since they are identical, it assumes we are /not/ in a chroot.
--
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to the bug report.
https://bugs.launchpad.net/bugs/2056570
Title:
Installation failed when Subiquity installs SSH server on noble
Status in curtin:
Confirmed
Status in subiquity:
Confirmed
Bug description:
I'm trying to install Ubuntu Server 24.04 using today's daily +
changes related to deb822. I'm using the following curtin revision:
https://git.launchpad.net/~ogayot/curtin/commit/?id=2ac55b7f594c5e73891e04a11ebcf9b1f7ec9e3e
Subiquity fails when installing openssh-server. Unfortunately, the
logs do not help much to understand what's going on:
Mar 08 13:39:08 ubuntu-server subiquity_log.1547[11562]: system install failed for ['openssh-server']: Unexpected error while running comma>
Mar 08 13:39:08 ubuntu-server subiquity_log.1547[11562]: Command: ['unshare', '--fork', '--pid', '--mount-proc=/target/proc', '--', 'chroot>
Mar 08 13:39:08 ubuntu-server subiquity_log.1547[11562]: Exit code: 100
Mar 08 13:39:08 ubuntu-server subiquity_log.1547[11562]: Reason: -
Mar 08 13:39:08 ubuntu-server subiquity_log.1547[11562]: Stdout: ''
Mar 08 13:39:08 ubuntu-server subiquity_log.1547[11562]: Stderr: ''
I've added more logs to see what happens (see
https://code.launchpad.net/~ogayot/curtin/+git/curtin/+merge/462051)
and now I can see that dpkg fails during the postinst script:
Mar 08 14:51:10 ubuntu-server subiquity_log.1547[10961]: Stderr: perl: warning: Setting locale failed.
[...]
Mar 08 14:51:10 ubuntu-server subiquity_log.1547[10961]: Creating config file /etc/ssh/sshd_config with new version
Mar 08 14:51:10 ubuntu-server subiquity_log.1547[10961]: Creating SSH2 RSA key; this may take some time ...
Mar 08 14:51:10 ubuntu-server subiquity_log.1547[10961]: 3072 SHA256:tXx12vlm+iJZZUZzitch0ZdmXdYpmjw2eFG+vBmizWo root at ubuntu-server>
Mar 08 14:51:10 ubuntu-server subiquity_log.1547[10961]: Creating SSH2 ECDSA key; this may take some time ...
Mar 08 14:51:10 ubuntu-server subiquity_log.1547[10961]: 256 SHA256:aTGgNLJcS/gjoXyDbZGGw8Bksjm/ENHOcWwER6hZOYQ root at ubuntu-server >
Mar 08 14:51:10 ubuntu-server subiquity_log.1547[10961]: Creating SSH2 ED25519 key; this may take some time ...
Mar 08 14:51:10 ubuntu-server subiquity_log.1547[10961]: 256 SHA256:MZmITjwhkmfqHyu/U3x68Y9yw48UgJTfLAriavFznv4 root at ubuntu-server >
Mar 08 14:51:10 ubuntu-server subiquity_log.1547[10961]: Failed to connect to bus: No data available
Mar 08 14:51:10 ubuntu-server subiquity_log.1547[10961]: dpkg: error processing package openssh-server (--configure):
Mar 08 14:51:10 ubuntu-server subiquity_log.1547[10961]: installed openssh-server package post-installation script subprocess retu>
Mar 08 14:51:10 ubuntu-server subiquity_log.1547[10961]: Errors were encountered while processing:
Mar 08 14:51:10 ubuntu-server subiquity_log.1547[10961]: openssh-server
Mar 08 14:51:10 ubuntu-server subiquity_log.1547[10961]: E: Sub-process /usr/bin/dpkg returned an error code (1)
After adding set -x to the postinst script, we see that it is the call
to systemctl daemon-reload which causes the failure:
+ [ -d /run/systemd/system ]
+ systemctl daemon-reload
Failed to connect to bus: No data available
+ cleanup
+ [ /tmp/tmp.iebuhpLhg7 ]
+ rm -f /tmp/tmp.iebuhpLhg7
+ [ ]
dpkg: error processing package openssh-server (--configure):
installed openssh-server package post-installation script subprocess returned error exit status 1
Errors were encountered while processing:
openssh-server
E: Sub-process /usr/bin/dpkg returned an error code (1)
I think it is a regression introduced by
https://code.launchpad.net/~mitchellaugustin/curtin/+git/curtin/+merge/460960
but I have to confirm.
To manage notifications about this bug go to:
https://bugs.launchpad.net/curtin/+bug/2056570/+subscriptions
More information about the foundations-bugs
mailing list