[Bug 1804847] Re: systemd=229-4ubuntu21.8 use of fchownat failes on some systems (openvz)

Mon Jan 14 16:55:20 UTC 2019

Hello Seth as I'm also facing a similar issue which is also related to
this bug I'm now posting to this bug report as it has the highest heat
concerning issues with systemd and systemd-tmpfiles (service do not
start correctly). I don't think this is exclusively related to OpenVZ
because I'm also affected I don't use OpenVZ.

I have collected a few reports which also relate to the systemd changes
which result in services not starting on boot up:

https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1811580
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1804603
https://forum.armbian.com/topic/8852-ssh-doesnt-work-on-orange-pi-zero/

I'm on Armbian on an OrangePI One and I'm also affected by services that
won't start anymore. The same was the case with systemd-229-4ubuntu21.9
and now with 229-4ubuntu21.15. All other systemd version did work
without any issues. For the sake of completeness I will now also attach
my system information and terminal output which relates to the issue.

Distribution / Kernel
Linux xxx 3.4.113-sun8i #2 SMP PREEMPT Sat Jan 12 15:54:26 CET 2019 armv7l armv7l armv7l GNU/Linux
Distributor ID: Ubuntu
Description: Ubuntu 16.04.5 LTS
Release: 16.04
Codename: xenial

Output of journalctl -b 0 -u systemd-tmpfiles-setup.service

Jän 14 11:01:51 xxx systemd[1]: Starting Create Volatile Files and Directories...
Jän 14 11:01:51 xxx systemd-tmpfiles[581]: [/usr/lib/tmpfiles.d/var.conf:14] Duplicate line for path "/var/log", ignoring.
Jän 14 11:01:51 xxx systemd-tmpfiles[581]: Failed to validate path /var: Bad file descriptor
Jän 14 11:01:51 xxx systemd-tmpfiles[581]: Failed to validate path /var/log: Bad file descriptor
Jän 14 11:01:51 xxx systemd-tmpfiles[581]: Failed to validate path /var/lib: Bad file descriptor
Jän 14 11:01:51 xxx systemd-tmpfiles[581]: Failed to validate path /run/sendsigs.omit.d: Bad file descriptor
Jän 14 11:01:51 xxx systemd-tmpfiles[581]: Failed to validate path /home: Bad file descriptor
Jän 14 11:01:51 xxx systemd-tmpfiles[581]: Failed to validate path /srv: Bad file descriptor
Jän 14 11:01:51 xxx systemd-tmpfiles[581]: Failed to validate path /run/lock/subsys: Bad file descriptor
Jän 14 11:01:51 xxx systemd-tmpfiles[581]: Failed to validate path /var/run/lighttpd: Bad file descriptor
Jän 14 11:01:51 xxx systemd-tmpfiles[581]: Failed to validate path /var/cache: Bad file descriptor
Jän 14 11:01:51 xxx systemd-tmpfiles[581]: Failed to validate path /var/cache/man: Bad file descriptor
Jän 14 11:01:51 xxx systemd-tmpfiles[581]: Failed to validate path /run/openvpn: Bad file descriptor
Jän 14 11:01:51 xxx systemd[1]: systemd-tmpfiles-setup.service: Main process exited, code=exited, status=1/FAILURE
Jän 14 11:01:51 xxx systemd[1]: Failed to start Create Volatile Files and Directories.
Jän 14 11:01:51 xxx systemd[1]: systemd-tmpfiles-setup.service: Unit entered failed state.
Jän 14 11:01:51 xxx systemd[1]: systemd-tmpfiles-setup.service: Failed with result 'exit-code'.

Affected services:

# dnsmasq.service loaded failed failed dnsmasq - A lightweight DHCP and caching DNS server
# lighttpd.service loaded failed failed Lighttpd Daemon
# openvpn at server.service loaded failed failed OpenVPN connection to server
# ssh.service loaded failed failed OpenBSD Secure Shell server
# systemd-tmpfiles-setup-dev.service loaded failed failed Create Static Device Nodes in /dev
# systemd-tmpfiles-setup.service loaded failed failed Create Volatile Files and Directories

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to systemd in Ubuntu.
https://bugs.launchpad.net/bugs/1804847

Title:
  systemd=229-4ubuntu21.8 use of fchownat failes on some systems
  (openvz)

Status in systemd package in Ubuntu:
  Fix Released

Bug description:
  The following description is taken from:

  https://answers.launchpad.net/ubuntu/+source/systemd/+question/676237

  Hello everyone,
  I'm running 16.04 LTS on a virtual server which, I think, uses OpenVz. After a recent reboot I found most of my services to be in a failed state. The reason for that, I guess, are these log entries:

  Nov 17 04:47:42 h2118376 systemd-tmpfiles[165]: fchownat() of /run/elasticsearch failed: Invalid argument
  Nov 17 04:47:42 h2118376 systemd-tmpfiles[165]: fchownat() of /run/kopano failed: Invalid argument
  Nov 17 04:47:42 h2118376 systemd-tmpfiles[165]: fchownat() of /run/kopano failed: Invalid argument
  Nov 17 04:47:42 h2118376 systemd-tmpfiles[165]: fchownat() of /run/php failed: Invalid argument
  Nov 17 04:47:42 h2118376 systemd-tmpfiles[165]: fchownat() of /run/postgresql failed: Invalid argument
  Nov 17 04:47:42 h2118376 systemd-tmpfiles[165]: fchownat() of /run/redis failed: Invalid argument
  Nov 17 04:47:42 h2118376 systemd-tmpfiles[165]: fchownat() of /run/screen failed: Invalid argument
  Nov 17 04:47:42 h2118376 systemd-tmpfiles[165]: fchownat() of /run/utmp failed: Invalid argument
  Nov 17 04:47:42 h2118376 systemd-tmpfiles[165]: fchownat() of /run/systemd/netif failed: Invalid argument
  Nov 17 04:47:42 h2118376 systemd-tmpfiles[165]: fchownat() of /run/systemd/netif/links failed: Invalid argument
  Nov 17 04:47:42 h2118376 systemd-tmpfiles[165]: fchownat() of /run/systemd/netif/leases failed: Invalid argument
  Nov 17 04:47:42 h2118376 systemd-tmpfiles[165]: fchownat() of /run/log/journal failed: Invalid argument
  Nov 17 04:47:42 h2118376 systemd-tmpfiles[165]: fchownat() of /run/log/journal/bbad3a438f4b4fb49e5d0700bd5981e8 failed: Invalid argument
  Nov 17 04:47:42 h2118376 systemd-tmpfiles[165]: fchownat() of /run/log/journal/bbad3a438f4b4fb49e5d0700bd5981e8/system.journal failed: Invalid argument

  To verify I tried this:

  /usr/lib/tmpfiles.d# SYSTEMD_LOG_LEVEL=debug systemd-tmpfiles --create elasticsearch.conf
  Reading config file "elasticsearch.conf".
  Running create action for entry d /var/run/elasticsearch
  Found existing directory "/var/run/elasticsearch".
  "/run/elasticsearch" has right mode 40755
  chown "/run/elasticsearch" to 120.128
  fchownat() of /run/elasticsearch failed: Invalid argument

  I can manually chown the directories, e.g. "chown
  elasticsearch:elasticsearch /var/run/elasticsearch" and restart the
  service successfully.  My suspicion is, this is related to an upgrade
  of systemd to 229-4ubuntu21.8.

  At this point I don't know what to do.

  I'm also confused about the version I have installed, which I thought is systemd-229. Howver, I looked at https://github.com/systemd/systemd/blob/v229/src/tmpfiles/tmpfiles.c and found that fchownat() is only used from version 238+:
  Tag v237 (and earlier, including 229):
  /.../
                          if (chown(fn,
                                    i->uid_set ? i->uid : UID_INVALID,
                                    i->gid_set ? i->gid : GID_INVALID) < 0)
                                  return log_error_errno(errno, "chown(%s) failed: %m", path);
  }
  /.../

  Tag v238

  /.../
      if (fchownat(fd,
                               "",
                               i->uid_set ? i->uid : UID_INVALID,
                               i->gid_set ? i->gid : GID_INVALID,
                               AT_EMPTY_PATH) < 0)
  return log_error_errno(errno, "fchownat() of %s failed: %m", path);
  /.../

  Any help fixing this problem would be highly appreciated.
  Many thanks,
  Rafael

  === Notes ===
  fchownat() was added to Linux in kernel 2.6.16;
  library support was added to glibc in version 2.4.
  checkinf if it is blocked/filtered/sandboxed, rarther than unavailable.
  glibc in bionic requires minimum linux 3.2.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1804847/+subscriptions