Should Ubuntu systemd journal logs be persistent by default?
Ryan Harper
ryan.harper at canonical.com
Mon Nov 13 15:33:14 UTC 2017
On Mon, Nov 13, 2017 at 2:27 AM, Robie Basak <robie.basak at ubuntu.com> wrote:
> Hi Mark,
>
> On Tue, Nov 07, 2017 at 08:11:27PM +0000, Mark Stosberg wrote:
> > Some time around the 15.04 release, a policy change was made to quit
> > making some logging persistent by default.
>
> To help me understand the problem, exactly what logging was persistent
> previously, and isn't persistent now? Can you please give us an example?
>
Any data in systemd-journal that is not flush to /var/log/* is gone
forever on power fail due
to the journal being hosted on ephemeral tmpfs (/run/log/journal);
# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 14.04.5 LTS
Release: 14.04
Codename: trusty
root at t-nonvocational-azzie:/var/log# ls -al /var/log/
total 500
drwxrwxr-x 8 root syslog 4096 Nov 13 15:19 .
drwxr-xr-x 12 root root 4096 Nov 10 20:43 ..
drwxr-xr-x 2 root root 4096 Nov 10 20:43 apt
-rw-r----- 1 syslog adm 2884 Nov 13 15:19 auth.log
-rw-r--r-- 1 root root 7678 Nov 13 15:12 boot.log
-rw-rw---- 1 root utmp 0 Nov 10 20:43 btmp
-rw-r--r-- 1 syslog adm 153678 Nov 13 15:12 cloud-init.log
-rw-r--r-- 1 root root 3870 Nov 13 15:12 cloud-init-output.log
drwxr-xr-x 2 root root 4096 Nov 30 2016 dist-upgrade
-rw-r--r-- 1 root adm 30483 Nov 13 15:12 dmesg
-rw-r--r-- 1 root root 0 Nov 13 15:12 dmesg.0
drwxr-xr-x 2 root root 4096 Nov 10 20:41 fsck
-rw-r----- 1 syslog adm 52797 Nov 13 15:13 kern.log
drwxr-xr-x 2 landscape root 4096 Nov 13 15:12 landscape
-rw-rw-r-- 1 root utmp 292292 Nov 13 15:19 lastlog
-rw-r----- 1 syslog adm 55736 Nov 13 15:17 syslog
-rw-r--r-- 1 root root 148851 Nov 13 15:12 udev
drwxr-xr-x 2 root root 4096 May 8 2017 unattended-upgrades
drwxr-xr-x 2 root root 4096 Nov 13 15:19 upstart
-rw-rw-r-- 1 root utmp 5376 Nov 13 15:19 wtmp
$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.3 LTS
Release: 16.04
Codename: xenial
ubuntu at x-undiluvial-edith:/var/log$ ls -al /var/log/
total 312
drwxrwxr-x 7 root syslog 4096 Nov 13 15:15 .
drwxr-xr-x 13 root root 4096 Nov 10 14:25 ..
drwxr-xr-x 2 root root 4096 Nov 10 14:24 apt
-rw-r----- 1 syslog adm 3181 Nov 13 15:17 auth.log
-rw------- 1 root utmp 0 Nov 10 14:24 btmp
-rw-r--r-- 1 syslog adm 127616 Nov 13 15:15 cloud-init.log
-rw-r--r-- 1 root root 4271 Nov 13 15:15 cloud-init-output.log
drwxr-xr-x 2 root root 4096 Oct 20 10:35 dist-upgrade
drwxr-xr-x 2 root root 4096 Nov 10 14:22 fsck
-rw-r----- 1 syslog adm 52555 Nov 13 15:15 kern.log
-rw-rw-r-- 1 root utmp 292292 Nov 13 15:16 lastlog
drwxr-xr-x 2 root root 4096 Aug 23 02:06 lxd
-rw-r----- 1 syslog adm 82360 Nov 13 15:17 syslog
drwxr-x--- 2 root adm 4096 Sep 20 14:13 unattended-upgrades
-rw-rw-r-- 1 root utmp 2688 Nov 13 15:16 wtmp
boot.log includes services that start on boot,
These service starts are not part of syslog; ssh for example in boot.log
shows
it starting (but any service may include more information here in case of
failure).
# grep -i openssh boot.log
* Starting OpenSSH server [
OK ]
$ journalctl -o short-precise --no-pager -u ssh
-- Logs begin at Mon 2017-11-13 15:15:13 UTC, end at Mon 2017-11-13
15:17:01 UTC. --
Nov 13 15:15:33.816052 x-undiluvial-edith systemd[1]: Starting OpenBSD
Secure Shell server...
Nov 13 15:15:33.846646 x-undiluvial-edith sshd[1249]: Server listening on
0.0.0.0 port 22.
Nov 13 15:15:33.847790 x-undiluvial-edith sshd[1249]: Server listening on
:: port 22.
Nov 13 15:15:33.848100 x-undiluvial-edith systemd[1]: Started OpenBSD
Secure Shell server.
Nov 13 15:16:35.015383 x-undiluvial-edith sshd[1317]: Accepted publickey
for ubuntu from 10.5.0.10 port 54030 ssh2: RSA
SHA256:0cuEJGbgH9aUy12t0jKluNEwLeAF4SIbvimjeM1OLWw
Nov 13 15:16:35.018911 x-undiluvial-edith sshd[1317]:
pam_unix(sshd:session): session opened for user ubuntu by (uid=0)
/var/log/udev is another.
In general when systems fail to boot, users may not have console access but
may retain power control.
Issuing a shutdown or even hard stop will still preserve data that's been
written to the filesystem for later
inspection. With the bulk if boot and service status and output
information included in the journald held in
tmpfs; we lose that data unless there is some way to flush that data to
persistent storage.
I'm +1 for having journald be persistent by default; it certainly makes
debugging boot failures
much easier since one has actual logs to examine versus nothing at all.
> Thanks,
>
> Robie
>
> --
> ubuntu-devel mailing list
> ubuntu-devel at lists.ubuntu.com
> Modify settings or unsubscribe at: https://lists.ubuntu.com/
> mailman/listinfo/ubuntu-devel
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/ubuntu-devel/attachments/20171113/640fe636/attachment-0001.html>
More information about the ubuntu-devel
mailing list