[Bug 2092309] [NEW] undetected data loss creating .zip file

Fri Dec 20 20:27:13 UTC 2024

Public bug reported:

When using /usr/bin/zip to create a new .zip file, then data loss due to
EDQUOT (exceeding disk quota) or ENOSPC (no space on block device) is
undetected, and the whole file is trash.  You won't learn of the loss
until you try to read the .zip file, when /usr/bin/unzip complains about
not finding the internal directory which is stored at the end of the
file:  the very place that is discarded because of EDQUOT or ENOSPC.
The loss is particularly likely to occur when the created file is inside
a Virtual memory Container, because the Disk Quota typically is set
quite low for a container.

Evidence of the existence of the bug:
$ (date;date;date;) >foo.in  ## create a compressible file
$ rm -f foo.zip   ## start fresh
$ gdb -q ./zip  ## built with debug symbols from zip-3.0-13ubuntu0.1
(gdb) b main
(gdb) run foo.zip foo.in
   Breakpoint at main

## set breakpoints at the 3 system calls that could detect EDQUOT or ENOSPC
(gdb) b close
(gdb) b fdatasync
(gdb) b fsync
(gdb) run
 adding: foo.   ## message from zip program
Breakpoint 2, __GI___close (fd=0x4) at ../sysdeps/unix/sysv/linux/close.c:26

## Find out which file is fd 4
(gdb) info proc
process 4154
cmdline = '/home/jreiser/zip-3.0/zip foo.zip foo.in'
cwd = '/home/jreiser/zip-3.0'
exe = '/home/jreiser/zip-3.0/zip'
(gdb) shell ls -l /proc/4154/fd
total 0
lrwx------ 1 jreiser jreiser 64 Dec 20 11:50 0 -> /dev/pts/0
lrwx------ 1 jreiser jreiser 64 Dec 20 11:50 1 -> /dev/pts/0
lrwx------ 1 jreiser jreiser 64 Dec 20 11:50 2 -> /dev/pts/0
lrwx------ 1 jreiser jreiser 64 Dec 20 11:50 3 -> /home/jreiser/zip-3.0/zikdScEa
lr-x------ 1 jreiser jreiser 64 Dec 20 11:50 4 -> /home/jreiser/zip-3.0/foo.in

## Therefore fd 4 is the input file foo.in

(gdb) continue
Continuing.
 (deflated 63%)   ## message from zip program
[Inferior 1 (process 4154) exited normally]
(gdb) q

Notice that none of close(), fdatasync(), nor fsync() was called on the output file.
Those are the only system calls that can detect EDQUOT.  The implicit close()
that Linux calls during exit() deliberately ignores all errors.  So running out
of space will not be detected.  System calls write(), pwrite(), etc do not detect
EDQUOT because it would be too inefficient.

The command "unzip -t foo.zip" of "unzip -l foo.zip" might be a quick
way to check whether the internal directory is present.

ProblemType: Bug
DistroRelease: Ubuntu 24.04
Package: zip 3.0-13ubuntu0.1
ProcVersionSignature: Ubuntu 6.8.0-51.52-generic 6.8.12
Uname: Linux 6.8.0-51-generic x86_64
ApportVersion: 2.28.1-0ubuntu3.3
Architecture: amd64
CasperMD5CheckResult: pass
CurrentDesktop: ubuntu:GNOME
Date: Fri Dec 20 11:52:39 2024
InstallationDate: Installed on 2024-12-11 (9 days ago)
InstallationMedia: Ubuntu 24.04 LTS "Noble Numbat" - Release amd64 (20240424)
ProcEnviron:
 LANG=en_US.UTF-8
 PATH=(custom, no user)
 SHELL=/bin/bash
 TERM=xterm-256color
 XDG_RUNTIME_DIR=<set>
SourcePackage: zip
UpgradeStatus: No upgrade log present (probably fresh install)

** Affects: zip (Ubuntu)
     Importance: Undecided
         Status: New

** Tags: amd64 apport-bug noble wayland-session

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to zip in Ubuntu.
https://bugs.launchpad.net/bugs/2092309

Title:
  undetected data loss creating .zip file

Status in zip package in Ubuntu:
  New

Bug description:
  When using /usr/bin/zip to create a new .zip file, then data loss due
  to EDQUOT (exceeding disk quota) or ENOSPC (no space on block device)
  is undetected, and the whole file is trash.  You won't learn of the
  loss until you try to read the .zip file, when /usr/bin/unzip
  complains about not finding the internal directory which is stored at
  the end of the file:  the very place that is discarded because of
  EDQUOT or ENOSPC.  The loss is particularly likely to occur when the
  created file is inside a Virtual memory Container, because the Disk
  Quota typically is set quite low for a container.

  Evidence of the existence of the bug:
  $ (date;date;date;) >foo.in  ## create a compressible file
  $ rm -f foo.zip   ## start fresh
  $ gdb -q ./zip  ## built with debug symbols from zip-3.0-13ubuntu0.1
  (gdb) b main
  (gdb) run foo.zip foo.in
     Breakpoint at main

  ## set breakpoints at the 3 system calls that could detect EDQUOT or ENOSPC
  (gdb) b close
  (gdb) b fdatasync
  (gdb) b fsync
  (gdb) run
   adding: foo.   ## message from zip program
  Breakpoint 2, __GI___close (fd=0x4) at ../sysdeps/unix/sysv/linux/close.c:26

  ## Find out which file is fd 4
  (gdb) info proc
  process 4154
  cmdline = '/home/jreiser/zip-3.0/zip foo.zip foo.in'
  cwd = '/home/jreiser/zip-3.0'
  exe = '/home/jreiser/zip-3.0/zip'
  (gdb) shell ls -l /proc/4154/fd
  total 0
  lrwx------ 1 jreiser jreiser 64 Dec 20 11:50 0 -> /dev/pts/0
  lrwx------ 1 jreiser jreiser 64 Dec 20 11:50 1 -> /dev/pts/0
  lrwx------ 1 jreiser jreiser 64 Dec 20 11:50 2 -> /dev/pts/0
  lrwx------ 1 jreiser jreiser 64 Dec 20 11:50 3 -> /home/jreiser/zip-3.0/zikdScEa
  lr-x------ 1 jreiser jreiser 64 Dec 20 11:50 4 -> /home/jreiser/zip-3.0/foo.in

  ## Therefore fd 4 is the input file foo.in

  (gdb) continue
  Continuing.
   (deflated 63%)   ## message from zip program
  [Inferior 1 (process 4154) exited normally]
  (gdb) q

  Notice that none of close(), fdatasync(), nor fsync() was called on the output file.
  Those are the only system calls that can detect EDQUOT.  The implicit close()
  that Linux calls during exit() deliberately ignores all errors.  So running out
  of space will not be detected.  System calls write(), pwrite(), etc do not detect
  EDQUOT because it would be too inefficient.

  The command "unzip -t foo.zip" of "unzip -l foo.zip" might be a quick
  way to check whether the internal directory is present.

  ProblemType: Bug
  DistroRelease: Ubuntu 24.04
  Package: zip 3.0-13ubuntu0.1
  ProcVersionSignature: Ubuntu 6.8.0-51.52-generic 6.8.12
  Uname: Linux 6.8.0-51-generic x86_64
  ApportVersion: 2.28.1-0ubuntu3.3
  Architecture: amd64
  CasperMD5CheckResult: pass
  CurrentDesktop: ubuntu:GNOME
  Date: Fri Dec 20 11:52:39 2024
  InstallationDate: Installed on 2024-12-11 (9 days ago)
  InstallationMedia: Ubuntu 24.04 LTS "Noble Numbat" - Release amd64 (20240424)
  ProcEnviron:
   LANG=en_US.UTF-8
   PATH=(custom, no user)
   SHELL=/bin/bash
   TERM=xterm-256color
   XDG_RUNTIME_DIR=<set>
  SourcePackage: zip
  UpgradeStatus: No upgrade log present (probably fresh install)

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/zip/+bug/2092309/+subscriptions