[Bug 1757565] Re: btrfs and tar sparse truncate archives
Joseph Salisbury
joseph.salisbury at canonical.com
Fri Mar 23 19:44:09 UTC 2018
** Description changed:
+
+ == SRU Justification ==
+ This bug causes btrfs and tar sparse to truncate archives. This bug has been
+ fixed in v4.15-rc3 by commit e3b8a4858566, which has a prereq commit of f48bf66b662e.
+
+ These commits were cc'd to upstream stable and are already in Bionic. However, upstream
+ 4.13 is EOL, so Artful never recieved the fixes, which is the reason for
+ the SRU request.
+
+ == Fixes ==
+ f48bf66b662e ("Btrfs: move definition of the function btrfs_find_new_delalloc_bytes")
+ e3b8a4858566 ("Btrfs: fix reported number of inode blocks after buffered append writes")
+
+ == Regression Potential ==
+ Low. This fix was also sent and accepted to stable, so it has had additional upstream review.
+
+ == Test Case ==
+ A test kernel was built with these patches and tested by the original bug reporter.
+ The bug reporter states the test kernel resolved the bug.
+
+
+
root at ubuntu:~# lsb_release -rd
Description: Ubuntu 17.10
Release: 17.10
- root at ubuntu:~# apt-cache policy linux-image-virtual
+ root at ubuntu:~# apt-cache policy linux-image-virtual
linux-image-virtual:
- Installed: 4.13.0.37.40
- Candidate: 4.13.0.37.40
- Version table:
- *** 4.13.0.37.40 500
- 500 http://archive.ubuntu.com/ubuntu artful-updates/main amd64 Packages
- 500 http://security.ubuntu.com/ubuntu artful-security/main amd64 Packages
- 100 /var/lib/dpkg/status
- 4.13.0.16.17 500
- 500 http://archive.ubuntu.com/ubuntu artful/main amd64 Packages
+ Installed: 4.13.0.37.40
+ Candidate: 4.13.0.37.40
+ Version table:
+ *** 4.13.0.37.40 500
+ 500 http://archive.ubuntu.com/ubuntu artful-updates/main amd64 Packages
+ 500 http://security.ubuntu.com/ubuntu artful-security/main amd64 Packages
+ 100 /var/lib/dpkg/status
+ 4.13.0.16.17 500
+ 500 http://archive.ubuntu.com/ubuntu artful/main amd64 Packages
3. Taring files into an archive are not truncated
4. Files included in tar are filled with NULLs
To reproduce, run an Artful system with one spare disk:
- mkfs.btrfs -f /dev/sda
- mount /dev/sda /mnt
- - grep sda /proc/mounts
+ - grep sda /proc/mounts
/dev/sda /mnt btrfs rw,relatime,space_cache,subvolid=5,subvol=/ 0 0
Then run this script which copies a 4MB binary to a btrfs filesystem,
tars the directory up containing the binary; then untars to stdout and
md5sum compares, showing it's different.
- % SPARSE="-S"; rm -rf /mnt/tmp; md5sum /usr/bin/python3.6; mkdir -p /mnt/tmp; cp -a /usr/bin/python3.6 /mnt/tmp; tar -C /mnt/tmp $SPARSE -czf /mnt/test.tgz .; tar $SPARSE -xzf /mnt/test.tgz -O | md5sum
+ % SPARSE="-S"; rm -rf /mnt/tmp; md5sum /usr/bin/python3.6; mkdir -p /mnt/tmp; cp -a /usr/bin/python3.6 /mnt/tmp; tar -C /mnt/tmp $SPARSE -czf /mnt/test.tgz .; tar $SPARSE -xzf /mnt/test.tgz -O | md5sum
e4121d2f3126b8c364bfa1aaa82371a3 /usr/bin/python3.6
0ce8c4139740198926273853defcb12a -
And now without the sparse flag:
- # SPARSE=""; rm -rf /mnt/tmp; md5sum /usr/bin/python3.6; mkdir -p /mnt/tmp; cp -a /usr/bin/python3.6 /mnt/tmp; tar -C /mnt/tmp $SPARSE -czf /mnt/test.tgz .; tar $SPARSE -xzf /mnt/test.tgz -O | md5sum
+ # SPARSE=""; rm -rf /mnt/tmp; md5sum /usr/bin/python3.6; mkdir -p /mnt/tmp; cp -a /usr/bin/python3.6 /mnt/tmp; tar -C /mnt/tmp $SPARSE -czf /mnt/test.tgz .; tar $SPARSE -xzf /mnt/test.tgz -O | md5sum
e4121d2f3126b8c364bfa1aaa82371a3 /usr/bin/python3.6
e4121d2f3126b8c364bfa1aaa82371a3 -
-
This has been reported to both gnu-tar and linux-btrfs; I'm not aware of an actual fix.
Note that Xenial 4.4 kernels do not exhibit this behavior, and Bionic 4.15 kernel appears to be fixed as well though I'm not sure what the difference is.
References:
- https://patchwork.kernel.org/patch/10151037/
- https://www.spinics.net/lists/linux-btrfs/msg56768.html
+ https://patchwork.kernel.org/patch/10151037/
+ https://www.spinics.net/lists/linux-btrfs/msg56768.html
https://www.spinics.net/lists/linux-btrfs/msg57111.html
ProblemType: Bug
DistroRelease: Ubuntu 17.10
Package: linux-image-virtual 4.13.0.37.40
ProcVersionSignature: Ubuntu 4.13.0-37.42-generic 4.13.13
Uname: Linux 4.13.0-37-generic x86_64
AlsaDevices:
- total 0
- crw-rw---- 1 root audio 116, 1 Mar 21 22:55 seq
- crw-rw---- 1 root audio 116, 33 Mar 21 22:55 timer
+ total 0
+ crw-rw---- 1 root audio 116, 1 Mar 21 22:55 seq
+ crw-rw---- 1 root audio 116, 33 Mar 21 22:55 timer
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay': 'aplay'
ApportVersion: 2.20.7-0ubuntu3.7
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord': 'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
CRDA: N/A
Date: Wed Mar 21 23:08:05 2018
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig': 'iwconfig'
Lsusb: Error: command ['lsusb'] failed with exit code 1:
MachineType: QEMU Standard PC (i440FX + PIIX, 1996)
PciMultimedia:
-
+
ProcEnviron:
- TERM=xterm-256color
- PATH=(custom, no user)
- LANG=C.UTF-8
- SHELL=/bin/bash
+ TERM=xterm-256color
+ PATH=(custom, no user)
+ LANG=C.UTF-8
+ SHELL=/bin/bash
ProcFB:
-
+
ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.13.0-37-generic root=LABEL=cloudimg-rootfs ro console=tty1 console=ttyS0
RelatedPackageVersions:
- linux-restricted-modules-4.13.0-37-generic N/A
- linux-backports-modules-4.13.0-37-generic N/A
- linux-firmware N/A
+ linux-restricted-modules-4.13.0-37-generic N/A
+ linux-backports-modules-4.13.0-37-generic N/A
+ linux-firmware N/A
RfKill: Error: [Errno 2] No such file or directory: 'rfkill': 'rfkill'
SourcePackage: linux
UpgradeStatus: No upgrade log present (probably fresh install)
dmi.bios.date: 04/01/2014
dmi.bios.vendor: SeaBIOS
dmi.bios.version: Ubuntu-1.8.2-1ubuntu1
dmi.chassis.type: 1
dmi.chassis.vendor: QEMU
dmi.chassis.version: pc-i440fx-xenial
dmi.modalias: dmi:bvnSeaBIOS:bvrUbuntu-1.8.2-1ubuntu1:bd04/01/2014:svnQEMU:pnStandardPC(i440FX+PIIX,1996):pvrpc-i440fx-xenial:cvnQEMU:ct1:cvrpc-i440fx-xenial:
dmi.product.name: Standard PC (i440FX + PIIX, 1996)
dmi.product.version: pc-i440fx-xenial
dmi.sys.vendor: QEMU
** Changed in: linux (Ubuntu)
Status: Triaged => In Progress
** Changed in: linux (Ubuntu Artful)
Status: Triaged => In Progress
** Changed in: linux (Ubuntu)
Status: In Progress => Fix Released
--
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to tar in Ubuntu.
https://bugs.launchpad.net/bugs/1757565
Title:
btrfs and tar sparse truncate archives
Status in linux package in Ubuntu:
Fix Released
Status in tar package in Ubuntu:
New
Status in linux source package in Artful:
In Progress
Status in tar source package in Artful:
New
Bug description:
== SRU Justification ==
This bug causes btrfs and tar sparse to truncate archives. This bug has been
fixed in v4.15-rc3 by commit e3b8a4858566, which has a prereq commit of f48bf66b662e.
These commits were cc'd to upstream stable and are already in Bionic. However, upstream
4.13 is EOL, so Artful never recieved the fixes, which is the reason for
the SRU request.
== Fixes ==
f48bf66b662e ("Btrfs: move definition of the function btrfs_find_new_delalloc_bytes")
e3b8a4858566 ("Btrfs: fix reported number of inode blocks after buffered append writes")
== Regression Potential ==
Low. This fix was also sent and accepted to stable, so it has had additional upstream review.
== Test Case ==
A test kernel was built with these patches and tested by the original bug reporter.
The bug reporter states the test kernel resolved the bug.
root at ubuntu:~# lsb_release -rd
Description: Ubuntu 17.10
Release: 17.10
root at ubuntu:~# apt-cache policy linux-image-virtual
linux-image-virtual:
Installed: 4.13.0.37.40
Candidate: 4.13.0.37.40
Version table:
*** 4.13.0.37.40 500
500 http://archive.ubuntu.com/ubuntu artful-updates/main amd64 Packages
500 http://security.ubuntu.com/ubuntu artful-security/main amd64 Packages
100 /var/lib/dpkg/status
4.13.0.16.17 500
500 http://archive.ubuntu.com/ubuntu artful/main amd64 Packages
3. Taring files into an archive are not truncated
4. Files included in tar are filled with NULLs
To reproduce, run an Artful system with one spare disk:
- mkfs.btrfs -f /dev/sda
- mount /dev/sda /mnt
- grep sda /proc/mounts
/dev/sda /mnt btrfs rw,relatime,space_cache,subvolid=5,subvol=/ 0 0
Then run this script which copies a 4MB binary to a btrfs filesystem,
tars the directory up containing the binary; then untars to stdout and
md5sum compares, showing it's different.
% SPARSE="-S"; rm -rf /mnt/tmp; md5sum /usr/bin/python3.6; mkdir -p /mnt/tmp; cp -a /usr/bin/python3.6 /mnt/tmp; tar -C /mnt/tmp $SPARSE -czf /mnt/test.tgz .; tar $SPARSE -xzf /mnt/test.tgz -O | md5sum
e4121d2f3126b8c364bfa1aaa82371a3 /usr/bin/python3.6
0ce8c4139740198926273853defcb12a -
And now without the sparse flag:
# SPARSE=""; rm -rf /mnt/tmp; md5sum /usr/bin/python3.6; mkdir -p /mnt/tmp; cp -a /usr/bin/python3.6 /mnt/tmp; tar -C /mnt/tmp $SPARSE -czf /mnt/test.tgz .; tar $SPARSE -xzf /mnt/test.tgz -O | md5sum
e4121d2f3126b8c364bfa1aaa82371a3 /usr/bin/python3.6
e4121d2f3126b8c364bfa1aaa82371a3 -
This has been reported to both gnu-tar and linux-btrfs; I'm not aware of an actual fix.
Note that Xenial 4.4 kernels do not exhibit this behavior, and Bionic 4.15 kernel appears to be fixed as well though I'm not sure what the difference is.
References:
https://patchwork.kernel.org/patch/10151037/
https://www.spinics.net/lists/linux-btrfs/msg56768.html
https://www.spinics.net/lists/linux-btrfs/msg57111.html
ProblemType: Bug
DistroRelease: Ubuntu 17.10
Package: linux-image-virtual 4.13.0.37.40
ProcVersionSignature: Ubuntu 4.13.0-37.42-generic 4.13.13
Uname: Linux 4.13.0-37-generic x86_64
AlsaDevices:
total 0
crw-rw---- 1 root audio 116, 1 Mar 21 22:55 seq
crw-rw---- 1 root audio 116, 33 Mar 21 22:55 timer
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay': 'aplay'
ApportVersion: 2.20.7-0ubuntu3.7
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord': 'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
CRDA: N/A
Date: Wed Mar 21 23:08:05 2018
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig': 'iwconfig'
Lsusb: Error: command ['lsusb'] failed with exit code 1:
MachineType: QEMU Standard PC (i440FX + PIIX, 1996)
PciMultimedia:
ProcEnviron:
TERM=xterm-256color
PATH=(custom, no user)
LANG=C.UTF-8
SHELL=/bin/bash
ProcFB:
ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.13.0-37-generic root=LABEL=cloudimg-rootfs ro console=tty1 console=ttyS0
RelatedPackageVersions:
linux-restricted-modules-4.13.0-37-generic N/A
linux-backports-modules-4.13.0-37-generic N/A
linux-firmware N/A
RfKill: Error: [Errno 2] No such file or directory: 'rfkill': 'rfkill'
SourcePackage: linux
UpgradeStatus: No upgrade log present (probably fresh install)
dmi.bios.date: 04/01/2014
dmi.bios.vendor: SeaBIOS
dmi.bios.version: Ubuntu-1.8.2-1ubuntu1
dmi.chassis.type: 1
dmi.chassis.vendor: QEMU
dmi.chassis.version: pc-i440fx-xenial
dmi.modalias: dmi:bvnSeaBIOS:bvrUbuntu-1.8.2-1ubuntu1:bd04/01/2014:svnQEMU:pnStandardPC(i440FX+PIIX,1996):pvrpc-i440fx-xenial:cvnQEMU:ct1:cvrpc-i440fx-xenial:
dmi.product.name: Standard PC (i440FX + PIIX, 1996)
dmi.product.version: pc-i440fx-xenial
dmi.sys.vendor: QEMU
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1757565/+subscriptions
More information about the foundations-bugs
mailing list