[Bug 1929370] Re: Backport support for the -lln command-line option to unsquashfs for 18.04

Alex Murray 1929370 at bugs.launchpad.net
Tue Jun 1 04:31:34 UTC 2021 

I tested the update in -proposed as follows:

# enable -proposed in a fresh 18.04 LTS install
cat <<EOF | sudo tee /etc/apt/sources.list.d/ubuntu-$(lsb_release -cs)-proposed.list
# Enable Ubuntu proposed archive
deb http://archive.ubuntu.com/ubuntu/ $(lsb_release -cs)-proposed restricted main multiverse universe
EOF

cat <<EOF | sudo tee /etc/apt/preferences.d/proposed-updates
# Configure apt to allow selective installs of packages from proposed
Package: *
Pin: release a=$(lsb_release -cs)-proposed
Pin-Priority: 400
EOF

# get the test script from qa-regression-testing
sudo apt install git python-minimal
git clone https://git.launchpad.net/qa-regression-testing

# package the test script etc
cd qa-regression-testing/
./scripts/make-test-tarball ./scripts/test-squashfs-tools.py 

# run the test with the current unpatched version
cd /tmp
tar xf qrt-test-squashfs-tools.tar.gz 
cd qrt-test-squashfs-tools/
sudo ./test-squashfs-tools.py 
Skipping private tests
F.
======================================================================
FAIL: test_mksquashfs_unsquashfs (__main__.SquashfsToolsTest)
Test mksquashfs/unsquashfs - root
----------------------------------------------------------------------
Traceback (most recent call last):
  File "./test-squashfs-tools.py", line 254, in test_mksquashfs_unsquashfs
    self._verify_unsquashfs(numeric_uid, fn, listing)
  File "./test-squashfs-tools.py", line 218, in _verify_unsquashfs
    (raw, entries) = self._get_comparable_unsquashfs(numeric_uid, fn)
  File "./test-squashfs-tools.py", line 171, in _get_comparable_unsquashfs
    "unsquashfs " + switch + " failed: %s" % report)
AssertionError: unsquashfs -lln failed: SYNTAX: unsquashfs [options] filesystem [directories or files to extract]
	-v[ersion]		print version, licence and copyright information
	-d[est] <pathname>	unsquash to <pathname>, default "squashfs-root"
	-n[o-progress]		don't display the progress bar
	-no[-xattrs]		don't extract xattrs in file system
	-x[attrs]		extract xattrs in file system (default)
	-u[ser-xattrs]		only extract user xattrs in file system.
				Enables extracting xattrs
	-p[rocessors] <number>	use <number> processors.  By default will use
				number of processors available
	-i[nfo]			print files as they are unsquashed
	-li[nfo]		print files as they are unsquashed with file
				attributes (like ls -l output)
	-l[s]			list filesystem, but don't unsquash
	-ll[s]			list filesystem with file attributes (like
				ls -l output), but don't unsquash
	-f[orce]		if file already exists then overwrite
	-s[tat]			display filesystem superblock information
	-fstime			display filesystem superblock time
	-e[f] <extract file>	list of directories or files to extract.
				One per line
	-da[ta-queue] <size>	Set data queue to <size> Mbytes.  Default 256
				Mbytes
	-fr[ag-queue] <size>	Set fragment queue to <size> Mbytes.  Default
				256 Mbytes
	-r[egex]		treat extract names as POSIX regular expressions
				rather than use the default shell wildcard
				expansion (globbing)

Decompressors available:
	gzip
	lzma
	lzo
	lz4
	xz


----------------------------------------------------------------------
Ran 2 tests in 2.152s

FAILED (failures=1)


# update squashfs-tools from -proposed
sudo apt update
sudo apt install squashfs-tools/bionic-proposed

# re-run test
sudo ./test-squashfs-tools.py 
Skipping private tests
..
----------------------------------------------------------------------
Ran 2 tests in 2.648s

OK


# also test manually
snap download core
unsquashfs -lln core_11081.snap 
Parallel unsquashfs: Using 1 processor
11395 inodes (12946 blocks) to write

drwxr-xr-x 0/0                     333 2021-04-28 17:53 squashfs-root
drwxr-xr-x 0/0                    1975 2021-04-28 17:53 squashfs-root/bin
-rwxr-xr-x 0/0                 1037528 2019-07-13 04:56 squashfs-root/bin/bash
-rwxr-xr-x 0/0                   31352 2019-07-04 22:06 squashfs-root/bin/bunzip2
-rwxr-xr-x 0/0                   31352 2019-07-04 22:06 squashfs-root/bin/bzcat
lrwxrwxrwx 0/0                       6 2019-07-04 22:06 squashfs-root/bin/bzcmp -> bzdiff
-rwxr-xr-x 0/0                    2140 2019-07-04 22:06 squashfs-root/bin/bzdiff
...


** Tags removed: verification-needed verification-needed-bionic
** Tags added: verification-done verification-done-bionic

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to squashfs-tools in Ubuntu.
https://bugs.launchpad.net/bugs/1929370

Title:
  Backport support for the -lln command-line option to unsquashfs for
  18.04

Status in squashfs-tools package in Ubuntu:
  Fix Released
Status in squashfs-tools source package in Bionic:
  Fix Committed

Bug description:
  [Impact]
  unsquashfs gains a new flag to list numeric uid/gid in the contents of squashfs files. This can be used by the snap store dashboard and review-tools to accurately assess the owner of files within a snap etc.

  [Test Case]
  snap download core 
  unsquashfs -lln core_*.snap

  The above should succeed and output numeric uid/gid information, e.g.:

  ...
  drwxrwsr-x 0/50                      3 2016-04-13 05:44 squashfs-root/var/local
  drwxrwxr-x 0/114                    94 2021-04-28 17:53 squashfs-root/var/log
  -rw------- 0/43                      0 2021-04-28 17:49 squashfs-root/var/log/btmp
  -rw-r----- 0/4                      31 2021-04-28 17:49 squashfs-root/var/log/dmesg
  -rw-r----- 0/4                      31 2021-04-28 17:49 squashfs-root/var/log/fsck/checkfs
  -rw-r----- 0/4                      31 2021-04-28 17:49 squashfs-root/var/log/fsck/checkroot
  -rw-rw-r-- 0/43                  30660 2021-04-28 17:49 squashfs-root/var/log/lastlog
  -rw-rw-r-- 0/43                      0 2021-04-28 17:49 squashfs-root/var/log/wtmp
  drwxrwsr-x 0/8                       3 2021-04-28 17:49 squashfs-root/var/mail
  drwx-wx--T 0/102                     3 2016-04-06 07:29 squashfs-root/var/spool/cron/crontabs
  drwx------ 108/4                     3 2019-03-25 23:44 squashfs-root/var/spool/rsyslog
  ...

  As a more detailed test, the test-squashfs-tools.py script from qa-
  regression-testing can be used to exercise both this -lln output
  option as well as the existing -lls output option since
  https://git.launchpad.net/qa-regression-
  testing/commit/?id=2bc14c225b91b7283e57e2d63a88c164be43df5a to ensure
  both function as expected.

  
  [Where problems could occur]
  unsquashfs could fail to properly run and so the snap store review-tools could fail as a result. The review-tools and the store are coded to fail-closed such that any resulting problems with unsquashfs should not allow snaps that would otherwise be flagged by the review-tools to pass automated review. 

  ------- original description follows --------

  
  The snap store review-tools is changing to compare snaps based on their numeric uid/gids rather than user/group names (see https://code.launchpad.net/~alexmurray/review-tools/+git/review-tools/+merge/403103 for more details).

  Currently the snap store dashboard, where the review-tools are run, is
  running Ubuntu 18.04 - which is based on squashfs-tools 4.3 - however
  support for the -lln command-line option was only added in the 4.4
  release. This change https://github.com/plougher/squashfs-
  tools/commit/a78a589a55a8524ef31ba06ea7c752b8aa8b3604 is relatively
  simple and can be backported to squashfs-tools in 18.04 with minimal
  effort.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/squashfs-tools/+bug/1929370/+subscriptions

More information about the foundations-bugs mailing list