[Bug 1926437] Re: [SRU] Backport zstd support, fix bug in python-apt
Balint Reczey
1926437 at bugs.launchpad.net
Thu May 13 20:00:06 UTC 2021
Verified 1.6.5ubuntu0.6 on Bionic.
root at bb-lp1926437:~# wget https://people.canonical.com/~rbalint/zstd-debs/glibc-doc-reference_2.33-0ubuntu2~zstd1_all.deb
--2021-05-13 19:29:17-- https://people.canonical.com/~rbalint/zstd-debs/glibc-doc-reference_2.33-0ubuntu2~zstd1_all.deb
Resolving people.canonical.com (people.canonical.com)... 91.189.89.62
Connecting to people.canonical.com (people.canonical.com)|91.189.89.62|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 5618446 (5.4M) [application/x-debian-package]
Saving to: ‘glibc-doc-reference_2.33-0ubuntu2~zstd1_all.deb’
glibc-doc-reference_2.33-0ubuntu2~zstd1_all
100%[=========================================================================================>]
5.36M 4.80MB/s in 1.1s
2021-05-13 19:29:18 (4.80 MB/s) - ‘glibc-doc-
reference_2.33-0ubuntu2~zstd1_all.deb’ saved [5618446/5618446]
root at bb-lp1926437:~# python3
Python 3.6.9 (default, Jan 26 2021, 15:33:00)
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import apt_inst
>>> import apt_pkg
>>> apt_pkg.init_config()
>>> apt_inst.DebFile("glibc-doc-reference_2.33-0ubuntu2~zstd1_all.deb").control.extractall()
True
>>>
root at bb-lp1926437:~# dpkg -l python3-apt python-apt-common
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-=====================================-=======================-=======================-================================================================================
ii python-apt-common 1.6.5ubuntu0.6 all Python interface to libapt-pkg (locales)
ii python3-apt 1.6.5ubuntu0.6 amd64 Python 3 interface to libapt-pkg
root at bb-lp1926437:~#
Verified 2.0.0ubuntu0.20.04.5 on Focal.
root at f-lp1926437:~# wget https://people.canonical.com/~rbalint/zstd-debs/glibc-doc-reference_2.33-0ubuntu2~zstd1_all.deb
--2021-05-13 19:56:02-- https://people.canonical.com/~rbalint/zstd-debs/glibc-doc-reference_2.33-0ubuntu2~zstd1_all.deb
Resolving people.canonical.com (people.canonical.com)... 91.189.89.62
Connecting to people.canonical.com (people.canonical.com)|91.189.89.62|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 5618446 (5.4M) [application/x-debian-package]
Saving to: ‘glibc-doc-reference_2.33-0ubuntu2~zstd1_all.deb’
glibc-doc-reference_2.33-0ubuntu2~zstd1_all
100%[=========================================================================================>]
5.36M 4.61MB/s in 1.2s
2021-05-13 19:56:03 (4.61 MB/s) - ‘glibc-doc-
reference_2.33-0ubuntu2~zstd1_all.deb’ saved [5618446/5618446]
root at f-lp1926437:~# python3 -c 'import apt_inst; import apt_pkg; apt_pkg.init_config(); print(apt_inst.DebFile("glibc-doc-reference_2.33-0ubuntu2~zstd1_all.deb").control.extractall())'
True
root at f-lp1926437:~# dpkg -l python3-apt python-apt-common
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-=================-====================-============-========================================
ii python-apt-common 2.0.0ubuntu0.20.04.5 all Python interface to libapt-pkg (locales)
ii python3-apt 2.0.0ubuntu0.20.04.5 amd64 Python 3 interface to libapt-pkg
Verified 2.1.3ubuntu1.4 on Groovy.
root at gg-lp1926437:~# wget https://people.canonical.com/~rbalint/zstd-debs/glibc-doc-reference_2.33-0ubuntu2~zstd1_all.deb
--2021-05-13 19:51:34-- https://people.canonical.com/~rbalint/zstd-debs/glibc-doc-reference_2.33-0ubuntu2~zstd1_all.deb
Resolving people.canonical.com (people.canonical.com)... 91.189.89.62
Connecting to people.canonical.com (people.canonical.com)|91.189.89.62|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 5618446 (5.4M) [application/x-debian-package]
Saving to: ‘glibc-doc-reference_2.33-0ubuntu2~zstd1_all.deb’
glibc-doc-reference_2.33-0ubuntu2~zstd1_all
100%[=========================================================================================>]
5.36M 4.70MB/s in 1.1s
2021-05-13 19:51:36 (4.70 MB/s) - ‘glibc-doc-
reference_2.33-0ubuntu2~zstd1_all.deb’ saved [5618446/5618446]
root at gg-lp1926437:~# python3 -c 'import apt_inst; import apt_pkg; apt_pkg.init_config(); print(apt_inst.DebFile("glibc-doc-reference_2.33-0ubuntu2~zstd1_all.deb").control.extractall())'
True
root at gg-lp1926437:~# dpkg -l python3-apt python-apt-common
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-=================-==============-============-========================================
ii python-apt-common 2.1.3ubuntu1.4 all Python interface to libapt-pkg (locales)
ii python3-apt 2.1.3ubuntu1.4 amd64 Python 3 interface to libapt-pkg
Verified 2.2.0~ubuntu0.21.04.1 on Hirsute.
root at hh-lp1926437:~# wget https://people.canonical.com/~rbalint/zstd-debs/glibc-doc-reference_2.33-0ubuntu2~zstd1_all.deb
--2021-05-13 19:42:59-- https://people.canonical.com/~rbalint/zstd-debs/glibc-doc-reference_2.33-0ubuntu2~zstd1_all.deb
Resolving people.canonical.com (people.canonical.com)... 91.189.89.62
Connecting to people.canonical.com (people.canonical.com)|91.189.89.62|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 5618446 (5.4M) [application/x-debian-package]
Saving to: ‘glibc-doc-reference_2.33-0ubuntu2~zstd1_all.deb’
glibc-doc-reference_2.33-0ubuntu2~zstd1_all
100%[=========================================================================================>]
5.36M 4.78MB/s in 1.1s
2021-05-13 19:43:00 (4.78 MB/s) - ‘glibc-doc-
reference_2.33-0ubuntu2~zstd1_all.deb’ saved [5618446/5618446]
root at hh-lp1926437:~# python3 -c 'import apt_inst; import apt_pkg; apt_pkg.init_config(); print(apt_inst.DebFile("glibc-doc-reference_2.33-0ubuntu2~zstd1_all.deb").control.extractall())'
True
root at hh-lp1926437:~# dpkg -l python3-apt python-apt-common
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-=================-=====================-============-========================================
ii python-apt-common 2.2.0~ubuntu0.21.04.1 all Python interface to libapt-pkg (locales)
ii python3-apt 2.2.0~ubuntu0.21.04.1 amd64 Python 3 interface to libapt-pkg
** Tags removed: verification-needed verification-needed-bionic verification-needed-focal verification-needed-groovy verification-needed-hirsute
** Tags added: verification-done verification-done-bionic verification-done-focal verification-done-groovy verification-done-hirsute
--
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to apt in Ubuntu.
https://bugs.launchpad.net/bugs/1926437
Title:
[SRU] Backport zstd support, fix bug in python-apt
Status in apt package in Ubuntu:
Fix Released
Status in python-apt package in Ubuntu:
Fix Released
Status in apt source package in Xenial:
Fix Released
Status in python-apt source package in Xenial:
Fix Committed
Status in python-apt source package in Bionic:
Fix Committed
Status in python-apt source package in Focal:
Fix Committed
Status in python-apt source package in Groovy:
Fix Committed
Status in python-apt source package in Hirsute:
Fix Committed
Status in python-apt source package in Impish:
Fix Released
Bug description:
[Impact]
APT in xenial needs to learn about zstd support to enable Launchpad to work with zstd packages.
python-apt in all releases needs to be adjusted to pass the compressor
names instead of programs to ExtractTar, as otherwise, if
/usr/bin/zstd does not exist, it will pass "false" as the compressor,
and then ExtractTar fails (same for other compressors).
On hirsute, the python-apt upload also includes some new <!nocheck> annotations for build-dependencies, as it's a straight upload of the 2.2
stable series.
[Test plan]
The apt regression tests cover the compression methods extensively, including even stored compressed lists in /var/lib/apt and fetching Packages files compressed with such methods, for all compression methods.
The final test is to make apt_inst.DebFile("glibc-doc-
reference_2.33-0ubuntu2~zstd1_all.deb").control.extractall() from
1923845 work in python3-apt.
For hirsute, the build dependency changes will verify themselves
during the build ;)
[Where problems could occur]
APT:
The changes to APT are adding the zstd compression backend to FileFd
and then linking it into the configuration. The former is adding new
code, triggered only on zstd files, and should not be able to cause
regressions.
The latter is a bunch of small lines here and there, and it caused
issues during backporting in the test suite because it expected
compressors in a specific order. However, chances are that nothing
else has as stringent requirements as the test suite, so I don't
believe that is a problem for existing users.
Bugs can of course be in the code dealing with zstd, and that might
open you up to security vulnerabilities in the zstd code if you pass
APT zstd compressed files.
A regression can be if clients used apt-helper cat-file to cat a zstd-
compressed file ending in .zst - they'll now get uncompressed content,
rather than compressed.
python-apt:
The change just replaces Binary with Name, aligning the code with APT
itself, so it's so trivial, we don't expect regressions.
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/apt/+bug/1926437/+subscriptions
More information about the foundations-bugs
mailing list