[Bug 1985964] Re: UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 1: invalid start byte

Mon Dec 5 19:38:18 UTC 2022

The autopkgtest failure was resolved with a retry.

I verified the fix using ubuntu-release-upgrader 1:22.04.15 from jammy-
proposed:

nr at clean-focal-amd64:~$ do-release-upgrade --proposed
Checking for a new Ubuntu release
Get:1 Upgrade tool signature [819 B]                                           
Get:2 Upgrade tool [1,267 kB]                                                  
Fetched 1,267 kB in 0s (0 B/s)                                                 
authenticate 'jammy.tar.gz' against 'jammy.tar.gz.gpg' 
extracting 'jammy.tar.gz'

[ ... ]

 Continue [yN]  Details [d]N   

Restoring original system state

Aborting
Reading package lists... Done    
Building dependency tree          
Reading state information... Done
nr at clean-focal-amd64:~$ sudo vim /tmp/ubuntu-release-upgrader-wah0wx3w/DistUpgradeViewText.py 
nr at clean-focal-amd64:~$ grep -A 10 'def confirmChanges' /tmp/ubuntu-release-upgrader-wah0wx3w/DistUpgradeViewText.py 
    def confirmChanges(self, summary, changes, demotions, downloadSize,
                       actions=None, removal_bold=True):
      DistUpgradeView.confirmChanges(self, summary, changes, demotions, 
                                     downloadSize, actions)
      print(2/0) # Cause an exception
      print()
      print(twrap(summary))
      print(twrap(self.confirmChangesMessage))
      print(" %s %s" % (_("Continue [yN] "), _("Details [d]")), end="")
      while True:
        res = readline().strip().lower()
nr at clean-focal-amd64:~$ cd /tmp/ubuntu-release-upgrader-wah0wx3w/
nr at clean-focal-amd64:/tmp/ubuntu-release-upgrader-wah0wx3w$ sudo -E ./jammy --frontend=DistUpgradeViewText

Reading cache

Checking package manager
Reading package lists... Done
Building dependency tree        
Reading state information... Done
Hit http://archive.ubuntu.com/ubuntu focal InRelease                           
Hit http://security.ubuntu.com/ubuntu focal-security InRelease                 
Hit http://archive.ubuntu.com/ubuntu focal-updates InRelease                   
Fetched 0 B in 0s (0 B/s)                                                      
Reading package lists... Done    
Building dependency tree          
Reading state information... Done
None

Checking for installed snaps

Calculating snap size requirements

Updating repository information
Hit http://security.ubuntu.com/ubuntu jammy-security InRelease                 
Hit http://archive.ubuntu.com/ubuntu jammy InRelease                           
Hit http://archive.ubuntu.com/ubuntu jammy-updates InRelease                   
Fetched 0 B in 0s (0 B/s)                                                      

Checking package manager
Reading package lists... Done    
Building dependency tree          
Reading state information... Done

Calculating the changes

Calculating the changes

Traceback (most recent call last):
  File "./jammy", line 8, in <module>
    sys.exit(main())
  File "/tmp/ubuntu-release-upgrader-wah0wx3w/DistUpgrade/DistUpgradeMain.py", line 241, in main
    if app.run():
  File "/tmp/ubuntu-release-upgrader-wah0wx3w/DistUpgrade/DistUpgradeController.py", line 2042, in run
    return self.fullUpgrade()
  File "/tmp/ubuntu-release-upgrader-wah0wx3w/DistUpgrade/DistUpgradeController.py", line 1939, in fullUpgrade
    if not self.askDistUpgrade():
  File "/tmp/ubuntu-release-upgrader-wah0wx3w/DistUpgrade/DistUpgradeController.py", line 1139, in askDistUpgrade
    res = self._view.confirmChanges(_("Do you want to start the upgrade?"),
  File "/tmp/ubuntu-release-upgrader-wah0wx3w/DistUpgrade/DistUpgradeViewText.py", line 212, in confirmChanges
    print(2/0)
ZeroDivisionError: division by zero

After this, apport prompted me to report a bug as expected.

** Tags removed: verification-needed verification-needed-jammy
** Tags added: verification-done verification-done-jammy

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to ubuntu-release-upgrader in
Ubuntu.
https://bugs.launchpad.net/bugs/1985964

Title:
  UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position
  1: invalid start byte

Status in ubuntu-release-upgrader package in Ubuntu:
  Fix Released
Status in ubuntu-release-upgrader source package in Jammy:
  Fix Committed

Bug description:
  [Impact]

  Crashes in ubuntu-release-upgrader cannot be properly reported with
  apport.

  [Test plan]

  Seeing this issue requires causing another exception to be raised
  first. I do this like so:

  * Start an upgrade, but say no when prompted to start the upgrade, after the tarball has been downloaded.
  * Navigate to the tempdir where the tarball was unpacked, e.g. /tmp/ubuntu-release-upgrader-04y_8xfj
  * Edit DistUpgradeViewText.py, and add a divide-by-zero in `confirmChanges`
  * Observe that instead of an apport crash report being generated, there is an exception trying to do so:

  Error in sys.excepthook:
  Traceback (most recent call last):
    File "/usr/lib/python3/dist-packages/problem_report.py", line 477, in add_to_existing
      self.write(f)
    File "/usr/lib/python3/dist-packages/problem_report.py", line 430, in write
      block = f.read(1048576)
    File "/usr/lib/python3.8/codecs.py", line 322, in decode
      (result, consumed) = self._buffer_decode(data, self.errors, final)
  UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 1: invalid start byte

  [Where problems could occur]
  The patch makes it so that u-r-u passes filenames, instead of open files, to apport to generate the report. Regressions would be seen in the generation of reports, or the format of the resulting report.

  [Original Description]

  While attempting to do-release-upgrade from 20.04 to 22.04, the
  upgrader crashed, and then the problem report script also crashed,
  leaving behind a corrupted crash report in /var/crash. This bug is
  about the problem report crash. I will report the upgrader crash
  separately.

  Here are the last few lines of output:

  Running mktexlsr /var/lib/texmf ... done.
  Building format(s) --all.
          This may take some time... done.
  Processing triggers for dbus (1.12.20-2ubuntu4) ...
  Processing triggers for libvlc-bin:amd64 (3.0.16-1build7) ...
  Processing triggers for libgdk-pixbuf-2.0-0:amd64 (2.42.8+dfsg-1) ...
  Processing triggers for libgdk-pixbuf-2.0-0:i386 (2.42.8+dfsg-1) ...
  Processing triggers for rygel (0.40.3-1ubuntu2) ...
  Reading package lists... Done
  Building dependency tree
  Reading state information... Done

  Calculating the changes

  Calculating the changes

  Traceback (most recent call last):
    File "/tmp/ubuntu-release-upgrader-ksd7z8sv/jammy", line 8, in <module>
      sys.exit(main())
    File "/tmp/ubuntu-release-upgrader-ksd7z8sv/DistUpgrade/DistUpgradeMain.py", line 241, in main
      if app.run():
    File "/tmp/ubuntu-release-upgrader-ksd7z8sv/DistUpgrade/DistUpgradeController.py", line 2042, in run
      return self.fullUpgrade()
    File "/tmp/ubuntu-release-upgrader-ksd7z8sv/DistUpgrade/DistUpgradeController.py", line 1991, in fullUpgrade
      if not self.calcDistUpgrade():
    File "/tmp/ubuntu-release-upgrader-ksd7z8sv/DistUpgrade/DistUpgradeController.py", line 1100, in calcDistUpgrade
      if not self.cache.installTasks(self.tasks):
  AttributeError: 'DistUpgradeController' object has no attribute 'tasks'
  Error in sys.excepthook:
  Traceback (most recent call last):
    File "/usr/lib/python3/dist-packages/problem_report.py", line 477, in add_to_existing
      self.write(f)
    File "/usr/lib/python3/dist-packages/problem_report.py", line 430, in write
      block = f.read(1048576)
    File "/usr/lib/python3.8/codecs.py", line 322, in decode
      (result, consumed) = self._buffer_decode(data, self.errors, final)
  UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 1: invalid start byte

  Original exception was:
  Traceback (most recent call last):
    File "/tmp/ubuntu-release-upgrader-ksd7z8sv/jammy", line 8, in <module>
      sys.exit(main())
    File "/tmp/ubuntu-release-upgrader-ksd7z8sv/DistUpgrade/DistUpgradeMain.py", line 241, in main
      if app.run():
    File "/tmp/ubuntu-release-upgrader-ksd7z8sv/DistUpgrade/DistUpgradeController.py", line 2042, in run
      return self.fullUpgrade()
    File "/tmp/ubuntu-release-upgrader-ksd7z8sv/DistUpgrade/DistUpgradeController.py", line 1991, in fullUpgrade
      if not self.calcDistUpgrade():
    File "/tmp/ubuntu-release-upgrader-ksd7z8sv/DistUpgrade/DistUpgradeController.py", line 1100, in calcDistUpgrade
      if not self.cache.installTasks(self.tasks):
  AttributeError: 'DistUpgradeController' object has no attribute 'tasks'

  ProblemType: Bug
  DistroRelease: Ubuntu 20.04
  Package: apport 2.20.11-0ubuntu27.24
  ProcVersionSignature: Ubuntu 5.4.0-124.140-generic 5.4.195
  Uname: Linux 5.4.0-124-generic x86_64
  NonfreeKernelModules: nvidia_modeset nvidia
  ApportVersion: 2.20.11-0ubuntu27.24
  Architecture: amd64
  CasperMD5CheckResult: skip
  CrashReports:
   644:0:119:0:2022-08-12 11:36:12.283871223 +0100:2022-08-12 11:36:12.283871223 +0100:/var/crash/_usr_bin_do-release-upgrade.0.upload
   640:0:119:15237203:2022-08-12 11:36:10.283871173 +0100:2022-08-12 11:36:44.453251015 +0100:/var/crash/_usr_bin_do-release-upgrade.0.crash
   600:112:119:5:2022-08-12 11:36:12.355865391 +0100:2022-08-12 11:36:12.355865391 +0100:/var/crash/_usr_bin_do-release-upgrade.0.uploaded
   640:1000:119:2093621:2022-08-12 11:33:46.059253454 +0100:2022-08-12 11:33:47.059253454 +0100:/var/crash/_usr_lib_x86_64-linux-gnu_xfce4_panel_wrapper-2.0.1000.crash
  CurrentDesktop: XFCE
  Date: Fri Aug 12 11:37:21 2022
  InstallationDate: Installed on 2018-05-07 (1557 days ago)
  InstallationMedia: Xubuntu 18.04 LTS "Bionic Beaver" - Release amd64 (20180426)
  PackageArchitecture: all
  SourcePackage: apport
  UpgradeStatus: Upgraded to focal on 2022-08-12 (0 days ago)

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/ubuntu-release-upgrader/+bug/1985964/+subscriptions