[Bug 1243065] Re: Invalid encoding returned for ca at valencia locale

[David Planella]

I think the issue here is that while the fix for the upstream bug was
committed [1] on the 19th Dec 2013, there hasn't been an upstream 2.7
point release that includes it, and thus that it can be packaged for
Ubuntu.

If I'm not mistaken, the latest upstream point release (2.7.6) was on
the 10th of November [2]. While it hasn't yet made it to Ubuntu (we've
got 2.7.5, and 2.7.6 might not qualify for a Stable Release Upload), the
main point is that the latest upstream release was made about one month
before the fix was committed, thus there is no release from the 2.7.x
series that includes it.

I think there is not an easy fix for Ubuntu 13.10 (backporting the fix
might will most probably not qualify as SRU, and porting ubuntu-release-
upgrader to Python 3.3, which might have the fix in Ubuntu 13.10 is
probably not feasible for a stable release).

So in summary, this will probably be a "Won't fix" for Ubuntu 13.10. Not
sure what to do with 14.04, as it might have exactly the same issue and
we'll have the same constraints now that it has been released.

[1] http://hg.python.org/cpython/rev/3d805bee06e2
[2] https://www.python.org/download/releases/2.7.6
[3] https://launchpad.net/ubuntu/saucy/+source/python2.7/+changelog

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to ubuntu-release-upgrader in
Ubuntu.
https://bugs.launchpad.net/bugs/1243065

Title:
  Invalid encoding returned for ca at valencia locale

Status in Python:
  Confirmed
Status in “ubuntu-release-upgrader” package in Ubuntu:
  Triaged

Bug description:
  While upgrading from 13.04 to 13.10 using locale ca at valencia, the
  process fails with the following traceback:

  Traceback (most recent call last):

  File "/tmp/ubuntu-release-upgrader-vh1jgw/saucy", line 10, in 
  <module> 
  sys.exit(main()) 

  File 
  "/tmp/ubuntu-release-upgrader-vh1jgw/DistUpgrade/DistUpgradeMain.py", 
  line 243, in main 
  if app.run(): 

  File 
  "/tmp/ubuntu-release-upgrader-vh1jgw/DistUpgrade/DistUpgradeController.py", 
  line 1791, in run 
  return self.fullUpgrade() 

  File 
  "/tmp/ubuntu-release-upgrader-vh1jgw/DistUpgrade/DistUpgradeController.py", 
  line 1744, in fullUpgrade 
  if not self.askDistUpgrade(): 

  File 
  "/tmp/ubuntu-release-upgrader-vh1jgw/DistUpgrade/DistUpgradeController.py", 
  line 996, in askDistUpgrade 
  self.cache.required_download) 

  File 
  "/tmp/ubuntu-release-upgrader-vh1jgw/DistUpgrade/DistUpgradeViewText.py", 
  line 165, in confirmChanges 
  downloadSize, actions) 

  File 
  "/tmp/ubuntu-release-upgrader-vh1jgw/DistUpgrade/DistUpgradeView.py", 
  line 387, in confirmChanges 
  downloadSizeStr = downloadSizeStr.decode(ENCODING) 

  LookupError: unknown encoding: utf_8_valencia

  Looking at it in more detail, this comes from the following code in
  DistUpgrade/DistUpgradeView.py [1]

  try:
      locale.setlocale(locale.LC_ALL, "")
      (code, ENCODING) = locale.getdefaultlocale()
  except:
      logging.exception("getting the encoding failed")
      ENCODING = "utf-8"   #pyflakes

  if not ENCODING:
      ENCODING = "utf-8"

  And digging deeper, it seems getdefaultlocale does not return the
  correct encoding when the locale is set to ca_ES

  >>> import locale
  >>> locale.setlocale(locale.LC_ALL, "")
  'LC_CTYPE=ca_ES.UTF-8 at valencia;LC_NUMERIC=ca_ES.UTF-8;LC_TIME=ca_ES.UTF-8;LC_COLLATE=ca_ES.UTF-8 at valencia;LC_MONETARY=ca_ES.UTF-8;LC_MESSAGES=ca_ES.UTF-8 at valencia;LC_PAPER=ca_ES.UTF-8;LC_NAME=ca_ES.UTF-8;LC_ADDRESS=ca_ES.UTF-8;LC_TELEPHONE=ca_ES.UTF-8;LC_MEASUREMENT=ca_ES.UTF-8;LC_IDENTIFICATION=ca_ES.UTF-8'
  >>> locale.getdefaultlocale()
  ('ca_ES', 'utf_8_valencia')

  Instead of 'utf_8_valencia', here I'd expect 'UTF-8' to be returned.
  The returned locale is also wrong: it should be ca_ES at valencia.

  While this needs to be fixed upstream, it might be worth adding a
  workaround (and a check for a valid encoding) in ubuntu-release-
  upgrader in the meantime.

  This will also affect other @ locales, such as sr at Latin (Serbian with
  Latin encoding).

  [1] http://bazaar.launchpad.net/~ubuntu-core-dev/ubuntu-release-
  upgrader/trunk/view/head:/DistUpgrade/DistUpgradeView.py#L41

To manage notifications about this bug go to:
https://bugs.launchpad.net/python/+bug/1243065/+subscriptions