[Bug 1694261] Re: Crashes with UnicodeDecodeError when trying to handle paths with non-ascii chars

[Robie Basak]

10:53 <mapreri> rbasak: besides that I can't see why anybody would want
to limit its filesystem to ASCII-only (which, btw, would be a purely
artificial limitation I can't even think where to set), reading it as
UTF-8 would Just Work.

10:57 <mapreri> Besides, I'm not convinced that I should be setting
LANG=C.UTF-8 anywhere to just open a file that happens to have a non-
ASCII filename.

10:57 <mapreri> and yes, setting LANG=C.UTF-8 would workaround the bug,
but it's very much not the fix here.

10:58 <mapreri> (i.e. it worked before, it works after with the new
upstream, it just doesn't work in-between…)

12:49 <rbasak> mapreri: would setting LANG correctly be a bug
workaround, or would it be fixing the root cause? Do you know how it is
that your system came to have LANG be unset?

12:49 <rbasak> mapreri: I'm concerned because the patch introduces a
behaviour change, and some users may have problems with that in a way I
can't necessarily predict.

12:50 <rbasak> mapreri: on the other hand, if you're impacted (I think
by having a misconfigured server in the first place), then you can fix
the behaviour on your own server without having something forcibly
pushed out to everyone else.

12:51 <rbasak> mapreri: if you're concerned about doing it globally, you
could arrange to set LANG just for that process even.

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to python-werkzeug in Ubuntu.
https://bugs.launchpad.net/bugs/1694261

Title:
  Crashes with UnicodeDecodeError when trying to handle paths with non-
  ascii chars

Status in python-werkzeug package in Ubuntu:
  Fix Released
Status in python-werkzeug source package in Trusty:
  In Progress
Status in python-werkzeug source package in Xenial:
  In Progress

Bug description:
  [Impact]

   * I've discovered this in a MoinMoin instance I admin, where there
  are pages with non-ascii character (i.e. àèéìòù), there trying to open
  a page with said name yields

  [:info] mod_wsgi (pid=29522, process='', application='wiki.ubuntu-it.org:8801|'): Loading WSGI script '/srv/wiki.ubuntu-it.org/www/moin.wsgi'.
  [:error] mod_wsgi (pid=29522): Exception occurred processing WSGI script '/srv/wiki.ubuntu-it.org/www/moin.wsgi'.
  [:error] Traceback (most recent call last):
  [:error]   File "/srv/wiki.ubuntu-it.org/www/moin.wsgi", line 71, in __call__
  [:error]     return self.app(environ, start_response)
  [:error]   File "/usr/lib/python2.7/dist-packages/werkzeug/wsgi.py", line 558, in __call__
  [:error]     cleaned_path = cleaned_path.encode(sys.getfilesystemencoding())
  [:error] UnicodeEncodeError: 'ascii' codec can't encode character u'\\xe0' in position 37: ordinal not in range(128)

  [Test Case]

   * https://bugs.launchpad.net/ubuntu/+source/python-
  werkzeug/+bug/1694261/+attachment/4886272/+files/test.py

  [Regression Potential]

   * The patch only changes all sys.getfilesystemencoding() calls with a local function, wrapping the same original call.  The overall changeset seems low on regression potential.
   * I've installed the proposed package in the wiki.ubuntu-it.org server and it is working :)

  [Other Info]

   * This is upstream bug https://github.com/pallets/werkzeug/issues/635
  - fixed with upstream commit
  https://github.com/pallets/werkzeug/commit/bba0cdcc67d4a1160d4ed9d3f99aef170a79dd88
  and released in version 0.11 and present in yakkety+.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/python-werkzeug/+bug/1694261/+subscriptions