[Bug 1763179] Re: Returned strings should not be forced into unicode

OpenStack Infra 1763179 at bugs.launchpad.net
Mon Apr 30 16:40:33 UTC 2018 

Reviewed:  https://review.openstack.org/561928
Committed: https://git.openstack.org/cgit/openstack/oslo.versionedobjects/commit/?id=c4e7defc88a79952c93b151310688f6cf39d3e6d
Submitter: Zuul
Branch:    stable/pike

commit c4e7defc88a79952c93b151310688f6cf39d3e6d
Author: Sean McGinnis <sean.mcginnis at gmail.com>
Date:   Wed Apr 11 21:02:53 2018 +0000

    Don't force unicode strings for UUID coercion
    
    Change Ic6b6308fb1960ec40407e6efde30137b64543e72 attempts to
    fix difference between Python 2 and Python 3 values by switching
    from using str() to formatting into a unicode string (u"%s").
    
    This is equivalent to changing str() to unicode(), but that is
    not correct for expected default string types for Python 2.
    This requires either using six.text_type(), or just formatting
    into a string, without forcing unicode ("%s"), to be correct on
    either runtime.
    
    Change-Id: I178f14cdc670d7a696778891e587ef75de208fc2
    Closes-bug: #1763179
    (cherry picked from commit b719764ba85d5e689d8276ad40c82985c84f03f9)


** Changed in: cloud-archive/pike
       Status: Triaged => Fix Committed

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to python-oslo.versionedobjects in
Ubuntu.
https://bugs.launchpad.net/bugs/1763179

Title:
  Returned strings should not be forced into unicode

Status in Ubuntu Cloud Archive:
  Fix Released
Status in Ubuntu Cloud Archive mitaka series:
  Triaged
Status in Ubuntu Cloud Archive ocata series:
  Fix Committed
Status in Ubuntu Cloud Archive pike series:
  Fix Committed
Status in Ubuntu Cloud Archive queens series:
  Fix Released
Status in oslo.versionedobjects:
  Fix Released
Status in python-oslo.versionedobjects package in Ubuntu:
  Fix Released
Status in python-oslo.versionedobjects source package in Xenial:
  Triaged
Status in python-oslo.versionedobjects source package in Artful:
  Triaged
Status in python-oslo.versionedobjects source package in Bionic:
  Fix Released

Bug description:
  [Impact]

  In Kilo, with ldap integrated, user_id can have non uuid format ( e.g. Gāo )
  Then after upgrading kilo to mitaka ( in trusty env )

  Gāo user can't create cinder volume with unicode error.

  Affected Xenial, UCA Mitaka ( this is modified after i re-testing all
  )

  Please refer to error code like below [1]

  [Test Case]

  1. deploy openstack env
  2. source novarc ( as admin )
  3. keystone user-create --name test
  4. openstack user set --password-prompt test
  4. log into keystone node and install mysql-client
  5. connect to mysql with keystone user
  6. do below query
  - set foreign_key_checks = 0;
  - update local_user set user_id='Gāo' where user_id='UUID';
  - update user set id='Gāo' where id='UUID';
  - set foreign_key_checks = 1;
  7. keystone user-role-add --user test --tenant admin --role Admin
  8. source novarc ( as test )
  9. cinder create --name test 1

  got unicode error on cinder-api node logs

  [Regression Potential]
  This patch make code handles unicode non uuid form user id. Basically this has little risk. but cinder-api service need to be restarted.

  [Others]

  [1] ###########

  2018-03-14 10:58:56.932 2198 ERROR cinder.api.middleware.fault File "/usr/lib/python2.7/dist-packages/taskflow/types/failure.py", line 343, in reraise
  2018-03-14 10:58:56.932 2198 ERROR cinder.api.middleware.fault six.reraise(*self._exc_info)
  2018-03-14 10:58:56.932 2198 ERROR cinder.api.middleware.fault File "/usr/lib/python2.7/dist-packages/taskflow/engines/action_engine/executor.py", line 82, in _execute_task
  2018-03-14 10:58:56.932 2198 ERROR cinder.api.middleware.fault result = task.execute(**arguments)
  2018-03-14 10:58:56.932 2198 ERROR cinder.api.middleware.fault File "/usr/lib/python2.7/dist-packages/cinder/volume/flows/api/create_volume.py", line 527, in execute
  2018-03-14 10:58:56.932 2198 ERROR cinder.api.middleware.fault volume = objects.Volume(context=context, **volume_properties)
  2018-03-14 10:58:56.932 2198 ERROR cinder.api.middleware.fault File "/usr/lib/python2.7/dist-packages/cinder/objects/volume.py", line 180, in __init__
  2018-03-14 10:58:56.932 2198 ERROR cinder.api.middleware.fault super(Volume, self).__init__(*args, **kwargs)
  2018-03-14 10:58:56.932 2198 ERROR cinder.api.middleware.fault File "/usr/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 304, in __init__
  2018-03-14 10:58:56.932 2198 ERROR cinder.api.middleware.fault setattr(self, key, kwargs[key])
  2018-03-14 10:58:56.932 2198 ERROR cinder.api.middleware.fault File "/usr/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 72, in setter
  2018-03-14 10:58:56.932 2198 ERROR cinder.api.middleware.fault field_value = field.coerce(self, name, value)
  2018-03-14 10:58:56.932 2198 ERROR cinder.api.middleware.fault File "/usr/lib/python2.7/dist-packages/oslo_versionedobjects/fields.py", line 192, in coerce
  2018-03-14 10:58:56.932 2198 ERROR cinder.api.middleware.fault return self._type.coerce(obj, attr, value)
  2018-03-14 10:58:56.932 2198 ERROR cinder.api.middleware.fault File "/usr/lib/python2.7/dist-packages/oslo_versionedobjects/fields.py", line 325, in coerce
  2018-03-14 10:58:56.932 2198 ERROR cinder.api.middleware.fault "for further details" % value, FutureWarning)
  2018-03-14 10:58:56.932 2198 ERROR cinder.api.middleware.fault File "/usr/lib/python2.7/logging/__init__.py", line 1709, in _showwarning
  2018-03-14 10:58:56.932 2198 ERROR cinder.api.middleware.fault s = warnings.formatwarning(message, category, filename, lineno, line)
  2018-03-14 10:58:56.932 2198 ERROR cinder.api.middleware.fault File "/usr/lib/python2.7/warnings.py", line 38, in formatwarning
  2018-03-14 10:58:56.932 2198 ERROR cinder.api.middleware.fault s = "%s:%s: %s: %s\n" % (filename, lineno, category.__name__, message)
  2018-03-14 10:58:56.932 2198 ERROR cinder.api.middleware.fault UnicodeEncodeError: 'ascii' codec can't encode character u'\xe1' in position 9: ordinal not in range(128)

  
  Original Description
  --------------------
  Change https://review.openstack.org/#/c/557778/ changed the return value for coerce() to explicitly force the value to be unicode. This causes problems with Python 2 runtimes, where the expected value of a string is no longer a string but now unicode (see: https://github.com/openstack/cinder/blob/b89e73a2ac315927d2eee42129597b14e87ba685/cinder/tests/unit/volume/drivers/dell_emc/vmax/test_vmax.py#L3412)

  The correct fix should be to use either six.text_type to convert it,
  or format it into a string as is being done, but without forcing it to
  be a unicode string. "%s" will end up being the correct type depending
  on whether it is running under py2 or py3.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/1763179/+subscriptions

More information about the Ubuntu-openstack-bugs mailing list