[Bug 1805332] Re: [Swift backend] Upload image hit error: Unicode-objects must be encoded before hashing
Liam Young
liam.young at canonical.com
Wed May 15 16:03:40 UTC 2019
Looks good to me. Tested 0.28.0-0ubuntu1.1~cloud0 from cloud-archive
:stein-proposed
$ openstack image create --public --file /home/ubuntu/images/bionic-server-cloudimg-amd64.img bionic-test
500 Internal Server Error: The server has either erred or is incapable of performing the requested operation. (HTTP 500)
$ juju run --unit glance/0 "add-apt-repository cloud-archive:stein-proposed --yes --update"
Reading package lists...
Building dependency tree...
Reading state information...
ubuntu-cloud-keyring is already the newest version (2018.09.18.1~18.04.0).
The following package was automatically installed and is no longer required:
grub-pc-bin
Use 'apt autoremove' to remove it.
0 upgraded, 0 newly installed, 0 to remove and 17 not upgraded.
Get:1 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB]
Hit:2 http://nova.clouds.archive.ubuntu.com/ubuntu bionic InRelease
Get:3 http://nova.clouds.archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB]
Ign:4 http://ubuntu-cloud.archive.canonical.com/ubuntu bionic-updates/stein InRelease
Ign:5 http://ubuntu-cloud.archive.canonical.com/ubuntu bionic-proposed/stein InRelease
Get:6 http://nova.clouds.archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB]
Get:7 http://ubuntu-cloud.archive.canonical.com/ubuntu bionic-updates/stein Release [7882 B]
Get:8 http://ubuntu-cloud.archive.canonical.com/ubuntu bionic-proposed/stein Release [7884 B]
Get:9 http://ubuntu-cloud.archive.canonical.com/ubuntu bionic-updates/stein Release.gpg [543 B]
Get:10 http://ubuntu-cloud.archive.canonical.com/ubuntu bionic-proposed/stein Release.gpg [543 B]
Get:11 http://ubuntu-cloud.archive.canonical.com/ubuntu bionic-proposed/stein/main amd64 Packages [179 kB]
Fetched 448 kB in 1s (358 kB/s)
Reading package lists...
$ juju run --unit glance/0 "apt install --yes python3-glance-store; systemctl restart glance-api"
Reading package lists...
Building dependency tree...
Reading state information...
The following package was automatically installed and is no longer required:
grub-pc-bin
Use 'apt autoremove' to remove it.
The following additional packages will be installed:
glance-store-common
The following packages will be upgraded:
glance-store-common python3-glance-store
2 upgraded, 0 newly installed, 0 to remove and 17 not upgraded.
Need to get 102 kB of archives.
After this operation, 3072 B of additional disk space will be used.
Get:1 http://ubuntu-cloud.archive.canonical.com/ubuntu bionic-proposed/stein/main amd64 python3-glance-store all 0.28.0-0ubuntu1.1~cloud0 [95.9 kB]
Get:2 http://ubuntu-cloud.archive.canonical.com/ubuntu bionic-proposed/stein/main amd64 glance-store-common all 0.28.0-0ubuntu1.1~cloud0 [6320 B]
Fetched 102 kB in 1s (190 kB/s)
(Reading database ... 77276 files and directories currently installed.)
Preparing to unpack .../python3-glance-store_0.28.0-0ubuntu1.1~cloud0_all.deb ...
Unpacking python3-glance-store (0.28.0-0ubuntu1.1~cloud0) over (0.28.0-0ubuntu1~cloud0) ...
Preparing to unpack .../glance-store-common_0.28.0-0ubuntu1.1~cloud0_all.deb ...
Unpacking glance-store-common (0.28.0-0ubuntu1.1~cloud0) over (0.28.0-0ubuntu1~cloud0) ...
Setting up glance-store-common (0.28.0-0ubuntu1.1~cloud0) ...
Setting up python3-glance-store (0.28.0-0ubuntu1.1~cloud0) ...
WARNING: apt does not have a stable CLI interface. Use with caution in
scripts.
$ openstack image create --public --file /home/ubuntu/images/bionic-server-cloudimg-amd64.img bionic-test
+------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| checksum | c8994590c7d61dc68922e461686ef936 |
| container_format | bare |
| created_at | 2019-05-15T16:00:48Z |
| disk_format | raw |
| file | /v2/images/52320ad5-353b-4da6-a876-9f054535155f/file |
| id | 52320ad5-353b-4da6-a876-9f054535155f |
| min_disk | 0 |
| min_ram | 0 |
| name | bionic-test |
| owner | 21dc90c52e3347bbb4441307c699904a |
| properties | os_hash_algo='sha512', os_hash_value='be4993640deb7eb99b07667213b1fe3a9145df2c0ed5c72cf786a621fe64e93fb543cbb3fafa9a130988b684da432d2a55493c50e77a9dfe336e7ed996be92d9', os_hidden='False' |
| protected | False |
| schema | /v2/schemas/image |
| size | 348127232 |
| status | active |
| tags | |
| updated_at | 2019-05-15T16:00:56Z |
| virtual_size | None |
| visibility | public |
+------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
** Tags removed: verification-stein-needed
** Tags added: verification-stein-done
--
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to Ubuntu Cloud Archive.
https://bugs.launchpad.net/bugs/1805332
Title:
[Swift backend] Upload image hit error: Unicode-objects must be
encoded before hashing
Status in Ubuntu Cloud Archive:
Fix Committed
Status in Ubuntu Cloud Archive stein series:
Fix Committed
Status in glance_store:
Triaged
Status in python-glance-store package in Ubuntu:
Fix Released
Status in python-glance-store source package in Disco:
Fix Committed
Status in python-glance-store source package in Eoan:
Fix Released
Bug description:
[Impact]
If we upload a large image (larger than 1G), the glance_store will hit a Unicode error. To fix this a patch has been merged in upstream master and backported to stable rocky.
[Test Case]
Deploy glance related to swift-proxy using the object-store relation. Then attempt to upload a large image (not cirros)
$ openstack image create --public --file /home/ubuntu/images/bionic-server-cloudimg-amd64.img bionic-test
500 Internal Server Error: The server has either erred or is incapable of performing the requested operation. (HTTP 500)
If the patch is manually applied and glance-api restarted then the
above command succeeds.
In order to avoid regression of existing consumers, the OpenStack team will
run their continuous integration test against the packages that are in
-proposed. A successful run of all available tests will be required before the
proposed packages can be let into -updates.
The OpenStack team will be in charge of attaching the output summary of the
executed tests. The OpenStack team members will not mark ‘verification-done’ until
this has happened.
[Regression Potential]
In order to mitigate the regression potential, the results of the
aforementioned tests are attached to this bug.
[Discussion]
n/a
[Original Description]
env: master branch, Glance using swift backend.
We hit a strange error， if we upload a large image (larger than 1G),
the glance_store will hit a error:Unicode-objects must be encoded
before hashing. But if the image is small enough, the error won't
happen.
error log:
https://www.irccloud.com/pastebin/jP3DapNy/
After dig into the code, it appears that when chunk reading the image
data, the date piece may be non-byte, so the checksum.updating will
raise the error.
encoding the date piece to ensure it's byte can solve the problem.
To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/1805332/+subscriptions
More information about the Ubuntu-openstack-bugs
mailing list