[Bug 2131790] Re: RGW - etag not returned on multipart upload

[Matthew Ruffell]

Sponsorship notes:

Modified file names to include patch number in their name:

    - d/p/lp2131790-1-publish-commit-with-etag.patch
    - d/p/lp2131790-2-CompleteMultipart-uses-s-object-instead.patch
    - d/p/lp2131790-3-CompleteMultipart-uses-s-object-for.patch
    - d/p/lp2131790-4-move-publish-complete-back-to.patch
    
Shortened file names as they were also too long.

Each patch had the origin tag set to a pull request, instead of the actual
commit. I changed each patch to the actual commit after cloning the upstream
source and finding the actual upstream commit hash:

old:
Origin: backport, https://github.com/ceph/ceph/pull/56453/changes/e6708c3ccc1436042c612b0507e2a881b5afd2bf
new:
Origin: backport, https://github.com/ceph/ceph/commit/18c202c979aefe7c34de3688036586a070addff8

I checked each commit, and all of them are present in v19.2.3 tag, so good to
go for N,Q,R.

I combined with the changes from bug 2064717.

-updates Build:
https://launchpad.net/~mruffell/+archive/ubuntu/lp2131790-updates
-proposed Build:
https://launchpad.net/~mruffell/+archive/ubuntu/lp2131790-proposed

Suggested final debdiff attached.

Luciano, can I please get a +1 on these changes from the Ceph Team?

** Patch added: "Suggested debdiff for sponsoring"
   https://bugs.launchpad.net/ubuntu/+source/ceph/+bug/2131790/+attachment/5962107/+files/lp2131790-jammy.debdiff

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to ceph in Ubuntu.
https://bugs.launchpad.net/bugs/2131790

Title:
  RGW - etag not returned on multipart upload

Status in Ubuntu Cloud Archive:
  Fix Released
Status in Ubuntu Cloud Archive yoga series:
  New
Status in ceph package in Ubuntu:
  Fix Released
Status in ceph source package in Jammy:
  In Progress
Status in ceph source package in Noble:
  Fix Released

Bug description:
  Thank you @pponnuvel for the original description!

  [ Impact ]
  Currently in ceph 17.2.9, when doing multipart uploads, the etag of the multipart upload is not returned. This causes issues with some applications that rely on ceph and the etag which now have to calculate the etag manually causing performance issues.
  The issue was fixed upstream through https://github.com/ceph/ceph/pull/56453.
  However this commit caused regressions and three other commits that are part of this PR are also needed to resolve them: https://github.com/ceph/ceph/pull/57257

  A total of 4 commits are needed to resolve the issue.

  [ Test Plan ]
  1- Deploy ceph 17.2.9
  2- Use the awscli tool to upload a file large enough that it would be sent as part of a multipart upload. Make sure to use the "--debug" argument to verify if the etag is being returned or not:
  aws s3 --profile ceph --endpoint-url <ceph URL> cp ./test.txt s3://test-bucket/ --debug

  Currently the following output is seen towards the end of the debug
  log:

  2026-04-14 22:47:57,150 - ThreadPoolExecutor-0_3 - botocore.parsers - DEBUG - Response body:
  b'<?xml version="1.0" encoding="UTF-8"?><CompleteMultipartUploadResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Location>http://10.159.7.70/test-bucket/test.txt</Location><Bucket>test-bucket</Bucket><Key>test.txt</Key><ETag></ETag></CompleteMultipartUploadResult>'

  Where the <ETag></ETag> clause is empty.

  The output should look like the following:

  2026-04-14 22:47:57,150 - ThreadPoolExecutor-0_3 - botocore.parsers - DEBUG - Response body:
  b'<?xml version="1.0" encoding="UTF-8"?><CompleteMultipartUploadResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Location>http://10.159.7.70/test-bucket/test.txt</Location><Bucket>test-bucket</Bucket><Key>test.txt</Key><ETag>9609801fffde7f91a34c9686065d9785-2</ETag></CompleteMultipartUploadResult>'

  With the <ETag> clause being populated.

  [ Where problems could occur ]

  * Since the fix touches on the upload logic of ceph, there is a chance that issues related to uploading data to ceph fails.
  * ETag might still fail to be included similarly to previous patches that tried resolving this issue but failed

  [Original Description]
  With 17.2.9, when doing multipart uploads, etags don't get returned. And the etag body is empty.

  This was supposed to be fixed in the bug:
  https://github.com/ceph/ceph/pull/51446

  However, we still see the issue with that fix too (which is part of
  17.2.9).

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/2131790/+subscriptions