[Bug 1961427] Re: zlib: compressBound() returns an incorrect result on z15

[Frank Heimes]

Good to have this follow-up, indeed ...
(For a fix in htslib either a new LP bug is needed or htslib needs to be added to this one as 'affecting package'.)

So I did some investigations and found that the "Remove compressBound assertions." merge request #1258 got upstream accepted with htslib v1.13:
$ git log --oneline --grep "Remove compressBound assertions"
515f6df Remove compressBound assertions. (PR #1258)
$ git tag --contains 515f6df
1.13
1.14
1.15
1.15.1
But we already have v1.13 in the latest Ubuntu releases:
$ rmadison -a source htslib
 htslib | 0.2.0~rc3-1     | trusty/universe  | source
 htslib | 1.2.1-2ubuntu1  | xenial/universe  | source
 htslib | 1.7-2           | bionic/universe  | source
 htslib | 1.10.2-3        | focal/universe   | source
 htslib | 1.13+ds-2       | impish/universe  | source
 htslib | 1.13+ds-2build1 | jammy/universe   | source
 htslib | 1.13+ds-2build1 | kinetic/universe | source

So PR #1258 is already included in the htslib version of kinetic/22.10, jammy/22.04 and impish/21.10 (and I even double-checked the code).
It's not included in focals/20.04 v1.10.x - and it looks like the PR cannot be simply applied to v1.10, since the code in v1.10 is too different - which will be a problem.

Since the regression was found in jammy's zlib 1.2.11.dfsg-2ubuntu8 in combination with 1.13+ds-2build1 - I am unsure if PR #1258 is really the fix or if there something else needed on top?
Any thoughts?

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to zlib in Ubuntu.
https://bugs.launchpad.net/bugs/1961427

Title:
  zlib: compressBound() returns an incorrect result on z15

Status in Ubuntu on IBM z Systems:
  Incomplete
Status in zlib package in Ubuntu:
  Incomplete
Status in zlib source package in Focal:
  New
Status in zlib source package in Impish:
  New
Status in zlib source package in Jammy:
  Incomplete

Bug description:
  SRU Justification:
  ==================

  [Impact]

  * zlib: compressBound() returns an incorrect result on IBM z15
  hardware.

  * Passing the result of compressBound() to compress() results
    in an error code.

  * This is because compressBound() is not adjusted for DFLTCC.

  [Fix]

  * Adjust compressBound() for DFLTCC like it's already done
    for deflateBound().

  * Since zlib project does not accept patches at the moment,
    the fix has been integrated into the DFLTCC pull request:
    https://github.com/madler/zlib/pull/410
    The commitid is b25781e735363e04f6c56e21431c47e4afc50b17.

  * The fix extracted out of the above is:
    https://launchpadlibrarian.net/589857296/debdiff_zlib_1.2.11.dfsg-2ubuntu7_to_zlib_1.2.11.dfsg-2ubuntu8_jammy.diff

  [Test Plan]

  * An IBM z15 system (LPAR, z/VM guest or KVM virtual machine)
    with Ubuntu Server 21.10 (or 22.04).

  * A test can be done  based on the following C test program:
        #include <assert.h>
        #include <stdlib.h>
        #include <zlib.h>
        int main() {
            Bytef in_buf[128], out_buf[1024];
            for (size_t i = 0; i < sizeof(in_buf); i++)
                in_buf[i] = rand();
            uLongf dest_len = compressBound(sizeof(in_buf));
            assert(dest_len <= sizeof(out_buf));
            int ret = compress(out_buf, &dest_len,
                               in_buf, sizeof(in_buf));
            assert(ret == Z_OK);
            }

  * The test needs to be done by IBM, due to the requirements
    for the special z15 hardware.

  * A successful test was just completed, based on the version in jammy-
  proposed, which is at the same code level that the impish version this
  SRU is targeted for.

  [Where problems could occur]

  * If the adjustment of compressBound() for DFLTCC is done
    erroneously the issue can still be present or in worst case
    even affect Z systems other than z15 only.

  * The compression can become errorneous with the new changes,
    e.g. in compressBound.

  * Mistakes in dfltcc_free_window OF and especially DEFLATE_BOUND_COMPLEN,
    (incl. the bit definitions), may cause various and unforseen defects.

  * Any build time issues that might have been introduced by this patch
    can be identified by a test build; this was done and is available here:
    https://launchpad.net/~fheimes/+archive/ubuntu/lp1961427

  [Other Info]

  * Ubuntu jammy, impish and focal are affected.
  __________

  Description:   zlib: compressBound() returns an incorrect result on z15
  Symptom:       Passing the result of compressBound() to compress()
                 results in an error code.
  Problem:       compressBound() is not adjusted for DFLTCC.
  Solution:      Adjust compressBound() for DFLTCC like it's already done
                 for deflateBound(). Since zlib project does not accept
                 patches at the moment, the fix has been integrated into
                 the DFLTCC pull request:
                 https://github.com/madler/zlib/pull/410
                 The commitid is b25781e735363e04f6c56e21431c47e4afc50b17.

  Reproduction:  z15 only:
                 #include <assert.h>
                 #include <stdlib.h>
                 #include <zlib.h>
                 int main() {
                     Bytef in_buf[128], out_buf[1024];
                     for (size_t i = 0; i < sizeof(in_buf); i++)
                         in_buf[i] = rand();
                     uLongf dest_len = compressBound(sizeof(in_buf));
                     assert(dest_len <= sizeof(out_buf));
                     int ret = compress(out_buf, &dest_len,
                                        in_buf, sizeof(in_buf));
                     assert(ret == Z_OK);
                 }

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu-z-systems/+bug/1961427/+subscriptions