[Bug 1961427] Re: zlib: compressBound() returns an incorrect result on z15

[Frank Heimes]

@xnox Well, initially I was also a bit puzzled noticing that not the standalone htslib is used, but an embedded htslib in bedtools.
But this is also upstream the case and htslib is not the only component that is bundled:
$ ls ./src/utils/
BamTools            BlockedIntervals  fileType    lineFileUtilities  tabFile
BamTools-Ancillary  chromsweep        general     NewChromsweep      ToolBase
bedFile             Contexts          GenomeFile  Point              VectorOps
bedFilePE           driver            gzstream    RecordOutputMgr    version
bedGraphFile        Fasta             htslib      sequenceUtilities
BinTree             FileRecordTools   KeyListOps  stringUtilities
And solving the situation for all of these will be a nighmare with a high risk of introducing problems on top and more difficulties on maintaining and making sure that the components are in place in the correct version ...

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to zlib in Ubuntu.
https://bugs.launchpad.net/bugs/1961427

Title:
  zlib: compressBound() returns an incorrect result on z15

Status in Ubuntu on IBM z Systems:
  Incomplete
Status in bedtools package in Ubuntu:
  New
Status in zlib package in Ubuntu:
  Incomplete
Status in bedtools source package in Focal:
  New
Status in zlib source package in Focal:
  New
Status in bedtools source package in Impish:
  New
Status in zlib source package in Impish:
  New
Status in bedtools source package in Jammy:
  New
Status in zlib source package in Jammy:
  Incomplete

Bug description:
  SRU Justification:
  ==================

  [Impact]

  * zlib: compressBound() returns an incorrect result on IBM z15
  hardware.

  * Passing the result of compressBound() to compress() results
    in an error code.

  * This is because compressBound() is not adjusted for DFLTCC.

  [Fix]

  * Adjust compressBound() for DFLTCC like it's already done
    for deflateBound().

  * Since zlib project does not accept patches at the moment,
    the fix has been integrated into the DFLTCC pull request:
    https://github.com/madler/zlib/pull/410
    The commitid is b25781e735363e04f6c56e21431c47e4afc50b17.

  * The fix extracted out of the above is:
    https://launchpadlibrarian.net/589857296/debdiff_zlib_1.2.11.dfsg-2ubuntu7_to_zlib_1.2.11.dfsg-2ubuntu8_jammy.diff

  [Test Plan]

  * An IBM z15 system (LPAR, z/VM guest or KVM virtual machine)
    with Ubuntu Server 21.10 (or 22.04).

  * A test can be done  based on the following C test program:
        #include <assert.h>
        #include <stdlib.h>
        #include <zlib.h>
        int main() {
            Bytef in_buf[128], out_buf[1024];
            for (size_t i = 0; i < sizeof(in_buf); i++)
                in_buf[i] = rand();
            uLongf dest_len = compressBound(sizeof(in_buf));
            assert(dest_len <= sizeof(out_buf));
            int ret = compress(out_buf, &dest_len,
                               in_buf, sizeof(in_buf));
            assert(ret == Z_OK);
            }

  * The test needs to be done by IBM, due to the requirements
    for the special z15 hardware.

  * A successful test was just completed, based on the version in jammy-
  proposed, which is at the same code level that the impish version this
  SRU is targeted for.

  [Where problems could occur]

  * If the adjustment of compressBound() for DFLTCC is done
    erroneously the issue can still be present or in worst case
    even affect Z systems other than z15 only.

  * The compression can become errorneous with the new changes,
    e.g. in compressBound.

  * Mistakes in dfltcc_free_window OF and especially DEFLATE_BOUND_COMPLEN,
    (incl. the bit definitions), may cause various and unforseen defects.

  * Any build time issues that might have been introduced by this patch
    can be identified by a test build; this was done and is available here:
    https://launchpad.net/~fheimes/+archive/ubuntu/lp1961427

  [Other Info]

  * Ubuntu jammy, impish and focal are affected.
  __________

  Description:   zlib: compressBound() returns an incorrect result on z15
  Symptom:       Passing the result of compressBound() to compress()
                 results in an error code.
  Problem:       compressBound() is not adjusted for DFLTCC.
  Solution:      Adjust compressBound() for DFLTCC like it's already done
                 for deflateBound(). Since zlib project does not accept
                 patches at the moment, the fix has been integrated into
                 the DFLTCC pull request:
                 https://github.com/madler/zlib/pull/410
                 The commitid is b25781e735363e04f6c56e21431c47e4afc50b17.

  Reproduction:  z15 only:
                 #include <assert.h>
                 #include <stdlib.h>
                 #include <zlib.h>
                 int main() {
                     Bytef in_buf[128], out_buf[1024];
                     for (size_t i = 0; i < sizeof(in_buf); i++)
                         in_buf[i] = rand();
                     uLongf dest_len = compressBound(sizeof(in_buf));
                     assert(dest_len <= sizeof(out_buf));
                     int ret = compress(out_buf, &dest_len,
                                        in_buf, sizeof(in_buf));
                     assert(ret == Z_OK);
                 }

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu-z-systems/+bug/1961427/+subscriptions