[Bug 2144593] Re: SRU: io.TextIOWrapper.write: write during flush causes pending_bytes length mismatch leading to crash/corruption

[Vladimir Petko]

Questing is not affected.

** Also affects: python3.12 (Ubuntu Questing)
   Importance: Undecided
       Status: New

** Changed in: python3.12 (Ubuntu Noble)
       Status: In Progress => Invalid

** Changed in: python3.12 (Ubuntu Noble)
       Status: Invalid => New

** Changed in: python3.12 (Ubuntu Questing)
       Status: New => Invalid

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to python3.12 in Ubuntu.
https://bugs.launchpad.net/bugs/2144593

Title:
  SRU: io.TextIOWrapper.write: write during flush causes pending_bytes
  length mismatch leading to crash/corruption

Status in python3.12 package in Ubuntu:
  Fix Released
Status in python3.12 source package in Noble:
  New
Status in python3.12 source package in Questing:
  Invalid

Bug description:
  [Impact]

  Under certain conditions, _io.TextIOWrapper.write can cause a hard
  crash (SIGABRT) of the Python interpreter or potential state
  corruption.

  This occurs when a nested or concurrent write() is triggered during a
  flush operation. If TextIOWrapper.write() tries to store more data
  than its chunk_size, it flushes its buffer. If the underlying stream's
  write() then triggers another write to the same TextIOWrapper object,
  the internal pending_bytes and pending_bytes_count variables get
  reset.

  When the outer write() resumes, it unconditionally assigns the new
  data to pending_bytes without realizing the internal state was
  altered, causing a length mismatch. In debug builds of Python, this
  immediately triggers a C-level assertion failure (Assertion
  PyUnicode_GET_LENGTH(pending) == self->pending_bytes_count failed). In
  standard release builds, this mismatch can lead to unexpected
  behavior, corrupted text output, or memory unsafety.

  This fix has been merged upstream and backported to the Python 3.12
  branch.

  [Test Plan]

  The upstream reporter provided a repro:

  ```
  import _io

  class MyIO(_io.BytesIO):
      def __init__(self):
          _io.BytesIO.__init__(self)
          self.writes = []

      def write(self, b):
          self.writes.append(b)
          tw.write("c")
          return len(b)

  buf = MyIO()
  tw = _io.TextIOWrapper(buf)

  CHUNK_SIZE = 8192

  tw.write("a" * (CHUNK_SIZE - 1))
  tw.write("b" * 2)

  tw.flush()

  assert b''.join(tw.buffer.writes) == b"a" * (CHUNK_SIZE - 1) + b"b" * 2 + b"c"
  ```

  $ python3 repro.py
  Traceback (most recent call last):
    File "/tmp/repro.py", line 23, in <module>
      assert b''.join(tw.buffer.writes) == b"a" * (CHUNK_SIZE - 1) + b"b" * 2 + b"c"
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  AssertionError

  [Where problems can occur]

  This fix makes modifications in Modules/_io/textio.c.

  The patch is very isolated. It only does a strict check to see if the
  nested call left data inside pending_bytes and concatenates it
  properly, instead of just overwrite it.

  Furthermore, the fix was reviewed by the upstream CPython developers
  and tested against the full Python test suite. It is also already
  included in the later stable upstream releases (3.12.4+).

  [Other Info]
  Upstream Issue: https://github.com/python/cpython/issues/119506
  Upstream PR: https://github.com/python/cpython/pull/119507
  Affected Ubuntu Release: Ubuntu 24.04 LTS (Noble Numbat)
  Affected Package: python3.12 (currently at version 3.12.3 in Noble)

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/python3.12/+bug/2144593/+subscriptions