[Bug 2144593] [NEW] SRU: io.TextIOWrapper.write: write during flush causes pending_bytes length mismatch leading to crash/corruption

[Joao Andre Simioni]

Public bug reported:

[Impact]

Under certain conditions, _io.TextIOWrapper.write can cause a hard crash
(SIGABRT) of the Python interpreter or potential state corruption.

This occurs when a nested or concurrent write() is triggered during a
flush operation. If TextIOWrapper.write() tries to store more data than
its chunk_size, it flushes its buffer. If the underlying stream's
write() then triggers another write to the same TextIOWrapper object,
the internal pending_bytes and pending_bytes_count variables get reset.

When the outer write() resumes, it unconditionally assigns the new data
to pending_bytes without realizing the internal state was altered,
causing a length mismatch. In debug builds of Python, this immediately
triggers a C-level assertion failure (Assertion
PyUnicode_GET_LENGTH(pending) == self->pending_bytes_count failed). In
standard release builds, this mismatch can lead to unexpected behavior,
corrupted text output, or memory unsafety.

This fix has been merged upstream and backported to the Python 3.12
branch.

[Test Plan]

The upstream reporter provided a repro:

```
import _io

class MyIO(_io.BytesIO):
    def __init__(self):
        _io.BytesIO.__init__(self)
        self.writes = []

    def write(self, b):
        self.writes.append(b)
        tw.write("c")
        return len(b)

buf = MyIO()
tw = _io.TextIOWrapper(buf)

CHUNK_SIZE = 8192

tw.write("a" * (CHUNK_SIZE - 1))
tw.write("b" * 2)

tw.flush()

assert b''.join(tw.buffer.writes) == b"a" * (CHUNK_SIZE - 1) + b"b" * 2 + b"c"
```

$ python3 repro.py 
Traceback (most recent call last):
  File "/tmp/repro.py", line 23, in <module>
    assert b''.join(tw.buffer.writes) == b"a" * (CHUNK_SIZE - 1) + b"b" * 2 + b"c"
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError

[Regression Risks]

This fix makes modifications in Modules/_io/textio.c.

The patch is very isolated. It only does a strict check to see if the
nested call left data inside pending_bytes and concatenates it properly,
instead of just overwrite it.

Furthermore, the fix was reviewed by the upstream CPython developers and
tested against the full Python test suite. It is also already included
in the later stable upstream releases (3.12.4+).

[Other Info]
Upstream Issue: https://github.com/python/cpython/issues/119506
Upstream PR: https://github.com/python/cpython/pull/119507
Affected Ubuntu Release: Ubuntu 24.04 LTS (Noble Numbat)
Affected Package: python3.12 (currently at version 3.12.3 in Noble)

** Affects: python3.12 (Ubuntu)
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to python3.12 in Ubuntu.
https://bugs.launchpad.net/bugs/2144593

Title:
  SRU: io.TextIOWrapper.write: write during flush causes pending_bytes
  length mismatch leading to crash/corruption

Status in python3.12 package in Ubuntu:
  New

Bug description:
  [Impact]

  Under certain conditions, _io.TextIOWrapper.write can cause a hard
  crash (SIGABRT) of the Python interpreter or potential state
  corruption.

  This occurs when a nested or concurrent write() is triggered during a
  flush operation. If TextIOWrapper.write() tries to store more data
  than its chunk_size, it flushes its buffer. If the underlying stream's
  write() then triggers another write to the same TextIOWrapper object,
  the internal pending_bytes and pending_bytes_count variables get
  reset.

  When the outer write() resumes, it unconditionally assigns the new
  data to pending_bytes without realizing the internal state was
  altered, causing a length mismatch. In debug builds of Python, this
  immediately triggers a C-level assertion failure (Assertion
  PyUnicode_GET_LENGTH(pending) == self->pending_bytes_count failed). In
  standard release builds, this mismatch can lead to unexpected
  behavior, corrupted text output, or memory unsafety.

  This fix has been merged upstream and backported to the Python 3.12
  branch.

  [Test Plan]

  The upstream reporter provided a repro:

  ```
  import _io

  class MyIO(_io.BytesIO):
      def __init__(self):
          _io.BytesIO.__init__(self)
          self.writes = []

      def write(self, b):
          self.writes.append(b)
          tw.write("c")
          return len(b)

  buf = MyIO()
  tw = _io.TextIOWrapper(buf)

  CHUNK_SIZE = 8192

  tw.write("a" * (CHUNK_SIZE - 1))
  tw.write("b" * 2)

  tw.flush()

  assert b''.join(tw.buffer.writes) == b"a" * (CHUNK_SIZE - 1) + b"b" * 2 + b"c"
  ```

  $ python3 repro.py 
  Traceback (most recent call last):
    File "/tmp/repro.py", line 23, in <module>
      assert b''.join(tw.buffer.writes) == b"a" * (CHUNK_SIZE - 1) + b"b" * 2 + b"c"
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  AssertionError

  [Regression Risks]

  This fix makes modifications in Modules/_io/textio.c.

  The patch is very isolated. It only does a strict check to see if the
  nested call left data inside pending_bytes and concatenates it
  properly, instead of just overwrite it.

  Furthermore, the fix was reviewed by the upstream CPython developers
  and tested against the full Python test suite. It is also already
  included in the later stable upstream releases (3.12.4+).

  [Other Info]
  Upstream Issue: https://github.com/python/cpython/issues/119506
  Upstream PR: https://github.com/python/cpython/pull/119507
  Affected Ubuntu Release: Ubuntu 24.04 LTS (Noble Numbat)
  Affected Package: python3.12 (currently at version 3.12.3 in Noble)

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/python3.12/+bug/2144593/+subscriptions