[Bug 2053225] [NEW] py312: unit tests fail with MemoryError

[James Page]

Public bug reported:

When running the unit tests under Python 3.12, the tests get to about
94% complete and then error out with a MemoryError - the python process
memory usage jumps rapidly just before this:

test/unit/proxy/controllers/test_container.py::TestContainerController::test_GET_bad_requests PASSED [ 94%]
test/unit/proxy/controllers/test_container.py::TestContainerController::test_container_cache_cleared_after_PUT PASSED [ 94%]
test/unit/proxy/controllers/test_container.py::TestContainerController::test_container_info_got_cached PASSED [ 94%]
test/unit/proxy/controllers/test_container.py::TestContainerController::test_handoff_has_deleted_database PASSED [ 94%]
test/unit/proxy/controllers/test_container.py::TestContainerController::test_node_errors PASSED [ 94%]
test/unit/proxy/controllers/test_container.py::TestContainerController::test_reseller_admin PASSED [ 94%]
test/unit/proxy/controllers/test_container.py::TestContainerController::test_response_code_for_DELETE PASSED [ 94%]
test/unit/proxy/controllers/test_container.py::TestContainerController::test_response_code_for_POST PASSED [ 94%]
test/unit/proxy/controllers/test_container.py::TestContainerController::test_response_code_for_PUT PASSED [ 94%]
test/unit/proxy/controllers/test_container.py::TestContainerController::test_response_codes_for_GET PASSED [ 94%]
test/unit/proxy/controllers/test_container.py::TestContainerController::test_swift_owner PASSED [ 94%]
test/unit/proxy/controllers/test_container.py::TestContainerController::test_sys_meta_headers_POST PASSED [ 94%]
test/unit/proxy/controllers/test_container.py::TestContainerController::test_sys_meta_headers_PUT PASSED [ 94%]
test/unit/proxy/controllers/test_container.py::TestGetShardedContainer::test_GET_record_type_object PASSED [ 94%]
test/unit/proxy/controllers/test_container.py::TestGetShardedContainer::test_GET_record_type_shard PASSED [ 94%]
test/unit/proxy/controllers/test_container.py::TestGetShardedContainer::test_GET_record_type_shard_with_listing_state PASSED [ 94%]
test/unit/proxy/controllers/test_container.py::TestGetShardedContainer::test_GET_sharded_container_empty_shard_no_memcache PASSED [ 94%]
test/unit/proxy/controllers/test_container.py::TestGetShardedContainer::test_GET_sharded_container_gap_in_shards_no_memcache PASSED [ 94%]
test/unit/proxy/controllers/test_container.py::TestGetShardedContainer::test_GET_sharded_container_gap_in_shards_with_memcache PASSED [ 94%]
test/unit/proxy/controllers/test_container.py::TestGetShardedContainer::test_GET_sharded_container_marker_beyond_end_marker_memcache PASSED [ 94%]
test/unit/proxy/controllers/test_container.py::TestGetShardedContainer::test_GET_sharded_container_mix_ok_and_unavailable_shards PASSED [ 94%] 
test/unit/proxy/controllers/test_container.py::TestGetShardedContainer::test_GET_sharded_container_mixed_policies_error PASSED [ 94%]
test/unit/proxy/controllers/test_container.py::TestGetShardedContainer::test_GET_sharded_container_no_memcache PASSED [ 94%]
test/unit/proxy/controllers/test_container.py::TestGetShardedContainer::test_GET_sharded_container_overlapping_shards_no_memcache PASSED [ 94%]
test/unit/proxy/controllers/test_container.py::TestGetShardedContainer::test_GET_sharded_container_shard_errors_no_memcache PASSED [ 94%]E: pybuild pybuild:391: test: plugin distutils failed with: exit code=1: cd /<<PKGBUILDDIR>>/.pybuild/cpython3_3.12_swift/build; python3.12 -m pytest --verbose -p no:requests_mock test/unit
dh_auto_test: error: pybuild --test --test-pytest -i python{version} -p "3.11 3.12" returned exit code 13


This is 100% reproducable directly against the update swift project on noble using:

tox -e py312 --
test/unit/proxy/controllers/test_container.py::TestGetShardedContainer

the tests may not fail but % memory will increase rapidly (13% on a 32GB
machine for example).

The same tests under py311 consume less 0.5% of RAM.

** Affects: swift
     Importance: Undecided
         Status: New

** Affects: swift (Ubuntu)
     Importance: Undecided
         Status: New

** Also affects: swift
   Importance: Undecided
       Status: New

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to swift in Ubuntu.
https://bugs.launchpad.net/bugs/2053225

Title:
  py312: unit tests fail with MemoryError

Status in OpenStack Object Storage (swift):
  New
Status in swift package in Ubuntu:
  New

Bug description:
  When running the unit tests under Python 3.12, the tests get to about
  94% complete and then error out with a MemoryError - the python
  process memory usage jumps rapidly just before this:

  test/unit/proxy/controllers/test_container.py::TestContainerController::test_GET_bad_requests PASSED [ 94%]
  test/unit/proxy/controllers/test_container.py::TestContainerController::test_container_cache_cleared_after_PUT PASSED [ 94%]
  test/unit/proxy/controllers/test_container.py::TestContainerController::test_container_info_got_cached PASSED [ 94%]
  test/unit/proxy/controllers/test_container.py::TestContainerController::test_handoff_has_deleted_database PASSED [ 94%]
  test/unit/proxy/controllers/test_container.py::TestContainerController::test_node_errors PASSED [ 94%]
  test/unit/proxy/controllers/test_container.py::TestContainerController::test_reseller_admin PASSED [ 94%]
  test/unit/proxy/controllers/test_container.py::TestContainerController::test_response_code_for_DELETE PASSED [ 94%]
  test/unit/proxy/controllers/test_container.py::TestContainerController::test_response_code_for_POST PASSED [ 94%]
  test/unit/proxy/controllers/test_container.py::TestContainerController::test_response_code_for_PUT PASSED [ 94%]
  test/unit/proxy/controllers/test_container.py::TestContainerController::test_response_codes_for_GET PASSED [ 94%]
  test/unit/proxy/controllers/test_container.py::TestContainerController::test_swift_owner PASSED [ 94%]
  test/unit/proxy/controllers/test_container.py::TestContainerController::test_sys_meta_headers_POST PASSED [ 94%]
  test/unit/proxy/controllers/test_container.py::TestContainerController::test_sys_meta_headers_PUT PASSED [ 94%]
  test/unit/proxy/controllers/test_container.py::TestGetShardedContainer::test_GET_record_type_object PASSED [ 94%]
  test/unit/proxy/controllers/test_container.py::TestGetShardedContainer::test_GET_record_type_shard PASSED [ 94%]
  test/unit/proxy/controllers/test_container.py::TestGetShardedContainer::test_GET_record_type_shard_with_listing_state PASSED [ 94%]
  test/unit/proxy/controllers/test_container.py::TestGetShardedContainer::test_GET_sharded_container_empty_shard_no_memcache PASSED [ 94%]
  test/unit/proxy/controllers/test_container.py::TestGetShardedContainer::test_GET_sharded_container_gap_in_shards_no_memcache PASSED [ 94%]
  test/unit/proxy/controllers/test_container.py::TestGetShardedContainer::test_GET_sharded_container_gap_in_shards_with_memcache PASSED [ 94%]
  test/unit/proxy/controllers/test_container.py::TestGetShardedContainer::test_GET_sharded_container_marker_beyond_end_marker_memcache PASSED [ 94%]
  test/unit/proxy/controllers/test_container.py::TestGetShardedContainer::test_GET_sharded_container_mix_ok_and_unavailable_shards PASSED [ 94%] 
  test/unit/proxy/controllers/test_container.py::TestGetShardedContainer::test_GET_sharded_container_mixed_policies_error PASSED [ 94%]
  test/unit/proxy/controllers/test_container.py::TestGetShardedContainer::test_GET_sharded_container_no_memcache PASSED [ 94%]
  test/unit/proxy/controllers/test_container.py::TestGetShardedContainer::test_GET_sharded_container_overlapping_shards_no_memcache PASSED [ 94%]
  test/unit/proxy/controllers/test_container.py::TestGetShardedContainer::test_GET_sharded_container_shard_errors_no_memcache PASSED [ 94%]E: pybuild pybuild:391: test: plugin distutils failed with: exit code=1: cd /<<PKGBUILDDIR>>/.pybuild/cpython3_3.12_swift/build; python3.12 -m pytest --verbose -p no:requests_mock test/unit
  dh_auto_test: error: pybuild --test --test-pytest -i python{version} -p "3.11 3.12" returned exit code 13

  
  This is 100% reproducable directly against the update swift project on noble using:

  tox -e py312 --
  test/unit/proxy/controllers/test_container.py::TestGetShardedContainer

  the tests may not fail but % memory will increase rapidly (13% on a
  32GB machine for example).

  The same tests under py311 consume less 0.5% of RAM.

To manage notifications about this bug go to:
https://bugs.launchpad.net/swift/+bug/2053225/+subscriptions