[Bug 2105503] [NEW] Uncontrolled memory usage

[Skia]

Public bug reported:

Here is a simplified snippet, taken from `cnf-extractor`:


```
$ cat apt_file_0ad-data.py
import apt_inst

files = []
debfile = apt_inst.DebFile("0ad-data_0.27.0-1_all.deb")
debfile.data.go(lambda item, data: files.append(item.name))
print(files)
```

To run this, you need the latest `0ad-data` package downloaded next to
the script: `apt download 0ad-data`.

When running this "normally", I can see very high memory usage, sometimes leading to OOM kill (how I digged into that in the first place):
python3 apt_file_0ad-data.py  & ; htop -p $!

Interestingly, when wrapping that with some kind of constraints, the issue doesn't appear anymore at all, and the memory usage remains low:
prlimit -v536870912 python3 apt_file_0ad-data.py  & ; htop -p $!
With that previous command, even the time to finish is divided by two (~consistently 8s vs 15s without wrapping on my machine), but I didn't take precise measurements there, so might be noise, but still...

And then one final interesting result:
systemd-run --scope -p MemoryMax=500M --user python3 apt_file_0ad-data.py
Sometimes this works just as with `prlimit`, with slightly higher memory usage, and sometimes it correctly triggers oom kill, after more than one minute. Again, the timing benchmark is inaccurate, but still...

ProblemType: Bug
DistroRelease: Ubuntu 25.04
Package: python3-apt 2.9.9build1
ProcVersionSignature: Ubuntu 6.12.0-12.12-generic 6.12.4
Uname: Linux 6.12.0-12-generic x86_64
NonfreeKernelModules: zfs
ApportVersion: 2.32.0-0ubuntu3
Architecture: amd64
CasperMD5CheckResult: unknown
CurrentDesktop: sway
Date: Mon Mar 31 15:58:25 2025
InstallationDate: Installed on 2023-08-02 (607 days ago)
InstallationMedia: Ubuntu 23.04 "Lunar Lobster" - Release amd64 (20230418)
ProcEnviron:
 LANG=fr_FR.UTF-8
 PATH=(custom, no user)
 SHELL=/bin/zsh
 TERM=alacritty
 XDG_RUNTIME_DIR=<set>
SourcePackage: python-apt
UpgradeStatus: Upgraded to plucky on 2024-06-12 (292 days ago)

** Affects: python-apt (Ubuntu)
     Importance: Undecided
         Status: New


** Tags: amd64 apport-bug plucky wayland-session

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to python-apt in Ubuntu.
https://bugs.launchpad.net/bugs/2105503

Title:
  Uncontrolled memory usage

Status in python-apt package in Ubuntu:
  New

Bug description:
  Here is a simplified snippet, taken from `cnf-extractor`:

  
  ```
  $ cat apt_file_0ad-data.py
  import apt_inst

  files = []
  debfile = apt_inst.DebFile("0ad-data_0.27.0-1_all.deb")
  debfile.data.go(lambda item, data: files.append(item.name))
  print(files)
  ```

  To run this, you need the latest `0ad-data` package downloaded next to
  the script: `apt download 0ad-data`.

  When running this "normally", I can see very high memory usage, sometimes leading to OOM kill (how I digged into that in the first place):
  python3 apt_file_0ad-data.py  & ; htop -p $!

  Interestingly, when wrapping that with some kind of constraints, the issue doesn't appear anymore at all, and the memory usage remains low:
  prlimit -v536870912 python3 apt_file_0ad-data.py  & ; htop -p $!
  With that previous command, even the time to finish is divided by two (~consistently 8s vs 15s without wrapping on my machine), but I didn't take precise measurements there, so might be noise, but still...

  And then one final interesting result:
  systemd-run --scope -p MemoryMax=500M --user python3 apt_file_0ad-data.py
  Sometimes this works just as with `prlimit`, with slightly higher memory usage, and sometimes it correctly triggers oom kill, after more than one minute. Again, the timing benchmark is inaccurate, but still...

  ProblemType: Bug
  DistroRelease: Ubuntu 25.04
  Package: python3-apt 2.9.9build1
  ProcVersionSignature: Ubuntu 6.12.0-12.12-generic 6.12.4
  Uname: Linux 6.12.0-12-generic x86_64
  NonfreeKernelModules: zfs
  ApportVersion: 2.32.0-0ubuntu3
  Architecture: amd64
  CasperMD5CheckResult: unknown
  CurrentDesktop: sway
  Date: Mon Mar 31 15:58:25 2025
  InstallationDate: Installed on 2023-08-02 (607 days ago)
  InstallationMedia: Ubuntu 23.04 "Lunar Lobster" - Release amd64 (20230418)
  ProcEnviron:
   LANG=fr_FR.UTF-8
   PATH=(custom, no user)
   SHELL=/bin/zsh
   TERM=alacritty
   XDG_RUNTIME_DIR=<set>
  SourcePackage: python-apt
  UpgradeStatus: Upgraded to plucky on 2024-06-12 (292 days ago)

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/python-apt/+bug/2105503/+subscriptions