[Bug 2073787] Re: apport-retrace needs more than 1 GB

[Benjamin Drung]

Quick size comparison:

```
$ ./convert 
[   0.000001] pickle.load
[   2.983616] write sqlite3
[   8.993280] write DBM (GNU)
[  61.849732] write DBM (NDBM)
[  87.584065] done
$ ls -Slh contents_mapping-jammy-amd64.*
-rw-rw-r-- 1 bdrung bdrung 2,1G Jul 23 15:10 contents_mapping-jammy-amd64.ndbm.db
-rw-rw-r-- 1 bdrung bdrung 1,2G Jul 23 15:09 contents_mapping-jammy-amd64.dbm
-rw-r--r-- 1 bdrung bdrung 802M Jul 23 15:08 contents_mapping-jammy-amd64.db
-rw-rw-r-- 1 bdrung bdrung 753M Jul 23 14:49 contents_mapping-jammy-amd64.pickle
```

Script for it:

```python3
#!/bin/python3

import dbm.gnu
import dbm.ndbm
import pickle
import sqlite3
import sys
import time

START_TIME = time.perf_counter()


def log(text: str) -> None:
    time_ = time.perf_counter() - START_TIME
    sys.stderr.write(f"[{time_:11.6f}] {text}\n")


def write_sqlite(database, file2pkg):
    connection = sqlite3.connect(database)
    cursor = connection.cursor()
    cursor.execute("CREATE TABLE file2deb(file, package)")
    cursor.executemany("INSERT INTO file2deb VALUES(?, ?)", file2pkg.items())
    connection.commit()
    connection.close()


def write_dbm_gnu(database, file2pkg):
    db = dbm.gnu.open(database, "cf")
    for filename, package in file2pkg.items():
        db[filename] = package
    db.sync()
    db.close()


def write_ndbm(database, file2pkg):
    db = dbm.ndbm.open(database, "c")
    for filename, package in file2pkg.items():
        db[filename] = package
    db.close()


def main(file):
    log("pickle.load")
    with open(f"{file}.pickle", "rb") as file2pkg_file:
        file2pkg = pickle.load(file2pkg_file)
    log("write sqlite3")
    write_sqlite(f"{file}.db", file2pkg)
    log("write DBM (GNU)")
    write_dbm_gnu(f"{file}.dbm", file2pkg)
    log("write DBM (NDBM)")
    write_ndbm(f"{file}.ndbm", file2pkg)
    log("done")


if __name__ == "__main__":
    main("contents_mapping-jammy-amd64")
```

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to apport in Ubuntu.
https://bugs.launchpad.net/bugs/2073787

Title:
  apport-retrace needs more than 1 GB

Status in apport package in Ubuntu:
  Triaged

Bug description:
  The system tests test_retrace_system_sandbox and
  test_retrace_jammy_sandbox fail on arm64, ppc64el, s390x, because
  apport-retrace is killed by the OOM killer. Example:

  autopkgtest kernel: Out of memory: Killed process 3597 (apport-
  retrace) total-vm:1512420kB, anon-rss:1241460kB, file-rss:2592kB,
  shmem-rss:0kB, UID:0 pgtables:2554kB oom_score_adj:0

  Log: https://autopkgtest.ubuntu.com/results/autopkgtest-oracular-
  bdrung-apport/oracular/s390x/a/apport/20240722_145904_d3c2f@/log.gz

  apport-retrace should be able to retrace crashes without needing
  multiple gigabytes of memory.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/apport/+bug/2073787/+subscriptions