[Bug 828730] Re: apt-get source memory leak

[Mark Menkhus]

I am using both local filesystem - a disk and san disks.  The problem
shows itself on 6.04 and 10.04. I use ext3 on 6.04, and ext4 on 10.04.

I don't know enough internals or memory monitoring to be effective here.
I setup machines, and start using them.  I don't set any kernel
configuration parameters (surely there are some knobs, but I don't
adjust anything). This may be attributable to some other 'known
problem', and tuning gotcha.

What I did see, and reproduce at will with this hinky method for getting
source code is an unrecoverable use of sytem memory.  This was not your
garden variety, applications use memory and after they die or crash, you
get the memory back.  I ran 10-26 separate processes all getting on
package source after another using apt-get source.   What happens is
that the processes can complete, but some of the memory never shows back
up in up in free & it stays in used.  The buffers values continually
increased.  Free, for the most part continually decreases.

Without stretching, you could certainly characterize this as a potential
local denial of service by a userland application.

If you are interested, I can share the scripts that run all these jobs
in paralllel.  Maybe this is not an apt leak, but something else a
little bit more problematic.  I have not looked for anything articles
for how to isolate what appears to be a kernel memory leak.  I can let
this set overnight after all the processes have completed, and the
memory does not come back.

Mark

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to apt in Ubuntu.
https://bugs.launchpad.net/bugs/828730

Title:
   apt-get source memory leak

Status in “apt” package in Ubuntu:
  Invalid

Bug description:
  I used apt-get source to download a bunch of source and noticed I
  would run out of memory.  I tried this on several machines, including
  LTS 10.04.03, with apt 0.7.25.3ubuntu9.6 for amd64 compiled on Jul 13
  2011 11:09:35.

  I used apt-get source in a script where several thousand downloads are
  done, the system will be unrecoverably out of memory, requiring
  reboot.  You can not download the whole source tree this way, without
  a lot of reboots.

  example:
  for package in `dpkg -l | grep 'ii' | tr -s '[:blank:]' '\t' | cut --fields=2`
  do
  mkdir $package
  cd $package
  apt-get --download-only source $package
  cd ..
  done

  Monitor the memory use using top or in /proc, eventually the system
  will start thrashing.

  Work around: unknown , how does one download the ubuntu or debian
  source tree ?

  Mark Menkhus, HP

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/apt/+bug/828730/+subscriptions