[Bug 973494] Re: Retracing is way too slow

[Evan Dandrea]

I followed up with James this afternoon. He doesn't want to drop core
dumps from the queue at random as the first solution to this, but he's
okay with a proper RED-like solution if we ultimately need one.

He also suggested that a cache of extracted debs would be a good idea. I
had tried this over the weekend and ultimately avoided it because I was
concerned about packages with Replaces fields. However, he pointed out
that's largely an academic exercise. Library packages wont be using
replaces on the paths to the libraries or their debugging symbols.

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to whoopsie-daisy in Ubuntu.
https://bugs.launchpad.net/bugs/973494

Title:
  Retracing is way too slow

Status in “whoopsie-daisy” package in Ubuntu:
  Confirmed

Bug description:
  We're currently only able to process about three core files per minute
  with each retracer. We can fire up more retracers and drop some core
  files at random when we're approaching a high load, both of which
  mitigate this problem to varying degrees. However, we should also
  address the problem that our current retracing code is taking far too
  long to process each core file:

  [4:40pm] jjo: ev: basically afaics we are losing the producer|consumer rate battle
  [4:40pm] mthaddon: ev: basically the rabbitmq queues are fairly consistently increasing in size, but the server itself is very lightly loaded - should we be firing up more retracers, or is there some way of making existing retracers do more work?
  [4:40pm] jjo: ev: while finfolk doesnt get squeezed for load
  [4:41pm] ev: the retracers process in serial, so bringing up more of them would be advisable
  [4:41pm] jjo: ev: FTR I had fired up to ~7 retracers in parallel , which hover'd loadavg~=8 -> got ~3 oopsen per min
  [4:41pm] jjo: ev: but still I couldn't manage to get more than ~3/min

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/whoopsie-daisy/+bug/973494/+subscriptions