[Bug 576590] Re: Command not found consumes lots of memory and CPU for longer input

Wed Aug 30 06:07:38 UTC 2017

The bug still exists in Ubuntu 16.04.

The examples below use a 1 GB memory limit.
3000-character input: 17 seconds, no crash.
4000-character input: 0.8 seconds, MemoryError crash.

$ (ulimit -v 1000000; time /usr/lib/command-not-found -- "$(python -c 'print "x"*3000')")
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx: command not found

real	0m17.407s
user	0m17.008s
sys	0m0.388s

$ (ulimit -v 1000000; time /usr/lib/command-not-found -- "$(python -c 'print "x"*4000')")
Sorry, command-not-found has crashed! Please file a bug report at:
https://bugs.launchpad.net/command-not-found/+filebug
Please include the following information with the report:

command-not-found version: 0.3
Python version: 3.5.2 final 0
Distributor ID:	Ubuntu
Description:	Ubuntu 16.04.3 LTS
Release:	16.04
Codename:	xenial
Exception information:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/CommandNotFound/util.py", line 24, in crash_guard
    callback()
  File "/usr/lib/command-not-found", line 90, in main
    if not cnf.advise(args[0], options.ignore_installed) and not options.no_failure_msg:
  File "/usr/lib/python3/dist-packages/CommandNotFound/CommandNotFound.py", line 267, in advise
    self.print_spelling_suggestion(command)
  File "/usr/lib/python3/dist-packages/CommandNotFound/CommandNotFound.py", line 143, in print_spelling_suggestion
    for w in similar_words(word):
  File "/usr/lib/python3/dist-packages/CommandNotFound/CommandNotFound.py", line 103, in similar_words
    inserts = [a + c + b     for a, b in s for c in alphabet]
  File "/usr/lib/python3/dist-packages/CommandNotFound/CommandNotFound.py", line 103, in <listcomp>
    inserts = [a + c + b     for a, b in s for c in alphabet]
MemoryError

real	0m0.766s
user	0m0.320s
sys	0m0.440s

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to command-not-found in Ubuntu.
https://bugs.launchpad.net/bugs/576590

Title:
  Command not found consumes lots of memory and CPU for longer input

Status in command-not-found:
  Fix Released
Status in command-not-found package in Ubuntu:
  Fix Released

Bug description:
  Binary package hint: command-not-found

  This concerns command-not-found 0.2.40ubuntu5 in Ubuntu Lucid.

  Once upon a time, I was working on my shiny new Lucid system, when
  things suddenly became very slow. I ran top(1) to check what was going
  on, and saw this:

  top - 13:34:17 up 15 min,  5 users,  load average: 3.79, 1.41, 0.54
  Tasks: 158 total,   2 running, 156 sleeping,   0 stopped,   0 zombie
  Cpu(s):  0.6%us,  1.5%sy,  0.0%ni, 15.8%id, 82.1%wa,  0.0%hi,  0.0%si,  0.0%st
  Mem:   3917500k total,  3890004k used,    27496k free,      480k buffers
  Swap:  4200956k total,  1919184k used,  2281772k free,    50180k cached

    PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND            
   2348 userme    20   0 4618m 3.2g  440 D    3 86.7   0:06.95 python             
   2370 userme    20   0 11000 1308  956 S    1  0.0   0:00.03 top                
   1587 root      20   0  101m 5788 1524 S    0  0.1   0:04.48 Xorg               

  Python was going out of control! What was it doing?

  # ps uww 2348
  USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
  userme    2348  6.5 84.6 4729240 3316468 tty1  D    13:32   0:07 /usr/bin/python /usr/lib/command-not-found -- mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm...

  (The m's went on for like two or three screens.)

  Now, I have no idea what caused this bizarre invocation of c-n-f. (I
  don't typically sleep with my forehead on the keyboard or anything.)
  However, I copied-and-pasted the complete invocation as reported by
  ps(1) into a script, and tried running it:

  $ time -p sh c-n-f-bug.sh 
  mmmm[...]mmmm: command not found
  real 42.60
  user 39.17
  sys 1.10

  During that time, c-n-f used over a gigabyte of memory. That's not
  right!

  I am attaching the c-n-f-bug.sh script. Interestingly, it is exactly
  4096 bytes long, which suggests to me that the original command
  invocation may have been even longer and not reported in full simply
  due to ps(1) limiting the output.

To manage notifications about this bug go to:
https://bugs.launchpad.net/command-not-found/+bug/576590/+subscriptions