[Bug 879334] Re: nfsd from nfs-kernel-server very slow and system load from 25%-100% from nfsd

Sun Feb 3 17:41:32 UTC 2013

Sorry, I did not know what to do with my report, so I am attaching it
here since it seems to be the same problem.

I am running desktop Ubuntu 12.04 lts on 4 separate gigabit-networked
machines, for my full-home media center, with the tuner installed in the
'server' (desktop install running NFS server) under mythtv, and 3
desktop install NFS clients in separate rooms. I had to upgrade the pre-
existing 'server' (my learning platform) from 11.10 to 12.04 to match
the clients because mythtv does not interoperate with differing
versions, and I did not want to downgrade the clients to 11.10, I wanted
a long term network install that is reliable and low-maintenance.

Now I have terrible network performance. The tuner works fine within the
'server', and I can view shows on the server, channel surf, record, play
back, etc. no problems. Over the NFS network at the desktop clients, the
media center system is almost completely broken.

If I start viewing a video media file, or listen to ripped audio, or
i.e. open any media file at all, that is stored on the server, viewing
on a client over the network, or if I attempt to edit the commercials
out of shows on a client from over the network within mythtv editor, or
even open a text file, the client will pause/hang for at least 30
seconds while 'loading' the file, and then finally it will start
sequential streaming the media with OK performance on one or maybe two
clients max - but when using Videolan VLC to view server media files on
a client,  I had to increase the buffer by 10X (from 3 to 30 seconds of
standard definition programming, approximately) to avoid long stuttering
pauses in playback. Within mythtv frontend application at the client
side, the video editing over the network is abominably slow, needing
tenths of seconds, to seconds, to minutes, to hours, to completely hung,
for the editor to respond to each keypress, getting slower all the time
until it eventually grinds to a halt.

Listing directories, editing files, viewing media, using any of the text
editors or media players I have installed, all have at least 30 seconds
of delay on 'opening' (sending a command, either from a terminal window,
or a nautilus window, or a text editor, or whatever), and the entire
network slowly grinds to a standstill eventually, with mythtv locked in
unusable state at the clients, even though it is still working fine on
the server.

My server is a core 2 duo and so is my main media center client. The
server is fully populated with 8gig of memory and terabytes of storage,
and the client is sparsely populated with 2 gig of memory. I realize
this is underpowered for hdtv media applications but surely a core 2 duo
should be able to serve at least one standard definition media file at a
time without any performance issues at all, and should be able to handle
text editors with its eyes closed. I also have an i7 laptop client with
8 gig of memory and a terabyte of storage that suffers from the same
poor network performance, even after disabling the troublesome Broadcom
wireless power management, or even after plugging in the 1 gigabit wired
connection and disabling of wireless.

I have no security at all configured on this network, and root squash is
turned off so that I can edit or delete files without having to
synchronize my clients accounts over NIS with Kerberos, which I do not
even understand how to install let alone set up. I am just learning
network admin, doing it on my own, slowly. This is by design a really
primitive, drop-dead simple network install with only a hardware
firewall protecting it. All transfers are synchronous, meaning there is
no automounting going on. Everything is hard mounted, so when the
desktop server hangs, so do all the desktop clients, if they happen to
be actively running an NFS-supplied media file or etc. It took me months
to learn how to do the hard mounting of encrypted volumes properly,
needing to let it time out and retry until the encrypted raid password
is entered. I did not encrypt the operating system just yet, not until I
have everything working in the open, and it is starting to look like I
will have to start over with another distribution entirely because of
this bug!!

It seems there is an NFS tuning issue because there are some lost
packets at the client side, and the NFS forums suggest increasing the
number of threads. I am just now learning to debug and I have a dim
perspective on these tuning issues at best. I am no Linux guru and have
not increased the number of threads, because there is no way I could
even use up the default number of threads with just a single user on
this network, unless attempting to transcode from multiple machines at
once and I do not have the disc space to support that just now anyway. I
do not see any reason why NFS defaults should perform so badly that they
lock up the system.

So far I have changed nothing on the NFS tuning, and started searching
for answers in the bug reports, because it seems to me that for my
pathetically underused hardware there should be no issues whatsoever
running a simple, hard-mounted, single-user household media center with
multiple networked clients and no security, even using conservative
default tuning that ships with NFS, even if I occasionally run out of
threads once in a blue moon there should be no performance issues
whatsoever due to NFS. If anything, I would expect performance issues to
be due to the hardware, but not of this character where things worsen
over time so that after one day the network freezes.

Now here I find eerily familiar, long-standing bug reports that seem to
incriminate kernel updates that changed the NFS auto tuning algorithm. I
felt compelled to write an explanation of my own scenario since it is so
different from the sophisticated, network-savvy implementations
mentioned here. I thought perhaps something of importance could be
learned, even from someone as ignorant as me, just because the
implementation I am using is so dirt simple, eliminating many potential
suspects.

I am wondering to myself, does Ubuntu intend to compete with Microsoft
in the home media center market, or just let Microsoft eat their lunch?
Or am I supposed to convert my Ubuntu network to run under Windows
networking via Samba, and just flush NFS down the toilet? I really,
really want to use NFS! I want the performance and I do not trust
Windows networking. But it seems that even advanced Ubuntu users have
been frustrated for over a year because of this bug! Should I change my
entire network to Red Hat? I am completely unfamiliar with even their
packaging application, I have no spare cash lying around, and my
impression of CentOS is that it is highly stripped-down. I would have to
build up all my proprietary hardware drivers from scratch and there is
nothing even approaching the level of the Ubuntu support community for
Red Hat.

Only solution I have found so far is to reboot all clients and server,
after which NFS recovers temporarily, but still has long delays when
starting up the streaming, and eventually grinds to a halt again,
especially when mythtv fills its allocated disc space, after which it is
a miracle if I can even task-switch out of its media player app in order
to reboot all the clients and start over. It seems that even 'disc full'
error messages are subject to the same limitations of this NFS bug,
whatever it is.

One other thing I noticed is that mp4 transcoding jobs running in
Handbrake on the clients, that seem to finish properly and close their
output files, often actually do not finish writing to the server,
leaving incomplete, corrupted files and requiring a second, third, or
fourth attempt at transcoding. Also, the 32-bit machine I am using seems
to have problems with the no-root-squash function -- while write-
protecting transcoded output, it chowns its transcoded files attributed
to user 'nobody', even though the other two (64-bit) clients have no
problem chowning to root via NFS! Apparently, the older and slower the
hardware, the worse the problems it experiences with NFS.

I apologize for the long, rambling comment on this bug report. I only
even wrote it to indicate to the folks at Canonical that this NFS
problem is affecting ordinary Ubuntu desktop users who are attempting
merely to run the same full-house media center scenario that is being
advertised on television for DirectTV and cable subscribers, and being
implemented independently by Windows users familiar with mythtv and
handbrake. Without a properly functioning NFS, Ubuntu is properly
crippled.

I am not a programmer or system administrator, just a retired engineer
with a little bit of hacking experience. A very little bit. After months
learning to install and use Ubuntu properly, then I had to read
documentation files and support group forums for a full year to get this
far with the network setup, and now to find that the operating system I
chose has a known, long-standing fatal networking flaw that I have been
battling with all along feels like a big fat stick in the eye. Would
some kind soul please remove that stick?

Does anyone consider it important or even kind to add warnings to the
package manager, or even to the Ubuntu download page, so we do not all
have to stumble across this networking show-stopper on our own? I chose
Ubuntu for its ease of use and support, but here is a big fat gaping
network hole that makes it impossible to even achieve Windows Media
Center level of performance. Adding insult to injury, there is no
advance warning anywhere about this networking problem.

Thanks for the learning experience, friends, but if this is the level of
functionality I can expect from Ubuntu going forward, I am going to have
no choice but to find another distribution that implements NFS
correctly. I intend to keep increasing the complexity of my network,
implementing full-network login accounts, printer sharing, etc. etc, and
no way do I intend to do it all under Samba! Sorry, this is all the
feedback I can provide on this bug, mainly that it affects me too and
that it seems basic to NFS and that it is a show-stopper for my intended
application. thx

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to nfs-utils in Ubuntu.
https://bugs.launchpad.net/bugs/879334

Title:
  nfsd from nfs-kernel-server very slow and system load from 25%-100%
  from nfsd

Status in “linux” package in Ubuntu:
  Incomplete
Status in “nfs-utils” package in Ubuntu:
  Confirmed
Status in “linux” package in Debian:
  Incomplete

Bug description:
  I have a diskless ubuntu 10.10 machine which I boot regularly using
  pxe-boot from another ubuntu machine where I have the root filesystem
  of the diskless machine exported over nfs.

  I set it up about a year ago using 10.10. In the mean while the server
  machine got upgraded to 11.04 and as of yesterday to 11.10.

  After the upgrade to 11.10 the diskless machine is dead slow (most of
  the times it wont even boot completely) and the load on the server
  machine is high (25%-100% as shown from top). If in the middle of the
  diskless computer booting I do a restart of the nfs server, the client
  computer proceeds with the boot a bit more and then it gets stuck
  again. I have to restart and nfs-server 3-4 times in order to get the
  gdm login screen at the client machine

  ProblemType: Bug
  DistroRelease: Ubuntu 11.10
  Package: nfs-kernel-server 1:1.2.4-1ubuntu2
  ProcVersionSignature: Ubuntu 3.0.0-12.20-generic 3.0.4
  Uname: Linux 3.0.0-12-generic i686
  ApportVersion: 1.23-0ubuntu3
  Architecture: i386
  Date: Fri Oct 21 12:53:02 2011
  ProcEnviron:
   LANG=en_US.UTF-8
   SHELL=/bin/bash
  SourcePackage: nfs-utils
  UpgradeStatus: Upgraded to oneiric on 2011-10-20 (1 days ago)

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/879334/+subscriptions