[Bug 1834994] Re: rsync: --xattrs is extremely slow

[Bryce Harrington]

This issue was fixed in Debian with the attached patch from upstream.

SRU process prohibits version jumps so would reject directly updating
the rsync package on Ubuntu 16.04 to 3.2.2-2.

However, if you can define a test case to reliably reproduce the bug,
then it may be possible to get the specific patch accepted.  Would you
be willing to help draft a set of steps to reproduce the bug -
preferably something that can be performed within a few minutes or an
hour, as opposed to a few weeks (else it'll be too hard for folks to
test)?

You can see other steps required for filing an SRU at
https://wiki.ubuntu.com/StableReleaseUpdates.  Anything you can do
towards making this bug report fulfill those requirements will make it
more likely for this to get attention.

One of the potential problems in this case is that the patch is rather
large so may be harder to review.  SRU's tend to go faster when the
patch is just a few lines.

** Patch added: "speedup-xattrs.diff"
   https://bugs.launchpad.net/ubuntu/+source/rsync/+bug/1834994/+attachment/5274955/+files/speedup-xattrs.diff

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to rsync in Ubuntu.
https://bugs.launchpad.net/bugs/1834994

Title:
  rsync: --xattrs is extremely slow

Status in rsync package in Ubuntu:
  New

Bug description:
  Ubuntu 16.04 LTS repositories contain only rsync version 3.1.1:

  $ apt policy rsync
  rsync:
    Installed: 3.1.1-3ubuntu1.2
    Candidate: 3.1.1-3ubuntu1.2
    Version table:
   *** 3.1.1-3ubuntu1.2 500
          500 http://fi.archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages
          500 http://security.ubuntu.com/ubuntu xenial-security/main amd64 Packages
          100 /var/lib/dpkg/status
       3.1.1-3ubuntu1 500
          500 http://fi.archive.ubuntu.com/ubuntu xenial/main amd64 Packages

  However, this version has a bug where flag --xattrs causes very very
  high CPU usage if source contains lots of unique extended attributes.
  I'm trying to sync 13 TB (roughly 6.5 M files) with extended
  attributes and this bug seems to make this slower and slower as the
  time goes by. The system has been copying the stuff for three weeks
  (!) now. It's getting slower every day but still seems to make
  progress so I try to run it to completion. First couple of terabytes
  suggested that this sync operation should have taken around 4 days.
  The source directory is used by glusterfs which creates checksums as
  extended attributes and practically every file has different checksum.

  Please, update rsync package on Ubuntu 16.04 to 3.1.2-2 or later.

  See also: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=799143

  ProblemType: Bug
  DistroRelease: Ubuntu 16.04
  Package: rsync 3.1.1-3ubuntu1.2
  ProcVersionSignature: Ubuntu 4.15.0-51.55~16.04.1-lowlatency 4.15.18
  Uname: Linux 4.15.0-51-lowlatency x86_64
  NonfreeKernelModules: nvidia_uvm nvidia_drm nvidia_modeset nvidia
  ApportVersion: 2.20.1-0ubuntu2.18
  Architecture: amd64
  CurrentDesktop: MATE
  Date: Tue Jul  2 09:07:49 2019
  InstallationDate: Installed on 2015-02-23 (1589 days ago)
  InstallationMedia: Ubuntu 14.04.2 LTS "Trusty Tahr" - Release amd64 (20150218.1)
  SourcePackage: rsync
  UpgradeStatus: Upgraded to xenial on 2016-06-10 (1116 days ago)

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/rsync/+bug/1834994/+subscriptions