[Bug 1862044] Re: packet loss, extra latency and lower bandwidth on bionic
Pauloone
1862044 at bugs.launchpad.net
Thu Feb 6 11:17:20 UTC 2020
Reproducing step:
Take 3 machines, connect them on the same network through a single switch (to limits network bandwith/latency effects)
run uperf on 3 of them, 2 slave and one master.
I am using the following uperf script:
<?xml version="1.0"?>
<profile name="xml">
<group nthreads="1">
<transaction iterations="1">
<flowop type="connect" options="remotehost=$h1 protocol=tcp
wndsz=50k tcp_nodelay"/>
</transaction>
<transaction duration="30s">
<flowop type="read" options="size=8k"/>
<flowop type="write" options="size=8k"/>
</transaction>
<transaction iterations="1">
<flowop type="disconnect" />
</transaction>
</group>
<group nthreads="1">
<transaction iterations="1">
<flowop type="connect" options="remotehost=$h2 protocol=tcp
wndsz=50k tcp_nodelay"/>
</transaction>
<transaction duration="30s">
<flowop type="write" options="size=8k"/>
<flowop type="read" options="size=8k"/>
</transaction>
<transaction iterations="1">
<flowop type="disconnect" />
</transaction>
</group>
<group nthreads="1">
<transaction iterations="1">
<flowop type="connect" options="remotehost=$h1 protocol=udp
wndsz=50k tcp_nodelay"/>
</transaction>
<transaction duration="30s">
<flowop type="read" options="size=1.4k"/>
<flowop type="write" options="size=1.4k"/>
</transaction>
<transaction iterations="1">
<flowop type="disconnect" />
</transaction>
</group>
<group nthreads="$n">
<transaction iterations="1">
<flowop type="connect" options="remotehost=$h2 protocol=udp
wndsz=50k tcp_nodelay"/>
</transaction>
<transaction duration="30s">
<flowop type="write" options="size=1.4k"/>
<flowop type="read" options="size=1.4k"/>
</transaction>
<transaction iterations="1">
<flowop type="disconnect" />
</transaction>
</group>
<group nthreads="$n">
<transaction iterations="1">
<flowop type="connect" options="remotehost=$h1 protocol=udp
wndsz=50k tcp_nodelay"/>
</transaction>
<transaction duration="30s">
<flowop type="read" options="size=0.2k"/>
<flowop type="write" options="size=0.2k"/>
</transaction>
<transaction iterations="1">
<flowop type="disconnect" />
</transaction>
</group>
<group nthreads="$n">
<transaction iterations="1">
<flowop type="connect" options="remotehost=$h2 protocol=udp
wndsz=50k tcp_nodelay"/>
</transaction>
<transaction duration="30s">
<flowop type="write" options="size=0.2k"/>
<flowop type="read" options="size=0.2k"/>
</transaction>
<transaction iterations="1">
<flowop type="disconnect" />
</transaction>
</group>
</profile>
if you put n big enough (50 in my case seems a good value) on the same hardware you will have significant throughput differences between Bionic and Xenial.
On hundred runs, I see an avergae throuhput 719,8 Mbits/s in and 719,9 Mbits/s out on xenial and 608,2 Mbits/s in and 608,3 Mbits/s out on Bionic. That represent 20% network performance on bionic.
Also if I run sudo tc -s qdisc or netstat, we see packets been dropped
on Bionic but not Xenial.
I ran that test because we see network performance issue (TCP
timeouts/packet drop) when validating our application on bionic in our
datacenter, which is using a completly different hardware.
--
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to systemd in Ubuntu.
https://bugs.launchpad.net/bugs/1862044
Title:
packet loss, extra latency and lower bandwidth on bionic
Status in systemd package in Ubuntu:
Incomplete
Bug description:
We are investigating migrating our servers from Xenial to Bionic.
We are seeing significant network differences between the 2 LTS versions.
Using the same hardware an machines connected to a single swith on the same network, we see:
10% less packets throuput on bionic in comparision to Xenial when running uperf.
Also "tc -s qdisc" report packets been dropped on Bionic but not Xenial.
The command iperf -c <host> -u -d -p 20 timeout on Bionic and not on Xenial, and if you reduce the value of p, you will see more packet drops on Bionic.
We tryied to upgrade the xenial kernel to the same version than
bionic. We did not see any improvement.
All the tests are performed with 4 identical physical machines
connected to the same physical switch. With one bionic machine, one
Xenial machine and 2 extra Xenial machine that act as iperf/uperf
clients.
All 4 nodes are configured as ntp peers with each others. Under load,
the bionic machine see siginificant jitter improvement with the 3
others machines. We don't see that with the Xenial hosts.
I doesn't seems to be an hardware compatibility issue as we see
significant network performance loss in our datacenter on fully
different hardware.
ProblemType: Bug
DistroRelease: Ubuntu 18.04
Package: iperf 2.0.10+dfsg1-1ubuntu0.18.04.2
ProcVersionSignature: Ubuntu 4.15.0-76.86-lowlatency 4.15.18
Uname: Linux 4.15.0-76-lowlatency x86_64
ApportVersion: 2.20.9-0ubuntu7.9
Architecture: amd64
Date: Wed Feb 5 17:14:25 2020
Dependencies:
gcc-8-base 8.3.0-6ubuntu1~18.04.1
libc6 2.27-3ubuntu1
libgcc1 1:8.3.0-6ubuntu1~18.04.1
libstdc++6 8.3.0-6ubuntu1~18.04.1
InstallationDate: Installed on 2020-01-29 (7 days ago)
InstallationMedia: Ubuntu-Server 18.04.3 LTS "Bionic Beaver" - Release amd64 (20190805)
ProcEnviron:
TERM=xterm-256color
PATH=(custom, no user)
XDG_RUNTIME_DIR=<set>
LANG=en_US.UTF-8
SHELL=/bin/bash
SourcePackage: iperf
UpgradeStatus: No upgrade log present (probably fresh install)
ProblemType: Bug
DistroRelease: Ubuntu 18.04
Package: systemd 237-3ubuntu10.29
ProcVersionSignature: Ubuntu 4.15.0-76.86-lowlatency 4.15.18
Uname: Linux 4.15.0-76-lowlatency x86_64
ApportVersion: 2.20.9-0ubuntu7.9
Architecture: amd64
Date: Wed Feb 5 17:29:53 2020
ExecutablePath: /lib/systemd/systemd
InstallationDate: Installed on 2020-01-29 (7 days ago)
InstallationMedia: Ubuntu-Server 18.04.3 LTS "Bionic Beaver" - Release amd64 (20190805)
MachineType: Dell Inc. XPS 8700
ProcEnviron:
TERM=linux
PATH=(custom, no user)
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-4.15.0-76-lowlatency root=UUID=400563c9-8749-47b7-8abb-1efdb57bc589 ro
SourcePackage: systemd
UpgradeStatus: No upgrade log present (probably fresh install)
dmi.bios.date: 05/16/2013
dmi.bios.vendor: Dell Inc.
dmi.bios.version: A01
dmi.board.name: 0KWVT8
dmi.board.vendor: Dell Inc.
dmi.board.version: A00
dmi.chassis.type: 3
dmi.chassis.vendor: Dell Inc.
dmi.modalias: dmi:bvnDellInc.:bvrA01:bd05/16/2013:svnDellInc.:pnXPS8700:pvr:rvnDellInc.:rn0KWVT8:rvrA00:cvnDellInc.:ct3:cvr:
dmi.product.family: To be filled by O.E.M.
dmi.product.name: XPS 8700
dmi.sys.vendor: Dell Inc.
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1862044/+subscriptions
More information about the foundations-bugs
mailing list