[Bug 1826857] Re: offloading disabled when using linuxbrige or bonding

James Page 1826857 at bugs.launchpad.net
Mon Jun 28 08:54:56 UTC 2021 

Revisiting this bug with new information.

I can confirm that when a bridge is involved in the data path, hardware
offloading of packets gets disabled as the network flow bumps to and
from the kernel and the NIC.

However with bonding and appropriate switchdev configuration, offloading
works fine and this is a more typical configuration.  Sticking to OVS
bridges rather than using Linux bridges as well will avoid this
situation.

AFAICT this is working as intended so marking this bug as a wont fix.

** Changed in: openvswitch (Ubuntu)
       Status: Confirmed => Won't Fix

** Changed in: cloud-archive
       Status: New => Won't Fix

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to openvswitch in Ubuntu.
https://bugs.launchpad.net/bugs/1826857

Title:
  offloading disabled when using linuxbrige or bonding

Status in Ubuntu Cloud Archive:
  Won't Fix
Status in openvswitch package in Ubuntu:
  Won't Fix

Bug description:
  Hi,

  I'm running Ubuntu 18.04 kernel 4.15.0-47-generic, openvswitch
  2.10.0-0ubuntu2~cloud0 set up by neutron (from openstack rocky) -
  VXLAN overlay.

  I've found out that as soon as a linuxbrige or bond is involved the
  network path of openvswitch, the offloading features of the NIC
  (checksumming, segmentation, VXLAN encap) get disabled, which reduces
  the throughput a lot.

  You can find the flows below. Examples of setup used :

  "Normal" setup :

  br-bond0 (192.168.11.11)
   bond0
    enp50s0f0
    enp113s0f0 (handling traffic)

  
  Only interface in the bond :

  br-bond0 (192.168.11.11)
   bond0
    enp50s0f0 (handling traffic)

  
  Interface directly in the bridge :

  br-bond0 (192.168.11.11)
   enp50s0f0

  With the setups above, there is NO offloading happening (tcpdump
  reports packet sizes of the MTU (~9k)). If the IP address is directly
  on the enp50s0f0 interface, then offloading works.

  For what it's worth, the NICs are Mellanox MT27800.

  Thanks !

  OVS flows :

  $ sudo ovs-vsctl show
  6f7e2040-e13b-4029-86b7-4f360aad2efe
      Manager "ptcp:6640:127.0.0.1"
          is_connected: true
      Bridge "br-physnet1"
          Controller "tcp:127.0.0.1:6633"
              is_connected: true
          fail_mode: secure
          Port "phy-br-physnet1"
              Interface "phy-br-physnet1"
                  type: patch
                  options: {peer="int-br-physnet1"}
          Port "br-physnet1"
              Interface "br-physnet1"
                  type: internal
      Bridge br-int
          Controller "tcp:127.0.0.1:6633"
              is_connected: true
          fail_mode: secure
          Port "sg-eecf32f7-da"
              tag: 2
              Interface "sg-eecf32f7-da"
                  type: internal
          Port "qvoc654e76e-de"
              tag: 2
              Interface "qvoc654e76e-de"
          Port "qr-84debef4-b7"
              tag: 2
              Interface "qr-84debef4-b7"
                  type: internal
          Port "qr-3e9ad9a7-02"
              tag: 3
              Interface "qr-3e9ad9a7-02"
                  type: internal
          Port "fg-997d43cb-27"
              tag: 5
              Interface "fg-997d43cb-27"
                  type: internal
          Port "qvoba76f8a7-50"
              tag: 2
              Interface "qvoba76f8a7-50"
          Port "qvod01a61c2-59"
              tag: 2
              Interface "qvod01a61c2-59"
          Port "enp50s0f0_60"
              tag: 2
              Interface "enp50s0f0_60"
          Port "tap15879e40-d9"
              tag: 2
              Interface "tap15879e40-d9"
          Port "int-br-physnet1"
              Interface "int-br-physnet1"
                  type: patch
                  options: {peer="phy-br-physnet1"}
          Port "qg-3f2f0aa4-fa"
              tag: 5
              Interface "qg-3f2f0aa4-fa"
                  type: internal
          Port "ha-a481b72f-6a"
              tag: 4
              Interface "ha-a481b72f-6a"
                  type: internal
          Port "enp50s0f0_62"
              tag: 2
              Interface "enp50s0f0_62"
          Port "sg-5e3c2880-79"
              tag: 3
              Interface "sg-5e3c2880-79"
                  type: internal
          Port "tap62452ada-90"
              tag: 3
              Interface "tap62452ada-90"
          Port br-int
              Interface br-int
                  type: internal
          Port "qvo75f1a5f8-e6"
              tag: 2
              Interface "qvo75f1a5f8-e6"
          Port "tapbc9a8481-c2"
              tag: 2
              Interface "tapbc9a8481-c2"
          Port "tapd7155195-25"
              tag: 1
              Interface "tapd7155195-25"
          Port patch-tun
              Interface patch-tun
                  type: patch
                  options: {peer=patch-int}
          Port "qvo1e23c34e-5f"
              tag: 2
              Interface "qvo1e23c34e-5f"
          Port "enp50s0f0_61"
              tag: 2
              Interface "enp50s0f0_61"
      Bridge br-ex
          Port br-ex
              Interface br-ex
                  type: internal
      Bridge br-tun
          Controller "tcp:127.0.0.1:6633"
              is_connected: true
          fail_mode: secure
          Port patch-int
              Interface patch-int
                  type: patch
                  options: {peer=patch-tun}
          Port br-tun
              Interface br-tun
                  type: internal
          Port "vxlan-c0a80a0b"
              Interface "vxlan-c0a80a0b"
                  type: vxlan
                  options: {df_default="true", in_key=flow, local_ip="192.168.11.11", out_key=flow, remote_ip="192.168.10.11"}
      ovs_version: "2.10.0"


  Features for all interfaces :
  $ for i in br-bond0 bond0 enp50s0f0 enp113s0f0; do sudo ethtool -k $i; echo "======"; done
  Features for br-bond0:
  rx-checksumming: off [fixed]
  tx-checksumming: on
          tx-checksum-ipv4: off [fixed]
          tx-checksum-ip-generic: on
          tx-checksum-ipv6: off [fixed]
          tx-checksum-fcoe-crc: off [fixed]
          tx-checksum-sctp: off [fixed]
  scatter-gather: on
          tx-scatter-gather: on
          tx-scatter-gather-fraglist: on
  tcp-segmentation-offload: on
          tx-tcp-segmentation: on
          tx-tcp-ecn-segmentation: on
          tx-tcp-mangleid-segmentation: on
          tx-tcp6-segmentation: on
  udp-fragmentation-offload: off
  generic-segmentation-offload: on
  generic-receive-offload: on
  large-receive-offload: off [fixed]
  rx-vlan-offload: off [fixed]
  tx-vlan-offload: on
  ntuple-filters: off [fixed]
  receive-hashing: off [fixed]
  highdma: on
  rx-vlan-filter: off [fixed]
  vlan-challenged: off [fixed]
  tx-lockless: on [fixed]
  netns-local: on [fixed]
  tx-gso-robust: off [requested on]
  tx-fcoe-segmentation: off [requested on]
  tx-gre-segmentation: on
  tx-gre-csum-segmentation: on
  tx-ipxip4-segmentation: on
  tx-ipxip6-segmentation: on
  tx-udp_tnl-segmentation: on
  tx-udp_tnl-csum-segmentation: on
  tx-gso-partial: on
  tx-sctp-segmentation: on
  tx-esp-segmentation: on
  fcoe-mtu: off [fixed]
  tx-nocache-copy: off
  loopback: off [fixed]
  rx-fcs: off [fixed]
  rx-all: off [fixed]
  tx-vlan-stag-hw-insert: on
  rx-vlan-stag-hw-parse: off [fixed]
  rx-vlan-stag-filter: off [fixed]
  l2-fwd-offload: off [fixed]
  hw-tc-offload: off [fixed]
  esp-hw-offload: off [fixed]
  esp-tx-csum-hw-offload: off [fixed]
  rx-udp_tunnel-port-offload: off [fixed]
  ======
  Features for bond0:
  rx-checksumming: off [fixed]
  tx-checksumming: on
          tx-checksum-ipv4: off [fixed]
          tx-checksum-ip-generic: on
          tx-checksum-ipv6: off [fixed]
          tx-checksum-fcoe-crc: off [fixed]
          tx-checksum-sctp: off [fixed]
  scatter-gather: on
          tx-scatter-gather: on
          tx-scatter-gather-fraglist: off [requested on]
  tcp-segmentation-offload: on
          tx-tcp-segmentation: on
          tx-tcp-ecn-segmentation: on
          tx-tcp-mangleid-segmentation: on
          tx-tcp6-segmentation: on
  udp-fragmentation-offload: off
  generic-segmentation-offload: on
  generic-receive-offload: on
  large-receive-offload: off
  rx-vlan-offload: on
  tx-vlan-offload: on
  ntuple-filters: off [fixed]
  receive-hashing: off [fixed]
  highdma: on
  rx-vlan-filter: on
  vlan-challenged: off [fixed]
  tx-lockless: on [fixed]
  netns-local: on [fixed]
  tx-gso-robust: off [fixed]
  tx-fcoe-segmentation: off [fixed]
  tx-gre-segmentation: on
  tx-gre-csum-segmentation: on
  tx-ipxip4-segmentation: on
  tx-ipxip6-segmentation: on
  tx-udp_tnl-segmentation: on
  tx-udp_tnl-csum-segmentation: on
  tx-gso-partial: off [fixed]
  tx-sctp-segmentation: off [fixed]
  tx-esp-segmentation: off [fixed]
  fcoe-mtu: off [fixed]
  tx-nocache-copy: off
  loopback: off [fixed]
  rx-fcs: off [fixed]
  rx-all: off [fixed]
  tx-vlan-stag-hw-insert: off [fixed]
  rx-vlan-stag-hw-parse: off [fixed]
  rx-vlan-stag-filter: off [fixed]
  l2-fwd-offload: off [fixed]
  hw-tc-offload: off [fixed]
  esp-hw-offload: off [fixed]
  esp-tx-csum-hw-offload: off [fixed]
  rx-udp_tunnel-port-offload: off [fixed]
  ======
  Features for enp50s0f0:
  rx-checksumming: on
  tx-checksumming: on
          tx-checksum-ipv4: on
          tx-checksum-ip-generic: off [fixed]
          tx-checksum-ipv6: on
          tx-checksum-fcoe-crc: off [fixed]
          tx-checksum-sctp: off [fixed]
  scatter-gather: on
          tx-scatter-gather: on
          tx-scatter-gather-fraglist: off [fixed]
  tcp-segmentation-offload: on
          tx-tcp-segmentation: on
          tx-tcp-ecn-segmentation: off [fixed]
          tx-tcp-mangleid-segmentation: off
          tx-tcp6-segmentation: on
  udp-fragmentation-offload: off
  generic-segmentation-offload: on
  generic-receive-offload: on
  large-receive-offload: off
  rx-vlan-offload: on
  tx-vlan-offload: on
  ntuple-filters: off
  receive-hashing: on
  highdma: on [fixed]
  rx-vlan-filter: on
  vlan-challenged: off [fixed]
  tx-lockless: off [fixed]
  netns-local: off [fixed]
  tx-gso-robust: off [fixed]
  tx-fcoe-segmentation: off [fixed]
  tx-gre-segmentation: on
  tx-gre-csum-segmentation: on
  tx-ipxip4-segmentation: off [fixed]
  tx-ipxip6-segmentation: off [fixed]
  tx-udp_tnl-segmentation: on
  tx-udp_tnl-csum-segmentation: on
  tx-gso-partial: on
  tx-sctp-segmentation: off [fixed]
  tx-esp-segmentation: off [fixed]
  fcoe-mtu: off [fixed]
  tx-nocache-copy: off
  loopback: off [fixed]
  rx-fcs: off
  rx-all: off
  tx-vlan-stag-hw-insert: on
  rx-vlan-stag-hw-parse: off [fixed]
  rx-vlan-stag-filter: on [fixed]
  l2-fwd-offload: off [fixed]
  hw-tc-offload: on
  esp-hw-offload: off [fixed]
  esp-tx-csum-hw-offload: off [fixed]
  rx-udp_tunnel-port-offload: on
  ======
  Features for enp113s0f0:
  rx-checksumming: on
  tx-checksumming: on
          tx-checksum-ipv4: on
          tx-checksum-ip-generic: off [fixed]
          tx-checksum-ipv6: on
          tx-checksum-fcoe-crc: off [fixed]
          tx-checksum-sctp: off [fixed]
  scatter-gather: on
          tx-scatter-gather: on
          tx-scatter-gather-fraglist: off [fixed]
  tcp-segmentation-offload: on
          tx-tcp-segmentation: on
          tx-tcp-ecn-segmentation: off [fixed]
          tx-tcp-mangleid-segmentation: off
          tx-tcp6-segmentation: on
  udp-fragmentation-offload: off
  generic-segmentation-offload: on
  generic-receive-offload: on
  large-receive-offload: off
  rx-vlan-offload: on
  tx-vlan-offload: on
  ntuple-filters: off
  receive-hashing: on
  highdma: on [fixed]
  rx-vlan-filter: on
  vlan-challenged: off [fixed]
  tx-lockless: off [fixed]
  netns-local: off [fixed]
  tx-gso-robust: off [fixed]
  tx-fcoe-segmentation: off [fixed]
  tx-gre-segmentation: on
  tx-gre-csum-segmentation: on
  tx-ipxip4-segmentation: off [fixed]
  tx-ipxip6-segmentation: off [fixed]
  tx-udp_tnl-segmentation: on
  tx-udp_tnl-csum-segmentation: on
  tx-gso-partial: on
  tx-sctp-segmentation: off [fixed]
  tx-esp-segmentation: off [fixed]
  fcoe-mtu: off [fixed]
  tx-nocache-copy: off
  loopback: off [fixed]
  rx-fcs: off
  rx-all: off
  tx-vlan-stag-hw-insert: on
  rx-vlan-stag-hw-parse: off [fixed]
  rx-vlan-stag-filter: on [fixed]
  l2-fwd-offload: off [fixed]
  hw-tc-offload: on
  esp-hw-offload: off [fixed]
  esp-tx-csum-hw-offload: off [fixed]
  rx-udp_tunnel-port-offload: on
  ======

  ProblemType: Bug
  DistroRelease: Ubuntu 18.04
  Package: openvswitch-switch 2.10.0-0ubuntu2~cloud0 [origin: Canonical]
  ProcVersionSignature: Ubuntu 4.15.0-47.50-generic 4.15.18
  Uname: Linux 4.15.0-47-generic x86_64
  ApportVersion: 2.20.9-0ubuntu7.6
  Architecture: amd64
  CrashDB:
   {
                  "impl": "launchpad",
                  "project": "cloud-archive",
                  "bug_pattern_url": "http://people.canonical.com/~ubuntu-archive/bugpatterns/bugpatterns.xml",
               }
  Date: Mon Apr 29 09:28:24 2019
  ProcEnviron:
   TERM=screen-256color
   PATH=(custom, no user)
   LANG=C.UTF-8
   SHELL=/bin/bash
  SourcePackage: openvswitch
  UpgradeStatus: No upgrade log present (probably fresh install)

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/1826857/+subscriptions

More information about the Ubuntu-openstack-bugs mailing list