[Bug 1918936] Re: ipset does NSS lookups even if ports are numeric

[Haw Loeung]

Tested on a Focal VM in Canonistack.

Using current ipset:

| ubuntu at juju-87625f-hloeung-110:~$ sudo apt-get install ipset
| ...
| Get:1 http://us.archive.ubuntu.com/ubuntu focal/main amd64 libipset13 amd64 7.5-1~exp1 [53.4 kB]
| Get:2 http://us.archive.ubuntu.com/ubuntu focal/main amd64 ipset amd64 7.5-1~exp1 [29.8 kB]

| ubuntu at juju-87625f-hloeung-110:~$ time sudo ipset restore < whitelist-ipv4-2021-06-29-with-proto-nums
|
| real    1m55.870s
| user    1m3.704s
| sys     0m26.604s

Using the one in -proposed:

| Get:1 http://us.archive.ubuntu.com/ubuntu focal-proposed/main amd64 ipset amd64 7.5-1ubuntu0.20.04.1 [29.8 kB]
| Get:1 http://us.archive.ubuntu.com/ubuntu focal-proposed/main amd64 libipset13 amd64 7.5-1ubuntu0.20.04.1 [53.6 kB]

| ubuntu at juju-87625f-hloeung-110:~$ time sudo ipset restore < whitelist-ipv4-2021-06-29-with-proto-nums
|
| real    0m41.372s
| user    0m18.724s
| sys     0m16.023s

Full output - https://paste.ubuntu.com/p/w74Kjvnn84/

** Tags removed: verification-needed-focal
** Tags added: verification-done-focal

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to the bug report.
https://bugs.launchpad.net/bugs/1918936

Title:
  ipset does NSS lookups even if ports are numeric

Status in ipset package in Ubuntu:
  Fix Released
Status in ipset source package in Bionic:
  Invalid
Status in ipset source package in Focal:
  Fix Committed
Status in ipset source package in Groovy:
  Won't Fix
Status in ipset source package in Hirsute:
  Fix Committed
Status in ipset source package in Impish:
  Fix Committed
Status in ipset source package in Jammy:
  Fix Released

Bug description:
  [Impact]
  A change included ipset 6.37 as a performance regression as all ip set rule incur a getprotocolbyname lookup, irrespective of whether the name of the protocol or the actual port number is specified in the set configuration.  For large sets this can double the speed of applying changes to ipset tables.

  [Test Plan]
  # Create a suitable large set of data to restore to the ipset
  for x in `seq 1 7`; do for y in `seq 1 254`; do for z in `seq 1 254`; do echo "add test 10.1.1.0/21,80,150.$x.$y.$z/32" >> whitelist-ipv4 ;done; done; done

  # Destroy,create, restore
  sudo ipset destroy test
  sudo ipset create test hash:net,port,net hashsize 4096 maxelem 786432
  time sudo ipset restore < ./whitelist-ipv4

  Large reduction in time taken to restore the ipset (32s-> 5s on an 8
  core machine).

  [Where problems could occur]
  The original patch to resolve this issue did introduce another bug which as subsequently been fixed as well (and is included in the updated packages).

  If the fix introduces issues its likely that iptable rules making use
  of ipset groups will start to fail in some way - probably rejecting
  traffic or suchlike.

  [Other Info]

  [Original Bug Report]
  Hi,

  Do you think we could get
  https://git.netfilter.org/ipset/commit/?id=dbeb20a667e82e4efb8b26b24a0ec641dab5c857
  SRUed to 20.04 ?

  This divides our ipset loading time by ~2 (from ~60s to ~25s).

  Thanks

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/ipset/+bug/1918936/+subscriptions