[Bug 2109303] Re: dhclient hang when tons of ip addresses exist

[Dongdong Wang]

After a long time of debugging, I think I found the root cause of the
hang.

The bug has been firstly introduced by the patch[1], see [2] for details.
The patch aimed to solve the multi-ip-addr-per-interface issue, but mistakely introduced another issue.

It mistakely calls getifaddrs() in the "while (next_iface(&info, &err, &ifaces))" loop.
As we reproduced above, there are 2500 ip addresses in the system, which will cause the loop runs 2500 x 2500 times, and totally calls getifaddrs() 2500 times too.

The getifaddrs() is not free, it allocates memories which need to call
freeifaddrs() to free them. But the patch[1] does not do that, causing
the dhclient's memory increases very fast, like a memory leak.

To fix the BUG, I propose a patch (the attachment), which reverts the patch[1] and append a tiny fix.
The fix is pretty simple and already exists in the source code of another next_iface() routine.
It just trims the "eth0:1" to "eth0" in the next_iface() routine, and returns the "eth0" to the caller, no need to loop 2500 times for each address.

After the fix, the dhclient hang issue disappears.

[1] https://launchpadlibrarian.net/66470806/dhcpd.workaround.patch
[2] https://bugs.launchpad.net/eucalyptus/+bug/717166

** Patch added: "dhclient-fix-hang.patch"
   https://bugs.launchpad.net/ubuntu/+source/isc-dhcp/+bug/2109303/+attachment/5874217/+files/dhclient-fix-hang.patch

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to isc-dhcp in Ubuntu.
https://bugs.launchpad.net/bugs/2109303

Title:
  dhclient hang when tons of ip addresses exist

Status in isc-dhcp package in Ubuntu:
  New

Bug description:
  Ubuntu
  ======

  No LSB modules are available.
  Description:    Ubuntu 24.04.2 LTS
  Release:        24.04

  Package
  =======

  isc-dhcp-client:
    Installed: 4.4.3-P1-4ubuntu2
    Candidate: 4.4.3-P1-4ubuntu2
    Version table:
   *** 4.4.3-P1-4ubuntu2 500
          500 http://archive.ubuntu.com/ubuntu noble/universe amd64 Packages
          100 /var/lib/dpkg/status

  Steps to Reproduce
  ==================

  # ip l add dev dummy type dummy
  # ip l set dev dummy up
  # for i in {1..50}; do for j in {1..50}; do ip a a 192.168.$i.$j/24 dev dummy; done; done
  # ip -o a show dev dummy | wc -l
  2500
  # dhclient -v dummy

  Result
  ======

  After the operations above, we could see the dhclient hanging. 
  The top command showed the dhclient's memory was increasing,
  seemed a memory leak happened. Also CPU usage was pretty high.

  top - 02:13:48 up 12 days, 23:59,  0 user,  load average: 2.99, 2.62, 2.43
  Tasks:   5 total,   2 running,   3 sleeping,   0 stopped,   0 zombie
  %Cpu(s):  1.8 us,  1.9 sy,  0.0 ni, 96.3 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
  MiB Mem : 772016.8 total, 614554.4 free, 111778.8 used,  50785.8 buff/cache
  MiB Swap:      0.0 total,      0.0 free,      0.0 used. 660238.0 avail Mem

      PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
     2870 root      20   0  283.4m 282.3m   2.3m R 100.0   0.0   5:05.29 dhclient
        1 root      20   0    4.5m   3.8m   3.2m S   0.0   0.0   0:00.00 bash
       10 root      20   0    4.6m   4.1m   3.4m S   0.0   0.0   0:00.59 bash
     2869 root      20   0    3.2m   1.3m   1.2m S   0.0   0.0   0:00.00 dhclient
     2882 root      20   0    8.7m   5.0m   2.9m R   0.0   0.0   0:00.01 top

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/isc-dhcp/+bug/2109303/+subscriptions