[Bug 1940908] Re: resolved: closes listening socket too rapidly and sends Destination port unreachable

[Dan Streetman]

> The local systemd-resolved fails to resolve the name so client
applications return failures. Most notably it shows up in web browsers
but also for apt package updates.

still not following you on why it's causing systemd-resolved to fail to
resolve the name.

the new socket graveyard helps avoid sending icmp errors to upstream
nameservers, when resolved is performing a lookup to multiple
nameservers in parallel. it should make no difference to the local
client that just wants to know the details of the dns lookup.

maybe it would help if you gave more details about how to reproduce the
actual local dns lookup failure.

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to systemd in Ubuntu.
https://bugs.launchpad.net/bugs/1940908

Title:
  resolved: closes listening socket too rapidly and sends Destination
  port unreachable

Status in systemd package in Ubuntu:
  Incomplete

Bug description:
  Afffects Ubuntu 18.04 through 21.04 (fixes are in systemd v248)

  With systemd v245 (and v247) and systemd-resolved we're seeing
  frequent problems due to resolved rapidly closing the socket on which
  it sends out a query before the server has answered. The server
  answers and then resolved sends an ICMP Destination Unreachable (Port
  Unreachable) response!

  This breaks name lookups frequently. In our case the DNS server is
  reached via a Wireguard tunnel over a satellite link and latencies can
  vary.

  A typical example captured via tcpdump:

  07:22:03.446919 IP6 fddc:7e00:e001:ee00:fffe:f875:a4f3:42b4.45338 > fddc:7e00:e001:ee00::1.53: 2963+ [1au] AAAA? contile-images.services.mozilla.com. (64)
  07:22:03.501089 IP6 fddc:7e00:e001:ee00::1.53 > fddc:7e00:e001:ee00:fffe:f875:a4f3:42b4.45338: 2963 1/0/1 AAAA 2a01:7e00:e001:ee64::2278:7366 (92)
  07:22:03.501152 IP6 fddc:7e00:e001:ee00:fffe:f875:a4f3:42b4 > fddc:7e00:e001:ee00::1: ICMP6, destination unreachable, unreachable port, fddc:7e00:e001:ee00:fffe:f875:a4f3:42b4 udp port 45338, length 148

  The time difference here is only 0.054170 and there is no way to alter
  the timeout in resolved.

  There are recent upstream commits to fix this which ought to be
  cherry-picked. See:

  https://github.com/systemd/systemd/issues/17421

  https://github.com/systemd/systemd/pull/17535

  https://github.com/systemd/systemd/commit/e03d156f78cb5a0cac85d1e1310d89fdfa4f1b88

  If I am reading the code correctly the timeout is very short:

  src/resolve/resolved-dns-transaction.c:22:#define DNS_TIMEOUT_USEC
  (SD_RESOLVED_QUERY_TIMEOUT_USEC / DNS_TRANSACTION_ATTEMPTS_MAX)

  src/resolve/resolved-def.h:79:#define SD_RESOLVED_QUERY_TIMEOUT_USEC
  (120 * USEC_PER_SEC)

  src/resolve/resolved-dns-transaction.h:212:#define
  DNS_TRANSACTION_ATTEMPTS_MAX 24

  So in micro-seconds that is 120 /24 = 5 per query with, as inferred,
  up to 24 attempts (I don't see multiple duplicate requests on the wire
  so not sure DNS_TRANSACTION_ATTEMPTS_MAX affects this.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1940908/+subscriptions