[Bug 2019094] Re: [SRU] Focal: TLS 1.3 doesn't work on strict firewall/middlebox
Mauricio Faria de Oliveira
2019094 at bugs.launchpad.net
Thu May 11 15:35:22 UTC 2023
Summary:
-------
The root cause is probably the firewall/middlebox not liking the TLS 1.3
handshake w/out Session ID, sending a RST packet to close the connection.
Output:
------
Jammy (works):
- Handshake was completed
...
|<10>| READ: Got 5 bytes from 0x7ffd40a3e620
Focal (fails):
- Handshake was completed
...
|<10>| READ: -1 returned from 0x7ffc4d8cc6c0, errno=104 gerrno=0
Source:
------
pull_func() returned -1 with EBADMSG errno, thus the error message.
Error message:
309 static ssize_t
310 _gnutls_stream_read(...)
...
346 i = pull_func(fd, &ptr[size - left], left);
347
348 if (i < 0) {
...
351 _gnutls_read_log
352 ("READ: %d returned from %p, errno=%d gerrno=%d",
353 (int) i, fd, errno,
354 session->internals.errnum);
...
Called function:
342 static ssize_t pull_func(...)
...
351 ret =
352 recvfrom(priv->fd, data, size, 0,
353 (struct sockaddr *) &cli_addr, &cli_addr_size);
354 if (ret == -1)
355 return ret;
...
Error number:
# ifndef EBADMSG
# define EBADMSG 104
# define GNULIB_defined_EBADMSG 1
# endif
Packet capture:
--------------
(The packet capture cannot be provided for privacy/security reasons.)
** Description changed:
+ < IN PROGRESS >
+
[ Impact ]
- When registering a focal VM to landscape server in a private network by
- landscape-config, it fails to register and shows:
+ * On Focal, the TLS 1.3 handshake might fail on strict
+ (or misbehaving) proprietary firewall/middlebox that
+ requires a non-empty Session ID (as TLS 1.2) per RFC.
- We were unable to contact the server.
- Your internet connection may be down. The landscape client will continue to try and contact the server periodically.
+ * The RFC specifies the ClientHello should always have
+ a non-empty session ID, but this _is_ empty in Focal.
- But registration works on bionic and jammy in the same network
+ * RFC 8446, Appendix D.4. Middlebox Compatibility Mode [1]
+ """
+ ... a significant number of middleboxes misbehave
+ when a TLS client/server pair negotiates TLS 1.3.
+ ... handshake look more like a TLS 1.2 handshake:
+
+ - The client always provides a non-empty session ID
+ in the ClientHello, ...
+ """
+
+ * Reverse build dependencies that link against the
+ static libraries in libgnutls28-dev (check needed)
+ would need No-Change Rebuilds to pick up this fix.
+ (see `reverse-depends -b -r focal libgnutls28-dev`)
+
+ [ Fix ]
+
+ * Analysis summary based on packet capture and source
+ code is provided in comment #3.
+
+ * GnuTLS 3.7.1 includes fix commit [2], Focal is 3.6.13.
+
+ $ git describe --contains e0bb98e1f71f94691f600839ff748d3a9f469d3e
+ 3.7.1~32^2
+
+ * There's an incremental fix-up commit [3]:
+
+ $ git describe --contains 05ee0d49fe93d8812ef220c7b830c4b3553ac4fd
+ 3.7.1~17^2~2
+
+ [ Test Plan ]
+
+ * Check whether TLS 1.3 handshake has `Session ID:`
+
+ - Focal (no):
+ $ gnutls-cli --priority NORMAL:-VERS-ALL:+VERS-TLS1.3 ubuntu.com </dev/null
+ ...
+ - Description: (TLS1.3-X.509)-...
+ - Options:
+ - Handshake was completed
+ ...
+
+ - Jammy (yes):
+ $ gnutls-cli --priority NORMAL:-VERS-ALL:+VERS-TLS1.3 ubuntu.com </dev/null
+ ...
+ - Description: (TLS1.3-X.509)-...
+ - Session ID: CB:7D:DF:...
+ - Options:
+ - Handshake was completed
+ ...
+
+ * Check tests run at build time (`Testsuite summary for GnuTLS`).
+
+ Tests passed per the build log from PPA with test packages:
+
+ ============================================================================
+ Testsuite summary for GnuTLS 3.6.13
+ ============================================================================
+ # TOTAL: 25
+ # PASS: 8
+ # SKIP: 17
+ # XFAIL: 0
+
+ * Check autopkgtests from gnutls28 against PPA/SRU [4].
+
+ Tests passed against PPA with test packages:
+
+ autopkgtest [13:47:40]: @@@@@@@@@@@@@@@@@@@@ summary
+ run-upstream-testsuite PASS
+
+ * Check autopkgtests from reverse test triggers against PPA/SRU
+
+ $ reverse-depends --build-depends --release focal libgnutls28-dev
+ Reverse-Testsuite-Triggers
+ * apt
+ * cmake
+ * gnustep-base
+ * knot-resolver
+ * lxc
+ * riemann-c-client
+ * squid
+ ...
+
+ [ Regression Potential ]
+
+ * TLS 1.3 handshake now includes non-empty Session ID
+ in ClientHello, so there's a behavior change in the
+ Client side-only, but it does affect how particular
+ Servers handle the client, depending on Session ID.
+
+ * Thus, theoretically, if issues were to occur, that
+ likely would manifest as client connection errors
+ with TLS 1.3 (failures would be realized early and
+ fast), and a workaround available is using TLS 1.2.
+
+ * Even though changes to TLS handshake understandably
+ may be scary (considering the impact of regressions),
+ the proposed change is specified by the RFC (and is
+ there to help w/ wider compatibility) and is already
+ implemented in later versions (3.7.1 in Hirsute [5]).
+
+ [ Other Info ]
+
+ * Bionic is not impacted (TLS 1.2 only)
+ * Jammy and later already fixed (TLS 1.3 on GnuTLS 3.7+)
+
+ [ Links ]
+
+ [1] https://www.rfc-editor.org/rfc/rfc8446#appendix-D.4
+ [2] https://gitlab.com/gnutls/gnutls/-/commit/e0bb98e1f71f94691f600839ff748d3a9f469d3e
+ [3] https://gitlab.com/gnutls/gnutls/-/commit/05ee0d49fe93d8812ef220c7b830c4b3553ac4fd
+ [4] https://autopkgtest.ubuntu.com/packages/g/gnutls28
+ [5] https://launchpad.net/ubuntu/+source/gnutls28/3.7.1-3ubuntu1
+
+
+ Issue: https://gitlab.com/gnutls/gnutls/-/issues/1074
+ FixMR: https://gitlab.com/gnutls/gnutls/-/merge_requests/1350
+
+ [ Original Description ]
+
+ <Removed bits related to Landscape, which is not at fault.>
Use gnutls-cli to check, it shows:
...
- Description: (TLS1.3-X.509)-(ECDHE-SECP256R1)-(RSA-PSS-RSAE-SHA256)-(AES-256-GCM)
- Options:
- Handshake was completed
- Simple Client Mode:
*** Fatal error: Error in the pull function.
*** Server has terminated the connection abnormally.
gnutls version:
bionic 3.5.18
focal 3.6.13
jammy 3.7.3
gnutls 3.5 doesn't support TLS1.3 so it's using TLS1.2
and both 3.6.x and 3.7.x are using TLS1.3, but only 3.7.x works
We built gnutls from upstream and found the same issue:
3.16.3 isn't working and 3.7.3 is working
- so we did a bisect and found this commit fixes the issue for TLS1.3 on 3.6.x:
+ so we did a bisect and found this commit fixes the issue for TLS1.3 on 3.7.x:
commit e0bb98e1f71f94691f600839ff748d3a9f469d3e
Author: Norbert Pocs <npocs at redhat.com>
Date: Fri Oct 30 17:18:30 2020 +0100
Fix non-empty session id (TLS13_APPENDIX_D4)
When TLS1.3 is used with middlebox compatible mode, the session id should be filled with random session id,
but remained empty.
Signed-off-by: Norbert Pocs <npocs at redhat.com>
Closes #1074
We need to SRU this commit to libgnutls28 on focal
-
- [ Test Plan ]
-
- I build a test package with above commit on focal in this PPA
- https://launchpad.net/~gerald-yang-tw/+archive/ubuntu/359157
-
- and confirmed it fixes the issue, TLS1.3 works on focal in the same
- network
-
- [ Where problems could occur ]
-
- With this commit, both public network and private network work fine
- and it fixes TLS1.3 used in middlebox compatible mode
- I couldn't see any potential issue here
-
- [ Other Info ]
-
- https://gitlab.com/gnutls/gnutls/-/merge_requests/1350
- https://gitlab.com/gnutls/gnutls/-/issues/1074
** Description changed:
< IN PROGRESS >
[ Impact ]
- * On Focal, the TLS 1.3 handshake might fail on strict
- (or misbehaving) proprietary firewall/middlebox that
- requires a non-empty Session ID (as TLS 1.2) per RFC.
+ * On Focal, the TLS 1.3 handshake might fail on strict
+ (or misbehaving) proprietary firewall/middlebox that
+ requires a non-empty Session ID (as TLS 1.2) per RFC.
- * The RFC specifies the ClientHello should always have
- a non-empty session ID, but this _is_ empty in Focal.
+ * The RFC specifies the ClientHello should always have
+ a non-empty session ID, but this _is_ empty in Focal.
- * RFC 8446, Appendix D.4. Middlebox Compatibility Mode [1]
- """
- ... a significant number of middleboxes misbehave
- when a TLS client/server pair negotiates TLS 1.3.
- ... handshake look more like a TLS 1.2 handshake:
+ * RFC 8446, Appendix D.4. Middlebox Compatibility Mode [1]
+ """
+ ... a significant number of middleboxes misbehave
+ when a TLS client/server pair negotiates TLS 1.3.
+ ... handshake look more like a TLS 1.2 handshake:
- - The client always provides a non-empty session ID
- in the ClientHello, ...
- """
-
- * Reverse build dependencies that link against the
- static libraries in libgnutls28-dev (check needed)
- would need No-Change Rebuilds to pick up this fix.
- (see `reverse-depends -b -r focal libgnutls28-dev`)
+ - The client always provides a non-empty session ID
+ in the ClientHello, ...
+ """
+
+ * Reverse build dependencies that link against the
+ static libraries in libgnutls28-dev (check needed)
+ would need No-Change Rebuilds to pick up this fix.
+ (see `reverse-depends -b -r focal libgnutls28-dev`)
[ Fix ]
- * Analysis summary based on packet capture and source
- code is provided in comment #3.
-
- * GnuTLS 3.7.1 includes fix commit [2], Focal is 3.6.13.
-
- $ git describe --contains e0bb98e1f71f94691f600839ff748d3a9f469d3e
- 3.7.1~32^2
+ * Analysis summary based on packet capture and source
+ code is provided in comment #3.
- * There's an incremental fix-up commit [3]:
-
- $ git describe --contains 05ee0d49fe93d8812ef220c7b830c4b3553ac4fd
- 3.7.1~17^2~2
+ * GnuTLS 3.7.1 includes fix commit [2], Focal is 3.6.13.
+ commit e0bb98e1f71f ("Fix non-empty session id (TLS13_APPENDIX_D4)")
+
+ $ git describe --contains e0bb98e1f71f
+ 3.7.1~32^2
+
+ * There's an incremental fix-up commit [3]:
+ commit 05ee0d49fe93 ("handshake: TLS 1.3: don't generate session ID in resumption mode")
+
+ $ git describe --contains 05ee0d49fe93
+ 3.7.1~17^2~2
[ Test Plan ]
- * Check whether TLS 1.3 handshake has `Session ID:`
+ * Check whether TLS 1.3 handshake has `Session ID:`
- - Focal (no):
- $ gnutls-cli --priority NORMAL:-VERS-ALL:+VERS-TLS1.3 ubuntu.com </dev/null
- ...
- - Description: (TLS1.3-X.509)-...
- - Options:
- - Handshake was completed
- ...
-
- - Jammy (yes):
- $ gnutls-cli --priority NORMAL:-VERS-ALL:+VERS-TLS1.3 ubuntu.com </dev/null
- ...
- - Description: (TLS1.3-X.509)-...
- - Session ID: CB:7D:DF:...
- - Options:
- - Handshake was completed
- ...
+ - Focal (no):
+ $ gnutls-cli --priority NORMAL:-VERS-ALL:+VERS-TLS1.3 ubuntu.com </dev/null
+ ...
+ - Description: (TLS1.3-X.509)-...
+ - Options:
+ - Handshake was completed
+ ...
- * Check tests run at build time (`Testsuite summary for GnuTLS`).
-
- Tests passed per the build log from PPA with test packages:
-
- ============================================================================
- Testsuite summary for GnuTLS 3.6.13
- ============================================================================
- # TOTAL: 25
- # PASS: 8
- # SKIP: 17
- # XFAIL: 0
+ - Jammy (yes):
+ $ gnutls-cli --priority NORMAL:-VERS-ALL:+VERS-TLS1.3 ubuntu.com </dev/null
+ ...
+ - Description: (TLS1.3-X.509)-...
+ - Session ID: CB:7D:DF:...
+ - Options:
+ - Handshake was completed
+ ...
- * Check autopkgtests from gnutls28 against PPA/SRU [4].
-
- Tests passed against PPA with test packages:
-
- autopkgtest [13:47:40]: @@@@@@@@@@@@@@@@@@@@ summary
- run-upstream-testsuite PASS
+ * Check tests run at build time (`Testsuite summary for GnuTLS`).
- * Check autopkgtests from reverse test triggers against PPA/SRU
-
- $ reverse-depends --build-depends --release focal libgnutls28-dev
- Reverse-Testsuite-Triggers
- * apt
- * cmake
- * gnustep-base
- * knot-resolver
- * lxc
- * riemann-c-client
- * squid
- ...
+ Tests passed per the build log from PPA with test packages:
+
+ ============================================================================
+ Testsuite summary for GnuTLS 3.6.13
+ ============================================================================
+ # TOTAL: 25
+ # PASS: 8
+ # SKIP: 17
+ # XFAIL: 0
+
+ * Check autopkgtests from gnutls28 against PPA/SRU [4].
+
+ Tests passed against PPA with test packages:
+
+ autopkgtest [13:47:40]: @@@@@@@@@@@@@@@@@@@@ summary
+ run-upstream-testsuite PASS
+
+ * Check autopkgtests from reverse test triggers against PPA/SRU
+
+ $ reverse-depends --build-depends --release focal libgnutls28-dev
+ Reverse-Testsuite-Triggers
+ * apt
+ * cmake
+ * gnustep-base
+ * knot-resolver
+ * lxc
+ * riemann-c-client
+ * squid
+ ...
[ Regression Potential ]
- * TLS 1.3 handshake now includes non-empty Session ID
- in ClientHello, so there's a behavior change in the
- Client side-only, but it does affect how particular
- Servers handle the client, depending on Session ID.
-
- * Thus, theoretically, if issues were to occur, that
- likely would manifest as client connection errors
- with TLS 1.3 (failures would be realized early and
- fast), and a workaround available is using TLS 1.2.
-
- * Even though changes to TLS handshake understandably
- may be scary (considering the impact of regressions),
- the proposed change is specified by the RFC (and is
- there to help w/ wider compatibility) and is already
- implemented in later versions (3.7.1 in Hirsute [5]).
-
+ * TLS 1.3 handshake now includes non-empty Session ID
+ in ClientHello, so there's a behavior change in the
+ Client side-only, but it does affect how particular
+ Servers handle the client, depending on Session ID.
+
+ * Thus, theoretically, if issues were to occur, that
+ likely would manifest as client connection errors
+ with TLS 1.3 (failures would be realized early and
+ fast), and a workaround available is using TLS 1.2.
+
+ * Even though changes to TLS handshake understandably
+ may be scary (considering the impact of regressions),
+ the proposed change is specified by the RFC (and is
+ there to help w/ wider compatibility) and is already
+ implemented in later versions (3.7.1 in Hirsute [5]).
+
[ Other Info ]
- * Bionic is not impacted (TLS 1.2 only)
- * Jammy and later already fixed (TLS 1.3 on GnuTLS 3.7+)
+ * Bionic is not impacted (TLS 1.2 only)
+ * Jammy and later already fixed (TLS 1.3 on GnuTLS 3.7+)
[ Links ]
[1] https://www.rfc-editor.org/rfc/rfc8446#appendix-D.4
[2] https://gitlab.com/gnutls/gnutls/-/commit/e0bb98e1f71f94691f600839ff748d3a9f469d3e
[3] https://gitlab.com/gnutls/gnutls/-/commit/05ee0d49fe93d8812ef220c7b830c4b3553ac4fd
[4] https://autopkgtest.ubuntu.com/packages/g/gnutls28
[5] https://launchpad.net/ubuntu/+source/gnutls28/3.7.1-3ubuntu1
-
Issue: https://gitlab.com/gnutls/gnutls/-/issues/1074
FixMR: https://gitlab.com/gnutls/gnutls/-/merge_requests/1350
[ Original Description ]
<Removed bits related to Landscape, which is not at fault.>
Use gnutls-cli to check, it shows:
...
- Description: (TLS1.3-X.509)-(ECDHE-SECP256R1)-(RSA-PSS-RSAE-SHA256)-(AES-256-GCM)
- Options:
- Handshake was completed
- Simple Client Mode:
*** Fatal error: Error in the pull function.
*** Server has terminated the connection abnormally.
gnutls version:
bionic 3.5.18
focal 3.6.13
jammy 3.7.3
gnutls 3.5 doesn't support TLS1.3 so it's using TLS1.2
and both 3.6.x and 3.7.x are using TLS1.3, but only 3.7.x works
We built gnutls from upstream and found the same issue:
3.16.3 isn't working and 3.7.3 is working
so we did a bisect and found this commit fixes the issue for TLS1.3 on 3.7.x:
commit e0bb98e1f71f94691f600839ff748d3a9f469d3e
Author: Norbert Pocs <npocs at redhat.com>
Date: Fri Oct 30 17:18:30 2020 +0100
Fix non-empty session id (TLS13_APPENDIX_D4)
When TLS1.3 is used with middlebox compatible mode, the session id should be filled with random session id,
but remained empty.
Signed-off-by: Norbert Pocs <npocs at redhat.com>
Closes #1074
We need to SRU this commit to libgnutls28 on focal
** Changed in: gnutls28 (Ubuntu)
Status: In Progress => Invalid
** Changed in: gnutls28 (Ubuntu)
Importance: High => Undecided
** Changed in: gnutls28 (Ubuntu)
Assignee: gerald.yang (gerald-yang-tw) => (unassigned)
** Changed in: gnutls28 (Ubuntu Focal)
Importance: High => Medium
--
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to gnutls28 in Ubuntu.
https://bugs.launchpad.net/bugs/2019094
Title:
[SRU] Focal: TLS 1.3 doesn't work on strict firewall/middlebox
Status in gnutls28 package in Ubuntu:
Invalid
Status in gnutls28 source package in Focal:
In Progress
Bug description:
< IN PROGRESS >
[ Impact ]
* On Focal, the TLS 1.3 handshake might fail on strict
(or misbehaving) proprietary firewall/middlebox that
requires a non-empty Session ID (as TLS 1.2) per RFC.
* The RFC specifies the ClientHello should always have
a non-empty session ID, but this _is_ empty in Focal.
* RFC 8446, Appendix D.4. Middlebox Compatibility Mode [1]
"""
... a significant number of middleboxes misbehave
when a TLS client/server pair negotiates TLS 1.3.
... handshake look more like a TLS 1.2 handshake:
- The client always provides a non-empty session ID
in the ClientHello, ...
"""
* Reverse build dependencies that link against the
static libraries in libgnutls28-dev (check needed)
would need No-Change Rebuilds to pick up this fix.
(see `reverse-depends -b -r focal libgnutls28-dev`)
[ Fix ]
* Analysis summary based on packet capture and source
code is provided in comment #3.
* GnuTLS 3.7.1 includes fix commit [2], Focal is 3.6.13.
commit e0bb98e1f71f ("Fix non-empty session id (TLS13_APPENDIX_D4)")
$ git describe --contains e0bb98e1f71f
3.7.1~32^2
* There's an incremental fix-up commit [3]:
commit 05ee0d49fe93 ("handshake: TLS 1.3: don't generate session ID in resumption mode")
$ git describe --contains 05ee0d49fe93
3.7.1~17^2~2
[ Test Plan ]
* Check whether TLS 1.3 handshake has `Session ID:`
- Focal (no):
$ gnutls-cli --priority NORMAL:-VERS-ALL:+VERS-TLS1.3 ubuntu.com </dev/null
...
- Description: (TLS1.3-X.509)-...
- Options:
- Handshake was completed
...
- Jammy (yes):
$ gnutls-cli --priority NORMAL:-VERS-ALL:+VERS-TLS1.3 ubuntu.com </dev/null
...
- Description: (TLS1.3-X.509)-...
- Session ID: CB:7D:DF:...
- Options:
- Handshake was completed
...
* Check tests run at build time (`Testsuite summary for GnuTLS`).
Tests passed per the build log from PPA with test packages:
============================================================================
Testsuite summary for GnuTLS 3.6.13
============================================================================
# TOTAL: 25
# PASS: 8
# SKIP: 17
# XFAIL: 0
* Check autopkgtests from gnutls28 against PPA/SRU [4].
Tests passed against PPA with test packages:
autopkgtest [13:47:40]: @@@@@@@@@@@@@@@@@@@@ summary
run-upstream-testsuite PASS
* Check autopkgtests from reverse test triggers against PPA/SRU
$ reverse-depends --build-depends --release focal libgnutls28-dev
Reverse-Testsuite-Triggers
* apt
* cmake
* gnustep-base
* knot-resolver
* lxc
* riemann-c-client
* squid
...
[ Regression Potential ]
* TLS 1.3 handshake now includes non-empty Session ID
in ClientHello, so there's a behavior change in the
Client side-only, but it does affect how particular
Servers handle the client, depending on Session ID.
* Thus, theoretically, if issues were to occur, that
likely would manifest as client connection errors
with TLS 1.3 (failures would be realized early and
fast), and a workaround available is using TLS 1.2.
* Even though changes to TLS handshake understandably
may be scary (considering the impact of regressions),
the proposed change is specified by the RFC (and is
there to help w/ wider compatibility) and is already
implemented in later versions (3.7.1 in Hirsute [5]).
[ Other Info ]
* Bionic is not impacted (TLS 1.2 only)
* Jammy and later already fixed (TLS 1.3 on GnuTLS 3.7+)
[ Links ]
[1] https://www.rfc-editor.org/rfc/rfc8446#appendix-D.4
[2] https://gitlab.com/gnutls/gnutls/-/commit/e0bb98e1f71f94691f600839ff748d3a9f469d3e
[3] https://gitlab.com/gnutls/gnutls/-/commit/05ee0d49fe93d8812ef220c7b830c4b3553ac4fd
[4] https://autopkgtest.ubuntu.com/packages/g/gnutls28
[5] https://launchpad.net/ubuntu/+source/gnutls28/3.7.1-3ubuntu1
Issue: https://gitlab.com/gnutls/gnutls/-/issues/1074
FixMR: https://gitlab.com/gnutls/gnutls/-/merge_requests/1350
[ Original Description ]
<Removed bits related to Landscape, which is not at fault.>
Use gnutls-cli to check, it shows:
...
- Description: (TLS1.3-X.509)-(ECDHE-SECP256R1)-(RSA-PSS-RSAE-SHA256)-(AES-256-GCM)
- Options:
- Handshake was completed
- Simple Client Mode:
*** Fatal error: Error in the pull function.
*** Server has terminated the connection abnormally.
gnutls version:
bionic 3.5.18
focal 3.6.13
jammy 3.7.3
gnutls 3.5 doesn't support TLS1.3 so it's using TLS1.2
and both 3.6.x and 3.7.x are using TLS1.3, but only 3.7.x works
We built gnutls from upstream and found the same issue:
3.16.3 isn't working and 3.7.3 is working
so we did a bisect and found this commit fixes the issue for TLS1.3 on 3.7.x:
commit e0bb98e1f71f94691f600839ff748d3a9f469d3e
Author: Norbert Pocs <npocs at redhat.com>
Date: Fri Oct 30 17:18:30 2020 +0100
Fix non-empty session id (TLS13_APPENDIX_D4)
When TLS1.3 is used with middlebox compatible mode, the session id should be filled with random session id,
but remained empty.
Signed-off-by: Norbert Pocs <npocs at redhat.com>
Closes #1074
We need to SRU this commit to libgnutls28 on focal
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/gnutls28/+bug/2019094/+subscriptions
More information about the foundations-bugs
mailing list