[Bug 2055450] [NEW] Uploading package to server with self-signed certificate on https fails despite adding cert to trust-store

[Mark Cunningham]

Public bug reported:

On Ubuntu 22.04 with dput version 1.1.0ubuntu2.1, and python3 3.10.x,
customers using a self-signed SSL for https are getting the following:

  File "/usr/bin/dput", line 37, in <module>
    sys.exit(load_entry_point('dput==1.1.0+ubuntu2.1', 'console_scripts', 'execute-dput')())
  File "/usr/share/dput/dput/dput.py", line 1235, in main
    upload_methods[method](
  File "/usr/share/dput/dput/methods/https.py", line 18, in upload
    return http.upload(
  File "/usr/share/dput/dput/methods/http.py", line 138, in upload
    conn.endheaders()
  File "/usr/lib/python3.10/http/client.py", line 1278, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.10/http/client.py", line 1038, in _send_output
    self.send(msg)
  File "/usr/lib/python3.10/http/client.py", line 976, in send
    self.connect()
  File "/usr/lib/python3.10/http/client.py", line 1455, in connect
    self.sock = self._context.wrap_socket(self.sock,
  File "/usr/lib/python3.10/ssl.py", line 513, in wrap_socket
    return self.sslsocket_class._create(
  File "/usr/lib/python3.10/ssl.py", line 1100, in _create
    self.do_handshake()
  File "/usr/lib/python3.10/ssl.py", line 1371, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate (_ssl.c:1007)

This seems to be an issue in how the SSL for the https connection is
validated. Even after adding the self-signed certificate to the trust
store with update-ca-certificates, this is not being read by the python
code for validation of the cert on the mirror.

The immediate solution has been to modify the main dput file to import
the ssl library, and tell it to not validate the certificate for the
connection:

import ssl
ssl._create_default_https_context = ssl._create_unverified_context

This is discussed further at the following link:

https://stackoverflow.com/questions/77639570/ssl-verification-problem-
when-uploading-a-deb-package-using-dput

This seems like a change in python behavior given this discussion:

https://stackoverflow.com/questions/35569042/ssl-certificate-verify-
failed-with-python3

I am not sure what the best path forward is, I would think that ideally
there may be an environment variable to tell python to read the
certificate from the standard trust-store /etc/ssl/certs/ca-
certificates.crt, or otherwise to skip certificate validation, without
needing to modify dput directly.

I do not see this happening on 20.04 with python 3.8.x and dput
1.0.3ubuntu1.1, so this seems to be a relatively recent change in
behavior.

** Affects: dput (Ubuntu)
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to dput in Ubuntu.
https://bugs.launchpad.net/bugs/2055450

Title:
  Uploading package to server with self-signed certificate on https
  fails despite adding cert to trust-store

Status in dput package in Ubuntu:
  New

Bug description:
  On Ubuntu 22.04 with dput version 1.1.0ubuntu2.1, and python3 3.10.x,
  customers using a self-signed SSL for https are getting the following:

    File "/usr/bin/dput", line 37, in <module>
      sys.exit(load_entry_point('dput==1.1.0+ubuntu2.1', 'console_scripts', 'execute-dput')())
    File "/usr/share/dput/dput/dput.py", line 1235, in main
      upload_methods[method](
    File "/usr/share/dput/dput/methods/https.py", line 18, in upload
      return http.upload(
    File "/usr/share/dput/dput/methods/http.py", line 138, in upload
      conn.endheaders()
    File "/usr/lib/python3.10/http/client.py", line 1278, in endheaders
      self._send_output(message_body, encode_chunked=encode_chunked)
    File "/usr/lib/python3.10/http/client.py", line 1038, in _send_output
      self.send(msg)
    File "/usr/lib/python3.10/http/client.py", line 976, in send
      self.connect()
    File "/usr/lib/python3.10/http/client.py", line 1455, in connect
      self.sock = self._context.wrap_socket(self.sock,
    File "/usr/lib/python3.10/ssl.py", line 513, in wrap_socket
      return self.sslsocket_class._create(
    File "/usr/lib/python3.10/ssl.py", line 1100, in _create
      self.do_handshake()
    File "/usr/lib/python3.10/ssl.py", line 1371, in do_handshake
      self._sslobj.do_handshake()
  ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate (_ssl.c:1007)

  This seems to be an issue in how the SSL for the https connection is
  validated. Even after adding the self-signed certificate to the trust
  store with update-ca-certificates, this is not being read by the
  python code for validation of the cert on the mirror.

  The immediate solution has been to modify the main dput file to import
  the ssl library, and tell it to not validate the certificate for the
  connection:

  import ssl
  ssl._create_default_https_context = ssl._create_unverified_context

  This is discussed further at the following link:

  https://stackoverflow.com/questions/77639570/ssl-verification-problem-
  when-uploading-a-deb-package-using-dput

  This seems like a change in python behavior given this discussion:

  https://stackoverflow.com/questions/35569042/ssl-certificate-verify-
  failed-with-python3

  I am not sure what the best path forward is, I would think that
  ideally there may be an environment variable to tell python to read
  the certificate from the standard trust-store /etc/ssl/certs/ca-
  certificates.crt, or otherwise to skip certificate validation, without
  needing to modify dput directly.

  I do not see this happening on 20.04 with python 3.8.x and dput
  1.0.3ubuntu1.1, so this seems to be a relatively recent change in
  behavior.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/dput/+bug/2055450/+subscriptions