[Bug 2055450] Re: Uploading package to server with self-signed certificate on https fails despite adding cert to trust-store

[Mark Cunningham]

Update: after a lot of discussion with Mitch Burton on the Landscape
team, he was able to demonstrate this working with a self-signed
certificate. We think that this may actually not be strictly an issue
with the self-signed SSL, but rather that the name in the cert is not an
FQDN, and instead is just the bare hostname.

Upon further testing myself, I swapped the hostname on my test instance
from landscape-2310-jammy to landscape-2310-jammy.lxd just as a test. I
then updated my /etc/hosts file, the certificates configured in Apache,
and imported the newly generated cert into ca-certificates. After this
dput worked just fine.

dput lds:ubuntu/jammy/upload hello.changes

D: Splitting host argument out of  lds:ubuntu/jammy/upload.
D: Setting host argument.
Checking signature on .changes
gpg: /root/hello.changes: Valid signature from 5E1E964200F3EA3D
Uploading to lds (via https to landscape-2310-jammy.lxd):
  Uploading hello_2.10-2ubuntu4+esm1_amd64.deb: done.  
  Uploading hello.changes: done.
Successfully uploaded packages.

This seems to confirm that the issue is not necessarily with dput
directly, but in how python's urllib is checking the domain/cert on the
connection. This may be something that can be worked around in dput to
allow for a bare hostname that is not an FQDN, but either way figured it
would be relevant to add this information.

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to dput in Ubuntu.
https://bugs.launchpad.net/bugs/2055450

Title:
  Uploading package to server with self-signed certificate on https
  fails despite adding cert to trust-store

Status in dput package in Ubuntu:
  New

Bug description:
  On Ubuntu 22.04 with dput version 1.1.0ubuntu2.1, and python3 3.10.x,
  customers using a self-signed SSL for https are getting the following:

    File "/usr/bin/dput", line 37, in <module>
      sys.exit(load_entry_point('dput==1.1.0+ubuntu2.1', 'console_scripts', 'execute-dput')())
    File "/usr/share/dput/dput/dput.py", line 1235, in main
      upload_methods[method](
    File "/usr/share/dput/dput/methods/https.py", line 18, in upload
      return http.upload(
    File "/usr/share/dput/dput/methods/http.py", line 138, in upload
      conn.endheaders()
    File "/usr/lib/python3.10/http/client.py", line 1278, in endheaders
      self._send_output(message_body, encode_chunked=encode_chunked)
    File "/usr/lib/python3.10/http/client.py", line 1038, in _send_output
      self.send(msg)
    File "/usr/lib/python3.10/http/client.py", line 976, in send
      self.connect()
    File "/usr/lib/python3.10/http/client.py", line 1455, in connect
      self.sock = self._context.wrap_socket(self.sock,
    File "/usr/lib/python3.10/ssl.py", line 513, in wrap_socket
      return self.sslsocket_class._create(
    File "/usr/lib/python3.10/ssl.py", line 1100, in _create
      self.do_handshake()
    File "/usr/lib/python3.10/ssl.py", line 1371, in do_handshake
      self._sslobj.do_handshake()
  ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate (_ssl.c:1007)

  This seems to be an issue in how the SSL for the https connection is
  validated. Even after adding the self-signed certificate to the trust
  store with update-ca-certificates, this is not being read by the
  python code for validation of the cert on the mirror.

  The immediate solution has been to modify the main dput file to import
  the ssl library, and tell it to not validate the certificate for the
  connection:

  import ssl
  ssl._create_default_https_context = ssl._create_unverified_context

  This is discussed further at the following link:

  https://stackoverflow.com/questions/77639570/ssl-verification-problem-
  when-uploading-a-deb-package-using-dput

  This seems like a change in python behavior given this discussion:

  https://stackoverflow.com/questions/35569042/ssl-certificate-verify-
  failed-with-python3

  I am not sure what the best path forward is, I would think that
  ideally there may be an environment variable to tell python to read
  the certificate from the standard trust-store /etc/ssl/certs/ca-
  certificates.crt, or otherwise to skip certificate validation, without
  needing to modify dput directly.

  I do not see this happening on 20.04 with python 3.8.x and dput
  1.0.3ubuntu1.1, so this seems to be a relatively recent change in
  behavior.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/dput/+bug/2055450/+subscriptions