[Bug 2023531] Re: [MIR] dotnet6

Mon Aug 21 09:28:29 UTC 2023

Response to the review for Source Package: dotnet6

First of all, thanks Christian (~paelzer) & Nishit (~0xnishit) for the
extensive review!

Required TODOs:
> #1a
> - Please further clarify the future handling of updates
>   Thanks Dviererbe for sharing more details on what happens after EOL.
>   Is there a prior example of "providing a transitional package that uninstalls
>   all binaries". While I totally understand the motivation it is also
>   rather rude.
>   People might say "I didn't care for detail X" but now I can't use it
>   anymore - or similar.
>   Is that approach an established pattern that I missed so far?
>   If not we should probably run this by the SRU team (and they decide if also by the TB).
I discussed the EoL Strategy again with Security, Foundations & Microsoft. I documented the outcome in the README.source [1]. TL;DR: We keep the package as is in the archive and demote it to universe. We asked Microsoft to warn the User about EoL. Note that this is also the RedHat EoL strategy for .NET.
You can also find more context about the discussion in the JIRA ticket FR-4855 [2]

> #1b
> - And just to be double sure, you do not intent to e.g. move them
>   dotnet6 -> dotnet8, but uninstall it right?
We (Foundations – Toolchains) discussed this approch and rejected it, because this may break some users' use-case or may change the behavior of .NET applications. Especially for users that install multiple runtimes/sdks in parallel.

> #1c
> - Finally, generally and given the impact of the above, is there a draft
>   of that "with a clear indication of the MS end of support date in the
>   package itself" content? Because with the plan of removal it isn't just
>   "the MS end of support" it is "the MS and Ubuntu end of support
>   and existence".
As mentioned in the answer for #1a above; this is now documented in the README.source [1].

> #3a
> - You mentioned to need lld and llvm. Please be aware that I've only seen
>   build time dependencies to those which since a long time you no more
>   strictly need in main. Even then runtime dependencies of things embedding
>   llvm features are usually just needing libllvm13/14/15... and those are
>   in main already. So do you need to correct your template or have I missed
>   how you depend on them?
> #3b
> - If I somehow missed them, even then the MIRs for lld and llvm should not
>   be in this bug (it is complex enough) but a new one each and refer to here.
>   Just like with the versions here (dotnet6, dotnet7, ...) please ensure you
>   keep the list of fully supported releases at a well selected minimum. For
>   example rustc pulls llvm-13 into jammy. You use the default which is v14
>   on jammy - which would imply two versions in main. I do not even mind on
>   which side this is sorted out, but it would help to use the same
>   where possible.
This was an error on my side. lld and llvm are not needed. I just thought that they are needed, because the check-mir tool pointed out that they aren't in main. 

#4
> - Please point me to the build time tests (More details below in the
>   [Common blockers] section.
This was also a confusion on my side. There are no build time tests and I don't know if we want  to automatically run the tests during build time, because a build of .NET or running the autopkgtest suite can take hours. There are multiple occasions were a coupled execution of the autopkgtest suite would have slowed down our ability to iterate.
I think this is fine, because we always trigger the autopkgtests in a PPA when a change occurs and the autopkgtests in the proposed pocket have to pass before a new version is deployed to end-users.

Recommended TODOs:
> #2
> - I understand that this oddity of needing .git structures is unavoidable
>   right now. But it would be great if you would find a way to patch this
>   out in a nice way or at least chime in and adopt upstream issue 3359
>   once resolved. I say "a nice way" because if the alternative makes it worse
>   there is no benefit.
Robie Basak also suggested this to us, because the need for the .git folder interferes with git-ubuntu. I don't know if we are able to patch this for .NET 6 before EoL, but this is definitely on our TODO for .NET 8.

> #5
> - have a look at filtering d/watch to only one major version (more see in the
>   [Packaging red flags] section below)
I "fixed" the watch file. It now can tell correctly if a new upstream version exists, but it will intentionally fail. The reason is also explained in the README.source [3] and there is a comment
in the watch file [4]. TL;DR: The .NET 6 source code is distributed over several public repositories and has no single public place to download the source from. This will change with .NET 8.

> #6
> - while classic symbols tracking isn't very applicable this has way too many
>   shared objects that not thinking about it would be ok. More details about
>   this request below the [Packaging red flags] section.
> #7
> - In the context of tests (#4) and ABI tests (#6) I've seen mentioning of a
>   JIT test (https://github.com/dotnet/runtime/issues/68837) which refers to
>   https://github.com/dotnet/runtime/blob/main/docs/project/jit-testing.md
>   I've not seen that on build not autopkgtest time.
>   They mention running on Ubuntu 18.04 so it should generally work, could
>   adding that to autopkgtests could be a great addition? Please have a look
>   at that. Only recommended as I'm not having enough experience on this, the
>   request might be unfulfillable.
I have to read up on these topics, but I agree that both should be added if they are fulfillable

[Summary of our TODO-List]
1. Integrate (upstream) smoke tests into autopkgtest suite.
2. Investigate embedded third party dependencies that should/can be patched out of dotnet packages.
3. Patch out need for .git folder.
4. Investigate symbols tracking for .so and .dll files.
5. Investigate enabling LTO.
6a. Investigate state of JIT Tests.
6b. Investigate feasibility of integration into autopkgtest suite

Note: These should not block the main inclusion as the TODOs are all non
trivial and not strictly required.

[1] https://git.launchpad.net/ubuntu/+source/dotnet6/tree/debian/README.source?id=1d83c4e1f527adcc0f9a30a39ad55c64260ef6bd#n39
[2] https://warthogs.atlassian.net/browse/FR-4855
[3] https://git.launchpad.net/ubuntu/+source/dotnet6/tree/debian/README.source?id=1d83c4e1f527adcc0f9a30a39ad55c64260ef6bd#n55
[4] https://git.launchpad.net/ubuntu/+source/dotnet6/tree/debian/watch?id=1d83c4e1f527adcc0f9a30a39ad55c64260ef6bd#n5

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to dotnet6 in Ubuntu.
https://bugs.launchpad.net/bugs/2023531

Title:
  [MIR] dotnet6

Status in dotnet6 package in Ubuntu:
  Incomplete

Bug description:
  [Availability]
   The package dotnet6 is already in Ubuntu universe.
   The package dotnet6 build for the architectures it is designed to work on.
   - See: https://github.com/dotnet/core/blob/main/release-notes/6.0/supported-os.md
   It currently builds and works for architetcures: amd64, arm64
   Link to package https://launchpad.net/ubuntu/+source/dotnet6

  [Rationale]
   - The package dotnet6 is required in Ubuntu main as part of
     Canonicals partnership with Microsoft to shorten the supply
     chain between Canonical and Microsoft and improve the .NET
     developer experience on Ubuntu. Read more here:
     - https://canonical.com/blog/install-dotnet-on-ubuntu
     - https://devblogs.microsoft.com/dotnet/dotnet-6-is-now-in-ubuntu-2204/
   - The package dotnet6 will generally be useful for a large part of
     our user base
   - It would be great and useful to community/processes to have the
     package dotnet6 in Ubuntu main, but there is no definitive deadline.

  [Security]
   - dotnet6 had security issues in the past that have been
     fixed, see trackers:
     - https://ubuntu.com/security/cves?package=dotnet6
     - https://github.com/dotnet/core/blob/main/release-notes/6.0/cve.md
     - NOTE: When searching for .NET CVEs in other trackers,
       keep in mind that .NET Framework and .NET (Core) is not
       the same and that many CVEs do not affect Linux distributions.
   - The Security Team and Foundations Toolchain Squad already
     work together with Microsoft to release security updates
     to Ubuntu.
   - Microsoft has weekly meetings with .NET Security Partners
     (including Canonical) where they get and keep informed
     about Security Issues.
   - .NET Security Partners (including Canonical) have early
     access to .NET releases containing CVE patches.
   - Microsoft and .NET Security Partners (including Canonical)
     coordinate releases to disclose and provide patches for
     security issues on all plattforms at the same time.
   - Microsoft informs Users about (security) issues in the
     monthly release notes where they aslo recommend actions
     to mitigate these issues.
     See example Release Note containing CVE warning:
     https://devblogs.microsoft.com/dotnet/february-2023-updates/
   - no `suid` or `sgid` binaries
   - no executables in `/sbin` and `/usr/sbin`
   - Packages does not open privileged ports (ports < 1024)
   - Packages does not contain extensions to security-sensitive software
     (filters, scanners, plugins, UI skins, ...)

  [Quality assurance - function/usage]
   - The package works well right after install

  [Quality assurance - maintenance]
   - The package is maintained well in Ubuntu/Upstream and does
     not have too many, long-term & critical, open bugs
     - Ubuntu https://bugs.launchpad.net/ubuntu/+source/dotnet6/+bug
     - There are multiple bug trackers upstream for the individual 
       components of the package https://github.com/dotnet
   - The package has no important open bugs
   - The package does not deal with exotic hardware we cannot support

  [Quality assurance - testing]
   - The package runs a test suite on build time, if it fails
     it makes the build fail, link to build logs:
     - mantic amd64: https://launchpad.net/ubuntu/+source/dotnet6/6.0.116-0ubuntu3/+build/26165948
     - mantic arm64: https://launchpad.net/ubuntu/+source/dotnet6/6.0.116-0ubuntu3/+build/26165949
     - lunar amd64: https://launchpad.net/~ubuntu-security/+archive/ubuntu/ppa/+build/25976292
     - lunar arm64: https://launchpad.net/~ubuntu-security/+archive/ubuntu/ppa/+build/25976293
     - kinetic amd64: https://launchpad.net/~ubuntu-security/+archive/ubuntu/ppa/+build/25964381
     - kinetic arm64: https://launchpad.net/~ubuntu-security/+archive/ubuntu/ppa/+build/25964382
     - jammy amd64: https://launchpad.net/~ubuntu-security/+archive/ubuntu/ubuntu-security-staging-private/+build/25974197
     - jammy arm64: https://launchpad.net/~ubuntu-security/+archive/ubuntu/ubuntu-security-staging-private/+build/25974198
   - The package runs an autopkgtest, and is currently passing
     on mantic/lunar/kinetic/jammy amd64/arm64 https://autopkgtest.ubuntu.com/packages/dotnet6
   - The package does NOT have failing autopkgtests tests right now.

  [Quality assurance - packaging]
   - debian/watch is present and works*
     (*Canonical has to work around the debian/watch file to
     consume embargoed releases before the official release)
   - debian/control defines a correct Maintainer field
   - This package does yield massive lintian Warnings/Errors,
     but they are either false-postives or acceptable.
   - Lintian overrides are present, but ok because of false-positive
     lintian warnings. The concrete reasons are explained as a
     comment in the overwrite files.
   - The package will not be installed by default
   - Packaging is complex, but that is ok because the software
     we are packaging is complex and we are working with
     Microsoft to reduce the complexity.

  [UI standards]
   - Application is end-user facing, Translation is NOT present,
     this is ok, as the application just provides a Command Line
     Interface for developers. The CLI output should not be
     translated to maintain online searchable error messages.
   - The exception messages of the .NET Runtime are localized.
   - End-user applications without desktop file, not needed,
     because it just provides libraries and command line tools

  [Dependencies]
   - There are further dependencies that are not yet in main, the MIR
     process for them is handled as part of this bug here.
     - lld binary and source package is in universe
     - llvm binary and source package is in universe
     - locales-all is in universe, but its source glibc is already in main

  [Standards compliance]
   - This package correctly follows FHS and Debian Policy (AFAICT: this package is huge and I have only limited experience)

  [Maintenance/Owner]
   - Team is already subscribed to the package
   - This package has embedded/vendorized dependencies. 
     We are aware of this problem and working on getting rid of them.
   - This package is not rust based
   - The package has been built in the archive more recently than the last
     test rebuild

  [Background information]
   - The Package description explains the package well
   - Upstream Name is ".NET 6"
   - Upstream project: https://github.com/dotnet/source-build
   - This MIR exists in parralel to the MIR for dotnet7

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/dotnet6/+bug/2023531/+subscriptions