[Ubuntu-US-CA] (ugly) workaround(s) - bug 1731762: Re: ubuntu-16.04.3-server-amd64.jigdo ... 210 files could not be downloaded - missing from archive!

[Michael Paoli]

Well, ... I did add some (ugly) workaround(s),
See:
https://bugs.launchpad.net/ubuntu/+source/jigdo/+bug/1731762
https://bugs.launchpad.net/ubuntu/+source/jigdo/+bug/1731762/comments/10
https://bugs.launchpad.net/ubuntu/+source/jigdo/+bug/1731762/+attachment/5010040/+files/workaround
At least with the program/script, it's not quite as ugly as having to
manually determine all the missing files and explicitly download all
those to some temporary location.  Still far from elegant "solution",
so yes, definitely still a bug (with the archive/.jigdo file itself, not
an issue with jigdo or jigdo-lite).

> From: "Michael Paoli" <Michael.Paoli at cal.berkeley.edu>
> Subject: Re: [Ubuntu-US-CA] ubuntu-16.04.3-server-amd64.jigdo ...  
> 210 files could not be downloaded - missing from archive!
> Date: Sun, 12 Nov 2017 06:31:16 -0800

> Internet connection is fine (but not particularly high speed).
> Those 210 files are missing from the archive - they return
> 404 Not Found
> Can't assemble ubuntu-16.04.3-server-amd64.iso using jigdo
> with 210 of the files missing (have the other 1264 files).
> The Jigdo files specify the hashes, and (preferred/suggested)
> Server(s) for obtaining the files, and the file paths relative to
> Server(s) locations given.  The Jigdo file specifies
> http://archive.ubuntu.com/ubuntu/
> I also tried http://us.archive.ubuntu.com/ubuntu/ - same results.
>
> $ curl -s  
> http://releases.ubuntu.com/16.04/ubuntu-16.04.3-server-amd64.jigdo |  
> sed -ne '/^\[Servers]$/,$p'
> [Servers]
> Debian=http://archive.ubuntu.com/ubuntu/ --try-last
> $
>
> If we take a random look at one of the missing files ...
> $ sort -R missing | head -n 1
> http://archive.ubuntu.com/ubuntu/pool/main/p/php7.0/php7.0-gmp_7.0.18-0ubuntu0.16.04.1_amd64.deb
> $ curl -s  
> http://releases.ubuntu.com/16.04/ubuntu-16.04.3-server-amd64.jigdo |  
> fgrep php7.0-gmp_7.0.18-0ubuntu0.16.04.1_amd64.deb
> LgEs7KGP4Mi1Jypu-VowTQ=Debian:pool/main/p/php7.0/php7.0-gmp_7.0.18-0ubuntu0.16.04.1_amd64.deb
> $ curl -I  
> http://archive.ubuntu.com/ubuntu/pool/main/p/php7.0/php7.0-gmp_7.0.18-0ubuntu0.16.04.1_amd64.deb
> HTTP/1.1 404 Not Found
> Date: Sun, 12 Nov 2017 14:00:28 GMT
> Server: Apache/2.4.18 (Ubuntu)
> Content-Type: text/html; charset=iso-8859-1
>
> $
> If we have a look more closely in
> http://archive.ubuntu.com/ubuntu/pool/main/p/php7.0/
> we find there is no php7.0-gmp_7.0.18-0ubuntu0.16.04.1_amd64.deb
> file there.
> It does have:
> php7.0-gmp_7.0.22-0ubuntu0.16.04.1_amd64.deb
> php7.0-gmp_7.0.18-0ubuntu0.16.10.1_amd64.deb
> but those are useless for assembling
> ubuntu-16.04.3-server-amd64.iso
> with jigdo - we specifically need
> php7.0-gmp_7.0.18-0ubuntu0.16.04.1_amd64.deb
> with a hash of:
> LgEs7KGP4Mi1Jypu-VowTQ
> ... so looks like someone at Ubuntu seriously messed up the
> archive (and/or jigdo file?).
> And, that's just 1 examples of the 210 missing files.
>
> ... anyway, opened bug:
> https://bugs.launchpad.net/ubuntu/+bug/1731762
> jigdo: files missing on archive http://archive.ubuntu.com/ubuntu/ -  
> cannot assemble, e.g. ubuntu-16.04.3-server-amd64.iso
>
>> From: charlie <contact at charlieluna.com>
>> Subject: Re: [Ubuntu-US-CA] ubuntu-16.04.3-server-amd64.jigdo ...  
>> 210 files could not be downloaded - missing from archive!
>> Date: Sun, 12 Nov 2017 04:50:31 -0800
>
>> hey! good morning Mr. Paoli! Awesome to meet you! Looks like you've  
>> got yourself an issue with some server software. I don't even know  
>> how to help with you that since I don't have any experience or  
>> knowledge of setting up a server. Did you lose your connection? a  
>> 404 sounds like the connection was lost or the server went down for  
>> some reason but you probably already know that.
>>
>>
>> On 11/12/2017 04:44 AM, Michael Paoli wrote:
>>> $ jigdo-lite  
>>> http://releases.ubuntu.com/16.04/ubuntu-16.04.3-server-amd64.jigdo
>>> ...
>>> http://archive.ubuntu.com/ubuntu/
>>> ...
>>> Aaargh - 210 files could not be downloaded.
>>>
>>> Dang, ... really? 404 Not Found on 210 of the required files? <sigh>
>>>
>>> These are what's missing: