RFC: Removal of ls-lR.gz files from Ubuntu mirrors

[Carlos Carvalho]

Colin Watson (cjwatson at ubuntu.com) wrote on 13 June 2012 08:46:
 >Could you be specific about the problem you're having, please?  This is
 >the first I've heard of any such problem.  (I rely on Debian tools to
 >keep my personal mirror up to date, and would have shouted pretty loudly
 >if we'd broken that.)

ls-lR and md5sums.txt don't match the repository. They either miss
entries or have entries that don't exist in the repo or what's listed
doesn't match the file attributes. I even filed a bug for it but it
was considered not important.

 >>  >This, then, is me asking our mirror world if anyone actually
 >>  >has an opinion on the future of ls-lR.gz files, or if you really
 >>  >don't care.
 >> 
 >> As it is it's better removed as far as I'm concerned.
 >> 
 >> Can you put a file list generated by
 >> 
 >> TZ=UTC rsync -r --no-h path/to/repo > path/to/repo/FILELIST-master
 >> 
 >> That'd be really more useful than the archaic ls-lR.
 >
 >There's only any point if there exist tools that consume it,
 >otherwise [...omitted...] this would just swap one problem for another.

Correct. Our mirroring script can use it. Even though it's not
released, if you do the scan once we'll not force the rsync scan when
we update, so your scan is compensated and you won't lose anything
doing it.

Another point is that you should configure archive.syncproxy to hold a
lot of metadata in ram. It should take a lot less than 10min to scan
the repo if it has enough ram and is well configured. It's more
important to have metadata than file pages in ram.