[Bug 972077] Re: apt repository disk format has race conditions
michael jones
bcmalloy at hotmail.com
Sun Feb 28 01:38:08 UTC 2016
mike at mike-desktop ~ $ sudo apt-get update
[sudo] password for mike:
Ign http://mirror.internode.on.net rosa InRelease
Ign http://mirror.optus.net trusty InRelease
Hit http://mirror.internode.on.net rosa Release.gpg
Hit http://mirror.optus.net trusty-updates InRelease
Hit http://mirror.internode.on.net rosa Release
Hit http://mirror.optus.net trusty Release.gpg
Hit http://mirror.internode.on.net rosa/main amd64 Packages
Hit http://mirror.internode.on.net rosa/upstream amd64 Packages
Hit http://mirror.internode.on.net rosa/import amd64 Packages
Hit http://mirror.optus.net trusty-updates/restricted amd64 Packages
Ign http://extra.linuxmint.com rosa InRelease
Hit http://mirror.internode.on.net rosa/main i386 Packages
Hit http://mirror.optus.net trusty-updates/universe amd64 Packages
Get:1 http://mirror.optus.net trusty-updates/main amd64 Packages [683 kB]
Hit http://mirror.internode.on.net rosa/upstream i386 Packages
Hit http://mirror.optus.net trusty-updates/multiverse amd64 Packages
Ign http://archive.canonical.com trusty InRelease
Hit http://mirror.internode.on.net rosa/import i386 Packages
Hit http://extra.linuxmint.com rosa Release.gpg
Hit http://mirror.optus.net trusty-updates/restricted i386 Packages
Hit http://mirror.optus.net trusty-updates/universe i386 Packages
Hit http://mirror.optus.net trusty-updates/multiverse i386 Packages
Hit http://mirror.optus.net trusty-updates/main Translation-en
Hit http://mirror.optus.net trusty-updates/multiverse Translation-en
Hit http://mirror.optus.net trusty-updates/restricted Translation-en
Hit http://extra.linuxmint.com rosa Release
Hit http://mirror.optus.net trusty-updates/universe Translation-en
Hit http://mirror.optus.net trusty Release
Get:2 http://security.ubuntu.com trusty-security InRelease [65.9 kB]
Hit http://mirror.optus.net trusty/restricted amd64 Packages
Get:3 http://mirror.optus.net trusty-updates/main i386 Packages [659 kB]
Hit http://mirror.optus.net trusty/multiverse amd64 Packages
Hit http://extra.linuxmint.com rosa/main amd64 Packages
Hit http://archive.canonical.com trusty Release.gpg
Hit http://mirror.optus.net trusty/restricted i386 Packages
Hit http://archive.canonical.com trusty Release
Hit http://mirror.optus.net trusty/multiverse i386 Packages
Hit http://mirror.optus.net trusty/main amd64 Packages
Hit http://mirror.optus.net trusty/main Translation-en_AU
Get:4 http://security.ubuntu.com trusty-security/main amd64 Packages [428 kB]
Hit http://extra.linuxmint.com rosa/main i386 Packages
Hit http://mirror.optus.net trusty/multiverse Translation-en_AU
Hit http://archive.canonical.com trusty/partner amd64 Packages
Hit http://mirror.optus.net trusty/multiverse Translation-en
Ign http://mirror.internode.on.net rosa/import Translation-en_AU
Hit http://mirror.optus.net trusty/restricted Translation-en_AU
Ign http://mirror.internode.on.net rosa/import Translation-en
Hit http://mirror.optus.net trusty/restricted Translation-en
Ign http://mirror.internode.on.net rosa/main Translation-en_AU
Ign http://mirror.internode.on.net rosa/main Translation-en
Hit http://archive.canonical.com trusty/partner i386 Packages
Ign http://mirror.internode.on.net rosa/upstream Translation-en_AU
Hit http://mirror.optus.net trusty/universe amd64 Packages
Ign http://mirror.internode.on.net rosa/upstream Translation-en
Hit http://mirror.optus.net trusty/main i386 Packages
Hit http://mirror.optus.net trusty/universe i386 Packages
Hit http://archive.canonical.com trusty/partner Translation-en
Hit http://mirror.optus.net trusty/main Translation-en
Get:5 http://security.ubuntu.com trusty-security/restricted amd64 Packages [13.0 kB]
Get:6 http://security.ubuntu.com trusty-security/universe amd64 Packages [124 kB]
Hit http://mirror.optus.net trusty/universe Translation-en_AU
Hit http://mirror.optus.net trusty/universe Translation-en
Get:7 http://security.ubuntu.com trusty-security/multiverse amd64 Packages [4,990 B]
Ign http://dl.google.com stable InRelease
Get:8 http://security.ubuntu.com trusty-security/main i386 Packages [399 kB]
Hit http://dl.google.com stable Release.gpg
Get:9 http://security.ubuntu.com trusty-security/restricted i386 Packages [12.7 kB]
Ign http://extra.linuxmint.com rosa/main Translation-en_AU
Get:10 http://security.ubuntu.com trusty-security/universe i386 Packages [124 kB]
Ign http://extra.linuxmint.com rosa/main Translation-en
Hit http://dl.google.com stable Release
Get:11 http://security.ubuntu.com trusty-security/multiverse i386 Packages [5,164 B]
Hit http://dl.google.com stable/main amd64 Packages
Hit http://security.ubuntu.com trusty-security/main Translation-en
Hit http://security.ubuntu.com trusty-security/multiverse Translation-en
Hit http://security.ubuntu.com trusty-security/restricted Translation-en
Hit http://dl.google.com stable/main i386 Packages
Hit http://security.ubuntu.com trusty-security/universe Translation-en
Ign http://dl.google.com stable/main Translation-en_AU
Ign http://dl.google.com stable/main Translation-en
Fetched 2,519 kB in 14s (170 kB/s)
W: Failed to fetch http://mirror.optus.net/ubuntu/dists/trusty-updates/main/binary-amd64/Packages Hash Sum mismatch
W: Failed to fetch http://mirror.optus.net/ubuntu/dists/trusty-
updates/main/binary-i386/Packages Hash Sum mismatch
E: Some index files failed to download. They have been ignored, or old ones used instead.
mike at mike-desktop ~ $
--
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to apt in Ubuntu.
https://bugs.launchpad.net/bugs/972077
Title:
apt repository disk format has race conditions
Status in APT:
New
Status in apt package in Ubuntu:
Confirmed
Bug description:
Apt archives are accessed over HTTP; this has resulted in a cluster of
bugs (reported here, and upstream) about problems behind intercepting
caches, problems with squid etc.
There are 3 interlocking issues:
A - mirror networks may be out of sync with each other (e.g. a file named on one mirror may no longer exist, or may not yet exist, on another mirror)
B - updating files on a single mirror is not atomic - and even small windows of inconsistency will, given enough clients, cause headaches.
C - caches exacerbate race conditions - when one happens, until the cached data expires, all clients of the cache will suffer from the race
Solving this requires one of several things:
- file system transactions
- an archive format that requires only weakly ordered updates to the files at particular urls with the assumption that only one file may be observed to change at a time (because a lookup of file A, then B, may get a cache miss on A and a cache hit on B, so even if all clients strictly go A, then B, updates may still see old files when paths are reused).
- super robust clients that repeatedly retry with progressively less cache friendly headers until they have a consistent view. (This is very tricky to do).
It may be possible to do a tweak to the apt repository format though,
which would allow publishing a race-free format in parallel with the
existing layout, while clients migrate. To be safe against issue (A)
the mirror network would need some care around handling of dns round-
robin mirrors [to minimise the situation where referenced data is not
available], but this should be doable - or alternatively clients doing
'apt-get update' may need to be willing to retry to accommodate round-
robin skew.
What would such an archive format look like?
It would have only one well known file name (InRelease), which would be internally signed. Rather than signing e.g. Packages.gz, it would sign a uniquely named packages and sources file - e.g. Packages-$HASH.gz or Packages-$serialno.gz.
Backwards compatibility is achieved by using the same filenames for
deb's and the like. We need to keep writing Packages.gz though, and
Releases, until we no longer worry about old apt clients. We can
optimise disk space a little by making Packages.gz a symlink to a
Packages-$HASH.gz (and so on for Sources..), but it may be simpler and
less prone to unexpected behaviour to keep using regular files.
tl;dr
* Unique file names for all unique file content with one exception
* InRelease, a self-signed file that provides hashes and names the index files (Packages, Sources, Translations etc)
* Coexists with existing archive layout
Related bugs:
* bug 804252: Please support InRelease files
* bug 1430011: support apt by-hash mirrors
To manage notifications about this bug go to:
https://bugs.launchpad.net/apt/+bug/972077/+subscriptions
More information about the foundations-bugs
mailing list