bzr-git, the git map, fetching revisions, and being very slow

[Jelmer Vernooij]

On Mon, 2009-11-23 at 15:44 +1000, Ian Clatworthy wrote: 
> Russel Winder wrote:
> 
> > The
> > user goal is to interact with a Git repository.  Current options are use
> > a Git clone or use a Bazaar branch. Major factors here are branch model
> > and speed.
> 
> Having spent most of my career working on very large 24x7
> enterprise-scale systems, I strongly believe we're under-utilising the
> "gateway" architecture for interacting with foreign systems. I wonder
> how many of the reasons are technical and how many are social?
> 
> I truly think bzr-svn, bzr-git and bzr-hg are awesome tools and they
> have a *huge* role for Bazaar going forward. OTOH, like any wrapper
> technology:
> 
> 1. Performance will (nearly) always be below the thing being wrapped.
This only matters if it's significantly slower though, and I think we should be able to get performance that's not noticeably worse.

> 2. User expectations are (nearly) always well above what's possible.
> 3. Initial excitement will give way to dissatisfaction over time.

> For large projects where the wrapper architecture doesn't scale, I feel
> we should be recommending a "gateway" architecture, i.e. converting from
> a git repo to a bzr one and sending back patches in 'git' format (mail
> patch sets). Ditto for hg and returning bundles.
This is the basic workflow that bzr-{svn,git,hg} support and how Russel
uses it I believe. The problems come in when he tries to push back a
local bzr branch (that is not the original import) to a remote git
branch. 

In this situation dpush needs the Git sha1's of the objects he's trying
to push into the remote git repository. bzr-git creates this map during
import time but it only stores it in the target bzr branch during the
import, the map is not copied around when branches based on that
original import are created (since there is no hook in bzr for this). If
dpush can't find an existing (or up-to-date) shamap it will generate
one, and this process is what Russel (IIUC) finds slow.

bzr-svn does not have this problem as Subversion doesn't rely on
content-based SHA's.

> You give two options for interacting with git but there's a third option
> as well: setup & maintain a bzr mirror using
> git-fast-export+bzr-fast-import. The "setup" bit is polished reasonably
> well now though the "maintain" bit is currently well below the
> ease-of-use of bzr-git. OTOH, it's pretty fast and will get faster (when
> I land some of jam's patches). Maybe fastimport could record the
> information dpush needed as well?
It would be nice to support this. The main thing dpush needs to be able to do this is that bzr-fastimport sets the converted-from revision property (http://bazaar-vcs.org/ConvertedRevisionProperty).

This won't make dpush faster though, as it will still need the shamap. 

Cheers,

Jelmer
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: This is a digitally signed message part
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20091124/70f3c1fb/attachment.pgp