bzr-pipeline, installers and explorer

[Barry Warsaw]

On Nov 20, 2009, at 2:45 AM, Ian Clatworthy wrote:
>
> * loom is great for managing patches over an upstream
> * pipeline is great for breaking a large change into logical pieces.
>
> Either can be used with varying degrees of success for either task
> though. As I spend nearly all of my time developing software rather  
> than
> packaging it, I find pipeline a better match for *my* needs.

This is a great thread because I've used looms fairly extensively and  
pipelines a lot recently. Both are very nice for managing a stack of  
related, logically separate branches, but neither feels "perfect".   
Let me see if I can collect my thoughts on the two technologies.

I use looms and pipelines for almost identical purposes, to manage  
both simple and complex tasks that can be decomposed into subtasks.  A  
simple example is fixing a bug.  I want a baseline that tracks trunk  
at the bottom of my stack because I'm going to update my trunk several  
times during the life of the bug fix, in order to resolve conflicts  
and ensure a clean application of my final patch set.  On top of that,  
I'll have the bug fix layer, which for simple bugs is relatively  
small.  Now, because I'm getting my bug fix reviewed, I will have a  
layer on top of that which will capture the conversation I'm having  
with my reviewer.  My top layer on the stack might track any  
peculiarities I have with actually landing the branch.  So something  
like:

* landing
* review
* bug-12345
* trunk

In this scenario, I will have relatively little need to migrate  
changes between the layers.  I'm basically using them as an easy way  
to generate diffs and track which changes I've made where in the  
development process.

For a complex example, I'm actually creating some similar layers  
(trunk, review, landing) but the bug-fix layer will decompose into  
many others.  I might be pairing with someone else who is working on  
part of the problem, so I need to track their branch.  I might realize  
that my current branch is too big to be reviewed and I need to migrate  
changes into a separate layer for an independent review.  I might  
realize that I have some unrelated tech debt reductions that can be  
segregated into a separate branch so that it's not seen as noise to  
the reviewer.  So I might have these layers:

* my-ui-work
* joes-ui-work
* my-model-work
* my-database-work
* joes-work-1
* tech-debt
* trunk

The layers are much more fluid because as joes-work-1 lands on trunk,  
I can get rid of that layer in my own branch.  Or, as I get the tech- 
debt layer reviewed, I might insert a new layer right above that named  
tech-debt-review.  Etc.  So here layers are coming and going much more  
often.

Both looms and pipes handle the basics of this pretty easily.  While  
I'm working on the branch, looms feel more natural because I'm  
definitely a heavyweight branch kind-of guy.  Lightweight checkouts  
just don't feel right to me.  Never have, and I can't quite explain  
why.  Something about the way I think about a problem wants all my  
work on that problem really self contained.

Looms have warts though.  'record' is a well-known one, but so is the  
requirement to 'export-loom' in order to publish the branches, say  
pushing them to Launchpad.  I also remember having some catastrophic  
failures with looms, where the changes are very difficult to extract,  
but its been a while and I don't remember the details.  I do really  
like that a loomed branch has no outside influences on my file system  
until I tell it too (i.e. export-loom).  Diffing among threads is  
pretty easy and is something I do all the time.  Navigating up and  
down the stack is pretty natural too, as is trunk-syncing.  Probably  
the biggest pain about looms is that they /aren't/ branches so need  
special treatment.  You're kind of always reminded that you're working  
in "something different".

Pipes improve this sense of working in "something different" because  
you're really not.  Once I'm over the lightweight checkout thing, it's  
pretty natural to create a new pipe, diff between them, trunk-sync,  
etc.  Pushing to Launchpad seems a bit weird in that it feels like I  
have to do that from the pipe branch, not from my working directory.

sync-pipeline is kind of magical and error prone.  It certainly  
doesn't feel natural.  It should be called 'push', and I've run into  
problems where sync-pipeline fails for mostly mysterious reasons  
(again, apologies for not having concrete examples).  Pipes don't seem  
to integrate well with 'bzr send' and pqm-submit, as again I can't  
seem to do that from my working tree, and instead have to be in the  
pipe branch directory to have any hope.

The thing that bugs me most about pipes is where it decides to put the  
branches for each layer. I really don't want to clutter up my shared  
repository directory with pipe branches because they don't follow my  
normal naming scheme.  What I mean is that when I'm about to work on a  
bug I do something like:

% cd shared-repo
% bzr co --lightweight trunk bug-12345
% cd bug-12345
% bzr add-pipe bug-fix

At this point I now have a directory called shared-repo/bug-fix but I  
really don't want that.  In my shared-repo directory I only want  
directories with names like bug-12345 bug-78900 bug-444321, etc.   
Suddenly my shared-repo directory is polluted with artifacts of one  
particular branch of work.  What if I start another bug fix branch and  
want to call its first layer "bug-fix"?  Collision.  Pipes should have  
a way of saying "all my pipe branches go in ./.pipes", in other words,  
they're unimportant artifacts and I don't want to see them, nor do I  
want them to collide with other work.

Ultimately, I think pipes are a better solution, and if some of the  
warts can be cleaned up, I can see myself using them to the exclusion  
of looms.  A rough list of improvements:

* Steal all of rockstar's aliases
   * first = switch-pipe :first
   * next = switch-pipe :next
   * prev = switch-pipe :prev
   * pipes = show-pipeline
* Make "diff" by default do "diff -r ancestor::prev"
* Make "send" by default do "send -r ancestor::prev.."
* By default, put pipe branches in ~/.pipes instead of as a sibling to  
the l/w co branch
* Make "push" Just Work and get rid of the need for sync-pipeline
* Somehow reduce the need to push all the underlying branches when  
pushing the current pipe

I know you're going to say that some of these are insane (like the  
last one), but honestly I don't really care about /how/ any of these  
work, I'm much more concerned with the workflow and feel of these  
features.  It's Friday after 3pm so when Warsaw's Second Law kicks in,  
all that's left is griping in email. :)

The truth is that both looms and pipes are very useful tools.  I see  
them as solving the same problem in different ways.  Both have their  
advantages and disadvantages.  I don't think they both need to exist  
(or be officially supported), but with a little time spent on  
smoothing out the workflow, I think you could end up with a really  
excellent, natural tool.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 194 bytes
Desc: This is a digitally signed message part
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20091120/fc44ee28/attachment.pgp