Some plans for scmproj

Fri Dec 11 16:23:32 GMT 2009

On Wednesday 09 December 2009 14:12:25 Alexander Belchenko wrote:
> > One question I have here is, does this mean that a bzr working tree will
> > have to be at the root of the project directory?  Or do you plan to
> > work-around this somehow?
>
> Yes, today you can have one branch in the root of project tree,
> specifying in the project.cfg:
>
> [component "foo"]
> RELPATH = .
> ...
>
> But it was mandatory. Now this will be mandatory to have *some* bzr tree
> at the root of the project. If you don't like this you may use your old
> .scmproj control branch as top tree. This is up to you. Such tree can be
> almost empty (only keep .bzrmeta/scmproj/config) and that's all.

Yes, that is what I meant actually.  I don't actually want to have a bzr tree 
at the root of the project and I thought it would now become mandatory to 
have some.

> I think there is still enough flexibility.

Absolutely, thanks for clearing that up.

> > I'm not sure what you mean that it's harder to track things?
>
> Commands like project-command, project-commit et al should be aware
> about existence of separate control branch. This is extra complexity for
> the code.

Understood.  However I imagine some of this will still have to stay there in 
order to be able to handle the scenario of project root separate from 
location of control directory?  E.g. to be able to do something like:

  $ cd ~/prj
  $ bzr project-commit --control=.scmproj

to specify that the control tree is ~/prj/.scmproj

> Snapshots will be updated every time any component will commit new
> revision or by user request. Snapshot saves the data about last
> committed revision-id for all components, so you can restore exactly the
> same state of the project later (modulo uncommitted changes of course).

OK, got it.

> > Yes, I see what you mean.  One option would be to make snapshots per-alt
> > but IIUC this wouldn't quite work for your use case.
> >
> > On the other hand if a snapshot recorded both branch and revision info,
> > that should work, no?
>
> Mmm, no. Problem in divergence of multiple snapshots. Resolving such
> conflicts will be real nightmare.
>
> Removing alts now also has another benefit: every top-level tree
> (branch) is the alt. They will be physically visible and more or less
> consistent (because there is snapshots).

But if you wanted to merge changes between them you'd have the divergence 
problem anyways, I guess.  I agree though, it does make them more visible.

The problem seems to be separate anyway.  I am actually experimenting with 
something similar recently as in the project I'm working on there are several 
separate diverged branches (like multiple mainlines), which will never be 
merged together, but still need some changes flowing between them.  Still 
looking for a good workflow for something like that.

> > I know alts as they are (were) could lead to problems but a concept of a
> > light 'variant' as a way of specifying multiple branches for a component
> > and specifying which one is 'active' (i.e. which one is used when
> > 'getting' the project) could still fit in, IMO.
>
> The recent idea of Ian to emulate colocated branches with light checkout
> and treeless repo with branches inside the tree (in .bzrbranches
> directory) makes me think we can have more or less good colocated
> branches. Alts is another way to emulate colocated branches but for
> nested branches case. It was intended to be use with boxed workspace,
> but Ian idea makes this boxed workspace idea less relevant.

Yes, I agree.  What I meant with the idea of light variant/alt was not for 
emulating colocated branches but instead using it so that each user can 
define their own 'variant' of the project.

For example, imagine you have a project 'project-x' which consists of 
components 'A' and 'B'.  The main variant 
uses 'bzr+ssh://server/{COMPONENT}/trunk' branch for each component.

While for example user 'joe' can have his own variant defined which uses his 
branch 'bzr+ssh://server/B/joesbranch' for component B (i.e. it does not 
fetch B's trunk at all).

However, you're probably right that it's better to handle this sort of thing 
as separate branches of the control branch.  Maintaining that information in 
main control branch probably wouldn't scale well and sooner or later cause 
issues.

> >> So my new attempt is to use different top-level project directories to
> >> keep different variants of projects (different alts). It will be similar
> >> to other concepts existing in bzr.
> >
> > Do you mean having separate diverged branches of the control branch for
> > each alt?
>
> Yes, something like that.
> Every alt is different from another by 3 parameters:
>
> 1) Set of components
> 2) Component branches origins
> 3) Snapshot (tip of every component branch)
>
> First 2 items specified by project config, this config belongs to top
> tree of the project, so top tree automatically forms alt. Snapshot lives
> in the top tree too.
>
> > That has it's own problems (maintaining common things for one, as you'd
> > have to cherrypick changes between branches manually), but could work if
> > there's no better option.
>
> Well, today I have to create new control branch if my new alt should
> have different set of components. So this problem of sync between two or
> more project.cfg exists and now, but less often. So yes, there is always
> pros and cons.

Yeah, I agree, this at least makes it easier to set up.  I'm sure issues with 
changes flowing between different branches of the control branch can be 
resolved somehow later.

> > My use case is more for maintaining a project composed of multiple
> > components, so I find scmproj's ability to run e.g. status for all
> > components very useful.  The only peculiarity here is that I can't have a
> > branch in project root but still need to keep some links there, pointing
> > to files within the actual component branches.
>
> You can have a branch in project root. Why you think you can't? I'm not
> quite understand your point.

I didn't actually mean that I can't have a bzr tree in project root because 
the software doesn't allow it, but because the peculiarities of my project 
don't allow it :)

> > I don't know if I understand how snapshots should work, but if they
> > record revision ids of component branches in a config file in project
> > control branch, wouldn't you have problems e.g. when a component is
> > removed from project.cfg in control branch?
>
> Snapshot will be stored separately of project config, and will be
> updated often enough. So, no. There should not be a problem if component
> will be removed. Snapshot file will be updated based on current config.

Note that I haven't yet read much doc/code regarding snapshots yet so I may be 
talking nonsense here.  I plan to catch-up this weekend.

What I meant is a scenario like that:

You have a control branch at rev 1 where project.cfg defines components A and 
B.

1. create snapshot: records components' revisions
2. do some more changes to the control branch, say up to rev 5
3. commit revision 6 to control branch which removes component B from 
project.cfg
4. now try to restore snapshot created in 1.  basically this snapshot refers 
to component B which doesn't exist in rev 6 of the control branch

> Not only you. No need to sorry, I appreciate your feedback regardless of
> delays.
>
> Thanks!

Thanks, as I mentioned I'll try to catch-up with snapshots and other changes 
this weekend and hopefully will be able to provide some more feedback next 
week.

-- 
Michael Gliwinski
Henderson Group Information Services
9-11 Hightown Avenue, Newtownabby, BT36 4RT
Phone: 028 9034 3319

**********************************************************************************************
The information in this email is confidential and may be legally privileged.  It is intended solely for the addressee and access to the email by anyone else is unauthorised.
If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful.
When addressed to our clients, any opinions or advice contained in this e-mail are subject to the terms and conditions expressed  in the governing client engagement leter or contract.
If you have received this email in error please notify support at henderson-group.com

John Henderson (Holdings) Ltd
Registered office: 9 Hightown Avenue, Mallusk, County Antrim, Northern Ireland, BT36 4RT.
Registered in Northern Ireland
Registration Number NI010588
Vat No.: 814 6399 12
*********************************************************************************