api stability policy (was Re: [qbzr] Re: API requests from a qbzr dev)

[Martin Pool]

2009/10/9 Stephen J. Turnbull <stephen at xemacs.org>:
> Martin Pool writes:
>
>  > Our approach used to be: there is a public api, which will be stable,
>  > and a private api which will be unstable.  This was worth a try but I
>  > think has not been successful in practice, so we should try something
>  > else.
>
> Why do you think that?  Just because various folks have complained
> about either the policy or the results?  This is a very hard problem.
> Before you drop the policy, I think you should think carefully about
> (1) what you hoped to accomplish (and now are saying has not been
> successully accomplished), and (2) what you realistically could have
> expected to accomplish.

1- We expected that:

 * clients would only use public APIs
 * they would ask us to make APIs public when they needed them
 * therefore, clients would mostly not have tight version dependencies on bzr

2- With the benefit of hindsight, for the reasons discussed in my
previous mail, I think this was not realistic.

istm that in practice:

 * clients use APIs with little regard for their nominal privacy
 * API users rarely if ever ask for things to be made public
 * the most interesting/important plugins (bzrtools, bzr-svn/hg/git)
are fairly tightly locked to bzr versions - and this is a source of
pain for users
 * the clients that are stable across versions, like qbzr, do so by
specifically accommodating API changes, or using APIs that rarely
change implementation, or (a special case of the preceding) using the
shell interface
 * we sometimes have to revert changes to keep compatibility, and we
do this roughly as often on nominally private as on public APIs
 * keeping api stability is a non-negligible cause of reviews being bounced

I would welcome contrary evidence, but that's how it seems to me


>  > Firstly, it's proved quite hard to keep good api compatibility while
>  > changing underneath, on any api.
>
> "If it hurts, that's good: it means you're not dead yet!" :-)

True, but it's also worth looking for an easier way.

>  > But third and most importantly, in practice plugin developers don't
>  > restrict themselves to public APIs.
>
> Sure.  Consenting adults.
>
>  > Not make a sharp distinction between public/stable and non-stable
>  > interfaces.  Whatever API clients depend on is what they depend upon.
>
> I think that's a bad idea.  You may make mistakes, but you can't let
> the individual plugins drive your decisions.  It needs to be the other
> way around.  If you think an API needs to be allowed to evolve or
> should be exposed to clients only if wrapped, make it private and any
> client that uses it should consider themselves warned.  If an API is
> public, OTOH, the developers of the client have a right to get upset
> if it changes.

I think making a distinction is fine, but what you outline is not what
I'd call a sharp distinction, or not as sharp as we have aimed for.

"Only if wrapped" is questionable - we previously have distinguished
"fast/powerful, unsafe" from "slower, safe" but that did not seem to
work well either.  If our own code doesn't use the wrapped version it
tends to not work well.

>  > Use underscores only to indicate 'relatively private' - something that
>  > normal users of the class shouldn't look at, not because they're not
>  > allowed to, but because there's probably a more appropriate interface.
>
> This begs the question of "more appropriate", though.  And it's not a
> question of "allowed" in Python.  It's what do you support.  I don't
> really see why you can't leave an API private, have a plugin developer
> find a use for it, and he develops his product simultaneously
> negotiating with you for a public interface.  Maybe it stays private,
> and he shoulders the whole risk; maybe you make it public, and suffer
> the inflexiblity; maybe you compromise on a new interface, giving him
> what he needs but reserving other aspects of the original interface
> and behavior.

That's what we expected would happen, but empirically it rarely
happens.  I think that's because of the fact they want to write code
now, not when the new bzr version comes out, and once they've written
it once there is little impediment to change.  But if you have an idea
how we could get that loop working better, I'd welcome it.

I think what people want is not just a stable API calling convention
but actually stable code to call into, and that is best satisfied by
having a stable branch.

None of this is an invitation to change APIs recklessly or with no
consideration for clients, just a recognition that the public/private
distinction is not enough.

-- 
Martin <http://launchpad.net/~mbp/>