Subject: Shell communication channel: simple, half-assed or fully-arsed?

From: Alan Griffiths <[email-masked]>
Date: Wed, 11 Dec 2013 14:00:38 +0000

I'm trying to summarise some discussions around the need for Mir to
provide a ABI stable channel of communications to shells and client
toolkits (such as unity8/platform-api) that make use of it. This should
allow messages to be passed that Mir has no need nor ability to interpret.

Future nice-to-haves:
  o consistency between shells and a framework for protocol extensions
  o some messages to be meaningful to Mir to enable optimizations
  o Migrating the current API (e.g. attribute setting) onto the same channel

There are several approaches to this.

*Simple/Simplistic

*This is what I've sketched out in
https://code.launchpad.net/~alan-griffiths/mir/message-passing-api/+merge/198422

Mir simply punts all concerns about protocol negotiation, versioning and
structure to the shell and relies on the shell to provide all
information that Mir will ever need.

The advantage of this approach is that we can do it without further
discussion and unblock unity8 work.

The disadvantage of this approach is that the "nice to haves" are left
as an exercise for the interested Mir user.

*Half-assed*

Mir can provide some helpful fields - e.g. a protocol-id,
version-number**and message-discriminator to pass with the message - and
define some base protocols (sizing, copy-paste, protocol-extension). Mir
(or "shellinator") can then implement defaults for the base protocols.

We discussed this last week and felt that it raised more issues than it
addressed.

*fully-arsed*

Projects like X11 and Wayland have effective protocol definition and
extension mechanisms. We should build on this - Wayland being the better
example as it builds on the X11 experience.

I don't yet have a clear vision of what a successful implementation of
this approach looks like - and suspect that different team members have
differing thoughts on how we would go about this.

*Conclusion*

I think that we should discuss the fully-arsed and flesh out some
specific ideas. But as that may not lead to a quick (or any) conclusion
it is worth implementing the simple approach in parallel so as to
unblock unity8 development.

---

From: Daniel van Vugt <[email-masked]>
Date: Thu, 12 Dec 2013 10:01:30 +0800

"Half-assed" is inaccurate I think. Under that heading you describe 
what we had already agreed to do (as much as the team can agree on 
something).

I suggest being careful trying to merge the existing areas of opacity in 
the protocol. I'm not sure they belong together.

Protobuf is quite extendable already. Are we not using it in a way 
that's compatible with extension? I'm not a fan of protobuf, and am even 
open to the idea of trying the Wayland protocol, but I don't yet see a 
serious problem with protobuf that needs solving.

---

From: Christopher James Halse Rogers <[email-masked]>
Date: Thu, 12 Dec 2013 12:32:25 +0800

Thanks for starting this off Alan,

> as an exercise for the interested Mir user.

I'd like to add that another disadvantage is that it's pushing the
responsibility for extension discovery, negotiation, and versioning onto
the shells and toolkits. That's non-trivial, and will be common to all
non-trivial shells/toolkits, so should be in Mir.

…
> unblock unity8 development.

I concur; unblock unity8 with the simplest possible thing, then replace
it with something long-term.

In that vein, let's discuss extension APIs.

We need three things:
1) An IDL to define the extensions,
2) A compiler to take the IDL and spit out appropriate code and headers,
and
3) A mechanism in core Mir to handle extensions

== IDL ==
For the IDL we need something more expressive than Protobuf's .proto
files; protobuf doesn't handle our requirement for fd passing, so we've
somewhat awkwardly wedged this alongside (see also MirEvent). This IDL
might be a simple extension of protobuf, or could be something entirely
different. The important things for the IDL are:

 *) Must be expressive enough to cover all the primitives we want to
send over - at least integers, strings, and fds.
 *) Should be expressive enough to handle the types of composites we use
- arrays and structs, primarily.
 *) Should be expressive enough to encode some minimal semantics -
request/response is what I'm thinking of here, but we may want more.
 *) Should support and encourage self-documentation; ideally we'd be
able to generate useful doxygen (or other format) from the IDL.

== IDL compiler ==
This should take the IDL and generate the requisite headers and code to
build a two shared libraries; the server side and the client side. I
like the fact that our interface is the shared library rather than any
particular wire protocol. We should treat extensions similarly.

The actual implementation should be pretty straightforward once we've
decided upon an IDL and core Mir mechanism.

== Core Mir mechanism ==

I've no particular current thoughts on this; we need a server-side
ability to register extensions (possibly multiple versions), and a
client-side ability to probe for them. This seems fairly
straightforward.

The main questions I can think of are: do we want to support the
implementation of multiple different versions of an extension
server-side (probably yes?), and do we want to support probing multiple
different versions of an extension client-side - use case is something
like ‘I can support DnD proto 2 or DnD proto 1, use what the server
supports’ (dunno?)

== Example ==

Here's a quick example of an IDL definition and generated headers. This
is not a proposal for what anything specific should look like. It's
merely intended to illustrate some properties of what a solution should
look like.

Given the example interface definition

/****************************************/
DnD {

version 1;

target surface {

/* Sent by shell when a DnD action targets a client surface.
 * May be sent more than once, with different coordinates.
 * To accept, client must respond with Accept
 */
event "Available" {
        surface_coordinate x;
        surface_coordinate y;
        string             available_formats;

        reply "Accept" {
                string acceptable_formats;
        };
};

}
}
/****************************************/

the interface compiler would generate the following header, and code
behind the header to interface with Mir's IPC mechanism in
libmirclientsurfacednd.so.1:

/***** CLIENT HEADER *********/
typedef void (*mir_surface_dnd_available_callback)(MirSurface *surf,
        int32_x, int32_y,
        const char *available_formats,
        MirEventCookie cookie,
        void *ctx);

void mir_surface_register_dnd_available_callback(MirSurface *surf,
        mir_surface_dnd_available_callback callback,
        void *ctx);

void mir_surface_dnd_accept(MirSurface *surf,
        MirEventCookie cookie,
        const char *acceptable_formats);

bool mir_connection_supports_surface_dnd(MirConnection* conn);

/******************************/

It would also generate the server interface, and code behind it to
interface with Mir's IPC mechanism in libmirserversurfacednd.so.1:

/***** SERVER HEADER **********/

class msh::Surface::DnD : public mir::Extension
{
public:
        void send_available (msh::Surface const& surf,
                             int32_t x, int32_t y,
                             std::string available_formats,
                             std::function<void(std::string)>
                                reply_handler);

};

---

From: Christopher James Halse Rogers <[email-masked]>
Date: Thu, 12 Dec 2013 12:46:29 +0800

> what we had already agreed to do (as much as the team can agree on 

*I* thought I was agreeing with the Simple/Simplistic method :). When
Alan brought up his stub implementation of Half-arsed it provoked a
further round of discussion, leading to here!

> serious problem with protobuf that needs solving.

I think we are, and have been, conflating a number of separate issues
here.

One is “what link-level IPC mechanism should we use? Sockets, shm, etc”.
We're currently using sockets+asio.

One is “what message serialisation mechanism should we use? Protobuf,
Cap'n'Proto, Wayland, etc”. We're currently using Protobuf.

A third is “what IPC interface definition language should we use?
Protobuf, Wayland, home-grown, etc”. We're currently not using *any*
IDL. For the bits where we are using an IDL we're using Protobuf's
descriptors, but we also have knowledge baked into the code - fd passing
because Protobuf just doesn't support that, and MirEvent, because of
reasons that I'm not aware of.

I don't mind if we continue to use Protobuf for our serialisation, but
an extension mechanism needs an actual IDL. Which could be as simple as
extending Protobuf; I don't mind.

---

From: Thomas Voß <[email-masked]>
Date: Thu, 12 Dec 2013 10:34:30 +0100

>> serious problem with protobuf that needs solving.

I don't see why we need a way to extend the protocol. We own every
part of the stack, top to bottom, and I don't see why the shell would
need a way to communicate to clients that is opaque to Mir. If it's an
issue of unaligned development pace, we should rather fix the process
as opposed to decoupling Mir from what Unity8 needs.

> reasons that I'm not aware of.

Let's fix it in one place in Mir then. MirEvent is a POD that was only
meant to communicate input events, it was never meant to carry over
arbitrary events to the client. This approach was triggered by our
choice of input stack. Btw., protobuf offers an IDL and a DDL. I'm not
sure why you think that we are not using it.

> extending Protobuf; I don't mind.

I guess it's less a question of IPC or serialization framework, but
more a question of: Why do we need to support arbitrary extensions
when we own the complete stack? In my understanding, this is a
contradiction. With our conversation at the sprint in mind: I think we
want an easy way to internally extend the protocol, that takes away
most of the tedious boilerplate generation by hand. Is that correct?

Thanks,

  Thomas

> --
> Mir-devel mailing list
> [email-masked]
> Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/mir-devel
>

---

From: Alan Griffiths <[email-masked]>
Date: Thu, 12 Dec 2013 10:02:52 +0000

> as opposed to decoupling Mir from what Unity8 needs.

The question I'm trying to resolve in this thread is how much support
for standardising protocols it is appropriate for Mir to provide.

What we've seen over the past months is that unity8 has a lot of
communication with the toolkits that do not represent anything that
matters directly to Mir (or to USC).  The past practice of providing an
implementation of these messages has proven problematic: every time
Unity8 needs a new message passing we get changes in the Mir client API
that are not driven by Mir, just by the need to pass these messages.

There is general agreement that a generic method for passing messages
through Mir is needed, The "poster child" from last week's meetings
being cut-and-paste.

Mir doesn't need to be involved in cut-and-paste beyond passing messages
- any intelligence lies in the communication between shell and toolkit.
While there are good reasons to standardise a cut-and-paste protocol the
Mir project isn't the place to do this. This protocol can develop and
change without changes to the Mir APIs.

---

From: Thomas Voß <[email-masked]>
Date: Thu, 12 Dec 2013 11:08:44 +0100

> that are not driven by Mir, just by the need to pass these messages.

Fair, but *some* client API has to change, why not Mir's then? As long
as we are conscious of the changes and bump version appropriately, we
are good. Coming up with an extension mechanism is an escape from a
strict and very helpful versioning scheme from my perspective and I do
not agree that we need a generic extension mechanism at this point.

> change without changes to the Mir APIs.

Agreed, but the real place to do this is not the shell but the content
hub (as discussed later on with Gerry and Ricmm). So taking this
example as a driving use-case might skew the conversation.

Thomas

---

From: Christopher James Halse Rogers <[email-masked]>
Date: Thu, 12 Dec 2013 18:11:15 +0800

On Thu, 2013-12-12 at 10:34 +0100, Thomas Voß wrote:
> need a way to communicate to clients that is opaque to Mir.

In part because we intend to *not* own every part of the stack - you
reiterated at the sprint a the goal is for Mir to be a generally-useful
library for writing display-server-compositor-shells, not simply a
unity8 subcomponent.

In part because we think there are things which need to be Mir-mediated
because they involve concepts like Surfaces, but which are Unity8
specific.

In part because we want to Mir to have a long and prosperous life, and
X11's extension mechanism is what allowed X to survive for 30-odd years.

>  If it's an issue of unaligned development pace, we should rather fix
>  the process as opposed to decoupling Mir from what Unity8 needs.

I don't think this *is* decoupling Mir from what Unity8 needs; it's
providing what Unity8 needs in a more flexible way.

> sure why you think that we are not using it.

We mostly use it, but we've gone outside protobuf in a way that means
mir_protobuf.proto is no longer a description of the interface.
Specifically, there's Event which is entirely opaque, and there's Buffer
which has ‘oh, by the way, there's stuff not in this interface’ field in
fds_on_side_channel.

If protobuf was expressive enough to use as our IDL we'd have a
‘repeated fd fds’ field and no fds_on_side_channel.

> most of the tedious boilerplate generation by hand. Is that correct?

We want that, yes.

But I think we do also want a general purpose extension mechanism,
designed in such a way that we can easily fold extensions into core
should we decide that they're useful in the general case.

---

From: Thomas Voß <[email-masked]>
Date: Thu, 12 Dec 2013 11:42:19 +0100

> library for writing display-server-compositor-shells, not simply a

Fully agreed, but I don't think having a generic protocol extension
mechanism helps in ensuring that we have a generally useful display
server for a simple reason: Application developers and toolkits would
then have to choose a shell that they specifically optimize for.

> X11's extension mechanism is what allowed X to survive for 30-odd years.

Fair point, but let's not make everything an extension. While it is
admittedly flexible to just come up with another extension (hey, it's
just an extension), it is likely to end in a maintenance nightmare if
it is overused. That being said, I think it's less a technical
question here, but more a question of policy and taking decisions
about functionality that should end up in the core.

>From a technical pov, just to make sure that we are on the same page,
I have the following approach in mind (U8 as an example):

Public facing API:
  // Can fail with: Not supported by current shell and it shouldn't be
called Mir* at all
  u8_enable_specific_feature_for_surface(MirSurface* surface);

Internally, this talks to Mir via a private API:

  // Check that U8 is actually on the other side.
  if (mir_connection_is_extension_supported(connection,
U8_SPECIFIC_EXTENSION_ID))
  {
      msg = mir_connection_create_message_for_surface(connection, surface);
      mir_message_set_destination(msg, U8_SPECIFIC_EXTENSION_RECEIVER_ID);
      // Fill in payload
      mir_connection_send(msg);
  }

Is that what you have in mind?

> providing what Unity8 needs in a more flexible way.

As pointed out earlier, it's a very delicate balance to strike and it
won't free us from the responsibility of deciding what is part of the
core and what is not. I think that if we introduce the ability to
extend the Mir API, we need to come up with a formal review process
for extensions, too, specifically to ensure the general usefulness of
Mir.

> should we decide that they're useful in the general case.

See before, let's not only think about technical feasibility but also
about our responsibility towards app & toolkit developers.

Cheers,

  Thomas

---

From: Thomas Voß <[email-masked]>
Date: Thu, 12 Dec 2013 13:35:14 +0100

>       mir_connection_send(msg);

This could quite easily be implemented on top of:

  https://developers.google.com/protocol-buffers/docs/reference/cpp/google.protobuf.dynamic_message

Cheers,

  Thomas

---

From: Gerry Boland <[email-masked]>
Date: Thu, 12 Dec 2013 16:33:34 +0000

On 11/12/13 14:00, Alan Griffiths wrote:
> I'm trying to summarise some discussions around the need for Mir
> to provide a ABI stable channel of communications to shells and
> client toolkits (such as unity8/platform-api) that make use of it.
> This should allow messages to be passed that Mir has no need nor
> ability to interpret.
> 
> Future nice-to-haves: o consistency between shells and a framework
> for protocol extensions o some messages to be meaningful to Mir to
> enable optimizations o Migrating the current API (e.g. attribute
> setting) onto the same channel
> 
>  Mir simply punts all concerns about protocol negotiation,
> versioning and structure to the shell and relies on the shell to
> provide all information that Mir will ever need.
> 
> The advantage of this approach is that we can do it without
> further discussion and unblock unity8 work.
> 
> The disadvantage of this approach is that the "nice to haves" are
> left as an exercise for the interested Mir user.
> 
> *Half-assed*
> 
> version-number**and message-discriminator to pass with the message
> - and define some base protocols (sizing, copy-paste,
> protocol-extension). Mir (or "shellinator") can then implement
> defaults for the base protocols.
> 
> We discussed this last week and felt that it raised more issues
> than it addressed.
> 
> *fully-arsed*
> 
> Projects like X11 and Wayland have effective protocol definition
> and extension mechanisms. We should build on this - Wayland being
> the better example as it builds on the X11 experience.
> 
> I don't yet have a clear vision of what a successful implementation
> of this approach looks like - and suspect that different team
> members have differing thoughts on how we would go about this.
> 
> *Conclusion*
> 
> specific ideas. But as that may not lead to a quick (or any)
> conclusion it is worth implementing the simple approach in parallel
> so as to unblock unity8 development.

Hey Alan,
I'm of the opinion that initially we will need to rapidly iterate on
the protocol between shell and client, which is something an opaque
channel lets us do with ease. Since right now I think that none of
those messages will interest Mir directly (shell will inform Mir if
any client message is relevant to it) so it can be a just simple
messaging channel.

I don't really see the advantage of your "half-assed" idea, as it
blurs the boundary between what Mir should define, and what it should
not. As we iterate, this boundary I imagine will move again and again!

However I do agree that longer term, having a client/shell messaging
channel without any defined protocol will introduce many problems once
other toolkits and shells start using Mir. We don't want each
shell/client pair having their own protocol, incompatible with our own.

So I would propose that at some point in future, at the point we have
shell performing to our requirements, that we review the client/shell
protocol that organically grew to satisfy those requirements, to see
how to clean it up and standardize it.

My thinking is that the client/shell protocol definition would live in
a project shared between unity-mir and platform-api (and probably
"shellinator"?). Then when it has stabilized, it can be moved directly
into Mir itself, or at the very least labelled a standard.

What do you think?

Also, should we intend to completely eradicate the opaque messaging
channel from Mir in future, then I think we should enable extensions
on top of the standard client/shell protocol. This is mainly to leave
an open door for 3rd party shell developers, to prove that Mir is not
just designed to satisfy Unity8's requirements, but is a flexible yet
powerful foundation for their own display server.

Thanks
- -Gerry

---

From: Alan Griffiths <[email-masked]>
Date: Fri, 13 Dec 2013 11:33:22 +0000

>   https://developers.google.com/protocol-buffers/docs/reference/cpp/google.protobuf.dynamic_message

Summarizing discussion from hangout...

The use of a definition language and code generation isn't a short term
option - but is nice for the future.

Short term we're going with:

Mir client and server provide a "-private-def" package with C++
interfaces into the communications protocol that allows shell+toolkit(s)
to pass arbitrary protobuf messages.

This will allow unity8/platform-api to prototype and refine interfaces
before they are migrated into the core Mir APIs

This mechanism will need some methods for identifying and transmitting
FDs across the socket (e.g. a special field name.) This is something
that was not allowed for in the raw buffer approach described in the OP.

---

From: Daniel van Vugt <[email-masked]>
Date: Mon, 16 Dec 2013 09:39:33 +0800

I assume you're not talking about exposing the fact that Mir uses 
protobuf to shell+toolkits? You still mean hiding extendability behind a 
library, right?

I'm still mostly trying to not have an opinion on this thread, but to 
break the information hiding we have now and expose the underlying 
protocol library to the shell or toolkit sounds like a bad idea. 
Information hiding is useful.

---

From: Alan Griffiths <[email-masked]>
Date: Tue, 17 Dec 2013 14:56:07 +0000

On 16/12/13 01:39, Daniel van Vugt wrote:
> protobuf to shell+toolkits? You still mean hiding extendability behind
> a library, right?

Wrong.

> Information hiding is useful.

It is a private interface that can be used to develop new protocols at
the level of unity-mir/platform-api prior to migrating them into Mir.

---

From: Christopher James Halse Rogers <[email-masked]>
Date: Wed, 18 Dec 2013 11:01:04 +0800

> the level of unity-mir/platform-api prior to migrating them into Mir.

Expanding on this, it's explicitly a prototyping interface, and
approximately the cheapest possible prototyping interface that allows
sending fds.

It'll go away the moment we have a more solid solution implemented.