Subject: What is a MirRenderSurface?

From: Kevin DuBois <[email-masked]>
Date: Wed, 7 Sep 2016 13:34:38 -0400

Two ways to answer this question make sense to me:

1) A MirRenderSurface is an EGLNativeWindowType
2) A MirRenderSurface is a mf::BufferStreamId (ie, it is the token
representing server-side)

We currently seem to be saying that MirRenderSurface are both. [1]

(crux of my point)
Given that there exists software, vulkan, multimedia, and
nested-passthrough, it doesn't make sense to me to answer that "A
MirRenderSurface is 1) AND 2)", because all of those technologies don't
care about EGL.
We shouldn't make an assumption that the user will be using EGL, because we
have clients who specifically want a place to draw without having anything
to do with EGL.

So, given that, I wrote up how the headers would look if we decide
MirRenderSurface is an EGLNativeWindowType or if we decide MirRenderSurface
is a mf::BufferStreamId. I attached a sketch of the headers to this mail,
and also catted the contents to pastebin
<http://pastebin.ubuntu.com/23146845/>. [2]

I think first we have to agree that the answer is "1) OR 2)" instead of "1)
AND 2)".
Both versions of the headers make sense from the client's perspective,
although we should go back-and-forth on the merits of both approaches of
course, once we've figured out what MirRenderSurface is.

Thanks,
Kevin

[1]
In the last version of the headers that I've seen, we have a conversion to
a MirBufferStream (ie, MirRenderSurface is a mf::BufferStreamId) here
<http://bazaar.launchpad.net/%7Ecemil-azizoglu/mir/mir-render-surface-v3/view/head:/src/include/client/mir_toolkit/mir_render_surface.h#L55>,
and we have an implicit conversion here
<http://bazaar.launchpad.net/%7Ecemil-azizoglu/mir/mir-render-surface-v3/view/head:/playground/eglflash_render_surface.c#L112>
(ie, MirRenderSurface is also a EGLNativeWindowType)

P.S.
Topic has come up in discussions here:
https://code.launchpad.net/~cemil-azizoglu/mir/use_egl_native_window/+merge/304726
and the WIP branch here:
https://code.launchpad.net/~cemil-azizoglu/mir/mir-render-surface-v3/+merge/304536

P.P.S
We also discussed a bit about client vs driver interfaces. I think
answering this mail is the primary topic, and will send a follow up mail to
make sure we're on the same page about the different technologies/semantics
going on.

---

From: Cemil Azizoglu <[email-masked]>
Date: Wed, 7 Sep 2016 14:09:08 -0500

Regardless ​of what EGL (or any other API) needs, there needs to be a way
for Mir platform to generically represent a unit of renderable (hw or sw)
with a distinct type. That's what MirRenderSurface provides. "Generically"
in the sense that it doesn't matter what it's backed by (BS or PC or
whatever), and it can be passed around  (to other APIs, not just
EGL) freely. Whereas, if we pass around, say, a BS handle then we become
dependent on a backend that we may want to change or even deprecate in the
future.

Also, EGL doesn't impose platforms to have an EGL specific type. A platform
has its own type, and EGL is *made* to work with it. Here's a comment from
the EGL header :

/* The types NativeDisplayType, NativeWindowType, and NativePixmapType
 * are aliases of window-system-dependent types, such as X Display * or
 * Windows Device Context. They must be defined in platform-specific
 * code below. The EGL-prefixed versions of Native*Type are the same
 * types, renamed in EGL 1.3 so all types in the API start with "EGL".

That means we should have one consistent type that may take different forms
depending on the backend. So saying "A MirRenderSurface is an
EGLNativeWindowType" is not the right way to look at it. Because there is
nothing EGL-specific about a MirRenderSurface.

As you know, ANativeWindow in Android, for instance, is not specific to
EGL. It just represents a native window for Android for s/w and h/w.
MirRenderSurface should be considered the same. You have a point in saying
that we already expose clients with too much info on BS & PC. Perhaps we
shouldn't, but that's a separate discussion.

Regards,
-Cemil

Cemil Azizoglu
Mir Display Server - Team Lead
Canonical USA

On Wed, Sep 7, 2016 at 12:34 PM, Kevin DuBois <[email-masked]>
wrote:

> We shouldn't make an assumption that the user will be using EGL, because
> we have clients who specifically want a place to draw without having
> anything to do with EGL.
>
> I think first we have to agree that the answer is "1) OR 2)" instead of
> "1) AND 2)".
> --
> Mir-devel mailing list
> [email-masked]
> Modify settings or unsubscribe at: https://lists.ubuntu.com/
> mailman/listinfo/mir-devel
>
>

---

From: Christopher James Halse Rogers <[email-masked]>
Date: Thu, 08 Sep 2016 10:01:11 +1000

On Thu, Sep 8, 2016 at 5:09 AM, Cemil Azizoglu 
<[email-masked]> wrote:
> Regardless ​of what EGL (or any other API) needs, there needs to be 
> a way for Mir platform to generically represent a unit of renderable 
> (hw or sw) with a distinct type. That's what MirRenderSurface 
> provides. "Generically" in the sense that it doesn't matter what it's 
> backed by (BS or PC or whatever), and it can be passed around  (to 
> other APIs, not just EGL) freely.

Right. There was a point (long ago) at which MirSurface was sort-of 
this atomic-unit-of-renderable, but with cursor-renderables and 
multiple-renderables-per-surface this went away.

It is useful to have an absolutely minimal ‘this is a thing that can 
have image content posted to it’ concept; MirRenderSurface should be 
that. (Indeed, I'm not sure that that MirRenderSurface should even have 
a *size*, and it definitely shouldn't have a pixel format; I've 
remarked as such on the prototype branches).

> Whereas, if we pass around, say, a BS handle then we become dependent 
> on a backend that we may want to change or even deprecate in the 
> future.
> 
> Also, EGL doesn't impose platforms to have an EGL specific type. A 
> platform has its own type, and EGL is *made* to work with it. Here's 
> a comment from the EGL header :
> That means we should have one consistent type that may take different 
> forms depending on the backend. So saying "A MirRenderSurface is an 
> EGLNativeWindowType" is not the right way to look at it. Because 
> there is nothing EGL-specific about a MirRenderSurface.
> 
> As you know, ANativeWindow in Android, for instance, is not specific 
> to EGL. It just represents a native window for Android for s/w and 
> h/w. MirRenderSurface should be considered the same. You have a point 
> in saying that we already expose clients with too much info on BS & 
> PC. Perhaps we shouldn't, but that's a separate discussion.

Right. EGLNativeWindowType is defined by the platform as “a handle to 
the thing you can render to”.

This is why Mir's Android support is misleading. On the Android 
platform Mir pretends to be SurfaceFlinger, and this is obviously 
awkward, even though Google has deliberately made it possible.

But this really is going to be a special case. We're not going to 
pretend to be Wayland to get closed-source Mali support; we're not 
going to pretend to be X11 to get closed-source NVIDIA support, 
although we could (at the cost of significant madness) in both cases.

---

From: Kevin DuBois <[email-masked]>
Date: Thu, 8 Sep 2016 07:36:34 -0400

On Wed, Sep 7, 2016 at 8:01 PM, Christopher James Halse Rogers <
[email-masked]> wrote:

>
>
> On Thu, Sep 8, 2016 at 5:09 AM, Cemil Azizoglu <
> [email-masked]> wrote:
>
>> whatever), and it can be passed around  (to other APIs, not just EGL)
>> freely.
>>
>
> Right. There was a point (long ago) at which MirSurface was sort-of this
> atomic-unit-of-renderable, but with cursor-renderables and
> It is useful to have an absolutely minimal ‘this is a thing that can have
> image content posted to it’ concept; MirRenderSurface should be that.
> (Indeed, I'm not sure that that MirRenderSurface should even have a *size*,
> and it definitely shouldn't have a pixel format; I've remarked as such on
> the prototype branches).
>
>

I'm on board here, right up until we try to cast MirRenderSurface to
EGLNativeWindowType. (This is what I called "Option 2 in the OP). The
'absolutely minimal' concept is a mf::BufferStreamId, and then the user has
to tell us what semantics they want to use to upload their content.

Being able to cast MirRenderSurface to differing native rendering
technologies:
1) Is not scalable. In the current spike we're really only casting to
EGLNativeWindowType, but will we cast to gstreamer handles? Can I just make
all the mir_presentation_chain() functions just take a MirRenderSurface
instead of a MirPresentationChain? (same with the mir_buffer_stream()
functions). Its really handy for EGL, but unnecessarily ties all other ways
of using Mir to EGL.

2) Its not obvious how to use this. The casting to an EGLNativeWindowType
has to be pointed out in comment-headers, or gleaned from reading example
code.

3) It puts MirPresentationChain and MirBufferStream as second-class
citizens. Having to create an EGLNativeWindowType first, and then create a
MirPresentationChain doesn't make sense (unless from the vantage of
favoring EGL to the detriment of the other technologies). It is more work
for us to create an EGLNativeWindowSurface up-front (or via an even more
complicated system of deferred type realization) regardless of whether the
user wants to use EGL or not.

So, by tweaking the header like this: (repost:
http://pastebin.ubuntu.com/23149827/)
These problems are resolved, and internally and externally, it becomes
clearer what's going on.

> Whereas, if we pass around, say, a BS handle then we become dependent on a
>> backend that we may want to change or even deprecate in the future.
>>
>> platform has its own type, and EGL is *made* to work with it. Here's a
>> comment from the EGL header :
> Right. EGLNativeWindowType is defined by the platform as “a handle to the
> thing you can render to”.
>
> This is why Mir's Android support is misleading. On the Android platform
> Mir pretends to be SurfaceFlinger, and this is obviously awkward, even
> though Google has deliberately made it possible.
>
> But this really is going to be a special case. We're not going to pretend
> to be Wayland to get closed-source Mali support; we're not going to pretend
> to be X11 to get closed-source NVIDIA support, although we could (at the
> cost of significant madness) in both cases.
>
>
This is a second topic from the one I originally brought up (having to do
with that we can cast MirRenderSurface to different rendering technologies
types). This topic is more what do we provide to drivers and how we define
our responsibilities to the different platform. It needs to be addressed,
but I think clearing up the MirRenderSurface wip header is step one. Maybe
I'll diverge the email chain to talk about Mir's relationship to
EGL/drivers.

---

From: Christopher James Halse Rogers <[email-masked]>
Date: Thu, 08 Sep 2016 22:39:17 +1000

On Thu, Sep 8, 2016 at 9:36 PM, Kevin DuBois 
<[email-masked]> wrote:
> 
> 
> On Wed, Sep 7, 2016 at 8:01 PM, Christopher James Halse Rogers 
> <[email-masked]> wrote:
>> 
>> 
>>> Regardless ​of what EGL (or any other API) needs, there needs to 
>>> be a way for Mir platform to generically represent a unit of 
>>> renderable (hw or sw) with a distinct type. That's what 
>>> MirRenderSurface provides. "Generically" in the sense that it 
>>> doesn't matter what it's backed by (BS or PC or whatever), and it 
>>> can be passed around  (to other APIs, not just EGL) freely.
>> 
>> It is useful to have an absolutely minimal ‘this is a thing that 
>> can have image content posted to it’ concept; MirRenderSurface 
>> should be that. (Indeed, I'm not sure that that MirRenderSurface 
>> should even have a *size*, and it definitely shouldn't have a pixel 
>> format; I've remarked as such on the prototype branches).
>> 
> 
> EGLNativeWindowType.

This is what I mean by the Android platform being a poor model. We're 
*not* casting MirRenderSurface to EGLNativeWindowType. We're *defining* 
EGLNativeWindowType to be MirRenderSurface. We are the windowing 
system. We define what EGLNativeWindowType is.

Now, this is awkward on Android because we're pretending to be a 
different windowing system, SurfaceFlinger, but it's a quirk of the 
Android platform.

> (This is what I called "Option 2 in the OP). The 'absolutely minimal' 
> concept is a mf::BufferStreamId, and then the user has to tell us 
> what semantics they want to use to upload their content.
> 
> EGLNativeWindowType, but will we cast to gstreamer handles? Can I 
> just make all the mir_presentation_chain() functions just take a 
> MirRenderSurface instead of a MirPresentationChain? (same with the 
> mir_buffer_stream() functions). Its really handy for EGL, but 
> unnecessarily ties all other ways of using Mir to EGL.

Again, this *doesn't* tie Mir to EGL. We're not casting 
MirRenderSurface to EGLNativeWindowType anymore than you cast Window* 
to EGLNativeWindowType on X11. Likewise, we're not tying 
MirRenderSurface to EGL anymore than Window* is tied to EGL on X11.

It *does* have implications for how we implement MirRenderSurface on 
Android; specifically, it means that MirRenderSurface needs to be a 
SurfaceFlinger window. But even this doesn't tie Mir to EGL here - if 
MirRenderSurface can masquerade as a SurfaceFlinger window it should 
also work for Android Vulcan, which expects a SurfaceFlinger window, 
and *does* also work for the HW video encoding, which expects a 
SurfaceFlinger window.

> 2) Its not obvious how to use this. The casting to an 
> EGLNativeWindowType has to be pointed out in comment-headers, or 
> gleaned from reading example code.
> 
> citizens. Having to create an EGLNativeWindowType first, and then 
> create a MirPresentationChain doesn't make sense (unless from the 
> vantage of favoring EGL to the detriment of the other technologies). 
> It is more work for us to create an EGLNativeWindowSurface up-front 
> (or via an even more complicated system of deferred type realization) 
> regardless of whether the user wants to use EGL or not.
> 
> These problems are resolved, and internally and externally, it 
> becomes clearer what's going on.

This:

EGLNativeWindowType mir_render_surface_create_eglnativewindowtype(
    MirRenderSurface* render_surface,
    int physical_width, int physical_height,
    void *egldisplay, void* eglconfig); //see 
mir_connection_get_pixel_format()

is a really confusing function. We're passing in an EGLDisplay and an 
EGLConfig to get a Mir-defined-type, the only purpose of which is to 
pass to EGLCreateWindowSurface to create an EGLSurface? Why aren't we 
just returning an EGLSurface?

Put like that, I think this makes it clear what's wrong - EGL *is* the 
layer that takes a window-system specific type (here, MirRenderSurface) 
and presents it as something cross-platform. It makes no sense to take 
a window-system specific type, convert it into *another* EGL-specific 
window-system specific type, and then create a cross-platform EGL 
object from it.

> 
> 
>>> Whereas, if we pass around, say, a BS handle then we become 
>>> dependent on a backend that we may want to change or even deprecate 
>>> in the future.
>>> 
>>> platform has its own type, and EGL is *made* to work with it. 
>>> Here's a comment from the EGL header :
>>> That means we should have one consistent type that may take 
>>> different forms depending on the backend. So saying "A 
>>> MirRenderSurface is an EGLNativeWindowType" is not the right way to 
>>> look at it. Because there is nothing EGL-specific about a 
>>> MirRenderSurface.
>>> 
>>> As you know, ANativeWindow in Android, for instance, is not 
>>> specific to EGL. It just represents a native window for Android for 
>>> s/w and h/w. MirRenderSurface should be considered the same. You 
>>> have a point in saying that we already expose clients with too much 
>>> info on BS & PC. Perhaps we shouldn't, but that's a separate 
>>> discussion.
>> 
>> Right. EGLNativeWindowType is defined by the platform as “a handle 
>> to the thing you can render to”.
>> 
> This is a second topic from the one I originally brought up (having 
> to do with that we can cast MirRenderSurface to different rendering 
> technologies types). This topic is more what do we provide to drivers 
> and how we define our responsibilities to the different platform. It 
> needs to be addressed, but I think clearing up the MirRenderSurface 
> wip header is step one. Maybe I'll diverge the email chain to talk 
> about Mir's relationship to EGL/drivers.

---

From: Kevin DuBois <[email-masked]>
Date: Thu, 8 Sep 2016 09:16:23 -0400

On Thu, Sep 8, 2016 at 8:39 AM, Christopher James Halse Rogers <
[email-masked]> wrote:

>
>
> On Thu, Sep 8, 2016 at 9:36 PM, Kevin DuBois <[email-masked]>
> wrote:
>
>>
>>
>>>> Regardless ​of what EGL (or any other API) needs, there needs to be a
>>>> way for Mir platform to generically represent a unit of renderable (hw or
>>>> sw) with a distinct type. That's what MirRenderSurface provides.
>>>> "Generically" in the sense that it doesn't matter what it's backed by (BS
>>>> or PC or whatever), and it can be passed around  (to other APIs, not just
>>>> EGL) freely.
>>>>
>>>
>>> have image content posted to it’ concept; MirRenderSurface should be that.
> EGLNativeWindowType to be MirRenderSurface. We are the windowing system. We
> define what EGLNativeWindowType is.
>
> Now, this is awkward on Android because we're pretending to be a different
> windowing system, SurfaceFlinger, but it's a quirk of the Android platform.
>
>> concept is a mf::BufferStreamId, and then the user has to tell us what
>> semantics they want to use to upload their content.
>>
> Again, this *doesn't* tie Mir to EGL. We're not casting MirRenderSurface
> to EGLNativeWindowType anymore than you cast Window* to EGLNativeWindowType
> on X11. Likewise, we're not tying MirRenderSurface to EGL anymore than
> Window* is tied to EGL on X11.
>

We quite literally cast MirRenderSurface* to EGLNativeWindowSurface in the
current WIP:
http://bazaar.launchpad.net/%7Ecemil-azizoglu/mir/mir-render-surface-v3/view/head:/playground/eglflash_render_surface.c#L112

Currently, we're a window-system-independent windowing system. This is from
our charter to be as flexible as possible, especially when we don't have
any control over the binaries we're trying to run. Once we define an object
and try to dictate that all technologies out there use it, we've lost a
great deal of flexibility.

This is why I'd much rather create an EGLNativeWindowType for the user than
try to tell all the drivers out there that they have to adopt
MirRenderSurface as their windowing type.

Its mesa-specific thinking to think that we can define the window interface
for every platform/technology that we've been told is important. We do
define our mesa interface here (and we carry unaccepted upstream patches
for it):
http://bazaar.launchpad.net/~mir-team/mir/development-branch/view/head:/include/platforms/mesa/mir_toolkit/mesa/native_display.h

If we want to redefine our mesa interface, that's alright by me, and its
even fine to define our mesa interface to /be/ the mir client interface.
The problem is that we're trying to define our windowing type for platforms
where we cannot define the windowing type.

> MirRenderSurface can masquerade as a SurfaceFlinger window it should also
> work for Android Vulcan, which expects a SurfaceFlinger window, and *does*
> also work for the HW video encoding, which expects a SurfaceFlinger window.
>
>    void *egldisplay, void* eglconfig); //see mir_connection_get_pixel_forma
> t()
>
> EGLConfig to get a Mir-defined-type, the only purpose of which is to pass
> to EGLCreateWindowSurface to create an EGLSurface? Why aren't we just
> returning an EGLSurface?
>

Once EGL_KHR_platform_android is defined, its not a mir-defined type
anymore.
I'd be fine to return a EGLSurface, (although EGLNativeWindowType is really
what the user wants)

> layer that takes a window-system specific type (here, MirRenderSurface) and
> presents it as something cross-platform. It makes no sense to take a
> window-system specific type, convert it into *another* EGL-specific
> window-system specific type, and then create a cross-platform EGL object
> from it.
>
>
>
>>
>> Whereas, if we pass around, say, a BS handle then we become dependent on
>>>> a backend that we may want to change or even deprecate in the future.
>>>>
>>> pretend to be Wayland to get closed-source Mali support; we're not going to
>>> pretend to be X11 to get closed-source NVIDIA support, although we could
>>> (at the cost of significant madness) in both cases.
>>>
>>>

---

From: Cemil Azizoglu <[email-masked]>
Date: Thu, 8 Sep 2016 12:59:20 -0500

I think we are at an impasse. I'd appreciate it if others joined in the
conversation and provided their feedback so we can form a consensus and
move forward. We can have a special call for this next week (sorry I'm out
this week) if needed.

Thanks
Cemil

​

---

From: Cemil Azizoglu <[email-masked]>
Date: Thu, 8 Sep 2016 13:18:10 -0500

> http://bazaar.launchpad.net/%7Ecemil-azizoglu/mir/mir-
> render-surface-v3/view/head:/playground/eglflash_render_surface.c#L112
>

​Perhaps, I should point out that our EGLNativeWindowType is not (and
cannot be) 'MirRenderSurface'. It's void*.​ This is because we (have to)
pretend to be other platforms (Android). That's what the cast is for. So if
we get technical, our EGLNativeWindowType is 'void*'. We are not painting
ourselves in a corner with a type that is not universal enough. So if Mir
appeared in an EGL header, it would have 'void*' not 'MirRenderSurface*' as
its native window type.

>
> This is why I'd much rather create an EGLNativeWindowType for the user
> than try to tell all the drivers out there that they have to adopt

I disagree. ​A platform absolutely has to define its own type, and the
drivers must adopt it. (Modulo the fact that we don't impose the
MirRenderSurface type as that because of the need to support Android.)

> Its mesa-specific thinking to think that we can define the window
> interface for every
>

​There is nothing Mesa-specific here. A platform defines the type and the
driver implementer adds support for it around that type. This is what every
platform has to do. (Except, for platforms that we pretend to be...like
Android).

As you see I'm having to add 'except' to the statements above, because
Android is the exception, not Mesa.

> where we cannot define the windowing type.

​We absolutely can and should....except again for Android.​

>>> has to be pointed out in comment-headers, or gleaned from reading example
Mir native window type is not in EGL headers yet. This "non-obvious-ness"
can easily be fixed when we add it in
https://www.khronos.org/registry/egl/api/EGL/eglplatform.h
 as a platform.​

>>> citizens. Having to
​Who says that they should be first-class citizens? They are specific
implementations that are prone to change and deprecation. That said, the
fact that you can't use a MirRenderSurface without a backing of BS or PC
would still make them a "first class citizen" anyways.

​Regards,​

​Cemil​

---

From: Christopher James Halse Rogers <[email-masked]>
Date: Fri, 09 Sep 2016 10:13:32 +1000

On Fri, Sep 9, 2016 at 4:18 AM, Cemil Azizoglu 
<[email-masked]> wrote:
>> 
>> We quite literally cast MirRenderSurface* to EGLNativeWindowSurface 
>> in the current WIP:
> ​Perhaps, I should point out that our EGLNativeWindowType is not 
> (and cannot be) 'MirRenderSurface'. It's void*.​ This is because we 
> (have to) pretend to be other platforms (Android).

Hm, this got me thinking.

*Does* Mir need to pretend to be SurfaceFlinger?

As I understand it, the only relevant constraint is that we need to be 
able to bring up Unity8 and USC using unmodified Android drivers.
 - This means we need to use Android libEGL at *some* point in the 
stack.
 - We (as in Ubuntu) control what libEGL.so.1 resolves to.
  + This does *not* have to be Android's libEGL - indeed, it's 
currently *not* Android's libEGL, it's libhybris' wrapper.
 - This means we have the opportunity to interpose code between 
EGL-using clients and Android's EGL.

I looked at the code in platforms/android/client, and there doesn't 
appear to be any code there that can *only* be implemented inside a 
platform module. Indeed, it looks like the NBS API was basically 
designed to allow code linking against libmirclient to do exactly what 
is done inside the EGLNativeWindow producing bits of the Android 
platform.

So I think we *could* #define EGLNativeWindowType MirRenderSurface* 
(and likewise #define EGLNativeDisplayType MirConnection*) without a 
great deal of effort required. Mostly this would require moving code 
from platforms/android/client into a new libEGL that implements 
eglGetDisplay() and eglCreateWindowSurface() and links to the hybris 
libEGL to satisfy all other EGL symbols.

This would make Mir look like all other EGL platforms. To create an 
EGLDisplay you'd eglGetDisplay(connection); to create an EGLSurface 
you'd eglCreateWindowSurface(render_surface, ...).

I'm not sure if this is worth doing, but it does not appear that it 
would be terribly difficult to do.

---

From: Christopher James Halse Rogers <[email-masked]>
Date: Fri, 09 Sep 2016 12:26:42 +1000

Thinking about this further, I think we'll *have* to implement a shim 
libEGL in the not-too-distant future. We can't propose or use 
EGL_KHR_platform_mir unless we implement it on the Android stack. (And 
we want to, because it's useful in certain cases).

So I therefore think we *should*:
#define EGLNativeWindowType MirRenderSurface*
#define EGLNativeDisplayType MirConnection*

implement our libEGL shim providing the Mir platform on Android, and 
deprecate mir_connection_get_egl_native_display() and not implement 
mir_render_surface_get_egl_native_surface().

---

From: Kevin DuBois <[email-masked]>
Date: Fri, 9 Sep 2016 07:25:36 -0400

I'm glad that the topic of EGL platforming was split off, its a different
question than the one I was raising in the "What is a MirRenderSurface?"

So really the question here is "should Mir have its own
EGL_KHR_platform_mir". I'm not opposed to this in general.

This goes back to the charter that we were given at the start of Mir, which
is to accept drivers as they come to us, especially if they come to use in
binary form with no intention from the driver authors to change anything
about their drivers. The notable example of this is the Android platform,
but the wider intention is to stay flexible for closed-source binary
drivers that have been coming our way.

So if we re-examine that question and decide that we're our own platform,
then we can bolt other technologies on top via the use of shims and such.
It is a bit more work for 'non-mir' platforms, especially when we have to
deal with hybris and creating shim libraries and such to wrangle things
into a mir-platform-interface (which at this point we don't define).

I'd rather just continue saying that Mir is a WindowManager that can take
plugins to different platforms, as opposed to saying Mir is a WindowManager
and NativeWindowType, and non-mir platforms have to be adapted to the mir
platform. It also is 'staying the course' as far as the story that we've
been promulgating to driver developers.

IMHO, The decision to define our own windowing type is something that will
affect more than just the mir team (it has to do a bit with strategy), so
would be good to get input from a wider audience.

On Thu, Sep 8, 2016 at 10:26 PM, Christopher James Halse Rogers <
[email-masked]> wrote:

> EGL_KHR_platform_mir unless we implement it on the Android stack. (And we
> want to, because it's useful in certain cases).
>
> Modify settings or unsubscribe at: https://lists.ubuntu.com/mailm
> an/listinfo/mir-devel
>

---

From: Kevin DuBois <[email-masked]>
Date: Fri, 9 Sep 2016 07:33:03 -0400

On Thu, Sep 8, 2016 at 8:13 PM, Christopher James Halse Rogers <
[email-masked]> wrote:

> On Fri, Sep 9, 2016 at 4:18 AM, Cemil Azizoglu <
>>> We quite literally cast MirRenderSurface* to EGLNativeWindowSurface in
>>> the current WIP:
>>> http://bazaar.launchpad.net/%7Ecemil-azizoglu/mir/mir-render
>>> -surface-v3/view/head:/playground/eglflash_render_surface.c#L112
>>>
>>
>> pretend to be other platforms (Android).
>>
>
> - This means we need to use Android libEGL at *some* point in the stack.
>  + This does *not* have to be Android's libEGL - indeed, it's currently
> *not* Android's libEGL, it's libhybris' wrapper.
> - This means we have the opportunity to interpose code between EGL-using
> clients and Android's EGL.
>
> I looked at the code in platforms/android/client, and there doesn't appear
> to be any code there that can *only* be implemented inside a platform
> module. Indeed, it looks like the NBS API was basically designed to allow
> code linking against libmirclient to do exactly what is done inside the
> EGLNativeWindow producing bits of the Android platform.
>

Right. EGL windowing support can be built on top of NBS, and so can
multimedia, vulkan, nested passthrough, software rendering.

> So I think we *could* #define EGLNativeWindowType MirRenderSurface* (and
> likewise #define EGLNativeDisplayType MirConnection*) without a great deal
> of effort required. Mostly this would require moving code from
> platforms/android/client into a new libEGL that implements eglGetDisplay()
> and eglCreateWindowSurface() and links to the hybris libEGL to satisfy all
> other EGL symbols.
>

> EGLDisplay you'd eglGetDisplay(connection); to create an EGLSurface you'd
> eglCreateWindowSurface(render_surface, ...).
>
> I'm not sure if this is worth doing, but it does not appear that it would
> be terribly difficult to do.
>
>

---

From: Christopher James Halse Rogers <[email-masked]>
Date: Fri, 09 Sep 2016 21:58:57 +1000

On Fri, Sep 9, 2016 at 9:25 PM, Kevin DuBois 
<[email-masked]> wrote:
> I'm glad that the topic of EGL platforming was split off, its a 
> different question than the one I was raising in the "What is a 
> MirRenderSurface?"
> 
> This goes back to the charter that we were given at the start of Mir, 
> which is to accept drivers as they come to us, especially if they 
> come to use in binary form with no intention from the driver authors 
> to change anything about their drivers. The notable example of this 
> is the Android platform, but the wider intention is to stay flexible 
> for closed-source binary drivers that have been coming our way.
> 
> So if we re-examine that question and decide that we're our own 
> platform, then we can bolt other technologies on top via the use of 
> shims and such. It is a bit more work for 'non-mir' platforms, 
> especially when we have to deal with hybris and creating shim 
> libraries and such to wrangle things into a mir-platform-interface 
> (which at this point we don't define).

Mir is *already* our own platform. There are Mir things that you pass 
to eglGetDisplay to get an EGLDisplay and Mir things you pass to 
eglCreateWindowSurface to get an EGLSurface. We cannot support EGL and 
*not* be a platform.

The fact that you can sometimes cast the result of 
mir_buffer_stream_get_egl_native_window() to an ANativeWindow* and 
sometimes not doesn't change the fact that the return type of 
mir_buffer_stream_get_egl_native_window() is Mir's EGLNativeWindowType.

2) Android is really, really special. Android drivers are the only 
binary drivers we're *ever* going to be able to use without vendor 
support, and that's only because the Android platform was specifically 
designed to be plugged into without vendor support.

Closed-source binary drivers have to support *some* platform, and the 
set of platforms they can support is {Android, Windows, OS X, X11, 
Wayland}. Of those, there's exactly one we could use - Android.

Not being a platform doesn't make it easier for us to support 
non-Android closed source drivers. It makes it harder for closed-source 
drivers to support *us*.

> I'd rather just continue saying that Mir is a WindowManager that can 
> take plugins to different platforms, as opposed to saying Mir is a 
> WindowManager and NativeWindowType, and non-mir platforms have to be 
> adapted to the mir platform. It also is 'staying the course' as far 
> as the story that we've been promulgating to driver developers.

I don't think it's relevant for driver developers, with, again, the 
exception of Android. It's just not possible for us to take a, say, 
Xorg-targetting binary driver and support it in Mir without 
implementing Xorg; we can't take a Windows-targetting binary driver 
without implementing a bunch of the Windows kernel, and we can't take a 
Wayland-targetting binary driver without implementing a Wayland 
server¹.

Having MirRenderSurface* as our native window type doesn't constrain 
driver developers; their existing EGL was going to take an X11 Window*, 
or a wl_surface, or a HWND, and there's no way they're going to want to 
fake an X11 connection, a Wayland connection, or the Windows kernel 
inside their Mir client platform. They'll use Mir API², because 
there's literally nothing else they could more easily do.

That only works for Android because it's fundamentally designed for it 
- and because Android is fundamentally designed for it, it's no easier 
to do in the client platform than in a libEGL.

> 
> IMHO, The decision to define our own windowing type is something that 
> will affect more than just the mir team (it has to do a bit with 
> strategy), so would be good to get input from a wider audience.
> 
> 
> 
> On Thu, Sep 8, 2016 at 10:26 PM, Christopher James Halse Rogers 
>> Thinking about this further, I think we'll *have* to implement a 
>> shim libEGL in the not-too-distant future. We can't propose or use 
>> EGL_KHR_platform_mir unless we implement it on the Android stack. 
>> (And we want to, because it's useful in certain cases).
>> 

¹: However much of a good or bad idea that might be ☺.
²: It would be nice if we could give them a fully featured extension 
mechanism to use, rather than our current emaciated 
mir_platform_operation pseudo-extension mechanism.

---

From: Kevin DuBois <[email-masked]>
Date: Fri, 9 Sep 2016 07:59:27 -0400

Hmm, let me try to rephrase my point, I've lost track of what is in
disagreement. We have a new thread "Does Mir have to pretend to be
SurfaceFlinger" that we can talk about our relationship to drivers. So for
further discussion, let's just assume we only have the Mesa platform.

We have a few different technologies that we're supporting, or aim to
support:
EGL, Vulkan, software-rendering, multimedia decoders, multimedia encoders,
nested-passthrough [1]

The problem that I see with the current WIP header is that we assume for
the user that they want to use EGL. When they get a MirRenderSurface, they
can immediately plug that resource into the EGL driver. We then try to
support the other technologies by transmuting transmute to other
technologies from there. [2]

If the answer to "What is a MirRenderSurface" is "A MirRenderSurface is the
common interface that all technologies can use to upload pixel content to
the mirserver" (and stop me here if this is not the right answer)

Then I think that we've already built this in MirPresentationChain.

[1] nested-passthrough is really support for building a generic display
server on top of the mirclient api.
[2] We build MirBufferStream from MirRenderSurface here:
http://bazaar.launchpad.net/~cemil-azizoglu/mir/mir-render-surface-v3/view/head:/src/include/client/mir_toolkit/mir_render_surface.h#L60
and I'm going out on a limb to assume that there's a similar function for
MirPresentationChain in the works.

On Thu, Sep 8, 2016 at 2:18 PM, Cemil Azizoglu <[email-masked]

>>>> EGLNativeWindowType has to be pointed out in comment-headers, or gleaned
>>>> from reading example code.
>>>>
>>>
> can easily be fixed when we add it in https://www.khronos.org/
> registry/egl/api/EGL/eglplatform.h
>  as a platform.​
>
>
>

---

From: Kevin DuBois <[email-masked]>
Date: Fri, 9 Sep 2016 09:19:22 -0400

On Fri, Sep 9, 2016 at 7:59 AM, Kevin DuBois <[email-masked]>
wrote:

> If the answer to "What is a MirRenderSurface" is "A MirRenderSurface is
> the common interface that all technologies can use to upload pixel content
> to the mirserver" (and stop me here if this is not the right answer)
>

Synced up with Chris on IRC on this topic, and this seems to be the right
answer.

Internally, a MirRenderSurface is really just a mf::BufferStreamId. By
defining a EGL_KHR_platform_mir and adding appropriate hooks on the
driver-side for the various platforms, then we can pass the
mf::BufferStreamId to the drivers, and they can do what they want from
there.

The various non-egl technologies (software, encoders/decoders,
nested-passthrough) will take their mf::BufferStreamId and use
MirPresentationChain semantics on it.

> Then I think that we've already built this in MirPresentationChain.

And I'm wrong here because of eglstreams, which can't really use this idea.

> http://bazaar.launchpad.net/~cemil-azizoglu/mir/mir-render-
> surface-v3/view/head:/src/include/client/mir_toolkit/
> mir_render_surface.h#L60
> On Thu, Sep 8, 2016 at 2:18 PM, Cemil Azizoglu <
>> can easily be fixed when we add it in https://www.khronos.org/regist
>> ry/egl/api/EGL/eglplatform.h
>>  as a platform.​
>>
>>
>>