Subject: Improving next_buffer() rpc

From: Kevin DuBois <[email-masked]>
Date: Tue, 8 Jul 2014 09:10:03 -0400

Hello mir team,

In order to get the next buffer for the client, we currently have:

rpc next_buffer(SurfaceId) returns (Buffer);

which is problematic for me in working on [1] because this implicitly
releases the buffer from the client side, whereas in working on that
performance improvement, I have to send a fd back to the server. So I was
thinking of adding an rpc method more like:

rpc exchange_buffer(Buffer) returns (Buffer);

This would be sufficient to pass the fd fence back, and the buffer id in
the Buffer protocol message would be sufficient for the server to figure
out which surface has sent back its buffer. (given the global buffer id's
we're using)

This does not address the problem noted in:
https://bugs.launchpad.net/mir/+bug/1253868
but I think that might be better addressed by having an exchange type rpc
call (explicit or implicit) and negotiating/increasing how many buffers the
client owns somehow else.

This seems like something that could have diverse opinions, so I'm hoping
to get some input on the protocol change here first.

Thanks!
Kevin

[1]
https://blueprints.launchpad.net/ubuntu/+spec/client-1410-mir-performance
item:
"[kdub] fencing improvements for clients add the ipc plumbing"

---

From: Daniel van Vugt <[email-masked]>
Date: Wed, 09 Jul 2014 10:00:22 +0800

Sounds better to just pass buffers around although I'm not keen on any 
change that doesn't make progress on the performance bottleneck LP: 
#1253868. The bottleneck is the swapping/exchanging approach which 
limits the client to holding only one buffer, so I don't think it's a 
good idea for new designs to still have that problem.

In order to improve parallelism per LP: #1253868 you'd really have to 
receive new buffers as soon as they're free, which means getting them as 
MirEvents. Then you only need an RPC function to release them back to 
the server:

    rpc release_buffer(Buffer) returns (Void);

Keep in mind the inter-process communication is the bottleneck here. If 
you allow a context switch between the server and client then that's 
half to one millisecond (see mirping) per RPC round trip. More than 
double that for nested servers and you see the protocol delay could be a 
significant factor. So I think any protocol enhancement should have 
parallelism designed in.

I also think we need to be careful about not landing any protocol 
changes to RTM candidate series' 0.4-0.5, so the foundation for RTM is 
maximally mature (albeit not yet optimal).

- Daniel

On 08/07/14 21:10, Kevin DuBois wrote:
> performance improvement, I have to send a fd back to the server. So I
> was thinking of adding an rpc method more like:
>
> out which surface has sent back its buffer. (given the global buffer
> id's we're using)
>
> but I think that might be better addressed by having an exchange type
> rpc call (explicit or implicit) and negotiating/increasing how many
> buffers the client owns somehow else.
>
> This seems like something that could have diverse opinions, so I'm
> hoping to get some input on the protocol change here first.
>
> Thanks!
> Kevin
>
> [1]
> https://blueprints.launchpad.net/ubuntu/+spec/client-1410-mir-performance item:

---

From: Daniel van Vugt <[email-masked]>
Date: Wed, 09 Jul 2014 15:41:57 +0800

Note that we're working on making double-buffering the default again and 
triple the exception. In that case fixing LP: #1253868 may seem 
pointless, but it is surprisingly still relevant. Because a fully 
parallelized design would significantly speed up double buffering too... 
client swap buffers would no longer have to wait for a round-trip before 
returning and would instead be almost instant.

---

From: Christopher James Halse Rogers <[email-masked]>
Date: Wed, 09 Jul 2014 17:54:48 +1000

On Wed, Jul 9, 2014 at 12:00 PM, Daniel van Vugt 
<[email-masked]> wrote:
> Sounds better to just pass buffers around although I'm not keen on 
> any change that doesn't make progress on the performance bottleneck 
> LP: #1253868. The bottleneck is the swapping/exchanging approach 
> which limits the client to holding only one buffer, so I don't think 
> it's a good idea for new designs to still have that problem.

I continue to disagree that this implies that clients cannot ever be 
more than double buffered. What it means is that there's the RPC delay 
before they get the third frame.

It is something we could do to increase performance, but it's nowhere 
near as bad as ‘triple buffering doesn't work’.

> receive new buffers as soon as they're free, which means getting them 
> as MirEvents. Then you only need an RPC function to release them back 
> to the server:
> 

I'm not averse to this approach, although I'd call this submit_buffer.

While we're at it, could we kindly let the platform decide whether the 
client can allocate buffers or not? :)

> 
> Keep in mind the inter-process communication is the bottleneck here. 
> If you allow a context switch between the server and client then 
> that's half to one millisecond (see mirping) per RPC round trip. More 
> than double that for nested servers and you see the protocol delay 
> could be a significant factor. So I think any protocol enhancement 
> should have parallelism designed in.
> 
> changes to RTM candidate series' 0.4-0.5, so the foundation for RTM 
> is maximally mature (albeit not yet optimal).

Ding!

> --
> Mir-devel mailing list
> [email-masked]
> Modify settings or unsubscribe at: 
> https://lists.ubuntu.com/mailman/listinfo/mir-devel

---

From: Daniel van Vugt <[email-masked]>
Date: Wed, 09 Jul 2014 16:08:09 +0800

Oops. I keep forgetting that the new BufferQueue disallows the 
compositor to own less than one buffer, so there would no longer be any 
benefit to double buffered clients from a more concurrent protocol :(

Maybe Kevin's suggestion is just fine then. So long as the server is 
able to figure out the surface(Id) from the Buffer struct.

---

From: Daniel van Vugt <[email-masked]>
Date: Wed, 09 Jul 2014 17:35:51 +0800

Forgive me for rambling on but I just had an important realisation...

Our current desire to get back to double buffering is only because the 
Mir protocol is synchronously waiting for a round trip every swap, and 
somehow I thought that the buffer queue length affected time spent in 
the ready_to_composite state. Now I'm not so sure that's true.

If we changed the protocol to implement parallelism, then in theory, 
keeping triple buffering with a fancy zero-latency swap buffers should 
perform better than the current protocol that has to wait for a round trip.

I cannot remember why I thought the length of the buffer queue affected 
the time from client-rendering to server-compositing. Perhaps we really 
do need to keep triple-buffering always-on so that the performance gain 
of a zero-latency client swap-buffers can be achieved...

In summary, I'm back to thinking any protocol change from next_buffer() 
needs to support parallelism and not be so synchronous.

- Daniel

---

From: Kevin Gunn <[email-masked]>
Date: Wed, 9 Jul 2014 10:39:29 -0500

First
Not sure we're still on topic necessarily wrt changing from id's to fd's
do we need to conflate that with the double/triple buffering topic ?
let's answer this first...

Second
while we're at it :) triple buffering isn't always a win. In the case of
small, frequent renders (as an example "8x8 pixel square follow my finger")
you'll have potentially 2 extra buffers that need their 16ms of fame on the
screen in the queue, 1 at session server, 1 at system server. Which can
look a little laggy. I'm willing to say in the same breath though, that
this may be lunatic fringe. The win for the triple buffering case is likely
more common, which is spikey render times (14+ms) amongst more normal
render times (9-12ms)
+1 on giving empty buffers back to the clients to allow them to have a
"queue" of empty buffers at their disposal (i'm not sure if RAOF is correct
or duflu in that its "synchronously waiting for a round trip every swap"...can
we already have an empty buffer queue on the client side ?)

On Wed, Jul 9, 2014 at 4:35 AM, Daniel van Vugt <
[email-masked]> wrote:

> Our current desire to get back to double buffering is only because the Mir
> protocol is synchronously waiting for a round trip every swap, and somehow
> I thought that the buffer queue length affected time spent in the
> ready_to_composite state. Now I'm not so sure that's true.
>
> the time from client-rendering to server-compositing. Perhaps we really do
> need to keep triple-buffering always-on so that the performance gain of a
> zero-latency client swap-buffers can be achieved...
>
> Modify settings or unsubscribe at: https://lists.ubuntu.com/
> mailman/listinfo/mir-devel
>

---

From: Kevin DuBois <[email-masked]>
Date: Wed, 9 Jul 2014 12:04:57 -0400

Attempting to steer the convo more towards the near term (kgunn's "First"),
I'm just trying to make sure that the protocol change is somewhat
forward-looking without wading into changing the buffer
distribution/swapping system too much.

Trying to distil the potential directions suggested a bit more:

#1 what we currently have:
rpc next_buffer(SurfaceId) returns (Buffer)

#2 what I've proposed:
rpc exchange_buffer(Buffer) returns (Buffer)

#3 "push buffers", the server pushes out buffers to the clients

#4 multiple buffers owned by the client/client allocated buffers

now,
#3 and #4 are interesting and potentially beneficial, but outside the scope
of what I want to accomplish [1] with the optimization I'm aiming for. Its
good to future-proof the protocol for when/if we decide to experiment.

#3 strikes me as something I'd want to avoid (especially in the near term),
because when you're pushing buffers to the client, the server is 'driving'
the activity of the client processes, which seems strange. Now, I'd be fine
with the client /making a request/ and then getting the buffer back
asynchronously, but couldn't think of an advantage to this.

I think #4 could be accommodated by having idea #5:
#5
rpc next_buffer(SurfaceId) returns (Buffer);
rpc submit_buffer(Buffer) returns (Void);

or supplementing idea #2 to make idea #6 (in the future)
rpc exchange_buffer(Buffer) returns (Buffer)
rpc request_client_buffer_count(int) returns (Void)

Effectively, #5 removes the implicit release of the client buffer, making
the submission of the filled buffer explicit. This seems a bit less nice to
me than #2/#6 from an RAII mindset, so I think I still prefer #2 with a
future expansion to #6 for multiple client buffers.

So I guess #2/#6 or  #5 are fine for my immediate purposes, and both seem
acceptible at the current time, and have some forward-thinking in them.
Interested if others share my slight preference for #2/#6

[1]
My aim is to let the android platform's clients delay waiting for the gpu
to finish with the buffer. It lets the client pass the fence and the
uncompleted buffer back to the server without waiting, and requires that
the fences are waited upon in the compositon pass. About a year or so ago,
doing something similar with internal clients in the first landing of
unity, we saw some noticeable performance improvements that makes us think
this would be useful in the USC/unity8 setup we have now.

On Wed, Jul 9, 2014 at 11:39 AM, Kevin Gunn <[email-masked]>
wrote:

>> somehow I thought that the buffer queue length affected time spent in the
>>>>> as

---

From: Kevin DuBois <[email-masked]>
Date: Wed, 9 Jul 2014 12:29:38 -0400

Alberto pointed out a gap in my suggestion #2/#6, which is that the client
wouldn't have a way of getting ownership of the additional buffers. So
maybe that idea (#2/#6) should become:

rpc exchange_buffer(Buffer) returns (Buffer)
rpc allocate_additional_buffer(Void) returns (Buffer)
rpc remove_additional_buffer(Buffer) returns (Void)

This still would have the submission happen on the exchange call, but make
additional buffer allocations more explicit than the #5 idea.

On Wed, Jul 9, 2014 at 12:04 PM, Kevin DuBois <[email-masked]>
wrote:

> Attempting to steer the convo more towards the near term (kgunn's
> "First"), I'm just trying to make sure that the protocol change is somewhat
> #3 and #4 are interesting and potentially beneficial, but outside the
> scope of what I want to accomplish [1] with the optimization I'm aiming
> for. Its good to future-proof the protocol for when/if we decide to
> experiment.
>
> #3 strikes me as something I'd want to avoid (especially in the near
> term), because when you're pushing buffers to the client, the server is
> 'driving' the activity of the client processes, which seems strange. Now,
> I'd be fine with the client /making a request/ and then getting the buffer
> back asynchronously, but couldn't think of an advantage to this.
>
>>>>>> If

---

From: Alberto Aguirre <[email-masked]>
Date: Wed, 9 Jul 2014 12:53:41 -0500

I think RAOF point is accurate, with the current existing approach (triple
buffered by default), BufferQueue has 2 buffers for clients, 1 for
compositor. rpc next_buffer(SurfaceId) returns (Buffer) will return
immediately as BufferQueue will have a free buffer to give out.
However Daniel's point I believe is the client should also be able to
pipeline the rpc wait with their rendering which I agree it should be a
goal.

I think I have a slight practical preference for #2/#6 as we are mostly
dealing with EGL glue code now however it doesn't let you pipeline the RPC
wait for next buffer but it's not any worse that what we currently have.

However, long-term I would favor a solution where the "free" signal could
just be done by an fd/fence that the client can check as it tries to
acquire a free buffer on its side (the client would initially get all
buffer references to maintain the pool) if none are free, it would just
epoll/wait on fence until free.
That way the server can signal free buffers asynchronously, which
potentially pipelines with client rendering and does not generate an rpc
from server to client.

--Alberto

On Wed, Jul 9, 2014 at 11:29 AM, Kevin DuBois <[email-masked]>
wrote:

>>>>>>> any
>>>>>>> is

---

From: Daniel van Vugt <[email-masked]>
Date: Thu, 10 Jul 2014 10:17:02 +0800

Triple buffering is relevant to the conversation because it appears to 
be the only way to make full concurrent use of buffers without a round 
trip on every swap. So if we want better performance that is directly 
relevant to the protocol change.

Presently a client swap_buffers is a delay (round trip) as well as a 
minimum of 2 context switches (probably more due to our generous use of 
threads) as it sends the old buffer and waits for a new one. We can get 
that down to a best case of no round-trip and thus zero context switches 
during swap_buffers. So long as the client already has the next "free" 
buffer at its disposal without having to wait at all. It's a goal worth 
aiming for.

I know triple buffering is traditionally laggy but in the context of 
Mir's design I'm now not sure it applies here. Because the server is 
always compositing new frames as soon as it has new frames from clients. 
Even the slightest pause then, and you can be certain that the "ready" 
sub-queue is emptied and thus it's only one frame lag from client to 
server, no matter how long the overall ring of buffers is. This would 
explain why I've struggled to see any visual difference in triple vs 
double buffering, but I will retest...

On 09/07/14 23:39, Kevin Gunn wrote:
> small, frequent renders (as an example "8x8 pixel square follow my
> finger") you'll have potentially 2 extra buffers that need their 16ms of
> fame on the screen in the queue, 1 at session server, 1 at system
> server. Which can look a little laggy. I'm willing to say in the same
> breath though, that this may be lunatic fringe. The win for the triple
> buffering case is likely more common, which is spikey render times
> (14+ms) amongst more normal render times (9-12ms)
> "queue" of empty buffers at their disposal (i'm not sure if RAOF is
> correct or duflu in that its "synchronously waiting for a round trip
> every swap"...can we already have an empty buffer queue on the client
> side ?)
>
>
> On Wed, Jul 9, 2014 at 4:35 AM, Daniel van Vugt
> <[email-masked] <mailto:[email-masked]>>
> wrote:
>
>     Our current desire to get back to double buffering is only because
>     the Mir protocol is synchronously waiting for a round trip every
>     swap, and somehow I thought that the buffer queue length affected
>     time spent in the ready_to_composite state. Now I'm not so sure
>     that's true.
>
>     keeping triple buffering with a fancy zero-latency swap buffers
>     should perform better than the current protocol that has to wait for
>     a round trip.
>
>     I cannot remember why I thought the length of the buffer queue
>     affected the time from client-rendering to server-compositing.
>     Perhaps we really do need to keep triple-buffering always-on so that
>     the performance gain of a zero-latency client swap-buffers can be
>     achieved...
>
>     In summary, I'm back to thinking any protocol change from
>     next_buffer() needs to support parallelism and not be so synchronous.
>
>     - Daniel
>
>
>
>                     This would be sufficient to pass the fd fence back,
>                     and the buffer
>                     id in
>                     the Buffer protocol message would be sufficient for
>                     the server to
>                     figure
>                     https://bugs.launchpad.net/__mir/+bug/1253868
>                     <https://bugs.launchpad.net/mir/+bug/1253868>
>                     https://blueprints.launchpad.__net/ubuntu/+spec/client-1410-__mir-performance
>                     <https://blueprints.launchpad.net/ubuntu/+spec/client-1410-mir-performance>
>
>
>                     item:
>     [email-masked] <mailto:[email-masked]>
>     https://lists.ubuntu.com/__mailman/listinfo/mir-devel
>     <https://lists.ubuntu.com/mailman/listinfo/mir-devel>
>
>

---

From: Daniel van Vugt <[email-masked]>
Date: Thu, 10 Jul 2014 10:23:00 +0800

Critically I think we need to avoid any RPC call which is an exchange or 
swap in a single call. Because that's the problem, and it prevents the 
client from ever owning more than one buffer. So performance bug 1253868 
would remain unsolved:
     https://bugs.launchpad.net/mir/+bug/1253868

Buffers need to arrive at the client asynchronously, probably inside a 
MirEvent. And then you only need a single message to send them back:
     rpc release_buffer(Buffer) returns (Void);

- Daniel

On 10/07/14 00:29, Kevin DuBois wrote:
> Alberto pointed out a gap in my suggestion #2/#6, which is that the
> client wouldn't have a way of getting ownership of the additional
> buffers. So maybe that idea (#2/#6) should become:
>
> This still would have the submission happen on the exchange call, but
> make additional buffer allocations more explicit than the #5 idea.
>
>
> On Wed, Jul 9, 2014 at 12:04 PM, Kevin DuBois
> <[email-masked] <mailto:[email-masked]>> wrote:
>
>     "First"), I'm just trying to make sure that the protocol change is
>     somewhat forward-looking without wading into changing the buffer
>     #3 and #4 are interesting and potentially beneficial, but outside
>     the scope of what I want to accomplish [1] with the optimization I'm
>     aiming for. Its good to future-proof the protocol for when/if we
>     decide to experiment.
>
>     term), because when you're pushing buffers to the client, the server
>     is 'driving' the activity of the client processes, which seems
>     strange. Now, I'd be fine with the client /making a request/ and
>     then getting the buffer back asynchronously, but couldn't think of
>     an advantage to this.
>
>     Effectively, #5 removes the implicit release of the client buffer,
>     making the submission of the filled buffer explicit. This seems a
>     bit less nice to me than #2/#6 from an RAII mindset, so I think I
>     still prefer #2 with a future expansion to #6 for multiple client
>     buffers.
>
>     So I guess #2/#6 or  #5 are fine for my immediate purposes, and both
>     seem acceptible at the current time, and have some forward-thinking
>     in them. Interested if others share my slight preference for #2/#6
>
>     [1]
>     My aim is to let the android platform's clients delay waiting for
>     the gpu to finish with the buffer. It lets the client pass the fence
>     and the uncompleted buffer back to the server without waiting, and
>     requires that the fences are waited upon in the compositon pass.
>     About a year or so ago,  doing something similar with internal
>     clients in the first landing of unity, we saw some noticeable
>     performance improvements that makes us think this would be useful in
>     the USC/unity8 setup we have now.
>
>
>     On Wed, Jul 9, 2014 at 11:39 AM, Kevin Gunn
>     <[email-masked] <mailto:[email-masked]>> wrote:
>
>         First
>         while we're at it :) triple buffering isn't always a win. In the
>         case of small, frequent renders (as an example "8x8 pixel square
>         follow my finger") you'll have potentially 2 extra buffers that
>         need their 16ms of fame on the screen in the queue, 1 at session
>         server, 1 at system server. Which can look a little laggy. I'm
>         willing to say in the same breath though, that this may be
>         lunatic fringe. The win for the triple buffering case is likely
>         more common, which is spikey render times (14+ms) amongst more
>         normal render times (9-12ms)
>         +1 on giving empty buffers back to the clients to allow them to
>         have a "queue" of empty buffers at their disposal (i'm not sure
>         if RAOF is correct or duflu in that its "synchronously waiting
>         for a round trip every swap"...can we already have an empty
>         buffer queue on the client side ?)
>
>
>         <[email-masked]
>         <mailto:[email-masked]>> wrote:
>
>             Our current desire to get back to double buffering is only
>             because the Mir protocol is synchronously waiting for a
>             round trip every swap, and somehow I thought that the buffer
>             queue length affected time spent in the ready_to_composite
>             state. Now I'm not so sure that's true.
>
>             If we changed the protocol to implement parallelism, then in
>             theory, keeping triple buffering with a fancy zero-latency
>             swap buffers should perform better than the current protocol
>             that has to wait for a round trip.
>
>             I cannot remember why I thought the length of the buffer
>             queue affected the time from client-rendering to
>             server-compositing. Perhaps we really do need to keep
>             triple-buffering always-on so that the performance gain of a
>                             This would be sufficient to pass the fd
>                             fence back, and the buffer
>                             id in
>                             the Buffer protocol message would be
>                             sufficient for the server to
>                             figure
>                             https://bugs.launchpad.net/__mir/+bug/1253868 <https://bugs.launchpad.net/mir/+bug/1253868>

---

From: Christopher James Halse Rogers <[email-masked]>
Date: Thu, 10 Jul 2014 14:46:20 +1000

On Thu, Jul 10, 2014 at 3:53 AM, Alberto Aguirre 
<[email-masked]> wrote:
> I think RAOF point is accurate, with the current existing approach 
> (triple buffered by default), BufferQueue has 2 buffers for clients, 
> 1 for compositor. rpc next_buffer(SurfaceId) returns (Buffer) will 
> return immediately as BufferQueue will have a free buffer to give out.
> pipeline the rpc wait with their rendering which I agree it should be 
> a goal.
> 
> I think I have a slight practical preference for #2/#6 as we are 
> mostly dealing with EGL glue code now however it doesn't let you 
> pipeline the RPC wait for next buffer but it's not any worse that 
> what we currently have.
> 
> However, long-term I would favor a solution where the "free" signal 
> could just be done by an fd/fence that the client can check as it 
> tries to acquire a free buffer on its side (the client would 
> initially get all buffer references to maintain the pool) if none are 
> free, it would just epoll/wait on fence until free.
> potentially pipelines with client rendering and does not generate an 
> rpc from server to client.

Oooh, yeah. That's very nice.

On mesa, at least, we should be able to get these fences for free, 
thanks to Maarten's dma-buf fence work. It seems that Android has a 
similar mechanism, so this looks gorgeous.

If this is what we want to end up at then any RPC call which exchanges 
a Buffer isn't going to be a part of The Glorious Future™.

I don't think The Glorious Future™ is going to arrive particularly 
soon, so I'm not averse to Kevin implementing a transient buffer 
exchange RPC call.

> --Alberto
> 
> 
> 
> 
> On Wed, Jul 9, 2014 at 11:29 AM, Kevin DuBois 
> <[email-masked]> wrote:
>> This still would have the submission happen on the exchange call, 
>> but make additional buffer allocations more explicit than the #5 
>> idea.
>> 
>> 
>> <[email-masked]> wrote:
>>> the scope of what I want to accomplish [1] with the optimization 
>>> I'm aiming for. Its good to future-proof the protocol for when/if 
>>> we decide to experiment.
>>> 
>>> #3 strikes me as something I'd want to avoid (especially in the 
>>> near term), because when you're pushing buffers to the client, the 
>>> server is 'driving' the activity of the client processes, which 
>>> seems strange. Now, I'd be fine with the client /making a request/ 
>>> and then getting the buffer back asynchronously, but couldn't think 
>>> of an advantage to this.
>>> 
>>> So I guess #2/#6 or  #5 are fine for my immediate purposes, and 
>>> both seem acceptible at the current time, and have some 
>>> forward-thinking in them. Interested if others share my slight 
>>> preference for #2/#6
>>> 
>>> [1]
>>> the gpu to finish with the buffer. It lets the client pass the 
>>> fence and the uncompleted buffer back to the server without 
>>> waiting, and requires that the fences are waited upon in the 
>>> compositon pass. About a year or so ago,  doing something similar 
>>> with internal clients in the first landing of unity, we saw some 
>>> noticeable performance improvements that makes us think this would 
>>> be useful in the USC/unity8 setup we have now.
>>> 
>>> 
>>> <[email-masked]> wrote:
>>>> First
>>>> willing to say in the same breath though, that this may be lunatic 
>>>> fringe. The win for the triple buffering case is likely more 
>>>> common, which is spikey render times (14+ms) amongst more normal 
>>>> render times (9-12ms)
>>>> have a "queue" of empty buffers at their disposal (i'm not sure if 
>>>> RAOF is correct or duflu in that its "synchronously waiting for a 
>>>> round trip every swap"...can we already have an empty buffer queue 
>>>> on the client side ?)
>>>> 
>>>> 
>>>>> because the Mir protocol is synchronously waiting for a round 
>>>>> trip every swap, and somehow I thought that the buffer queue 
>>>>> length affected time spent in the ready_to_composite state. Now 
>>>>> I'm not so sure that's true.
>>>>> 
>>>>> theory, keeping triple buffering with a fancy zero-latency swap 
>>>>> buffers should perform better than the current protocol that has 
>>>>> to wait for a round trip.
>>>>> 
>>>>> Perhaps we really do need to keep triple-buffering always-on so 
>>>>> that the performance gain of a zero-latency client swap-buffers 
>>>>> can be achieved...
>>>>> 
>>>>>>>>> This would be sufficient to pass the fd fence back, and the 
>>>>>>>>> buffer
>>>>>>>>> id in
>>>>>>>>> the Buffer protocol message would be sufficient for the 
>>>>>>>>> server to
>>>>>>>>> figure

---

From: Gerry Boland <[email-masked]>
Date: Thu, 10 Jul 2014 11:01:22 +0100

> we already have an empty buffer queue on the client side ?)

I also want to remind everyone that our default shipping configuration
is a root mir server with a nested mir server as a client, and that
nested mir server manages most client apps the user will be interacting
with.

Nesting will increase input latency, as now there's not just 3 buffers
in play, but more (5 yeah?).

I had thought that the double-buffering idea was to try reduce the
number of buffers being used in the nested case. Sounds like Daniel
isn't confident that'll work now, which is a pity.
Thanks
-G

---

From: Kevin DuBois <[email-masked]>
Date: Thu, 10 Jul 2014 10:23:28 -0400

Okay, so resynthesizing the concerns to try to come up with a plan...

It seems the practical thing to do is first implement:
rpc exchange_buffer(Buffer) returns (Buffer)
which is by-in-large just an evolution of what we have now. The difference
that lets me proceed is that the buffer release is now more explicit. I
won't be changing the swapping algorithm in BufferQueue with this.

We can supplement this when someone wants to increase or decrease the
client buffer count with something like:
rpc request_additional_buffer(Void) returns (Buffer)
rpc release_client_buffer(Buffer) returns (Void)
so that the client can have spare buffers modelled as owned-by the client.
Example [1]

And the longer term plan to address bug 1253868
<https://bugs.launchpad.net/mir/+bug/1253868> will involve deeper changes
to the ipc and the BufferQueue, perhaps with asynchronous buffers or with
fd signalling mechanisms. These do have advantages (I esp like the fd
idea), but also need expansive restructuring.

I think this is a decent plan that is reasonably forward-looking, while
letting me get to the delayed-wait/client-fence-passing goal that gave us
some good gains when u8 was an internal client.

[1]
client calls "rpc create_surface()" //has 1 buffer to play with, A
client calls "rpc request_additional_buffer() //has 2 buffers to play with,
A+B
client calls "rpc exchange_buffer()" //has 2 buffers to play with, and
could have submitted A or B back to the server, getting C back.

On Thu, Jul 10, 2014 at 6:01 AM, Gerry Boland <[email-masked]>
wrote:

> finger")
> correct

---

From: Daniel van Vugt <[email-masked]>
Date: Fri, 11 Jul 2014 16:55:06 +0800

Good news. I've done some visual testing and there definitely is visibly 
reduced lag from double-buffering.

My previous argument was both right and wrong:
"Even the slightest pause then, and you can be certain that the "ready" 
sub-queue is emptied and thus it's only one frame lag from client to 
server, no matter how long the overall ring of buffers is."

This statement remains true. However what I forgot was that during 
scrolling etc when latency matters most, there is no "slightest pause" 
for the compositor to catch up. Thus the client is allowed to pre-render 
too many frames and the age of a frame by the time it appears on screen 
is nbuffers-1.

So it looks like double-buffering is still a good idea. In fact, based 
on those findings it seems a more efficient protocol while increasing 
throughput, would only increase the problem of perceived lag as clients 
can render even further in advance of vsync.

But it's a delicate balance. On desktop for example one can hack around 
the parallel render/pageflipping logic and make it wait for vsync 
sooner. This reduces lag even further, however it increases the risk of 
the compositor missing the frame deadline (as seen in clone mode) 
unacceptably.