Subject: Multi-monitor thoughts

From: Alexandros Frantzis <[email-masked]>
Date: Thu, 27 Jun 2013 19:10:52 +0300

Hi all,

there are two related but slightly different aspects of multi-monitor
handling.  The first is reacting to HW configuration changes (e.g. user
plugs in/out a monitor). The second is reacting to configuration change
requests from clients.  It's a complex situation, and I am trying to
find some model that will elegantly and simply accommodate both of these
aspects. Here are my thoughts so far...

The display server works with a virtual coordinate space, which is used
for both display and input. Outputs are placed in rectangles in this
coordinate space. A display configuration is a particular placement of
the outputs in the coordinate space (of course, not placing an output is
a valid placement) with valid sizes for each output (corresponding to a
supported mode).

The display server will have a base display configuration. Each session
(aka application, Mir connection) will optionally have a custom display
configuration associated with it. Having such a configuration means that
the session has special modesetting needs, and that this configuration
will be applied when the session is focused (whatever this means in each
case), and reverted (i.e. the base configuration will be reapplied) when
we focus a normal session.

Here is how this would work in some interesting scenarios:

Scenario 1: Two XMir sessions
-----------------------------

We are in the greeter, with the base configuration applied.

XMir session 1 starts up, gets the base display configuration, sends
back a changed configuration and creates/changes its surfaces to match
the configuration. Mir applies the new config since XMir 1 has the
focus.

We switch away from XMir 1 to the greeter, therefore reverting to the
base configuration.

XMir session 2 starts up, gets the base display configuration, sends
back a changed configuration and changes its surfaces to match the
configuration. Mir applies the new config since XMir 2 has the focus.

Now, as we switch between XMir 1 and XMir 2 their respective custom
configurations are applied.

Scenario 2: Two XMir sessions, new monitor plugged/removed
----------------------------------------------------------

XMir 1 is active, a new monitor is plugged. Mir detects the change and
changes its base configuration (without applying it, since a session
with a custom config has the focus). It then notifies all sessions (or
perhaps only those who already have a special config).

XMir 1 gets the new (base) configuration, decides what it needs, sends a
new config back and changes its surfaces to match. Since XMir 1 is the
focused session, Mir applies the new configuration.

XMir 2 gets the new (base) configuration, decides what it needs, sends
new config back, changes its surfaces to match. The XMir 2 configuration
is not applied since XMir 2 is not focused.

Scenario 3: XMir requests new config (e.g. an XRandR request)
-------------------------------------------------------------

XMir sends a new config to Mir, and changes its surfaces to match. Mir
gets and applies the new configuration (since presumably XMir is
focused).

Scenario 4: Mir session/app messes up its custom display configuration
----------------------------------------------------------------------

No problem. When the session/app loses focus (or gets killed) the base
configuration will be applied.

Scenario 5: Native Mir app wants to become fullscreen in one of the
connected screens
-------------------------------------------------------------------

The app gets the base display configuration, creates or changes its
surfaces to become fullscreen.

Scenario 6: Native Mir app wants to become fullscreen in one of the
connected screens *and* wants to have a different resolution there.
-------------------------------------------------------------------

The app gets the base display configuration, changes what it needs (e.g.
changes modes and position in virtual space in one of the connected
screens) and changes surfaces to match. This is the same as the XMir
cases discussed above. Will we allow all apps to change the modes?

Note that for these scenarios to work, we need to allow clients to make
a surface fullscreen for a specific output (they can get output ids from
the configuration).

Thanks,
Alexandros

---

From: Daniel van Vugt <[email-masked]>
Date: Fri, 28 Jun 2013 10:02:09 +0800

I think we went through this in February. Yes a virtual coordinate space 
is a good idea. And it will accommodate just about anything you can put 
on top of it.

I'm a little concerned about "Each session (aka application, Mir 
connection) will optionally have a custom display configuration 
associated with it". This sounds like Mir-specific metadata that may not 
translate into anything existing toolkits use.

I think it could be dangerous to take an approach that requires all apps 
to be Mir-aware. I'm not sure, but if an app can change output 
resolutions cleanly through the toolkit or another API right now then it 
is usually done in such a way that you are changing the monitor 
resolution and not particularly associating that with your window. Your 
window just happens to be fullscreen on that monitor.

I think it's much more familiar for apps to just ask for monitor N to be 
resolution X*Y, than to define any "config" metadata. Also much more 
flexible and simpler to implement. What you're suggesting sounds like it 
requires apps to be modified, and not just the toolkits they use.

---

From: Thomi Richards <[email-masked]>
Date: Fri, 28 Jun 2013 16:09:44 +1200

Hi,

Thought I'd add my $0.02:

> requires apps to be modified, and not just the toolkits they use.

Implementation aside, one area I'd love to see improved is better
restoration of sane defaults after something goes wrong. For example, if I
start playing a game, it typically alters the resolution (and other
options). If the game crashes, I'm almost always left with a desktop in the
wrong resolution.

Alexandros' plan sounded to me like it would handle this display
restoration transparently, which would make me very happy indeed. Then
again, I'm sure there are many ways to implement that feature, so I'm not
suggesting that your concerns are invalid in any way.

Cheers,

-- 
Thomi Richards
[email-masked]

---

From: Daniel van Vugt <[email-masked]>
Date: Fri, 28 Jun 2013 12:35:37 +0800

You can easily track which client/connection/process a resolution change 
has come from. If the client disconnects/exits/crashes then just restore 
the old resolution. Automatically and transparently.

Then only apps which make permanent resolution changes, like in System 
Settings, will need to be modified to pass an extra flag that their 
changes are permanent.

Regular apps like games would not require modification.

On 28/06/13 12:09, Thomi Richards wrote:
> On Fri, Jun 28, 2013 at 2:02 PM, Daniel van Vugt
> <[email-masked] <mailto:[email-masked]>>
> wrote:
>
>     I think it's much more familiar for apps to just ask for monitor N
>     to be resolution X*Y, than to define any "config" metadata. Also
>     much more flexible and simpler to implement. What you're suggesting
>     sounds like it requires apps to be modified, and not just the
>     toolkits they use.
>
>
> restoration of sane defaults after something goes wrong. For example, if
> I start playing a game, it typically alters the resolution (and other
> options). If the game crashes, I'm almost always left with a desktop in
> the wrong resolution.
>
> again, I'm sure there are many ways to implement that feature, so I'm
> not suggesting that your concerns are invalid in any way.
>
> Cheers,
>
>
> --
> Thomi Richards
> [email-masked] <mailto:[email-masked]>
>
>

---

From: Alexandros Frantzis <[email-masked]>
Date: Fri, 28 Jun 2013 10:57:40 +0300

On Fri, Jun 28, 2013 at 10:02:09AM +0800, Daniel van Vugt wrote:
> I think we went through this in February. Yes a virtual coordinate
> space is a good idea. And it will accommodate just about anything
> you can put on top of it.
> 
> associated with it". This sounds like Mir-specific metadata that may
> not translate into anything existing toolkits use.
>
> I think it could be dangerous to take an approach that requires all
> apps to be Mir-aware. I'm not sure, but if an app can change output
> resolutions cleanly through the toolkit or another API right now
> then it is usually done in such a way that you are changing the
> monitor resolution and not particularly associating that with your
> window. Your window just happens to be fullscreen on that monitor.
>

The driving use case for this is having multiple xmir sessions under the
system compositor, each with its own display configuration, but can also
be applied to applications that need to set the mode under the session
compositor.

I think that the per-session configuration makes more sense as a model
for the future, and the only exception to this is the system settings
app which should set the global configuration.

Most toolkits don't have mode-setting APIs, you need to use windowing
system facilities directly, so apps will need to be rewritten for Mir
anyway if the require mode-setting. If a toolkit has such an API it can
decide how it will behave: set a session specific configuration or the
global/base configuration; mir can allow both.

Note that this does not affect how X apps under XMir work.

Thanks,
Alexandros

---

From: Robert Ancell <[email-masked]>
Date: Mon, 1 Jul 2013 16:50:48 +1200

Nice work Alexandros!

This solution captures the requirements that I know of.

--Robert

On Fri, Jun 28, 2013 at 4:10 AM, Alexandros Frantzis <
[email-masked]> wrote:

> Mir-devel mailing list
> [email-masked]
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/mir-devel
>