Subject: What we do and don't expose to client toolkits

From: Alan Griffiths <[email-masked]>
Date: Thu, 1 Sep 2016 12:02:38 +0100

When clients toolkits provide hints to place child surfaces using the
existing functions:

    mir_surface_spec_attach_to_foreign_parent();
    mir_connection_create_spec_for_tip();
    mir_connection_create_spec_for_menu(); or the proposed,
    mir_surface_spec_set_placement()

the toolkit wants to know where the child surface actually ends up in
order to render appropriately.

We currently have a policy not to provide any location information to
clients, so I want to be sure that I don't propose anything controversial.

In discussion with Chris he suggests that sending a
PlacementRelativeToParent{dx, dy} message is the right solution.

Doing this opens up an opportunity for clients to:

    1. probe the display boundaries using dummy placement requests. (The
    point is to provide the location before they render.)

    2. parent (and place) everything they do on a fullscreen surface
    (which they can conceal from the user). It does limit them to
    surface types that can be parented.

Thoughts and suggestions for the right way forward please.

---

From: Daniel van Vugt <[email-masked]>
Date: Fri, 2 Sep 2016 09:29:56 +0800

A relative {x,y} would make sense and be much simpler than the currently 
over-engineered "attach to a rectangle". In Xmir and toolkits we tend to 
see empty rectangles being used:
    MirRectangle placement = {dx, dy, 0, 0};
already. It should have been a simple relative {x,y} from the beginning...

---

From: Alan Griffiths <[email-masked]>
Date: Fri, 2 Sep 2016 14:40:17 +0100

> Thoughts and suggestions for the right way forward please.

A data point from Wayland
(https://cgit.freedesktop.org/wayland/wayland-protocols/tree/unstable/xdg-shell/xdg-shell-unstable-v6.xml#n1018):

    <event name="configure">
      <description summary="configure the popup surface">
    This event asks the popup surface to configure itself given the
    configuration. The configured state should not be applied immediately.
    See xdg_surface.configure for details.

    The x and y arguments represent the position the popup was placed at
    given the xdg_positioner rule, relative to the upper left corner of the
    window geometry of the parent surface.
      </description>
      <arg name="x" type="int"
       summary="x position relative to parent surface window geometry"/>
      <arg name="y" type="int"
       summary="y position relative to parent surface window geometry"/>
      <arg name="width" type="int" summary="window geometry width"/>
      <arg name="height" type="int" summary="window geometry height"/>
    </event>

So (in their current "unstable" anyway) they are returning the relative
placement. And running the risk of sneaky clients probing the display
boundaries.

I claim this as "prior art" and propose that we do the same by sending a
MirRectangle notification.

Any objections before the weekend please.

---

From: William Hua <[email-masked]>
Date: Fri, 2 Sep 2016 10:27:36 -0400

I asked Jonas Adahl how the Wayland side is dealing with this, and
roughly:

Wayland doesn't see any of this as a security concern and broadcasts
the geometry of the display to clients:

https://cgit.freedesktop.org/wayland/wayland/tree/protocol/wayland.xml#n
2364

So, the menu positioning feedback is no concern either, and they
currently have an unstable protocol that notifies the client of where
its popup surfaces were actually positioned, relative to the parent
surface:

https://cgit.freedesktop.org/wayland/wayland-protocols/tree/unstable/xdg
- -shell/xdg-shell-unstable-v6.xml#n1018
(same link referenced by Alan)

Jonas said there's some plans towards making this stable, but didn't
make any guarantees.

Wayland does prevent clients from knowing the absolute positions of
surfaces, but not for security reasons. More for being able to do
input transformations and non-traditional compositing. And if a client
goes out of its way to "probe" for its absolute position using this
feedback mechanism, that may end up working terribly since they're
making assumptions about undefined behaviour. For example, workspaces
larger than the display that are able to pan around.

Anyways, if we continue to specify display geometry as private
information, Mir needs to be smart enough to detect when a client is
abusing this new feature and disconnect them. Easiest thing I can
think of is a simple rate limit. We can also re-evaluate if it is
still worthwhile to prevent clients from knowing the geometry, but I
don't know the security implications there or the history behind this
decision.

On 2016-09-02 09:40 AM, Alan Griffiths wrote:
> (https://cgit.freedesktop.org/wayland/wayland-protocols/tree/unstable/
xdg-shell/xdg-shell-unstable-v6.xml#n1018):
>
>  <event name="configure"> <description summary="configure the popup
> surface"> This event asks the popup surface to configure itself
> given the configuration. The configured state should not be applied
> immediately. See xdg_surface.configure for details.
> 
> The x and y arguments represent the position the popup was placed
> at given the xdg_positioner rule, relative to the upper left corner
> of the window geometry of the parent surface. </description> <arg
> name="x" type="int" summary="x position relative to parent surface
> window geometry"/> <arg name="y" type="int" summary="y position
> relative to parent surface window geometry"/> <arg name="width"
> type="int" summary="window geometry width"/> <arg name="height"
> type="int" summary="window geometry height"/> </event>
> 
> So (in their current "unstable" anyway) they are returning the
> relative placement. And running the risk of sneaky clients probing
> the display boundaries.
> 
> I claim this as "prior art" and propose that we do the same by
> sending a MirRectangle notification.
>

---

From: Thomas Voß <[email-masked]>
Date: Fri, 2 Sep 2016 16:42:07 +0200

On Thu, Sep 1, 2016 at 1:02 PM, Alan Griffiths
<[email-masked]> wrote:
> the toolkit wants to know where the child surface actually ends up in order
> to render appropriately.
>
> 1. probe the display boundaries using dummy placement requests. (The point
> is to provide the location before they render.)
>
> 2. parent (and place) everything they do on a fullscreen surface (which they
> can conceal from the user). It does limit them to surface types that can be
> parented.
>

>From my pov, exposing relative place information is fine. Security
concerns do not apply as placement information
does not magically enable a client to eat away input. Moreover, even
if a client goes to great length to estimate the
display boundaries: This only works in the fullscreen case and I
cannot think of a scenario where this impacts overall
user experience. The app already is the one "owning" the display in fullscreen.

Cheers,

  Thomas

> --
> Mir-devel mailing list
> [email-masked]
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/mir-devel
>

---

From: Alan Griffiths <[email-masked]>
Date: Fri, 2 Sep 2016 15:44:18 +0100

On 02/09/16 15:42, Thomas Voß wrote:
> This only works in the fullscreen case and I

Why would you think it only work in the fullscreen case?

---

From: Christopher James Halse Rogers <[email-masked]>
Date: Sun, 04 Sep 2016 14:36:28 +1000

>> surface types that can be parented.

Note step 1 here is unnecessary, and clients can quite happily do this 
already - they can already position all their surfaces relative to a 
transparent fullscreen surface. The probing is unnecessary - the know 
where the boundary of their surface is, and that it's the boundary of 
the output.

---

From: Gerry Boland <[email-masked]>
Date: Sun, 4 Sep 2016 14:17:10 +0200

Hi,
replt inline

> PlacementRelativeToParent{dx, dy} message is the right solution.

That sounds ok to me. Client gave detailed information (rect+offsets) to
window manager about how to position the child surface relative to
parent, and WM replies with the final position (relative to parent).

Qt does want to know this information anyway.

>     surface types that can be parented.

I can't think of how to avoid these issues, as if policy has limits,
then those limits can be probed. And a fullscreen surface with alpha can
be easily used by a malicious client to give appearance of window
positioning, something we cannot prevent with Mir alone, but with shell
UI hints.

Thanks
-Gerry

---

From: Daniel van Vugt <[email-masked]>
Date: Mon, 5 Sep 2016 10:42:33 +0800

That's an important point worth highlighting again:

"Wayland does prevent clients from knowing the absolute positions of
surfaces, but not for security reasons. More for being able to do
input transformations and non-traditional compositing."

Indeed compositing is potentially free-form if you get creative like in 
Compiz. This means a surface doesn't have one position on screen - it 
might have multiple positions like "Always on visible workspace". And 
its pixels might not be the same size as those of the server either. 
Finally your screen might not even be a single two dimensional space; it 
might be multiple workspaces with different coordinate systems or even 
3D. So for many reasons it could create bugs in future if you were to 
let the client know its absolute position.

Nothing controversial here, just a reminder...

On 02/09/16 22:27, William Hua wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2
>
> iQIcBAEBCAAGBQJXyYxRAAoJENtfC4FFqZmIgQkP/24EVx69iWlyjhlWeScYexeF
> mVp8bhcORoLLxOn8ITKUGNqijrPYstDaaS8tr1QaMNZ/bZmqGi1pEUDnRTGitTn+
> pn26Y+csOqiV+NducloBYhxVFkQeOMNciV0eWSFvPR8OF0J/Kv0Jti+5MNb5JTSc
> y+L/zcHDcQFKb1Q9hmlinIZ97GSGiKP7vp8tzzdiehgFcTYncRYryAMqcIern6iD
> s27gQpZkRLGryMdRo1Bcl17zoH5m5E3X6dFxN0j4CxXTVK0enSY454ODVPYHIcdm
> Tm/Az5ufB4u1mP4yhWT4KN3FV9Obt1by7PiHfjChObPWoIhNObcA+2WfG0C4w+Kf
> sLLrO+gKRXqfz6XA8WW4clzxCMRdpIhbKNbModZiHjUSQ7e0ODmD/YRE3f7yy/LF
> NcU4mu6gAiobyqb1kbtI/BLUHPfFk2JCv61FUOnA3kzLcjf0OY8L/ySOSc+SXMxr
> L3Ew6MDkvQXpilDMHvuJjNPzu9nJQIIDAYJ3YLJQRGzPm9jjPVPVf0qnQG/ptUu1
> KbWE0WqAzEwcOtKuMNJO2+nZUHADV4OKzZjQJx8/hLoHN0e14wTk+aSBCsuo6y2v
> e2rwhIX8eTVfasmtDgb3XqlDbE9CTaqMwosEaPsCNq/ypI2EG2lcZXqv42+ornYc
> L0j0R3PUl6AJLODN4rMg
> =iIEA
> -----END PGP SIGNATURE-----
>

---

From: Alan Griffiths <[email-masked]>
Date: Mon, 5 Sep 2016 17:21:01 +0100

> Why would you think it only work in the fullscreen case?

there was an IRC conversation to sort this out between myself William
and Thomas.

I think the resulting consensus was:

1. Mir will constrain the placement anchor of the aux_rect to the parent
surface. I don't think we agreed exactly how (e.g. do we "clip" the
rect? What happens if there is *no* intersection?)

2. Mir will constrain the the offset placement anchor to the parent
surface. Again I don't think we agreed how. (Slide it horizontally
and/or vertically the minimum amount?)

3. Mir will provide a new notification of the placement rectangle.

4. Clients can then probe the display boundaries:

    E.g.

      MirRectangle aux { 0, 0, 0, 0 };

     auto const surface_spec = mir_connection_create_spec_for_tip(connection, a_big_width, ...);
     mir_surface_spec_set_placement(
        surface_spec, &aux, 
        mir_placement_gravity_northwest, // rect anchor
        mir_placement_gravity_northeast, // surface anchor
        mir_placement_hints_resize_x, 
        0,
        0);

    Will result in a "tip" surface whose width is the distance to the
    left hand side of the screen.

5. Clients using the result of such probing may get stupid results. We
don't care.

---

From: Christopher James Halse Rogers <[email-masked]>
Date: Tue, 06 Sep 2016 09:41:38 +1000

On Tue, Sep 6, 2016 at 2:21 AM, Alan Griffiths 
<[email-masked]> wrote:
> 1. Mir will constrain the placement anchor of the aux_rect to the 
> parent surface. I don't think we agreed exactly how (e.g. do we 
> "clip" the rect? What happens if there is *no* intersection?)
> 
This is easy. The client knows all the information required to ensure 
this, so if any part of the placement anchor is outside the parent 
anchor, we fail the request.

> and/or vertically the minimum amount?)
If any part of the offset placement anchor is outside the parent 
surface, we fail the request. ☺
>>  auto const surface_spec = 
>> mir_connection_create_spec_for_tip(connection, a_big_width, ...);
> 5. Clients using the result of such probing may get stupid results. 
> We don't care.

---

From: Alan Griffiths <[email-masked]>
Date: Tue, 6 Sep 2016 09:16:26 +0100

> anchor, we fail the request.

1.1 Things are asynchronous - so the client doesn't know in advance the
state the server will see.
1.2 I should have stated explicitly: we favoured making this a
post-condition, not a pre-condition.

> surface, we fail the request. ☺

2.1 Things are asynchronous - so the client doesn't know in advance the
state the server will see.
2.2 I should have stated explicitly: we favoured making this a
post-condition, not a pre-condition.

This also leads me to wonder how much use the server should make of
placement information when the parent surface is resized or moved. On
Unity7 tips and menus typically disappear in these cases. But with Mir
the client doesn't know about anything about position - even moves.

---

From: Alan Griffiths <[email-masked]>
Date: Tue, 6 Sep 2016 09:25:17 +0100

On 05/09/16 17:21, Alan Griffiths wrote:
> parent surface. I don't think we agreed exactly how (e.g. do we "clip"
> the rect? What happens if there is *no* intersection?)
>
I plan to start a second pass on "placement" later today to get
notification of the resulting placement to the client into the Mir API.
A third pass adding these post-conditions to MirAL will follow. This
would be a good time for any further thoughts or clarifications.