Subject: Mir on vmwgfx

From: Thomas Hellstrom <[email-masked]>
Date: Tue, 05 Nov 2013 08:22:28 +0100

Hi!

I'm new to this list and I'm trying to get Mir running in a VMware 
virtual machine on top of the vmwgfx driver stack.
The idea is to first get "mir_demo_server_basic" running with demo 
clients and then move on to Xmir, patching up our drivers as necessary.

So far, I've encountered a couple of issues that might need attention 
from MIR developers:

1) function mggh::DRMHelper::is_appropriate_device() in 
gbm_display_helpers.c checks whether a drm device has any children 
except itself. This is not true for vmwgfx, and the server will fail to 
start thinking that our drm device is not appropriate. Why the child 
requirement?

2) Once I get the basic server to start, the cursor disappears as soon 
as I move the mouse. This boils down to Mir thinking that the cursor is 
outside of the current mode bounding box. At Mir server startup, there 
is no KMS setup configured, hence 
DisplayConfigurationOutput::current_mode_index will be set to max (or 
-1) in mgg::RealKMSDisplayConfiguration::add_or_update_output().
The value of DisplayConfigurationOutput::current_mode_index then doesn't 
seem to change even when Mir sets a display configuration, and when the 
mode bounding box is calculated, an out of bounds array access is performed.

3) Minor thing: The "Virtual" connector type is not recognized by Mir. 
(actually it's not in xf86drmMode.h either, I'll see if I can fix that 
up), but it's in the kernel user-space api file "drm_mode.h" and is 
right after the "eDP" connector type. Should be added in 
connector_type_name() in real_kms_output.cpp

4) vmwgfx does not yet implement the drm "Prime" mechanism for sharing 
of dma buffers, which Mir relies on. I'm about to implement that.
However, it seems like Mir is using dma buffers in an illegal way:
1) Mir creates a GBM bufffer.
2) Mir uses Prime to export a dma_buf handle which it shares with its 
clients.
3) The client imports the dma_buf handle and uses drm to turn it into a 
drm buffer handle.
4) The buffer handle is typecast to a "dumb" buffer handle, and then 
mmap'ed. in struct GBMMemoryRegion : mcl::MemoryRegion.

It's illegal to typecast a GBM buffer to a dumb buffer in this way. It 
accidently happens to work on the major driver because deep inside, both 
a GBM buffer and a dumb buffer is represented by a GEM buffer object. 
With vmwgfx that's not the case either for a GBM buffer or a dumb 
buffer, and they are different objects.

In fact, currently the only way to mmap() a GBM buffer (unless it's a 
cursor buffer) is to export a dma_buf and use it's mmap() operation. But 
that is not implemented by any of the major drivers since it's not 
really desired. The reason is that a GBM buffer is completely opaque and 
may not even reside in mappable memory. Hence any attempt to map it may 
result in coherence issues and, in some cases, extremely costly 
operations. This results in awkward driver code that attempts to guess 
the usage-patterns of applications that mix mmap'ed cpu-access and 
accelerated access to the same object.

The correct way to do this is to have the client import the buffer to 
the appropriate API, and use texsubImage like operations (or perhaps 
readPixels / writePixels) to access the buffer from the CPU.

IMHO, I think this is something that needs to be addressed as soon as 
possible.

Thanks,
Thomas

---

From: Daniel van Vugt <[email-masked]>
Date: Tue, 05 Nov 2013 15:42:01 +0800

Thanks Thomas, nice work.

Would you be able to log the individual problems separately here; ?
https://bugs.launchpad.net/mir/+filebug

---

From: Thomas Hellstrom <[email-masked]>
Date: Tue, 05 Nov 2013 09:49:53 +0100

> https://bugs.launchpad.net/mir/+filebug

Sure, will do that.
/Thomas

---

From: Alexandros Frantzis <[email-masked]>
Date: Tue, 5 Nov 2013 12:04:06 +0200

> clients and then move on to Xmir, patching up our drivers as

Hi Thomas, thanks for looking into this. Feel free to also join
#ubuntu-mir on freenode if you need more direct information.

> So far, I've encountered a couple of issues that might need
> attention from MIR developers:
> 
> except itself. This is not true for vmwgfx, and the server will fail
> to start thinking that our drm device is not appropriate. Why the
> child requirement?

Will take a deeper look, probably an arbitrary requirement based
on what major hardware drivers expose.

> 2) Once I get the basic server to start, the cursor disappears as
> soon as I move the mouse. This boils down to Mir thinking that the
> cursor is outside of the current mode bounding box. At Mir server
> startup, there is no KMS setup configured, hence
> DisplayConfigurationOutput::current_mode_index will be set to max
> (or -1) in mgg::RealKMSDisplayConfiguration::add_or_update_output().
> The value of DisplayConfigurationOutput::current_mode_index then
> doesn't seem to change even when Mir sets a display configuration,
> and when the mode bounding box is calculated, an out of bounds array
> access is performed.

Will take a deeper look.

> 3) Minor thing: The "Virtual" connector type is not recognized by
> Mir. (actually it's not in xf86drmMode.h either, I'll see if I can
> fix that up), but it's in the kernel user-space api file
> "drm_mode.h" and is right after the "eDP" connector type. Should be
> added in connector_type_name() in real_kms_output.cpp

No problem.

> 4) vmwgfx does not yet implement the drm "Prime" mechanism for
> sharing of dma buffers, which Mir relies on. I'm about to implement
> that.
> 2) Mir uses Prime to export a dma_buf handle which it shares with
> its clients.
> 3) The client imports the dma_buf handle and uses drm to turn it
> into a drm buffer handle.
> It's illegal to typecast a GBM buffer to a dumb buffer in this way.
> It accidently happens to work on the major driver because deep
> inside, both a GBM buffer and a dumb buffer is represented by a GEM
> buffer object. With vmwgfx that's not the case either for a GBM
> buffer or a dumb buffer, and they are different objects.

This code path (i.e. mmap-ing the buffer on the client side) is only
valid when the client has requested a "software" buffer, which on the
server side leads to the creation of a "dumb" DRM buffer (i.e., with
DRM_IOCTL_MODE_CREATE_DUMB). Although mmaping non-dumb buffers doesn't
fail per se with the major hardware drivers, the returned pixel data
usually has a non-linear layout (e.g. some sort of tiling), so it's not
really usable for our purpose.

Note that the Mesa codebase we are using has some changes in the GBM code
(experimental, not upstream yet). Notably:
  * we allow creation of "dumb" drm buffers of arbitrary size (not just 64x64)
    when using GBM_BO_USE_WRITE
  * gbm buffers backed by a "dumb" DRM buffer also get a DRIimage

We have found that the major hardware drivers support these changes.
Do they pose a problem for the VMware driver?

See http://github.com/RAOF/mesa/ for the changed code base. It's still
based on the Mesa version shipped in Ubuntu 13.10, but we plan to rebase
on a more recent version.

Thanks,
Alexandros

---

From: Thomas Hellstrom <[email-masked]>
Date: Tue, 05 Nov 2013 12:13:42 +0100

Hi Alexandros.
Thanks for responding.

> really usable for our purpose.

Ah, that sounds good to hear! Will save us a lot of pain.

>      when using GBM_BO_USE_WRITE

It depends on the intended use-case. Currently these buffers are meaningful
and intended only as cursors when created from GBM, and the vmwgfx cursor
code will currently refuse any other cursors than 64x64 sized ones, but I
think that could be relaxed. Need to check internally.

>    * gbm buffers backed by a "dumb" DRM buffer also get a DRIimage

Again that depends on the use-case. Is there any coherence assumed or 
data-transfer
going on between the dumb drm buffer and the DRIimage? Or are you using 
it to be
able to render to cursors?

Thanks,
Thomas

---

From: Jakob Bornecrantz <[email-masked]>
Date: Tue, 5 Nov 2013 03:36:52 -0800 (PST)

----- Original Message -----
> > clients and then move on to Xmir, patching up our drivers as

Also new to this list, I work with Thomas but the mail address probably
made obvious.

> #ubuntu-mir on freenode if you need more direct information.

I'm on there, nick Prf_Jakob[1].

[SNIP]

>   * we allow creation of "dumb" drm buffers of arbitrary size (not just
>     64x64) when using GBM_BO_USE_WRITE

There is no technical limit on this.

>   * gbm buffers backed by a "dumb" DRM buffer also get a DRIimage

This will be a problem, at least to my knowledge DRIimages are backed
by a gallium resource/texture, in SVGA this is backed by a surface,
while dumb drm buffers would be backed by a dma-buffer (I think as
of right now vmwgfx does not support the dumb interface).

Taking a step back here, SVGA have two types of resources: a surface
which is in simplified terms a opaque handle to a GL texture on the
host side, which can not be mapped; and dma-buffers which are regions
of memory both the guest and the host and used for transferring data
to and from surfaces.

> Do they pose a problem for the VMware driver?

See above, and adding too that, if you are doing software a approach we
would like the storage to be a dma-buffer and not to be backed by a surface
(to avoid unnecessary data transfers and resource overhead). On the other
hand the way that the current vmwgfx xorg driver works is that it tries to
keep everything in software until the moment that the compositor asks for
the data and then transfers it to the host so the compositor can use 3D.
Basically per window shadowfb (for those familiar with X), finding a way
for mir to work like this (on vmwgfx) would be the best for us.

> on a more recent version.

Cheers, Jakob.

[1] Not really a professor

---

From: Alexandros Frantzis <[email-masked]>
Date: Tue, 5 Nov 2013 15:55:00 +0200

On Tue, Nov 05, 2013 at 03:36:52AM -0800, Jakob Bornecrantz wrote:

[SNIP]

Hi Jakob!

> There is no technical limit on this.

Good to hear.

> (to avoid unnecessary data transfers and resource overhead). 

The main use case for this is to allow direct surface pixel access for
clients that either can't, or prefer not to use GL to render, while
still keeping compositing performant.

In an early Mir server implementation, when the server had to deal with
"dumb" buffers it just mmap-ed the pixel data and uploaded them to a
texture with glTexImage2D(), so that the compositor could use them.
However, it turned out that this was very slow. To speed up things, we
added a backing DRIimage to the "dumb" gbm buffer, so that we could use
glEGLImageTargetTexture2DOES() with it to populate the texture (like we
do for non-dumb buffers), which is significantly faster.

If this is just a matter of reduced performance in the VMware driver for
this use case, then perhaps we should wait to see if it's actually a
problem before adding a special case for it in Mir. On the other hand,
if it is a matter of complicating the VMware driver excessively, we can
try to find a way to accommodate this elegantly in Mir. Would the
first approach (mmap-ing the dumb buffer and using glTexImage2D()) be
a better match for the VMware drivers?

Thanks,
Alexandros

---

From: Alexandros Frantzis <[email-masked]>
Date: Tue, 5 Nov 2013 15:57:13 +0200

On Tue, Nov 05, 2013 at 12:13:42PM +0100, Thomas Hellstrom wrote:
[snip]
> Again that depends on the use-case. Is there any coherence assumed
> or data-transfer
> going on between the dumb drm buffer and the DRIimage? Or are you
> using it to be

Please see my reply to Jakob's email, for more information about this.

Thanks,
Alexandros

---

From: Thomas Hellstrom <[email-masked]>
Date: Tue, 05 Nov 2013 15:17:49 +0100

> do for non-dumb buffers), which is significantly faster.

I still don't quite understand how you get the pixel data in the dumb 
buffer to the DRIimage?
On the glTexImage2D() approach, did you try ordinary shmem sharing 
rather than a dumb GBM buffer?
It might have been that the GBM buffer was residing in uncached memory, 
which makes reading painfully slow.

> a better match for the VMware drivers?

In this case glTexImage2D() might be better since GBM dumb buffers 
reside in cached memory. However,
it would be desirable only to copy the data that has been dirtied by the 
client. Our X server driver uses this approach.
Thanks,
Thomas

---

From: Alexandros Frantzis <[email-masked]>
Date: Tue, 5 Nov 2013 17:00:51 +0200

[snip]
> I still don't quite understand how you get the pixel data in the
> dumb buffer to the DRIimage?

When creating a dumb GBM buffer we create a DRIimage associated with the
dumb DRM buffer using DRI's createImageFromName(). After that we treat the
GBM buffer as a normal non-dumb buffer as far as rendering is concerned:
we create an EGLImage using the GBM buffer object as the native pixmap
type, and "upload" the data with glEGLImageTargetTexture2DOES().

> It might have been that the GBM buffer was residing in uncached
> memory, which makes reading painfully slow.

This was a long time ago, and I don't remember if I tried using shmem
to share the data. I will keep that in mind, though, if we need to
implement a glTexImage2D() fallback.

> it would be desirable only to copy the data that has been dirtied by
> the client. Our X server driver uses this approach.

As a first step, can we ignore this until it proves to be a performance
problem, or does even a simple implementation of supporting DRIimage for
dumb buffers fall into "complicating the VMware driver excessively"?

Thanks,
Alexandros

---

From: Thomas Hellstrom <[email-masked]>
Date: Tue, 05 Nov 2013 18:47:55 +0100

> type, and "upload" the data with glEGLImageTargetTexture2DOES().

Alexandros, unfortunately casting a dumb buffer to an accelerated GBM 
buffer is as much of an API
violation as the other way around, and works on some drivers for the 
same reason, but may stop working at
any point in time. I don't think any patches utilizing this approach 
will ever make it upstream.

The APIs for accelerated buffers deliberately try to avoid mmap. The 
reason is that there are coherency
problems. In some cases cache flushes, in other cases, like vmwgfx, 
expensive data transfers.

Imagine someone mmaping and writing a single character on a 3200x1200 
canvas, then try to make an
accelerated buffer out of this. Obviously a 3200x1200 texsubimage for 
each character will yield unacceptable performance,
A non-coherent driver will likewise need to perform a 3200x1200 cache 
flush or data transfer which will also yield
unacceptable performance. The correct solution here is a character size 
texSubImage or writePixels operation. If performed on the
server this means some form of damage tracking being communicated with 
the buffer, like Wayland does.

> dumb buffers fall into "complicating the VMware driver excessively"?

Unfortunately I don't think this is an option for us. Even if we'd 
ignore the API violation, without some form of synchronization 
information telling us transfer directions and dirty regions it would be 
too inefficient.

Thanks,
Thomas

---

From: Alexandros Frantzis <[email-masked]>
Date: Tue, 5 Nov 2013 22:54:11 +0200

On Tue, Nov 05, 2013 at 06:47:55PM +0100, Thomas Hellstrom wrote:

[snip]
> Alexandros, unfortunately casting a dumb buffer to an accelerated
> GBM buffer is as much of an API
[snip]

I think that being able to use accelerated buffers with linearly
accessible pixels (in the DRI case: dumb drm buffers with associated
DRIimages) should at least be an option, even if there is no guarantee
of general support. All major implementations support it relatively
efficiently at the moment, and although it's clear that using buffers in
such a way may come with a performance penalty, or not be possible at
all in some cases, I don't think we should preclude it from the API
completely.

> information telling us transfer directions and dirty regions it
> would be too inefficient.

Regardless of the GBM API concerns, I imagine that if a DRI
implementation doesn't support the aforementioned operation it should
report a failure, e.g., when trying to create the DRIimage for a dumb
DRM buffer. If the VMware driver works this way then it would be easy
for Mir to handle the error and fall back to some other supported
method, e.g., a plain dumb buffer plus glTex{Sub}Image2D().

Is this behavior feasible/reasonable from the VMware driver's
perspective?

Thanks,
Alexandros

---

From: Christopher James Halse Rogers <[email-masked]>
Date: Wed, 06 Nov 2013 10:21:27 +1100

> on what major hardware drivers expose.

This requirement is intended to be a rough-and-ready check for “does
this drm device have any output hardware”. The physical drm drivers all
expose their connectors as sub-objects of the main drm udev node.

We now also check to see whether any of the enumerated connectors are
actually connected, so the udev check should be able to go away.

Chris

---

From: Thomas Hellstrom <[email-masked]>
Date: Wed, 06 Nov 2013 09:08:11 +0100

> all in some cases, I don't think we should preclude it from the API

I understand your point, but the APIs look the way they look for a reason:
Instead of relying on mmaping a huge buffer, like old X, a modern 
implementation
should be smarter and use the preferred API method to transfer pixel
data to the accelerator. This is not just something that happened to be 
the way it is,
It's preceded and agreed upon by discussions by the various driver 
developers
and It wouldn't surprise me if this access path will be blocked even 
with the major
drivers developers once they find out about it.

The GBM WRITE buffers are intended for cursors, and as such it's legal 
for a driver
to block read access, and even if they don't drivers may and will 
probably put these buffers in
write-combined or uncacheable memory which will make things like 
software compositing
painfully slow. I took a brief look at the Wayland code and they are 
using shmem + damage tracking
to handle this, and IMO that seems like a good choice. Another reason is 
that shmem can handle processor
cache coherency problems efficiently, whereas dumb buffers generally 
can't. They'd probably need to rely on
coherent (uncached) memory for this.

Also I think it's important to deal with this now, because once software 
Mir clients that have no idea about
damage tracking start to emerge, and if this usage path gets blocked, 
this issue will be immediately raised
from a vmware problem to a serious Mir performance problem.

I'm trying to tell you this as politely as I can, but I urge you to 
rework this path.

> Is this behavior feasible/reasonable from the VMware driver's

I think vmware will already error here, (if the error is propagated up 
the DRI stack, that is), and we could probably
work something more efficient out, but damage tracking is needed.

Thanks,
Thomas

---

From: Thomas Hellstrom <[email-masked]>
Date: Wed, 06 Nov 2013 09:15:29 +0100

> expose their connectors as sub-objects of the main drm udev node.

OK. Need to check why they do that. Jakob, any idea?

> actually connected, so the udev check should be able to go away.

Sounds great. IIRC there is an IOCTL to check whether kms is actually 
supported, that might
come in handy.

Thanks,
Thomas

---

From: Alexandros Frantzis <[email-masked]>
Date: Wed, 6 Nov 2013 12:01:08 +0200

> I'm trying to tell you this as politely as I can, but I urge you to

OK, hint taken. I will investigate what's involved in using shmem
instead of dumb buffers in Mir, and also what the performance
implications are.

In any case, feel free to ignore the "software" buffer case, since one
way or the other it shouldn't affect the driver (e.g. we may temporarily
add the dumb buffer fallback, until shmem is supported). Plus, you won't
trigger this path anyway unless you run a client that asks for
"software" buffers.

Thanks,
Alexandros

---

From: Jakob Bornecrantz <[email-masked]>
Date: Wed, 6 Nov 2013 06:07:30 -0800 (PST)

----- Original Message -----
> instead of dumb buffers in Mir, and also what the performance

When using software is the application expected to still have
opened a EGL display with the mir-display/connection?

Cheers, Jakob

---

From: Alexandros Frantzis <[email-masked]>
Date: Wed, 6 Nov 2013 16:46:19 +0200

> opened a EGL display with the mir-display/connection?

No. Just a normal connection to the mir server is sufficient.
See examples/flicker.c for a client using software buffers.

Thanks,
Alexandros

---

From: Christopher James Halse Rogers <[email-masked]>
Date: Thu, 07 Nov 2013 14:41:58 +1100

> Sounds great. IIRC there is an IOCTL to check whether kms is actually 

There's a libdrm function, yeah, although I think its semantics have
become ‘is this drm device sane’. It'll return true on drm devices that
don't have any connectors, because the *driver* supports kms.

Hybrid laptops with GPUs entirely disconnected from any display hardware
post-date it :).