Subject: CI + touch images

From: Daniel van Vugt <[email-masked]>
Date: Thu, 23 Jan 2014 15:56:21 +0800

It seems we've had a couple of occasions where code has landed that 
makes mir_*_tests fail on touch devices. And yet CI passes and the merge 
proposals land. Certainly CI is ensuring our tests pass on armhf for 
some configurations, but it seems not enough...

Can we get better coverage of real touch images/devices for CI?

- Daniel

---

From: Kevin Gunn <[email-masked]>
Date: Thu, 23 Jan 2014 08:26:23 -0600

hey Daniel -
this is something I've been raising with the CI team in general.
They do have actual devices running the tests - so, is this a matter
intermittent failure of the tests? meaning we need to request repeat runs
of the tests ?

Or is your concern the actual models (galaxy vs n4 vs n7 etc) ?

br,kg

On Thu, Jan 23, 2014 at 1:56 AM, Daniel van Vugt <
[email-masked]> wrote:

> It seems we've had a couple of occasions where code has landed that makes
> mir_*_tests fail on touch devices. And yet CI passes and the merge
> proposals land. Certainly CI is ensuring our tests pass on armhf for some
> configurations, but it seems not enough...
>
> --
> Mir-devel mailing list
> [email-masked]
> Modify settings or unsubscribe at: https://lists.ubuntu.com/
> mailman/listinfo/mir-devel
>

---

From: Daniel van Vugt <[email-masked]>
Date: Fri, 24 Jan 2014 09:45:10 +0800

The recent issues are actually odd edge cases with reasonable 
explanations for slipping through CI:

https://bugs.launchpad.net/mir/+bug/1271434
CI runs tests on the build machine. Hence it has the required build deps 
installed. Until a couple of days ago we could also run tests on fresh 
touch images (which is an awesome feature I use all the time), but not 
any more.

https://bugs.launchpad.net/mir/+bug/1270685
Nexus 10 only. So CI would not have noticed.

- Daniel

On 23/01/14 22:26, Kevin Gunn wrote:
> intermittent failure of the tests? meaning we need to request repeat
> runs of the tests ?
>
> On Thu, Jan 23, 2014 at 1:56 AM, Daniel van Vugt
> <[email-masked] <mailto:[email-masked]>>
> wrote:
>
>     makes mir_*_tests fail on touch devices. And yet CI passes and the
>     merge proposals land. Certainly CI is ensuring our tests pass on
>     armhf for some configurations, but it seems not enough...
>
>     [email-masked] <mailto:[email-masked]>
>     Modify settings or unsubscribe at:
>     https://lists.ubuntu.com/__mailman/listinfo/mir-devel
>     <https://lists.ubuntu.com/mailman/listinfo/mir-devel>
>
>