[apparmor] [patch] make coverage should fail if one of the tests fails

[Steve Beattie]

On Sat, Nov 29, 2014 at 09:26:03PM +0100, Christian Boltz wrote:
> the subject says it all - make coverage should fail if one of the tests 
> fails. Currently it ignores failures and creates the coverage data, so
> the failed test has a good chance to go unnoticed.
> 
> The patch adds a "set -e" to let "make coverage" fail if one of the 
> tests fails.

In general, I support having tests fail the make target that triggered
them; however, in this instance I think it's okay that the coverage
results get generated even if testcases fail, as it can be useful
for someone in the middle of working on a patch set, where not all
the tests are passing due to something not having been implemented
completely, but still wants to see the progress they're making on
test coverage for another part of their code (I was doing exactly
this while re-working the capability rules patches). And patches
should be verified to pass 'make check' anyway, where failing tests
would get caught.

So a -1 from me, but I'm open to hearing others' opinions on it. Thanks.

> === modified file 'utils/test/Makefile'
> --- utils/test/Makefile 2014-11-04 21:01:14 +0000
> +++ utils/test/Makefile 2014-11-29 20:07:05 +0000
> @@ -43,8 +43,10 @@
>         export PYTHONPATH=.. ; $(foreach test, $(wildcard test-*.py), $(call pyalldo, $(test)))
>  
>  .coverage: $(wildcard ../aa-* ../apparmor/*.py test-*.py)
> -       export PYTHONPATH=.. ; $(foreach test, $(wildcard test-*.py), $(PYTHON) -m coverage run --branch -p $(test); )
> +       export PYTHONPATH=.. ; set -e ; $(foreach test, $(wildcard test-*.py), $(PYTHON) -m coverage run --branch -p $(test); )
>         $(PYTHON) -m coverage combine
>  
>  coverage: .coverage

-- 
Steve Beattie
<sbeattie at ubuntu.com>
http://NxNW.org/~steve/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <https://lists.ubuntu.com/archives/apparmor/attachments/20141202/e61d43cf/attachment.pgp>