selftest results for lp:bzr revno.4804

[John Arbash Meinel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Alexander Belchenko wrote:
> Vincent Ladeuil пишет:
>>>>>>> "bialix" == Alexander Belchenko <bialix at ukr.net> writes:
>>
>> <snip/>
>>     >> In summary: depending on the OS, passing the full test suite may
>>     >> *require* to use --parallel=fork and 4 CPUs so that the whole
>>     >> test suite is split in parts small enough to avoid the
>>     >> leak-related bugs.
>>
>>     bialix> Most the machines I have around have only one core.
>>
>>     bialix> I think it was my mistake to try run bzr selftest again.
>>
>> Certainly not ! What a strange idea :)
> 
> Well, I'm full of strange ideas.
> 
>>     bialix> It was frustration in the past, and is now.
>>
>> I know. I have a work-in-progress that should address these
>> problems.
>>
>>     >>     >> 4 is the magic number such as:
>>     >> - it's greater than 2 where my OSX slaves fails to pass,
>>     >> - is smaller than 8 where my main desktop pass
>>     >> - is equal to the number of CPUs most of my VMs use (hardy,
>>     >> jaunty, karmic, freebsd, gentoo) and pass
>>     >>     >>>>>>> "Gordon" == Gordon Tyler <gordon at doxxx.net> writes:
>>     >>     Gordon> John Arbash Meinel wrote:
>>     >> >> Alexander Belchenko wrote:
>>     >> >>> IIUC latest improvements for selftest on win32 from John
>> has landed.
>>     >> >>> I've tried to run tests on lp:bzr revno.4804.
>>     >> >>     >>> selftest running for almost 3 hours on my
>>     >> Windows XP and finished with
>>     >> >>> result:
>>     >> >>     >>> Ran 23081 tests in 9306.360s
>>     >> >>     >>> FAILED (failures=13, errors=500,
>>     >> known_failure_count=27)
>>     >>     >> How many leaks ? This can be very important as errors and
>>     >> failures can often be caused by leaks.
>>
>>     bialix> Next time I'll try to grab this info (if this next
>>     bialix> time will ever happens).
>>
>> Thanks in advance. And please, try to collect the whole output so
>> we can analyze the failures and errors, the raw numbers are an
>> indication but unless we fix the leaks, they are extremely noisy.
> 
> OK, I need redirect stderr too. But it is very noisy about locks et al.
> 

I grabbed a screen of the last test run on EC2. There was 1 test
failure, but I think it was spurious, and caused by some of the other
locking issues. (It failed with mismatched locks taken/released, but my
guess is that a thread called unlock late, rather than early.)

Anyway, I should start by saying that running the bzr test suite with a
single core on win32 should be *better* today than it used to be.
However, that doesn't mean it is perfect yet.

We have a bit of kipple from:
Exception RuntimeError: 'maximum recursion depth exceeded in
__subclasscheck__' in <type 'exceptions.RuntimeError'> ignored

I don't have a good answer for how to remove that. We intentionally
trigger a recursion depth exhaustion to make sure that we don't
segfault, etc. I'm a bit surprised it says that it is 'ignored'. My
guess is that 'isinstance()' wants to raise a max-recursion exception,
but it is suppressed, and then it gets raised by the next call into
'process()', which is what we catch and pass the test with.


We have several tests that fail to clean up.


We have 3 failures to unlock properly:
> Exception UserWarning: UserWarning("'LockableFiles(<bzrlib.transport.remote.RemoteTCPTransport url=bzr://127.0.0.1:2010/extra/tree3/.bzr/repository/>)' was gc'd while locked",) in <bound method _LockWarner.__del__ of <bzrlib.lockable_files._LockWarner object at 0x79FEBCF0>> ignored
> Exception UserWarning: UserWarning("'LockableFiles(<bzrlib.transport.remote.RemoteTCPTransport url=bzr://127.0.0.1:2010/extra/tree2/.bzr/repository/>)' was gc'd while locked",) in <bound method _LockWarner.__del__ of <bzrlib.lockable_files._LockWarner object at 0x7A20EC90>> ignored
> Exception UserWarning: UserWarning("'LockableFiles(<bzrlib.transport.remote.RemoteTCPTransport url=bzr://127.0.0.1:2010/extra/tree1/.bzr/repository/>)' was gc'd while locked",) in <bound method _LockWarner.__del__ of <bzrlib.lockable_files._LockWarner object at 0x79FFABB0>> ignored

Though it isn't clear yet what test is causing those failures. It is
somewhere between "test_log" and "test_repository". I'm guessing the latter.

I'll try to clean out some of these errors/warnings. I think there are
still some spurious sftp failures which I haven't been able to explain
yet. Doing:

=== modified file 'bzrlib/repofmt/pack_repo.py'
- --- bzrlib/repofmt/pack_repo.py 2009-10-29 05:54:49 +0000
+++ bzrlib/repofmt/pack_repo.py 2009-12-03 15:14:30 +0000
@@ -508,7 +508,13 @@
         new_name = self.name + '.pack'
         if not suspend:
             new_name = '../packs/' + new_name
- -        self.upload_transport.rename(self.random_name, new_name)
+        try:
+            self.upload_transport.rename(self.random_name, new_name)
+        except errors.PermissionDenied:
+            # Sleep and try again
+            time.sleep(0.001)
+            self.upload_transport.rename(self.random_name, new_name)
+
         self._state = 'finished'
         if 'pack' in debug.debug_flags:
             # XXX: size might be interesting?

Seemed to fix them for me, but that is a kludgey workaround. I think the
issue is that we aren't synchronously closing the file stream over sftp
(instead doing it asynchronously). And on a single CPU machine, the sftp
thread never gets control to actually run 'close'. Hence why
'time.sleep(0.001)' is enough to make it work.

I should note for Vincent that I never run with --parallel. I *do* tend
to run with --subunit, but it isn't required for the test suite to pass.

I also get spurious failures if I try to run with just -Werror. PyCrypto
raises deprecation warnings (by loading sha directly), and win32com
raises ImportWarning because of a missing "__init__.py".

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAksX1wQACgkQJdeBCYSNAAMaeQCcDPmFA/e+5r0H70qu5NAjg68v
pH4An1T+STUZ4943q3sxtS0uKuIseiqE
=MqIL
-----END PGP SIGNATURE-----