Work flow on large repositories

[Michael Hope]

Ah, no - I meant it to go to the list.  Sent again.

It is creating a working copy.  My machine is a Core 2 Duo P8600 with
3 GB of RAM and a 7200 RPM disk, so it's OK.  It's not a very fair
comparison but a tar xzf of the upstream tarball takes 35 s - that
kind of sets the upper limit on how fast bzr could be.

Although gunzip is single threaded... :)

-- Michael

On Thu, Jul 29, 2010 at 9:19 AM, John Arbash Meinel
<john at arbash-meinel.com> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Did you mean to reply off list?
>
> Michael Hope wrote:
>> Thank you for all of your comments.  I'll keep an eye on things over
>> the next few days and record a bit more (-Dmemory is quite
>> interesting) and see how things go.  There's a few comments inline
>> below...
>>
>
> If you really care, you can set:
>  debug_flags = memory
>
> In your ~/.bazaar/bazaar.conf and it will report memory consumption
> after every run.
>
> I suppose something that might be a bit more useful:
>  a) Always put it in the log file, but not in the terminal or
>  b) Computed memory consumption, but only dumped it if it was above a
>    given threshold.
>
>>
>> On Thu, Jul 29, 2010 at 8:17 AM, John Arbash Meinel
>> <john at arbash-meinel.com> wrote:
>>> -----BEGIN PGP SIGNED MESSAGE-----
>>> Hash: SHA1
>>>
>>>> Day-to-day work is done on topic branches.  Creating the branch takes
>>>> 46 s, 250 MB of RAM, and creates a 20 MB .bzr directory.  Pushing this
>>>> branch to LP for merging involves pushing the full 20 MB, but this is
>>>> acceptable.
>>> I can't say I fully understand this. Testing it here:
>>>  $ time bzr branch trunk test -Dmemory
>>>  PeakWorking          25824 KiB
>>>  real    0m0.974s
>>>  $ du -ksh test/.bzr/
>>>  265K    test/.bzr/
>>
>> michaelh at crucis:~/c/gcc/gcc-linaro$ time bzr branch 4.4 test -Dmemory
>> Branched 93541 revision(s).
>> VmPeak:   267768 kB
>>
>> real    3m49.509s
>> user    0m47.875s
>> sys     0m11.833s
>
> Do you have any interesting plugins like 'bzr-search' on your machine?
>
> Is this branch creating a working tree, or just a branch? If the former,
> I'm a bit surprised it is that efficient, if the latter, I'm surprised
> it is taking so long.
>
> What kind of machine are you running on?
>
>
>>
>> michaelh at crucis:~/c/gcc/gcc-linaro$ du -sk test/.bzr/
>> 20352   test/.bzr/
>>
>>>> Doing a bzr pull on the 4.4 mirror directory may more than half an
>>>> hour and more than 500 MB of memory.
>>> Is that with the mirror already up to date? To branch all of
>>> lp:gcc-linaro took me about 65minutes. To add 4.4 to it took 8.5min and
>>> 52MB of content transfer.
>>
>> A mix.  If there's nothing to pull then things are fast.  I was on the
>> 4.4 branch under a mirror branch, did a bzr pull, and waited.  The
>> branch isn't very active so it shouldn't have had much to do.
>>
>> -- Michael
>>
>
> Note that you can use Ctrl+\ (SIGQUIT) to drop into a debugger, and you
> should be able to do 'bt' to get the back trace of what is currently
> going on.
> (At times in the past ^\ has caused connections to drop out, so it isn't
> always ideal.)
>
> You can also run 'bzr command --lsprof-file foo.callgrind' which will
> generate an output file we can use with KCachegrind to determine where
> time was spent.
>
> John
> =:->
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.9 (Cygwin)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iEYEARECAAYFAkxQnvkACgkQJdeBCYSNAAOqBQCeMIy8KnYw7r0CqJ9b8MyQ8Lc5
> xXYAn0prOJQvQXq26p2H+4wWDz+5WyHp
> =NQwT
> -----END PGP SIGNATURE-----
>