Software errors

[Reinhold Rumberger]

On Sunday 23 May 2010, Mark Greenwood wrote:
> On Sunday 23 May 2010 19:35:45 Reinhold Rumberger wrote:
> > On Sunday 23 May 2010, Mark Greenwood wrote:

<nipping for brevity>

> True, but that's not what I mean by 'software quality' and it's
> not what the people with spreadsheets mean either. Good design is
> what engineers strive for, but it's not measurable and so the
> clipboard-ites don't understand it. They want numbers, more bugs
> fixed means more quality. This pushes one of my buttons :)

I'm not a spreadsheet guy - I find most of those metrics stupid and 
worse than no measurement of quality since it conveys an incorrect 
sense of security. Better metrics include some definitions of code 
complexity, but then there is no proper metric that I'm aware of.

<more snippage>

> It was an extreme example. But it happened to me on a monthly
> basis with Karmic. Every time I installed a new version of KDE4
> (from the ppa repo) there was some library incompatability
> (usually an update of Qt) that broke a lot of stuff. I ended up
> having to compile digikam, kdenlive, and one or two others from
> SVN source on a regular basis.

Now, this is funny. I also kept upgrading from PPA and despite having 
a highly customised system, things tended to get better (in fact, KDE 
4.4.3 in lucid is actually the first release in a while where I've 
experienced more problems than in a previous one). I guess one of us 
just got (un)lucky.

> > Also, most libraries will be backwards compatible
> > and/or installable side-by-side with older versions. That's what
> > the version numbers in the name are for.
> > 
> > My point is that the same problems occur in the CS world,
> > Windows being an example. Sure, not in the base release, but
> > then that is unusable because it contains little to no end-user
> > software.
> 
> And that's my point too. The base release of Kubuntu contains
> *all* the end user software. Testing it is an unimaginably
> complex problem. Breakage is assured at some point.

Just for the record, I was thinking more of testing the individual 
components. I think we both now stated at some point that everything 
else just isn't doable. In fact, even proper testing of the 
individual components is a challenge.

<snip>

> > > That's the way it is, and that is why things that worked in
> > > one release are often broken by the next. It's just the way
> > > it is - lots of individuals working to no particular plan
> > > with no resources for testing.
> > 
> > That, too, is incorrect. IME, testing in popular open source
> > software projects is better organised and more effective than
> > in most big software houses.
> 
> I simply disagree. Testing in a large software house is what I do.
> Believe me, we have resources and automation that any OSS project
> would kill for, and we still don't catch all the bugs. A 6 month
> testing cycle is standard for our large projects.

Well, I guess that makes you one of the lucky ones... :-P
Yet, in my experience, this is the exception rather than the norm, I 
hope I've just been unlucky. Still, the vast amount of testers the 
bigger OS project has outweighs all currently available automation 
that I'm aware of. (And, coming from a university context, I'm most 
certainly not aware of the higher-priced stuff.)

> Kubuntu has a
> 6-month *release* cycle, and most of the testing is done by
> volunteers who download the betas. You can't call that thorough
> or organised.

As I mentioned, that's not what I was thinking of. Still, if you take 
Ubuntu's Debian background into consideration, there is a lot of 
"hidden testing" Ubuntu eventually profits from.
You're also completely ignoring the fact that other distros' 
improvements will eventually help Ubuntu - there is a lot more 
context here to offset the problems than in the commercial world.

> > And your idea of the chaotic development in OSS is
> > rather strange to me - no software can grow in size to a point
> > where it seriously matters without some pretty well-defined
> > kind of plan behind it.
> 
> But surely nobody working on a particular oss project is working
> to deadlines for distro releases (unless they're employed by the
> distro of course)? Why would anybody do that?

I was thinking of "goal" when you wrote "plan", not "time 
constraints".

> I'm not knocking open source development, don't get me wrong. This
> all started because I told a guy that stuff like this just
> happens in Linux, it's the nature of the beast.

Yes, and I'm saying it's the nature of software development in 
general, and not specific to Linux, open source or any kind of 
software in particular.

I don't really care how this started - I'm enjoying the discussion, 
even though I'm wondering whether this is the right place for it.

> I speak from
> personal experience. I'm not trying to make any judgements, just
> pointing out a fact. It's a fact that my Kubuntu system breaks
> more often than my Windows XP

I can't speak for Macs, but for me, I've had more trouble with 
breakage in Windows than in Linux, even though my Linux systems tend 
to become heavily customised with time. But then, I seem to have more 
trouble with Windows than most people I deal with. Perhaps it's my 
fault for expecting it to be able to do stuff I do regularly with 
Linux.

> one or my MacOS X one, and I
> believe that the nature of the way they are released and
> developed is the reason for that. But I live with it, because I
> believe in it.

I live with it because it suits me better than either Windows or
MacOS X. I like to tinker with stuff, which is mostly impossible in 
Windows and pretty uncomfortable with Macs. Nothing to do with 
beliefs on my end... ;-)

  --Reinhold