Improving the mainline kernel testing process

[komputes]

On 05/04/2011 03:14 AM, Brad Figg wrote:
> On 05/03/2011 04:57 PM, komputes wrote:
>> On 04/25/2011 10:46 PM, Brad Figg wrote:
>>> On 04/25/2011 01:38 PM, Tim Gardner wrote:
>>>> On 04/22/2011 02:01 PM, komputes wrote:
>>>>> Hi Kernel Team,
>>>>>
>>>>> It has been some time that I have been talking to JFo about improving
>>>>> instructions or simplifying the process of testing the mainline 
>>>>> kernel.
>>>>> We use the response "can you test the mainline kernel" so much 
>>>>> that we
>>>>> should make it much simpler to test. The problem is that the current
>>>>> instructions [1] are complicated for average user to understand. 
>>>>> We have
>>>>> to remember that we have experience doing this, average users 
>>>>> (like my
>>>>> parents) do not.
>>>>>
>>>>> Why is this a problem? What happens to most bugs, they get reported
>>>>> against linux. Triager asks for the user to test mainline. At this 
>>>>> point
>>>>> many users give up and do not follow up causing expired bugs.
>>>>>
>>>>> Here are some suggestions I propose:
>>>>> - Rewrite instructions [1] to be more use friendly (dated and 
>>>>> could use
>>>>> some love)
>>>>> - Create a simpler process for testing the mainline build
>>>>> - Generate a LiveCD with the mainline kernel to simplify testing (not
>>>>> ideal for bandwidth, but very user friendly). Simply boot, test and
>>>>> report back.
>>>>> - Place a "mainline" metapackage in the repos for testing purposes
>>>>>
>>>>> What does the kernel team think of this proposal? Should it be 
>>>>> something
>>>>> to discuss at UDS or do we think we can correct this simply?
>>>>>
>>>>> Cheers,
>>>>>
>>>>> -komputes
>>>>>
>>>>> (]( -. .- )[)
>>>>>
>>>>> [1] https://wiki.ubuntu.com/Kernel/MainlineBuilds
>>>>>
>>>>> PS. Thanks to bjf and JFo for providing assistance and guiding me in
>>>>> this proposal.
>>>>>
>>>>
>>>> While the directions for installing a kernel are a bit technical, I 
>>>> don't think they are particularly complicated. If someone isn't 
>>>> comfortable following these directions, then they probably 
>>>> shouldn't be installing kernels. It _is_ a wiki, so feel to make
>>>> improvements where you see fit.
>>>>
>>>> I've long lusted after an easy customized Live CD build script. I 
>>>> think Brad has done something with customized kernels in Live CDs 
>>>> for suspend/resume testing, but it takes a lot of space and 
>>>> bandwidth (and time).
>>>>
>>>> rtg
>>>
>>>
>>> I think the issue is that we basically spam every new bug that comes 
>>> in with
>>> a request for upstream testing. We assume that if you are technical 
>>> enough
>>> to file a bug you can install a kernel. And if you don't do the 
>>> testing we
>>> won't look at your bug.
>>
>> I appreciate you answer Brad. You must understand that the assumption 
>> that the user is technical is incorrect. Apport and Launchpad make 
>> reporting a bug extremely simple meaning there is no technical 
>> requirement other than running ubuntu-bug and having a
>> Launchpad account. As part of crossing the chasm, we need to make 
>> reporting issues and assisting kernel developers a simpler process. I 
>> believe that changing the existing mainline test process will do just 
>> that and will result in less expired linux bugs.
>>
>
> I guess I should have made it clear that I thought the stated 
> assumption is
> is one we make but which does not actually fit the profile of our users.
>
>>>
>>> The main problem I have with the use of Live CDs is that to test a 
>>> kernel we
>>> ask that you download a 700+ MB image. This seems like a lot to ask.
>>
>> It does seem that way but if a weekly build can be scripted to make a 
>> minimal ISO it will help novice users and minimise risks as it will 
>> not affect the installation.
>>
>
> If you feel this is doable, give it a try. I have a script which takes 
> a daily
> iso and adds a custom kernel. It also does a little pairing down of 
> the installed
> applications. It is still quite large. However, you may be more 
> successful.
>
>>>
>>> Something like a meta package which would let people just download 
>>> and install
>>> the current mainline kernel would be a lower barrier.
>>
>> Great, how do you suggest we get started on this effort so that we 
>> may see more kernel bugs corrected instead of expired. Should this be 
>> discussed at UDS or is this mailing list enough to get this started?
>>
>> Note that there are risks with this. A user (not knowing that shift 
>> brings up GRUB's menu) may be stuck at a terminal, which is why I 
>> suggest a minimal yet simple CD image over the meta package idea.
>>
>
> I like this idea but don't know myself how to go about implementing 
> it. Again,
> feel free to take a stab at it yourself.
>
>> -komputes
>>
>> (]( -. .- )[)
>
>
Sure thing Brad. I am interested in seeing your script to see how you 
currently go about it. I would guess that taking a minimal image and 
pre-seeding certain necessary packages then inserting the mainline 
kernel packages should do the trick. Then again I'm not a dev but I'm 
interested in doing the necessary groundwork. I definitely don't want to 
lose sight of this idea.

Blueprint has been registered for UDS-O and key players have been 
invited to join:
https://blueprints.launchpad.net/ubuntu/+spec/qa-o-kernel-mainline-testing

Cheers,

-komputes

   (]( -. .- )[)