Gherkin DSL for testcase description and automation

Fri Dec 9 10:49:02 UTC 2011

On Fri, Dec 9, 2011 at 12:41 PM, Brendan Donegan <
brendan.donegan at canonical.com> wrote:

> Not sure I buy some of the criticisms of the ISTQB definitions, but it's
> certainly true that Gherkin encodes the tests in such a way as to make
> it possible for tools to parse them, whereas perhaps ISTQB definitions
> don't have that. I do think the language ends up looking slightly weird
> for manual testers who may not have that much experience with the
> software though. Here's a weakness, at least in the definitions given
> below:
>
>  Scenario: TC_CAL-002: Calculate sums
>   Given Calculator is in Basic mode
>   When I input "1+1"
>     And I calculate result <- How?
>   Then Result is "2" <- Where?
>    And No error occurs <- Where does the error not occur?
>
>
> I will concede that those things are probably rectifiable though. We
> just need to acknowledge that given a.) time and resource constraints
> and b.) the very nature of certain tests, we won't be able to automate
> everything and whereas ISTQB provides a definition which is probably
> easier to understand for the manual tester but not so much for tools,
> Gherkin DSL provides one that is easier for tools to understand but
> maybe vulnerable to providing definitions that are more difficult for
> manual testers.
>
>
So, in fact, we kind of need to decide who should we cater more with the
test cases - manual
testers or automatic ones? I think we need to handle manual ones now, and
try to move as much
as possible for automation later. So ISTQB definition would get my +1 for
the nearest future.

> It also seems that it's background is unit test generation and I'd be
> curious to see what a larger test case written in this syntax would look
> like.
>
>
And so would I :-)

On 09/12/11 10:05, Руаньяк wrote:
> > Hi all,
> >
> > After some hot debate in the mailing list about re-writing and
> > categorizing testcases, I've decided to propose an adoption of another
> > testcase DSL: Gherkin.
> > Current ISTQB definition doesn't seem to be elegant enough to quickly
> > describe the expected behavior of the system. As for me, these tests
> > with actions separated from expected results are not very readable, as
> > the tester has to do too many action while performing the test: a)
> > perform the action, b) look down for expected result, c) verify the
> > result, d) look up for another action and so on.
> > Also, current testcases can be used for manual testing only and it is
> > not directly suitable for Mago or any other automation framework.
> >
> > There is also another way to describe the behavior of the system,
> > developed to be readable and directly used for automation: Gherkin
> > DSL. See more info on the links sections
> >
> > This is a sample test case written using ISTQD definition for Gnome
> Calculator:
> >
> > ---
> > Test Case ID: TC-CAL-002
> > Test Case Description: Check that sums are calculated
> > Assumptions: Gnome Calculator is started, mode is switched to the Basic
> mode
> >
> > Actions:
> > 1. I input "1+1"
> > 2. I click '=' button
> > Expected Results:
> > 1. '1+1' appears in the calculator textbox
> > 2. Result is 2
> > 3. No error appears
> > ---
> > As you may see, there several flaws in this testcase description:
> > 1. As we are using lists in testcase description, it won't be easy to
> > add more steps in the middle of the scenario, especially if testcase
> > is long
> > 2. We have to use 2 expected results (result is 2 and no error
> > appears) for one action (I click '=' button), as there two things to
> > be checked after this action.
> >
> > And here is a sample in Gherkin DSL:
> > ---
> > Feature: Basic Mode
> >
> >  Scenario: TC_CAL-002: Calculate sums
> >    Given Calculator is in Basic mode
> >    When I input "1+1"
> >     And I calculate result
> >    Then Result is "2"
> >     And No error occurs
> > ---
> >
> > This description is much more readable and can be easily copy-pasted
> > in bugreport, as it doesn't have lost of unnecessary info, lists and
> > can be broken anywhere with actual error description.
> >
> > Note, that automation suites can easily use the same scenarios. The
> > only thing implemented is just actions for steps, which are basically
> > procedures (using i.e ldtp), which use regexp to parse steps. If you
> > guys need more info on this, I will write a separate post on
> > automation with Gherkin DSL.
> >
> > There also some more interesting stuff in Gherkin - such as tagging,
> > step argument transform and step reusing - I guess, I can describe
> > them in other posts, if this whole propasl will be interesting.
> >
> > Hope to hear for more ideas and thoughts.
> > Vadim Rutkovsky
> >
> > -----
> > Links:
> > Gherkin DSL examples and structure:
> >  https://github.com/cucumber/cucumber/wiki/Gherkin
> >  http://code.google.com/p/spectacular/wiki/WritingBDDTests
> >
> http://en.wikipedia.org/wiki/Behavior_Driven_Development#Application_examples_in_the_Gherkin_language
> >
> > Sample automation code for GCalc:
> >  https://code.launchpad.net/~roignac/+junk/freshen_mago
> >
>
>
> --
> Ubuntu-qa mailing list
> Ubuntu-qa at lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/ubuntu-qa
>

-- 
Alex Lourie
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/ubuntu-quality/attachments/20111209/379827a1/attachment.html>