requirements-testing

Requirements Testing Brainstorm

This is a concept for documenting acceptance tests and project requirements in XHTML. The microformat should exist to encourage and facilitate the progress and uptake of Agile development methodologies across a wide spectrum of projects and organizations.

Participants

• Mark Rickerby

The problem

There are numerous popular approaches to requirements definition and acceptance test/use case documentation, and various automated tools to run test fixtures and report back results.

Many of the accepted formats focus on diagram notation of use cases, but a lot of the time, test plans and documentation are captured in unstructured and arbitrary styles, often in Word documents or similar. One of the major issues with working with requirements in this document format is that it is very easy to fall out of step with revisions and changes, especially once development cycles are up and running on a project. Older requirements information tends to rot and become incorrect very quickly, unless documents evolve directly with the development process itself. Word documents are hard to author collaboratively in this way, and encourage clients and developers to take an adversarial stance.

Documenting Requirements

Could there be a delarative equivalent of a UML use case diagram in XHTML? What would this actually achieve? It's easy enough to do in XML, but this wouldn't be viewable directly in a document format. Good idea not to overcomplicate the simple description of a user role, and possible interactions with other roles/services.

A simple HTML vocabulary could make it much easier to share structured requirements information through a wiki type format, which could thus be updated in realtime to reflect changing requirements and new knowledge about the conditions of a system.

Test Vocabulary

A list of possible commands for a a web acceptance testing tool in basic English:

 - Start at [link] (Go to [link])
 - Click [text][submit]
 - Set field [name] to [value] (Add [name] to [value)
 - Should see (Expect to see)

Communicating and Sharing Acceptance Test Data

Who is the audience? Clients, developers, stakeholders. What do they want to see? An itemized list of test cases - whether the tests are completed or not, and if incomplete, where did the failures occur?

Current markup ideas:

 <div class="test">
  <h3 class="case">Local Login Test (incomplete)</h3>
  <ol class="results">
    <li>Start at http://localhost/login <span class="pass">(pass)</span></li>
    <li>Expect to see "Please enter your login" <span class="fail">(fail)</span></li>
  </ol>
 </div>

xoTest: XOXO outlines for defining macros

Example of how the compact attribute could be used to define a test "macro". In this case, "Login to localhost" becomes shorthand for referring to the sequence of nested steps.

<ol class="xoxo test">
  <li>Add A Link
    <dl>
      <dt>Tests the process of adding a link</dt>
        <dd>additional notes here</dd>
    </dl>
    <ol>
      <li>Login to localhost
         <ol compact="compact" class="macro">
            <li>Go to http://localhost/login</li>
            <li>Set username to "user"</li>
            <li>Set password to "pass"</li>
            <li>Click submit</li>
        </ol>
      </li>
      <li>Click "Add Link"</li>
      <li>Set url to "http://microformats.org/wiki/XOXO"</li>
      <li>Set description to "Extensible Open XHTML Outlines"</li>
      <li>Click submit</li>
      <li>Should see "Link was saved successfully"</li>
    </ol>
  </li>
</ol>

This raises some interesting possibilities for both documentation and test generation. As well as functioning as use case documentation, this basic list could be used as input for code generation tools.