confidence

The Confidence Axiom: The value of testing is measured by the confidence of stakeholder decision making

Summary

Testers must understand the relationship between test evidence and the decisions that stakeholders must make. Testing should focus on providing the evidence that stakeholders require to make decisions with confidence.

Consequence if ignored or violated

Stakeholders regard the evidence produced by testing as irrelevant, unreliable, incomplete and not aligned with their goals.

Questions

  • What evidence do stakeholders need to make decisions with confidence?
  • How will stakeholders use that evidence?
  • How are the goals of testing articulated in plans, specifications, meetings and other communications?
  • How will we ensure that evidence is delivered as early as practical to stakeholders?
  • How will we assure the accuracy and currency of the evidence produced?
  • What preferences for format, detail, frequency, precision, accuracy exist for the evidence produced?
  • How will the evidence be transmitted, acknowledged and reviewed by stakeholders?

The Good-Enough Axiom: The scope of testing and acceptance are always compromises

Summary

Stakeholders and testers must jointly appreciate that there is no limit to testing and that the acceptance decision will always be made on incomplete evidence. In fact, acceptance may occur in spite of evidence, based on information known only to stakeholders.

Consequence if ignored or violated

Stakeholders are frustrated by poor system quality or late delivery because their expectations are unrealistic. Testers are frustrated because they cannot finish testing, the system is imperfect and stakeholders decide to accept regardless.

Questions

  • How much evidence from testing will be required to make the acceptance decision?
  • Who is authorised to make the acceptance decision?
  • What is the mechanism for assessing the value of evidence gathered during testing?
  • What coverage model(s) can be used to judge that enough evidence has been gathered?
  • What criteria will be used to judge that the system under test is acceptable or unacceptable?
Syndicate content