The Coverage Axiom: Testing needs a test coverage model or models

Summary

Testers need a means of assessing the thoroughness or completeness of testing with respect to the chosen test models in ways that are meaningful to stakeholders.

Consequence if ignored or violated

Stakeholders do not understand the status of testing. No one knows what has been tested, what has not been tested or if testing is finished.

Questions

  • How will coverage definitions that describe the thoroughness or adequacy of testing be articulated?
  • Can these coverage definitions be used to define a quantifiable coverage measure?
  • How can the coverage measures be related to the goals and concerns of stakeholders?
  • Could these measures support estimation, planning and progress reporting?
  • How will the thoroughness/adequacy of testing be articulated to stakeholders?
  • With respect to the acceptability of the system, what interpretation(s) of these coverage measures could be made?