Software Testing Fundamentals

Software testing is crucial for ensuring product quality by revealing defects, exposing failures, and verifying that quality requirements are met, ultimately reducing costs and improving user satisfaction.

Core Principles

Quality has two faces: conformance to requirements (producer view) and fitness for use (customer view).
Testing provides evidence for product quality and supports quality management, assurance, and control.
Faults are static defects, errors are incorrect internal states, and failures are observable wrong behaviors.
The cost of finding and fixing defects increases exponentially the later they are found in the software development lifecycle.
Verification checks if the software is built right (against specifications), while validation checks if the right software is built (for user needs).

Action Steps

Exercise the software against its expected behavior to evaluate its functionality (Evaluate).
Observe and document any wrong behavior before real users experience it (Reveal failures).
Provide evidence that the product meets quality standards and is ready for release (Assure quality).

Key Terms

Fault: A static defect in the software, a mistake in the source code.
Error: An incorrect internal state produced when a fault is executed.
Failure: Observable wrong behavior relative to the requirements.
Verification: Checking if the software is being built correctly against specifications.
Validation: Checking if the right software is being built to meet user needs.

Pro Tips

Test early and continuously to significantly reduce the cost of fixing defects.
Prioritize testing efforts on areas with the highest risk, as exhaustive testing is impossible.
Refresh tests regularly to uncover new defects, as a few modules often contain most bugs (Pesticide Paradox & Defect Cluster).

Pitfalls to Avoid

Treating testing as mere debugging, blurring the lines between wrong behavior and code mistakes.
Assuming that passing tests proves complete correctness; tests only show the presence of defects, not their absence.
Believing bug-free software is inherently useful; if it's the wrong software, it's useless regardless of defect count.

Real World Examples

Mars Climate Orbiter failure: A metric-vs-imperial unit mix-up caused the probe to go off course, costing $327M.
Knight Capital trading loss: Untested trade-engine deployment led to a loss of $440M in 45 minutes due to a lack of testing.
THERAC-25 radiation overdoses: A race condition in the software delivered massive radiation overdoses, resulting in 3 deaths.