User Tools

Site Tools


software_carpentary2

This is an old revision of the document!


Quality Assurance

The more you invest in quality, the less time it takes to develop working software.
Quality is not just testing
Trying to improve the quality of software by doing more testing is like trying to lose weight by weighing yourself more often. (Steve McConnell)

Quality is:

  • Designed in
  • Monitored and maintained through the whole software life cycle

This lecture looks at basic things every developer can do to maintain quality.

Limits to Testing

  • Suppose you have a function that compares two 7-digit phone numbers, and returns True if the first is greater than the second
  • 1072 possible inputs
  • At ten million tests per second, that's 155 days
  • If they're 7-character alphabetic strings, it's 254 years

o Then you move on to the second function…

  • And how do you know that your tests are correct?
  • All a test can do is show that there may be a bug

Nomenclature

A unit test exercises one component in isolation

  • Developer-oriented: tests the program's internals

An integration test exercises the whole system

  • User-oriented: tests the software's overall behavior

Regression testing is the practice of rerunning tests to check that the code still works

  • I.e., make sure that today's changes haven't broken things that were working yesterday
  • Programs that don't have regression tests are difficult (sometimes impossible) to maintain

Test results specifications

Any test can have one of three outcomes:

  • Pass: the actual outcome matches the expected outcome
  • Fail: the actual outcome is different from what was expected
  • Error: something went wrong inside the test (i.e., the test contains a bug)

o Don't know anything about the system being tested

A specification is something that tells you how to classify a test's result

  • You can't test without some sort of specification

Writing Tests

  • How to write tests so that:

o It's easy to add or change tests

        o It's easy to see what's been tested, and what hasn't
  * A test consists of a fixture, an action, and an expected result
        o A fixture is something that a test is run on
        o Can be as simple as a single value, or as complex as a networked database
  * Every test should be independent
        o I.e., the outcome of one test shouldn't depend on what happened in another test
        o Otherwise, faults in early tests can distort the results of later ones
  * So each test:
        o Creates a fresh instance of the fixture
        o Performs the operation
        o Checks and records the result
software_carpentary2.1308541852.txt.gz · Last modified: 2018/03/24 11:13 (external edit)