Thursday, November 13, 2008

Thoughts On Software Testing

I have read a bunch of blogs lately about software testing, unit testing, functional testing, etc, including this and this.

There are circular arguments out there that go like this: "Developers can't possibly test their own software because they understand it too well." - "Yes, but developers have to test their own software because only they truly understand it (plus those tests make changes easier)."

These are both true of course.

In my view there are three kinds of testing and people confuse them all of the time.
  1. Unit testing. This is testing written by the developer at the object or module level that proves that the code does what the developer intended it to do. Ideally you should inject dependencies and make the tests fairly shallow.
  2. Integration testing. This is testing that proves that the objects/modules/bits of the software work together as expected. It proves that the internal contracts are being met. This can be written by the developer, but it doesn't have to be. They should probably be written by whoever decided on the contracts. Writing integration tests using JUnit doesn't make them unit tests. These tests can and should be deep. Call stuff at the highest level and prove that everything works all the way down to the lowest level.
  3. Functional testing. This is testing that demonstrates that the software works as it is supposed to. This should never be done by the developer because they do indeed know too much about the software. If the developer tries to do functional testing, they can only prove that the software works as they intended and that is not the point here at all.
These are different things and you need all of them. Automate them if you can with a priority of 1, then 2, then 3.



Doug Anderson said...

If the software is a commercial application, then there is a fourth kind of testing, and that is "User Acceptance Testing", where representatives of the business (i.e., non-programmers) test the software against the original requirements definitions.

More info here:
and here:

David said...

You are absolutely right of course Doug. We always do user acceptance testing. Even projects that don't do any other kind usually manage some sort of user acceptance testing.

I guess I was thinking of testing from the programmer's point of view, but I shouldn't have slighted UA.