How do you unit test code that does nothing?

I been working on a CPU emulator in ActionScript recently more as an exercise in how to write good code, try some methodologies and tools than I've not been using in other projects and improve my understanding of the CPU's instruction set. Someone will hopefully find it useful when I open source it.

We all know that code should be tested and that best practice involves writing unit tests and/or test driven development. Both Flash Builder and Flex Builder make it easy to write and run unit tests with FlexUnit so there's really no excuse not to. Despite that is the Flex SDK itself not having unit tests unit tests and the countless projects I've been involved in for one reason or the other that don't have unit tests (usually the excuse is we don't have time or budget to write tests - doh!), but that's another story.

I'm writing the tests for the instruction set and most of it is fairly straight forward, test adding two numbers together, test if a register is zero etc and then I come across this instruction "NOP". For those who don't know machine code mnemonics NOP is an instruction that does nothing, it's used as padding or to eat a few CPU cycles. How do you test something that does nothing? It's not easy to test. You could take a snapshot of the memory before and after running the instruction and compare byte by byte and check that nothing has changed. I know that from looking at the function that it does nothing so in this case I can be fairly confident that it does what is says. However it did make me think about what unit tests can and should test.

Generally unit tests will only test for correct behaviour, a few boundary conditions or null parameters and the like but they don't often test for side effects or the completely unexpected. In most cases your code should be written to prevent any side effects but every now and then you going to run into some code that you just can't test 100% without a large amount of effort. The solution is to be pragmatic (unless you're writing software for controlling rockets or life support machines) just go on to write the next test, you'll waste far too much time trying to test something that may not happen and your time can be better spent writing other tests that add value.

Also remember because some code has been unit tested doesn't mean it will work 100% of the time. Units tests give you increased confidence that you code will do what you expect it to do but that's all. They will not test everything but having unit tests is far better than having none at all.

TweetBacks
Comments (Comment Moderation is enabled. Your comment will not appear until approved.)
You could have 2 suites of test; one that does a basic check of the items it _should_ be doing and another more exhaustive set that checks for things it _shouldn't_ be doing to make sure an instruction doesn't modify any of the memory or doesn't alter registers etc.

I guess like memtest you'd have to run the opcode with various memory or register states depending on what actions it performs - and then just leaves the tests running every night or week (if they're going to take a while); then you can be sure you're running the tests but not wasting precious development time.

One thing I do like but haven't set up at home is a continuous integration server that constantly checks out your code and tests as things change and then just emails you or notifies you in some way when it broke; obviously you'd need to make sure the latest code passed before a release but during development you could just leave it running.
# Posted By Chris | 10/18/10 9:42 PM
Two (or more) test suites are are good idea.

However my point was it not always easy to test everything (may vary based on the language you are coding in) and in some cases it probably best (in terms of effort) to move on to writing other tests especially when the method you're testing is so simple.

Of course the flip side is that sometime important bugs are missed for a long time because people think code is simple enough to not be tested. Like most things in writing code it's risk vs effort.

Unit test are never going to catch all bugs but certainly give you increased confidence that your code will work and make future refactoring/development so much easier.

Running tests usually doesn't take too long and if they do most test runners give you a way of running a sets of tests or select between different test suites so I don't see that as a major issue (on medium sized projects anyway).

I use Hudson (http://hudson-ci.org/) for continuous integration. It's quite easy to set up and can run it on your development machine to get familiar with it/test it out. It's straight forward to make it compile and run your unit tests on check in to your source control system.
# Posted By Justin Mclean | 10/19/10 12:37 AM