One of the questions I’ve been asked after yesterday’s blog post about Phix’s ContractLib is why not just use PHP’s built-in assert() function? I think it’s a great question, and the best way to answer it is to take a look at the key differences between two solutions.
Side By Side Comparison
| Feature | assert() | ContractLib |
|---|---|---|
| Implementation | PHP extension written in C (ships as standard part of PHP) | PHP library written in PHP |
| Enable / disable execution | Partial (there is an overhead when disabled, but it’s low) | Partial (there is an overhead when disabled, but it’s higher) |
| Issues PHP4-style warning when tests fail | Yes (configurable) | No (throws a ContractFailedException instead) |
| Terminate PHP script when tests fail | Yes (configurable) | Only if the ContractFailedException is never caught |
| Quiet eval of test expression | Yes (configurable) | No (not required; test expressions are pure PHP code, not eval() strings) |
| Callback on failed test | Yes (configurable) | No (unwinds the stack instead by throwing ContractFailedException) |
| Throws Exception when tests fail | No (but can emulate if you write your own assert() callback method) | Yes (standard behaviour) |
| Tests are pure PHP code | No – recommended way is to pass strings into assert() to be eval()’d | Yes |
| Error report includes original value that failed the test | No | Yes |
| Support for per-test custom failure messages | No | Yes – are required to provide one |
| Support for Old Value storage and recall | No (but can emulate by hand) | Yes |
The Differences Explained
The key difference is one of philosophy. assert() sits well with the PHP4 style of error reporting and handling, whereas ContractLib is firmly in favour of the OO world’s approach to reporting errors.
It’s a personal preference, but I think that PHP4-style errors have no place in code that has any desire to be robust. Exceptions aren’t perfect, don’t get me wrong, but their core property of unwinding the call stack in an orderly fashion makes writing robust code much easier. And they also carry a payload – information about what went wrong and why – which PHP’s assert() cannot provide to the same extent.
It’s much quicker to debug something when there’s a record of the value that failed the test. For that reason alone, I’d always prefer something like ContractLib over the built-in assert() approach.
But we can’t ignore the fact that these are tests that get shipped to, and executed in, the production environment. Unlike unit tests, adopting programming by contract will slow down your PHP code in production. The question is: by how much?
What About The Performance?
I’ve done some benchmarking between the two, using the five tests listed in the final example in yesterday’s blog post. It’s a real-world example of the kind of tests that I would look to add to code to improve robustness.
Here are the results I gathered, calling the tests 10,000 times in a tight loop. The tests were run from the command line, and the times do include PHP start-up / shutdown time and the time taken to parse each test file. I assumed a best-case scenario, where the tests would always pass.
| Test Approach | Time w/ Tests Disabled | Time w/ Tests Enabled |
|---|---|---|
| Tests written using assert() | 1.103s (100%) | 5.989s (543%) |
| Tests written using ContractLib | 3.055s (277%) | 3.096s (281%) |
When tests are disabled, using assert() is much cheaper than using ContractLib today. That’s to be expected, as assert() is written in C. I imagine that we could get close to the same performance if ContractLib was rewritten in C as a PHP extension.
But, when tests are enabled, assert() is much slower than ContractLib. Why? Because the recommended way to use assert() is to pass the test in as a string. PHP has to convert that string into bytecode to execute, and that conversion appears to be quite expensive.
Given the choice, I’d rather trade things running a little slower in production for having much faster tests when I’m writing code, and that’s why I created ContractLib. Plus I get much better information to understand why the test failed, and if I wanted to run the tests in production, I can handle their failures in a much saner way.
Final Words
In my experience, the time it takes to develop and ship code is normally more critical than how fast the code runs in production. Developer time has become a scarcer resource than CPU time.
Used intelligently, these kinds of tests in your code can help your team deliver quicker, because the code they are using and reusing is more robust first time around. Programming by contract is different to, and complements, unit testing because contract tests catch errors in using the code.
Whether you use ContractLib, assert(), or you create your own solution, you should really consider how much it is costing you when you don’t use these kinds of tests.