Do not waste time pre-optimizing your code, but don't pre-pessimize it either!

Regardless of what language you are programming in and what system you are designing, until you have measured performance you cannot know what needs optimization beyond what the compiler already provides out of the box. That is perhaps a rather bold statement. It is true in my experience and I hope you will at least consider its truth if not agree with me by the end of this post. 

Donald Knuth in his paper entitled Structured Programming with go to Statements says
"There is no doubt that the grail of efficiency leads to abuse. Programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs, and these attempts at efficiency actually have a strong negative impact when debugging and maintenance are considered. We should forget about small efficiencies, say about 97% of the time: pre- mature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3 %. A good programmer will not be lulled into complacency by such reasoning, he will be wise to look carefully at the critical code; but only after that code has been identified. It is often a mistake to make a priori judgments about what parts of a program are really critical, since the universal experience of programmers who have been using measurement tools has been that their intuitive guesses fail."

Note: There is discussion on the internet that part or all of that quote should be attributed to C. A. R. Hoare. However that is incorrect as can be seen here. Given this is the internet we may never know who first said it, regardless of who should get credit for it, it is still good advice.

It is important to read that carefully. Many find the last line hard to believe and to consistently apply. My experience in this area parallels that of Mr. Knuth. "The universal experience of programmers who have been using measurement tools has been that their intuitive guesses fail" The only way to know what needs to be optimized is to measure and then only optimize the part of the system that needs it! One must measure the complete path and then find the piece that needs to be optimized.

Just because we know some code will often be executed, we don't know if improving the performance of that code will actually change the performance of the overall system. There may be other pieces of the system whose performance overshadows the performance of that piece of code. It could be that network hops, or latency or some other piece of code or component of the architecture needs to be addressed first to improve performance. Performance improvement is a cycle of measure, find the bottleneck, optimize to remove the bottleneck, and measure again. If the system (not the piece of code) is still not performant enough repeat the cycle until it is.

I have watched many programmers spend a lot of time optimizing code, because they can make it faster, not because there was a net benefit to the system. In many cases the optimization hurt the system because the optimization was at a micro level, it negatively impacted readability and therefore maintainability of the code and sometimes it prevented a macro optimization across a larger swath of the system. This is what Knuth refers to when he said "the grail of efficiency leads to abuse".

So what is an optimization? An optimization is any change, made for performance reasons, that makes designs and/or code more complex and less readable, and therefore harder to maintain. If the change doesn't lessen readability, then it isn't an optimization.

In their book C++ Coding Standards Herb Sutter and Andrei Alexandrescu propose two guidelines

• Don't optimize prematurely
• Don't pessimize prematurely

The first guideline is exactly what we have been discussing. It is very important to apply the second rule especially when it comes to design. In code this means use the idioms and practices of the language which are usually naturally efficient. If those were not used, changing the code to use those idioms will often increase performance but since using the idioms will increase the readability of the code it is not an optimization. We need to make sure we apply the second guideline to designs as well as code. Building something that will not scale to the required usage would be an instance of pre-pessizming.

There are many articles on the internet discussing the fallacy of the "evil of pre-optimization" in every cases I have seen to date they are really discussing issues that are cases of pre-pessimization including cases where the developers are not yet competent enough to know what the correct performant code and design idioms of the language in use are and there are no experienced developers to guide them.

Do not pre-optimize or pre-pessimize! Using the definition of optimization above it is clear how to differentiate. I have followed these guidelines for many years with great success at creating solid systems and only optimizing the parts that truly needed it. I hope you find these guidelines equally as helpful.