Premature Optimization - What is it and why does it sound bad?
"Premature Optimization" is a term I have heard quite a bit from Peter Bell and it popped up again today in comment on Brian Kotek's site.
The comment on Brian's blog made me say to myself, "Self, what constitutes 'premature optimization' (PO)?" From what I can gather, there are times when it is perfectly OK to write code that is efficient and follows best practices, and other times, its considered PO.
There are certain things I think everyone agrees with when it comes to qualifying as PO. For example, avoiding evaluate() whenever possible seems to be universally accepted as good coding practice as opposed to PO. Using 'Duck Typing' in your objects without knowing if a performance issue exists seems to be universally accepted as PO as opposed to good coding practice.
In a recent comment on one of his own posts, Peter Bell, states:
I mentioned the performance benefits of using cfqueryparam, but I'm always leery of premature optimization, so I wouldn't use cfqueryparam for performance reasons unless I found a performance bottleneck with my db queries.
OK, now here is where the line gets a little fuzzy. The use of <cfqueryparam> for performance, as well as security, benefits is PO? Following that train of thought, wouldn't the use of singletons in OOP also be PO? I mean, why not create a create a new object every time you need it if you are not having performance issues? One could also argue that unless you see performance issues, go right ahead and evaluate() to your heart's content.
When I am coding, I try to make my code as efficient as possible. Would some consider my coding practices to be PO? I am certain there would be. However, I like to think of PO as stuff like tuning your JVM or using 'duck typing' when no performance issues exist. I also think that using <cfqueryparam>, even if only for performance, is NOT PO.
Can you share other examples of what you think are best coding practices, but others consider PO, or even examples of what you consider PO that others consider good coding practices?
That came out as "you shouldn't use cfqueryparam purely for performance purposes" which I kind of support as for a lot of apps its impact on performance will be nominal, but if someone were to say they thought that cfqueryparam was just a best practice (even ignoring the security benefits) I wouldn't really disagree.
For me, there are a handful of no brainer issues to keep an eye out for. For example, I think Singletons should be used where appropriate and you've always got to look out for potential n+1 query issues.
I'm actually on the other side of the eval argument. It allows me to code more dynamically and while I don't use it much these days (mostly a deeper knowledge of the language allows you to do the same with other syntax) I do find the occasional case where it makes my code much simpler and so evaluate is a valuable part of my toolkit and my apps are still standing (and running) just fine!
A more generic term for premature optimization is simply inappropriate priorities. Most of the time your goal should be to write maintainable, flexible code to solve the business problem as elegantly and quickly as posisble. Balance of elegance, speed of development and flexibility will depend on situation. For many use cases performance should be fairly low down the list of priorities, so spending a bunch of time thinking about performance rather than focusing on functionality and maintainability would be PO in my mind.
Looking forward to other peoples comments, as a list of "possible" issues to consider early stage (a quick and dirty list of some of the most likely performance bottlenecks) would be a useful guide as long as you didn't spend too much time thinking about it!
BTW, great to finally meet you the other week at Frameworks!
Once you have this design ready, build it. And if you find yourself stopping in the middle of it to ponder "am I creating too many objects in this request", that's premature optimization. Most of the time, you can just leave it and keep going. There will be plenty of time to test and refactor to tweak things later.
In my experience, when an app has performance problems, it is almost guaranteed to be a SQL or database design issue. I'm talking 95% of the time. So when performance issues DO arise, this is the first place I go hunting. Remember the old adage that there is only one bottleneck in a system at a time, and if you aren't working on that one issue, you're mostly just wasting your time.
Now obviously, this doesn't give license to throw best-practices out the window. Use common sense. And this is probably the grey area that only becomes known through experience. Overall though, I'd say the message is: build what you designed, and wait for testing to reveal any actual performance issues before you start swinging in the dark. If you aren't basing your performance tweaks on good test data, you're very likely tweaking the wrong thing anyway.
For me premature optimization is when you make your code less readable and less maintainable for the sake of performance before you have had a performance problem.
Certainly, if you *know* something will cause a performance problem that is different. By problem, I mean something that will noticeably affect the user - not merely cost milliseconds.
Said simply, premature optimization is making your code ugly for the sake of performance without having a known performance problem.
Sean has a nice list of best-practices that his team at Adobe uses, and I agree with just about all of them. Most OO design principles and patterns (when used in the right context of course) would also fall into this group. So I don't think there's much there that would stun anyone.
I suppose the difference is between the "gee I wonder if that might cause some performance hit" and "holy crap that is going to blow out the JVM". Don't fix the former until you are sure there actually is a problem, but definitely fix the latter!
It's actually taking from a quote by Donald Knuth who said: "Premature Optimization is the root of all evil" (http://en.wikipedia.org/wiki/Donald_Knuth).
I think that creating clean, maintainable code is a worthy and important goal - but as a user, I constantly find myself wishing that the people who developed the tools I use had done so with more of an eye on performance, and maybe a little less on creating a masterwork of architecture.
As an aside, I think the notion of "Premature Optimization" is a little bit dangerous. There are plenty of developers who would use it as a smoke-screen to avoid doing necessary and sometimes difficult work. "Just add another server!!" :)
I know what you're saying, but in the general case (and the devil is in the details) I disagree. In my experience, the only time it is worth optimizing the heck out of a server is if you have a site right at the limits of the machine it is on that was never designed to be scaled across a web farm so the cost of adding the second server is huge - a major rewrite.
Otherwise $2k for a passable box (or $5k if you're an enterprise guy), $1200 for CF Pro and $100/mo co-lo is a pretty small cost when compared to the number of man hours at $50/hr in additional optimization and then the additional costs of maintaining an app that was designed for performance rather than maintainability.
I also have to question why you'd even consider using CF for a performance critical app - even Java is a little sluggish. If you want bare metal performance and think your time is worth less than the cost of a server, go for it and write the darn app in C++!
I would say that caching (regardless of using session/application/cachedwithin etc) would qualify as premature optimization. It can cause development frustration by staring at the screen wondering "why the hell isn't this working??!" A couple minutes (hours?) later.... "oh right, it's cached!"
It seems to me that if an app can't run relatively fast without caching (while running it locally):
1) You have a DB with more than say 500,000+ records (this is an exception)
2) There is an underlying flaw in the design
I just always find it both annoying and humorous when i randomly come across global cached queries that would normally run in 0ms uncached. I think this is the perfect definition of "premature optimization".
But then, on the other hand, if you are not quite sure where the code is going yet, you might not want to care about performance as you want to concentrate on just getting code to work first, then go back and optimize.
So, really, I think what it comes down to is being able to relate your new problem to previously solved problem (does this sound strangely like Design Patterns???). If its a related situation, I do NOT think it is PO to apply previously tested solutions. However, if your idea is new enough that you don't even know if it is plausible, then by all means, just get the damn thing working THEN go back and optimize once your concept has been proven.
For instance, I think of ListFind() vs. Struct look up. ListFind() takes string comparisons which can be pricey. Struct key loop up is blazingly fast. So, know this, does it makes sense to create an algorithm that converts a list to a struct w/ keys (of list items) even though it would be perhaps less readable that a bunch of ListFind() calls?
I would argue that NO, that is not PO as it is a proven situation that I have had to deal with before.... just my 2 cents.
That is the essence of it for me. But, I would also add what Knuth implied - that in solving problems that don't exist, you are wasting time and effort.
See also TSTTCPW and YAGNI. Violations of which, to me, can constitute premature optimization.
Finally, Ben, I would disagree with you about the list versus struct issue. I assume you've already considered the cost of creating the structs from the list, and that you still find it faster than using listfind. That is fine, but in my experience, I've never seen a problem with listfind() being too slow. And I use lists just about everywhere. Of course, your mileage may very, and if you find it useful, then by all means use it! =)
With that being said though, it would be more acceptable (to me as a reader of the code) if you abstracted it (I can't tell if you did) and made it look like:
fastListFind(list, "string")
Then, from the reader's point of view, it doesn't muck it up and make it hard to understand the intent. But, I wouldn't have written it until I noticed I was having a performance problem from listfind, and then I'd also have to notice a significant increase in performance after having written it, or I'd chunk it.
Sorry, I didn't mean to imply that Struct looks up are inherently faster than ListFind() all the time. In my mind I had a scenario that would require many list finds on the SAME set of data within one algorithm. This is something where the overhead of the struct creation would be negligible when compared to the many repeat-list find situations.
But that was just an example... my point was that it is not PO if you are applying a solution that you have found useful in the past.
Thanks for clarifying. The way I read it was that you were using the struct method all the time. I do agree with applying similar solutions to similar problems, though. So, if you were running up against a huge data set like you mention and it worked before, and you would be running against a new huge data set, I don't think I would call that premature optimization either.
In fact, you've already experienced the poor performance the first (or first few) times you did something similar, so it wouldn't be premature at all.
Thanks again for clarifying!
Seems to me that in a project where the client is balking at the price of the server is one where performance optimization is likely not in the budget at all.
It makes sense to optimize and code that is known to cause a problem (again, being defined as something that is significant enough to affect the user). Trying to write every single line in the code in the most performant manner doesn't.
Spending time worrying about the performance difference between isDefined() and StructKeyExists() is rarely worthwhile. Rewriting a query that takes five minutes is a different story.
Peter, this part of your comment inspired a blog post for me. :)
http://devnulled.com/content/2007/02/correcting-lo...
Developmnet best practices items I don't usualy consider PO.(as long as they do not greatly increase initial development time. Generaly if you cannot show a need for optimization (ie: in this situation the code is slow) then you should not be optimizing the code. More offten than not you will probably spend more time optimizing the results than total time saved. If you spend one hour optimizing a page to gain 100ms faster result it will take 36000 page loads to equal the amount of time you spend on that page. If that hour was spend on something that took 10 seconds to process and you reduced it to 1 second savings of 9 seconds then time save vs time spent would be recoved in only 400 pages loads. Plus you have made a noticable difference to the end user where 100ms is not usualy noticed.
That said 8 out of 10 times that I find a page that can be reduced from 10s to 1s the majority of the changes are all good coding practices vs "optimization tricks".
Examples: using good indexs on tables.
Not running a query in side of a loop
One of the worst pages performance wise that I have optimized all of the optimization done was chaning the code to use known best practices for coding. The page initialy took 10+ mins to render after implementation of good coding practices it took less than 1second to generate. None of the changes are what I would call optimizing. No caching was added no special code was added just using good changing so that it used good coding practices.
When you find that you have a speed problem and the code is doing all the standard best practices is when true optimization comes into play and you start looking at things to change in the code to improve performance. Such as adding caching possibly going against some best practices that are hindering the speed of the application (example: removing required and variable types attributes of arguments on functions). These are optimization type activities in my mind and they should only be done where you have a proven performance problem. That is not to say wait for the issue to come up but put time in your development cycle to do performance tuning. This should be a seperate step from developing code. If you find something that is always slow for your application and there is a standard way of doing it then it eventualy becomes a best practice and then you will not need to optimize those items as it is done during standard development. But as I mentioned above you should look at cost vs benefit when doing optimization. If there is little to gain in optimization then chances are it is not worth the time to do the optimization.
I think using SELECT * is good enough while developing.
Going back and entering all the field names is a non-trivial process, but coding the requirements is even more non-trivial.
1) make it work
2) make it good
3) make it fast
#1 is obvious. #2, especially in the CF community as compared to OO languages with more standards, means many different things to many different people. And #3, as most seem to suggest in these posts, should only be done when a performance problem rears it's ugly head or if you *know* from previous experience that one method is slower in certain situations than another. Unfortunately, in the real world #3 gets dropped completely until CF/Jrun come crashing down on the live sites, causing hours of outage and untold amounts of $$$ lost (depending on the traffic/sales of the site of course).
#2 is not a black & white issue. Ben Forta will say something much different about "best coding practices" than a CF newbie. In fact, Forta, Corfield, Camden, and whatever big names you want to throw out there, might believe completely different things about whether a "framework" should be used, when to use CFCs vs. CustomTags, whether to push all cfqueries to stored procs, etc. To further muddy the waters, it depends on if you're a one, two, 10, or 100 man shop... is everyone on the team CF gurus or are some newbies? Is the development all done by the CF developers, or is there an architect, 2 DBA, some "heavy-lifting apps" that can be done by the Java developers, and a UI team that deals with XHTML, CSS, and JS? Does the app need to be portable (i.e. are you selling it as software?)... does it need to work with multiple RDBMSs, should it work on Adobe CF 5, 6.1, & 7, as well as BD, BD .NET, etc.? First, answer all these questions. Then, you'll probably know what's the "best" approach to the coding of that particular app or enterprise. If you don't have answer at that point, get some advise from someone who's been there before.
#3 is also not a black & white issue either, but I'll put in my $.02 from the perspective of a developer who has been working in shops with 5 - 10 senior CF developers the past 3 years. This may contrast with the other posts in some ways because most of them may be more used to working in smaller teams. First of all, in a team the "Premature" in PO is only applicable if you're trying to optimize code that you're not familiar with *and* you don't know that it's a performance problem. Otherwise, if you're building something as part of a large, high-traffic enterprise app then I wouldn't even consider your code "done" unless it accomplishes #1, 2, & 3. Enterprise apps are often useless if they can't be easily understood by the other developers (a requirement for #2 in my mind) or if they don't scale to at least hundreds of concurrent users. In enterprise shops, thinking about the scalability and performance of even small pieces of code *up front* (consult the architect) saves many hours of time down the line, and may be the difference between it scaling to thousands of hits/per day and *millions*.
An example of a small tweak most of us would have never guessed that made a huge different under heavy load: http://www.webapper.net/index.cfm/2006/7/27/200607... I've filed that away in my list of "best practices", and whenever I'm working on an enterprise app that needs to scale at at, I will use cfif-cfelseif-cfelse blocks for string expressions instead of cfswitch...
In short, it all depends on what you're doing, with whom, and how well it needs to scale (try to think at least 2 years into the future). ;-)