This is a republication of a column I wrote for Linux Journal that’s generated a lot of fascinating feedback, about the siren song of perfection and how it can stall and even kill projects, whether software or even entrepreneurial businesses. I would love your thoughts and feedback on this too!
Last month marked the end of my series about writing a blackjack game as a shell script, and I don’t know about you but I had a good time with the development process and have even learned a bit more about the game itself. I received a number of fun email messages from readers about the column, but I also received one that I thought was most thought-provoking.
The author criticized me for using less than optimal algorithms for things like my shuffle routine, of using poor scripting style and generally questioned how much I really knew about shell script programming in the first place.
ANOTHER DAY, ANOTHER FLAME
Having lived on the Internet for almost 30 years now, I’m quite familiar with flames and hostile email, with people nitpicking, focusing on the molecules of the leaf without ever even knowing that there’s a forest ahead, but this message still got me thinking about the practice of scripting and of programming in general.
To quote a friend and colleague of mine, Ken McCarthy, what’s a better strategy, imperfect action or perfect inaction?
While the Bourne Again Shell is remarkably capable and certainly has all the basic programmatic structures of more sophisticated programming languages, I think it’s nonetheless fair to say that it’s a lightweight, even throwaway programming environment. You don’t write large, complex or mission-critical applications as shell scripts, do you?
That’s how I have always approached shell script writing, basically as a fast prototyping environment. You want to know how many lines are in a file? Use something like
lines=$(wc -l < $filename)
Is that the most elegant and efficient solution? Probably not. Indeed, if you’re doing that to test if the file has non-zero content, you should indeed be using the “test” command instead, but here’s my point: it doesn’t really matter.
That’s what I mean by imperfect action. It’s far, far better to get going with your script, to build a sloppy prototype, to get it done, than to tune, clean up, tweak, rewrite and optimize until it’s 3am two weeks after your deadline. That’s perfect inaction, right? The zeal to wait and wait and tweak and prod until it really is perfect, by which time you’ve missed your deadlines and goals.
So when I received the criticism from this reader that I hadn’t chosen the best possible algorithms and wasn’t using what they thought was optimal scripting technique, I was glad to see how they thought things should be done. But I was also unsurprised, as one of the greatest challenges I believe facing software developers is learning that in many cases and situations “good enough” really is, well, good enough.
WRITING A SHELL SCRIPT ISN’T DEVELOPING SOFTWARE
Am I advocating that the next time you’re writing the firmware for the in-flight controller on the Boeing 777 you should cut corners, skip testing and write crummy code so you can ship on time? Of course not. But y’know what? If you’re writing a testing framework cron script that will simply log the start of the test, invoke a series of MySQL queries, and log the end of the test, well, yeah, in that case relatively crummy code, code that works well enough might just be exactly what’s required.
I see this same perfectionist attitude with letters that I occasionally receive from people who have bought my Wicked Cool Shell Scripts book. They haven’t realized that writing a shell script is inherently an exercise in rough prototyping (hence the absence of sophisticated shell script development environments and testing frameworks), not a programming world where perfect code is the digital holy grail.
THE “NOT GOOD ENOUGH” DEVELOPER CROWD
I also think that these rather snobbish software developers who worship elegance and dismiss software developed by people who are still stumbling their way through programming are doing a significant disservice to the world of computing.
Don’t get me wrong, I appreciate an ingenious algorithm and snappy implementation when I read code, but most of the real innovations in software and applications come from the bubble gum and baling wire crowd anyway, from the rough prototypes and the “barely beta” software that works well enough to demonstrate concepts and get the community to start experimenting anyway.
SO I SHOULD BE WRITING SLOPPY CODE?
Some of you doubtless are a bit confused by this point in my column, puzzled that someone who is supposed to be teaching you tips and tricks of shell script programming is actually advocating sloppy coding and quick “throwaway” scripts. I ask instead why you’re surprised at the idea that getting something out the door, solving the problem quickly and reasonably well, is often better than wasting — yes, wasting — time writing “the perfect script”?
This reminds me of a computer programming course I took many years ago at UC San Diego. Our challenge was to figure out the optimal sorting algorithm for a given situation and write a program that implemented it. Just about everyone in the class thrashed about, but I pulled out Knuth’s Art of Computer Programming, picked out the algorithm he recommended, and typed it in, with an appropriate citation. I was penalized for “cheating” and had to argue adamantly that there was in fact no better way to learn how to pick the best sorting algorithm than to refer to the definitive work on the subject. Finally, the professor relented and gave me full credit.
Shell script programming is the same: shortcuts are always a good thing, efficiency is a measure of how fast you can solve a problem, and while tuning and tweaking can be rewarding as an intellectual exercise, 90% of the time it just doesn’t matter at the end of the day.
Think about that. And ask yourself how you’re working towards imperfect action rather than being trapped trying to achieve perfect inaction.
Next month we’ll go back to the nuts and bolts of shell scripting. But, please, don’t expect me to write perfect little scripts or use the absolute best algorithm in the world for a given task. Indeed, sometimes my code will be inefficient, will spawn more subshells or child processes than entirely necessary, or might even have unnecessary loops and conditionals. Maybe, just maybe, that’s okay…
I remember you, I, and argv debated this topic over email almost 20
years ago. I can appreciate your argument, but unfortunately I have
seen the dark side of it. Sometimes you get stuck with your good
enough solution because there’s no time/money/staff to build something
better. Then management changes and you lose your job because you
didn’t build the quality system you wanted to build in the first
place.