The Design Is Never Right The First Time

tacoma_narrows The design is never right the first time. Nor is it usually right the second time. This is one of those key lessons you learn over time building real software, and is the source of one of the chief complaints leveled at the Waterfall methodology for building software, which puts heavy emphasis on getting the design right up front.

We have to define “right” in this case. If by “right” you mean perfect, then I don’t think the design is ever right, as I’ve argued in the past that there is no perfect design.

Recently, this lesson was driven home as my feature team (ASP.NET MVC) were hashing out some tricky design issues. Many times, we find ourselves in a tight spot in which we have two different audiences demanding a different approach to a design problem. The tough part is that the solutions we come up with inevitable satisfy one camp at the expense of the other.

Let me be clear, it’s not our goal and intention to try and satisfy everyone as that is a sure means to satisfying nobody. As Bill Cosby once said...

I don’t know the key to success, but the key to failure is to trying to please everybody.

At the same time, I also think it’s a copout to simply give up immediately when a thorny design problem comes up with two disagreeing camps. It’s worth a good faith effort to try and gather ideas you haven’t considered before and see if you can’t just solve the problem in a manner that both camps are happy with. Or at least in a manner that one camp is happy, and the other is merely less happy.

An illustration of this point happened recently at work. After many iterations, I sent out our design meeting notes internally with our solution to a particular issue. ScottGu noticed the write-up and had some questions about the design (questions like WTF?) and called me and Eilon (the lead dev) into his office for a “short” meeting to discuss these recent design changes.

I made sure to call my wife before the meeting to let her know I would be late for dinner. Once we get a talking about MVC, I know it won’t be short. We spent two hours hashing out the design, going over options, shooting them down, trying out other ideas, and so on.

At some point, Scott proposed something my team had rejected earlier due to a major flaw. However this time, as he was proposing the idea, it suddenly occurred to me that there were two different ways to implement this idea. The one we thought of wouldn’t work, but a slight adjustment to the idea, the very one Scott was proposing, would make it workable.

Eilon and I left the meeting happy with the improved design, but now concerned with the fact that we had already implemented another solution, had a bunch of unit tests and automated web functional tests. Not to mention having to deal with explaining it to the team and getting buy in so close to our release deadline.

So we did the cowboy thing (which I generally don’t recommend) and decided instead of telling the rest of the team about it, we’d follow the old maxim: “Show, don’t tell”. So that night, we went and coded the whole implementation, updated all the unit and web tests, and had something to demo in the morning rather than talk about. Since we had updated all the tests, QA was much more amenable to signing off on making this change so close to the deadline.

In retrospect, the part that causes me pain in the missed opportunities sense was just how tantalizingly close we were to having implemented this very idea from the beginning. All that time spent prototyping, app building, etc... we could have saved by getting it right the first time.

But that’s the way design goes. Sometimes, you are so close to a great solution that if the wind blows a certain way, you’ll hit upon it, but if it blows another way, you dismiss your line of thought and move onto other plans. Try as you might, it is very difficult to get the design just right the very first time all of the time.

Luckily for us, our team had the agility to respond to a better design. Our suite of unit and integration tests gave us confidence that the changes we were making were not breaking the framework in other areas. To me, this is a real testament to the benefits of applying lean and agile principles in software development, as well as the benefits of having good unit test coverage.

What others have said

Requesting Gravatar... Chris May 31, 2008 4:29 PM
# re: The Design Is Never Right The First Time

any chance you could let us know which feature of the MVC framework this relates to?

Requesting Gravatar... Steve May 31, 2008 4:39 PM
# re: The Design Is Never Right The First Time
lol, yeah, throw us a bone :)

I suspect it has to do with the returning of the different 'result' object (Action, Content, Json)
Requesting Gravatar... Javier Lozano May 31, 2008 6:02 PM
# re: The Design Is Never Right The First Time
These sort of things is what keeps me in this field; there is no "right" answer.

I do agree that it is a pretty awesome feeling being able to change the design of the framework with a true knowledge that nothing is broken (thanks to your unit tests).
Requesting Gravatar... Mojtaba May 31, 2008 9:24 PM
# re: The Design Is Never Right The First Time
i think that design will always change and so i as a developer should be flexible to changes.so i use methods to damp the effect of changes such as using my own code generator
Requesting Gravatar... Robert G May 31, 2008 10:49 PM
# re: The Design Is Never Right The First Time
Phil, your post made me think of the idea of code ownership. Several years ago another team went through the same process (discussed the design to death, went ahead and implemented agreed upon solution). The team lead thought of an arguably better way and in the spirit of "show, don't tell" re-implemented the solution overnight with another dev. This was about 3 days before the milestone release. I was not on the team, but was sitting in a cubicle among them. So next morning I watched all hell break loose, as some people (who implemented the original solution) could not deal with the fact that someone took their code ownership away. To them, it was plain disrespect. Within weeks the team was history, people simply could no longer work together.

Anyway, I am glad your team was able to handle the change.
Requesting Gravatar... Nick Berardi Jun 01, 2008 5:38 AM
# re: The Design Is Never Right The First Time
Hi Phil,

Would you mind sharing the MVC feature with all of us. I would love to go in to the source code and see the outcome.
Requesting Gravatar... Tommy Jun 01, 2008 6:40 AM
# re: The Design Is Never Right The First Time
Hehe. So many of us wanna know what feature and code got changed as a result of SG's "outbreak". Nice post Phil! It's good to hear about this, because so many of us deal with these problems but probably keep thinking it is not supposed to be like that. "One day I will nail that perfect design the first time, one day!"

Our team are building an application from scratch and swapped out the web UI with ASP.Net Mvc from webforms. We worked evenings and weekends and when we felt confident that Mvc was the way to go, we replaced the entire UI with Mvc. A somewhat design change. ;-)
Requesting Gravatar... Leandro M Jun 02, 2008 6:42 AM
# re: The Design Is Never Right The First Time
Great point !
Could you please send an email to my bosses and tell them that ?
Requesting Gravatar... howardw Jun 02, 2008 9:48 AM
# re: The Design Is Never Right The First Time
The fact that your last-minute changes passed all the regression tests reduces the probability that you broke something by a factor of "N." The factor N depends on a huge number of things, but based on personal experience I'd say as large as 10 would be unusual and somewhere between 2 and 4 would be typical.
Requesting Gravatar... Michael Jun 02, 2008 10:53 AM
# re: The Design Is Never Right The First Time
I think the fact that the change came out of a meeting with ScottG makes a difference. I mean sometimes you have to suck it up and deal.
Requesting Gravatar... Eilon Lipton Jun 02, 2008 12:47 PM
# re: The Design Is Never Right The First Time
The feature had to do with ViewData - specifically, how we carry around strongly-typed ViewData for use with ViewPage<T>.
Requesting Gravatar... Peter Jun 02, 2008 8:11 PM
# re: The Design Is Never Right The First Time
I dove into MVC for a few days now, and my two main impressions are:

1) Strings = BAD, using generics / lambda fixes this, but it doesn't work in all places yet (generics ie).
2) Getting data out of form, responding to server side error, or validation is something that needs work.

Peter.
Requesting Gravatar... ShaharY Jun 02, 2008 10:58 PM
# re: The Design Is Never Right The First Time
It is OK that the design won't be right the first time, but it should be good at the first time.
Explanation: the fact that you could change the implementation over night tells me that your initial design was very good. It is not that simple to change the design and the implementation so close to the deadline... I know some projects with such a bad design where you can't replace some code just like that, it will take at least a month to do it...
Requesting Gravatar... JimS Jun 03, 2008 11:47 PM
# re: The Design Is Never Right The First Time
I just hope the new design was proposed as a new alternative to the existing as in "Hey, have a look at this. Do you think this could be a better solution?" and not as a "We redesigned the system, this is how it is from now on..." - the latter would just glow disrespect all over!
Requesting Gravatar... John Jun 04, 2008 6:26 AM
# re: The Design Is Never Right The First Time
Microsoft's new round of anti-MS hires sure come off as arrogant and self-righteous sometimes, don't they? I'd hate to be a grunt on that team. I wonder what the mainstream people at MS (ie. webforms, core language, etc) think of the MVC group's posturing. :P
Requesting Gravatar... Haacked Jun 04, 2008 4:01 PM
# re: The Design Is Never Right The First Time
The reason I didn't point out the original design problem is I didn't want it to detract from the main point of the post.
Requesting Gravatar... Haacked Jun 04, 2008 4:02 PM
# re: The Design Is Never Right The First Time
@John what do you mean? I thought this was a rather self effacing post. Would you like to clarify what you mean by that?
Requesting Gravatar... secretGeek Jun 05, 2008 8:35 PM
# re: The Design Is Never Right The First Time
@John: "I'd hate to be a grunt on that team."

It sounds like the sort of team where no-one is allowed to get away with being "just a grunt" -- but instead people are empowered and allowed to bring about immediate change.

They back it up not with "arrogance" but with "passing unit tests".

No wait, I take it back. Phil's a poser ;-)
Requesting Gravatar... lb Jun 06, 2008 7:24 AM
# re: The Design Is Never Right The First Time
A comment from Rick Brewster (Paint.net) at coding horror (http://www.codinghorror.com/blog/archives/000980.html)

I read something about an old version of Google Groups (I think that's what it was). The developers were trying to choose between two ways of sorting or organizing the data, and they couldn't come to a consensus about which one they should go with. So they shipped a beta with NEITHER. They then got 100 e-mails asking for method A, and only 1 asking for method B. That made the decison clear to them: they went ahead and implemented method A.

Rick Brewster on October 26, 2007 11:36 AM
Requesting Gravatar... the case for sanity in IT Jun 11, 2008 11:07 PM
# re: The Design Is Never Right The First Time

Luckily for us, our team had the agility to respond to a better design.

No your team does not. Your team has no such thing: in order to accomplish your task, you and the other guys who "did the cowboy thing" and "coded the whole implementation, updated all the unit and web tests".

You had to give up something in order to do this: you gave up your free time that night. You called your wife to tell her you'd be late to dinner; given the progression of events that night, did you in fact make it to dinner at all? How does she feel about that? You have a child: how is he impacted by your failure to come home at a reasonable hour? How is your Subtext Project affected?

Your team is not so "agile" in the business sense (I do not mean "agile" in the technical sense; this clearly is the case: you got away with making major design changes without, seemingly, doing documentation. That's classic "agile" development: write the code and don't give a damn about whether anybody else ever has to maintain it) that you could not work on the changes you made that night during the next day instead.

In few other professional fields (law, medicine, and business at the level of senior/strategic management are exceptions) do people avail themselves of their work at the expense of their own personal well-being and the well-being of friends and family as we do in IT. That said, I am not at all surprised that a Sr. Program Manager at MSFT would work such ridiculous hours (I have worked similar hours, and I think it's high-time we in IT assert our need for free time for our own purposes, whatever they may be)...

BTW, I agree that designs are rarely right the first time; that is why I am a TDDer (modified to include documentation, that is). :-)
Requesting Gravatar... haacked Jun 12, 2008 9:05 AM
# re: The Design Is Never Right The First Time
@thecaseforsanity You don't have the full context to make such accusations. I can't imagine any industry in which employees nver work a little later once in a while. This was a CHOICE we made in order to meet a preview deadline because we value the feedback. We didn't have to do it. We could've waited and got this code in the next iteration. This wasn't for a release product.

I think that's the key difference here from other cases of insane project schedules. We had a choice in this matter.

Also, I more than compensated by going home early the next few days. If anything, I've found Microsoft to be much more work-life balanced than almost anywhere I worked (except maybe Univision, but I was bored there). Partly because I've always worked at consulting companies or startups. ;)

Today for example, I have a softball game in the middle of the day, a Dr. appt for my son, and a soccer game in the evening. I won't be at work much today. :)
Requesting Gravatar... haacked Jun 12, 2008 9:06 AM
# re: The Design Is Never Right The First Time
p.s. I did update the specs as part of this late night coding section, so I did document it.
Requesting Gravatar... the case for sanity in IT Jun 12, 2008 6:42 PM
# re: The Design Is Never Right The First Time
Ah, fair enough; my apologies where my concerns are based on incomplete information - I made a few assumptions based on what I've observed as common practice in IT.

Carry on then. :)
Requesting Gravatar... Manuel Castro Jun 13, 2008 11:22 AM
# re: The Design Is Never Right The First Time
The obsession with design-all-first methodologies do not have a place today in modern software building, because the refactoring and unit testing tools that we have today let us be flexible without compromising the solution.
Requesting Gravatar... Senthil Kumar Jan 27, 2009 8:56 PM
# re: The Design Is Never Right The First Time
Hi Phil,

The Blog is very informative.

Thanks and Expecting to read more of your Blogs

What do you have to say?

(will show your gravatar)
Please add 1 and 8 and type the answer here: