The Siren Song of Backwards Compatibility

aspnet 18 comments suggest edit

This post is sort of a continuation of my post on Microsoft’s New Running Shoes.

The Importance of Backwards Compatibility

If anyone tells you that backwards compatibility isn’t important, they’re wrong. And in fact, if they use any software long enough, they’ll tell you themselves. Another upgrade of some framework they depend on will break their application and they’ll get real care mad about it. I know because I’ve been on both sides of this river. I’ve shipped a Framework that broke people who told me we should break compatibility and experienced the heat of their anger. Usually, when someone tells you breaking compatibility is fine, they mean as long as it doesn’t affect them.

Microsoft is famous for its tenacious dedication to backwards compatibility. In his post, How Microsoft Lost the API War Spolsky highlights this comment from Raymond Chen, famous for his stories of the crazy lengths they went to maintain backwards compat.

Look at the scenario from the customer’s standpoint. You bought programs X, Y and Z. You then upgraded to Windows XP. Your computer now crashes randomly, and program Z doesn’t work at all. You’re going to tell your friends, “Don’t upgrade to Windows XP. It crashes randomly, and it’s not compatible with program Z.” Are you going to debug your system to determine that program X is causing the crashes, and that program Z doesn’t work because it is using undocumented window messages? Of course not. You’re going to return the Windows XP box for a refund. (You bought programs X, Y, and Z some months ago. The 30-day return policy no longer applies to them. The only thing you can return is Windows XP.)

I’ve been there. I’ve written applications for friends that I hope I never have to change again just to move it to a new server. I’ve also had times where I depended on some 3rd party code where I didn’t have the source and the author was long gone. If that code breaks because of an operating system upgrade, I’m in a world of hurt. It’s situations like these where Microsoft’s adherence to backwards compatibility is a sanity saver.

Backwards Compatibility is a Tax

But there’s another side to the backwards compatibility story. All of those benefits I mentioned have a cost. Backwards compatibility is a tax that creates significant drag on a team’s agility and its ability to innovate. A long time ago I wrote a post that suggested this blind adherence to backwards compatibility was holding Microsoft back.

Wait, so now I’ve argued both for and against backwards compatibility. Does it sound like I want to have it both ways? Well of course I do! But good design is a series of trade-offs and good execution is knowing when to make one trade-off vs the other. Nobody said it would be easy and straightforward to compete in the software industry and give users what they need.

For example, many discussions about this topic miss another key consideration. Technical compatibility isn’t the sole factor in backwards compatibility. For example, if a company isn’t able to innovate and have its product stay relevant, it might need to remove investment in the product, or worse, go out of business. This also breaks backwards compatibility.

In this case, compatibility isn’t broken by new versions of the product. It’s broken by how the world continues to change around the now stagnant unsupported version of the product. We’ve seen this happen not long after Microsoft pulled support for Windows XP, a zero-day exploit was discovered. Microsoft relented and patched it, but makes no promise to patch the next zero day.

At this point, users of the product have to make a decision, switch, or stay and risk the exploits.

ASP.NET vNext

I believe this is the situation that ASP.NET found itself in recent years. When I was part of the team several years ago, I worked on a new product called ASP.NET MVC. Though it was “new”, it was really just a layer on top of the existing 13 year-old, at the time, System.Web stack.

This code had accumulated a LOT of cruft and any change to it was a slow and tedious process that required a huge test effort on multiple operating systems etc. There were compatibility fixes so old we were quite convinced they paradoxically predated the advent of computers. There were even fixes I wasn’t sure anyone understood the code, but we were afraid to change it nonetheless.

Around this time, my manager, Scott Hunter (heretofore known as “The Hu” to complement Scott Guthrie who is known as “The Gu”) and I often day dreamed (as one does) about a complete rewrite of the stack. As a joke, I coined the name “ASP2.NET” as the moniker for this new stack. At the time, we were Don Quixotes dreaming the impossible dreams. The disruption to existing customers would be too great. Backwards compatibility is monarch! It could never happen!

Don Quixote charging the windmills by Dave Winer CC BY-SA 2.0

But the world changed around Microsoft. Node.js and many other modern web frameworks, unencumbered by years of compatibility drag, exploded on the scene. These frameworks felt fresh and lightweight. Meanwhile, as I mentioned in my last post, Azure’s business model created new incentives.

Azure provides an environment that is not limited to hosting .NET web applications. Azure makes money whether you host ASP.NET, NodeJs, or whatever. This is analogous to how the release of Office for iPad is a sign that Office will no longer help prop up Windows. Windows must live or die on its own merits.

In this new environment, ASP.NET was starting to show its age. Continue on its current course and it would risk complete irrelevance with the next generation of developers. It reached a crossroads where it had two possible strategies:

  1. Continue to invest in the existing stack and try to slow the bleeding as much as possible.
  2. Disrupt everything and build something new.

The first strategy is appealing. It makes existing customers feel comfortable and happy. Heck, it would be profitable for a very long time. But it’s ultimately not sustainable. It hamstrings Microsoft from making inroads with new developers not already on its stack. Also, eventually, companies switch from old technologies to newer platforms. It might be the result of the fact that they can’t hire developers to work on the old platforms. Or it might be that the software that meets their new business needs are on newer platforms. Either way, what happens when they are ready to make this switch? What platform would they choose?

ASP.NET wants to be that next platform. It needs to be that next modern platform. It might rattle existing customers a bit, but that’s a calculated risk. After all, if you’re an existing customer, you know you have 10 years of support on the current stack. It goes by fast, but it’s still a long time. You might be angry at having to make that switch at that point, but Microsoft’s betting that might happen anyways over time and they’re hoping to provide the next platform that you switch to. As Steve Jobs famously said,

If you don’t cannibalize yourself, someone else will.

Microsoft wants to be the zombie, not the zombie food.

I left Microsoft a little under two years ago, but Scott Hunter stayed on and continued to execute on the impossible dream with his team and folks like David Fowler, Louis De Jardin, and others. That’s why I’m pretty excited about ASP.NET vNext. It’s not just another flavor of the day framework from Microsoft. It represents a new modular approach that makes it easier to swap out the parts you don’t like and keep the parts you do. And it’s a sign that Microsoft is more and more taking a page from Steve Jobs playbook and solve the Innovator’s Dilemma,

My passion has been to build an enduring company where people were motivated to make great products. The products, not the profits, were the motivation. Sculley flipped these priorities to where the goal was to make money. It’s a subtle difference, but it ends up meaning everything.

Found a typo or error? Suggest an edit! If accepted, your contribution is listed automatically here.

Comments

avatar

18 responses

  1. Avatar for Frank Quednau
    Frank Quednau May 27th, 2014

    Backward compatibility has the potential to be the end of civilization, it is important to break it once in a while. Without change there is no future. Yes, you get frustrated by the additional work of adapting, but that's called life.

  2. Avatar for Eric J. Smith
    Eric J. Smith May 27th, 2014

    The only frustrating thing is that they already created a new stack with Web API which is only a couple years old at this point and that is being tossed out as well. I really like the new stack so I will suck it up, but I do wish they had just skipped the whole Web API thing altogether and caused us a little less pain.

  3. Avatar for haacked
    haacked May 27th, 2014

    Heh. The Web API thing was a great example of [Conway's Law in effect](http://haacked.com/archive/....

    The team that built that and the ASP.NET team was reorganized into the same larger team under Guthrie when I was still at Microsoft. That's why you saw a concerted effort to merge the two with ASP.NET MVC 4. They were too different for that to fully happen, but it was the start of that process.

    Now that Guthrie is in charge of all of Cloud and Server, I don't think you'll see that happen again.

  4. Avatar for Stilgar
    Stilgar May 28th, 2014

    Wait isn't Web API built on OWIN and therefore compatible with ASP.NET vNext?

  5. Avatar for Eric J. Smith
    Eric J. Smith May 28th, 2014

    Web API and vnext can both be run in OWIN, but they would be completely separate and running 2 different pipelines.

  6. Avatar for pm
    pm May 28th, 2014

    Well, in GNU world this problem has been resolved once and for all some decades ago. It is as simple as releasing source. Hope that MS or at least their clients will understand it some day ;)

  7. Avatar for Binary Worrier
    Binary Worrier May 28th, 2014

    Cloying smugness overload! Can't . . . see . . . words . . .

  8. Avatar for Ollie Riches
    Ollie Riches May 28th, 2014

    'I've shipped a Framework that broke people who told me we should break compatibility and experienced the heat of their anger.'

    this doesn't make sense...

  9. Avatar for boriscallens
    boriscallens May 28th, 2014

    As much as I love open source, personally maintaining each and every piece of software I consume is just impractical. Open source has many merits, but it is not the holy grail.
    If winXP were open source, you think the people now stuck on an unsupported OS would bother?

  10. Avatar for pm
    pm May 28th, 2014

    Open source and more generally free software is not about moving liability of maintaining whole software to each and every user. It is about _letting_ people change that annoying little piece of code that makes their OS and their APP incompatible at some point. Instead of forcing them to forego either the former or the latter (or more frequently both of them at the same time). All closed source software is just a cage, that you get into believing, that the cage owner won't close doors behind you somewhere in the future.
    I'm not willing to offence anyone, but for me it looks funny how people try to resolve problems that shouldn't exist at all in the first place. On the other side it is sad, how people waste their time struggling with obstacles brought to them by others - in the name of making business.

  11. Avatar for Stilgar
    Stilgar May 28th, 2014

    So wouldn't you be able to take existing WebAPI code and run it under vNext with minimum changes. From the announcement I got the impression that changes would be minor like changing a few namespaces.

  12. Avatar for James Chaldecott
    James Chaldecott May 28th, 2014

    You're being ironic, right?
    https://github.com/aspnet/home

    :-)

  13. Avatar for Eli Weinstock-Herman
    Eli Weinstock-Herman May 29th, 2014

    My feeling, as a consumer of these technologies, is that Microsoft has started abandoning backwards compatibility. I get that it it is hard, but the total lack of a bridge or upgrade path in many cases is a tax on the end users and it has seemed in the past 10 years or so that MS has simultaneously turned up the delivery frequency and lowered the bar for breaking changes.

    Some of us have products and projects that continue to last and grow and make money across many versions of software. When a team decides to update their name space unexpectedly (Azure Storage SDK) or the upgrade path as conventions on file location and such change from version to version (MVC), etc, then they have decided that it is more important to make their own lives easier at the expense of the customer. It may be that this is for the customers own good, it may be that this is just easier for the developers, but in either case it is taxing the customers that spend developer time keeping the stack up to date instead of letting it fall into extended support and trying to innovate on 5+ year old software.

    I get it, designing public SDKs and frameworks is difficult, you learn as you grow them, the world changes, etc. But giving up backwards compatibility should be a hugely painful decision. It should be made with the awareness of how much pain it will cause the very customers that are successfully using that product, and some consideration should be given to tooling or walkthroughs on making that transition from one to the next as easy as possible.

  14. Avatar for Eric J. Smith
    Eric J. Smith May 29th, 2014

    The entire pipeline is different. If you are doing anything other than simple controller actions then you will need to do work migrating your filters, message handlers, formatters, auth, constraints, action results and who knows what else. It will definitely be an effort to move to the new pipeline.

  15. Avatar for Stilgar
    Stilgar May 29th, 2014

    I assume this is also true for the MVC side? In the light of this information I am suddenly skeptical about vNext's chances of success.

  16. Avatar for Eric J. Smith
    Eric J. Smith May 29th, 2014

    Yes, there will be work migrating from MVC as well.

    Don't get me wrong though. I am extremely excited about vNext and the new lightweight core runtime. I have been doing a lot of digging through the code and the system is very well designed and it looks like it will actually be a lot easier to implement a lot of the custom things that we have in vNext. The system has DI built in at the core and everything is very modular and easy to plug things in.

    I think they had to make these changes or they would have become irrelevant for new development.

  17. Avatar for aspx
    aspx June 12th, 2014

    Is asp.net broken so that it needs fixing?

  18. Avatar for Vidal Agraval Shah Ibn Hattab
    Vidal Agraval Shah Ibn Hattab July 8th, 2014

    "It represents a new modular approach that makes it easier to swap out the parts you don't like"

    Hm... to be honest the only part I don't like is ASP.NET vNext and would swap it out. It looks like a reckless gimmick scheme. People use ASP.NET because it is fast (async IO), stable and compatible. Those kiddish new frameworks are joke (with Node as the only pleasant exception).

    Other commenter would say "this is life" but I would say "this is the road to hell where everything is broken, limited, unstable and incompatible". Windows Store full screen apps are great example of pseudo-innovation. They are joke. The money they bring are joke. The fact that I can't resize them is joke.

    System.Web is beautiful. It is monolithic. It is well-designed. It is useful, optimized and nearly bug-free.

    vNext, what the heck. vBS is a better name for that thing.