Abstract Base Classes Have Versioning Problems Too

This is part 2 in an ongoing series in which I talk about various design and versioning issues as they relate to Abstract Base Classes (ABC), Interfaces, and Framework design. In part 1 I discussed some ways in which ABCs are more resilient to versioning than interfaces. I haven’t covered the full story yet and will address some great points raised in the comments.

In this part, I want to point out some cases in which Abstract Base Classes fail in versioning. In my last post, I mentioned you could simply add new methods to an Abstract Base Class and not break clients. Well that’s true, it’s possible, but I didn’t emphasize that this is not true for all cases and can be risky. I was saving that for another post (aka this one).

I had been thinking about this particular scenario a while ago, but it was recently solidified in talking to a coworker today (thanks Mike!). Let’s look at the scenario. Suppose there is an abstract base class in a framework named FrameworkContextBase. The framework also provides a concrete implementation.

public abstract class FrameworkContextBase
{
  public abstract void MethodOne();
}

Somewhere else in another class in the framework, there is a method that takes in an instance of the base class and calls the method on it for whatever reason.

public void Accept(FrameworkContextBase arg)
{
  arg.MethodOne();
}

With me so far? Good. Now imagine that you, as a consumer of the Framework write a concrete implementation of FrameworkContextBase. In the next release of the framework, the framework includes a method to FrameworkContextBase like so...

public abstract class FrameworkContextBase
{
  public abstract void MethodOne();
  public virtual void MethodTwo()
  {
    throw new NotImplementedException();
  }
}

And the Accept method is updated like so...

public void Accept(FrameworkContextBase arg)
{
  arg.MethodOne();
  arg.MethodTwo();
}

Seems innocuous enough. You might even be lulled into the false sense that all is well in the world and decide to go ahead and upgrade the version of the Framework hosting your application without recompiling. Unfortunately, somewhere in your application, you pass your old implementation of the ABC to the new Accept method. Uh oh! Runtime exception!

The fix sounds easy in theory, when adding a new method to the ABC, the framework developer need to make sure it has a reasonable default implementation. In my contrived example, the default implementation throws an exception. This seems easy enough to fix. But how can you be sure the implementation is reasonable for all possible implementations of your ABC? You can’t.

This is often why you see guidelines for .NET which suggest making all methods non-virtual unless you absolutely need to. The idea is that the Framework should provide checks before and after to make sure certain invariants are not broken when calling a virtual method since we have no idea what that method will do.

As you might guess, I tend to take the approach of buyer beware. Rather than putting the weight on the Framework to make sure that virtual methods don’t do anything weird, I’d rather put the weight on the developer overriding the virtual method. At least that’s the approach we’re taking with ASP.NET MVC.

Another possible fix is to also add an associated Supports{Method} property when you add a method to an ABC. All code that calls that new method would have to check the property. For example...

public abstract class FrameworkContextBase
{
  public abstract void MethodOne();
  public virtual void MethodTwo()
  {
    throw new NotImplementedException();
  }
  public virtual bool SupportsMethodTwo {get{return false;}}
}

//Some other class in the same framework
public void Accept(FrameworkContextBase arg)
{
  arg.MethodOne();
  if(arg.SupportsMethodTwo)
  {
    arg.MethodTwo();
  }
}

But it may not be clear to you, the framework developer, what you should do when the instance doesn’t support MethodTwo. This might not be clear nor straightforward.

This post seems to contradict my last post a bit, but I don’t see it that way. As I stated all along, there is no perfect design, we are simply trying to optimize for constraints. Not only that, I should add that versioning is a hard problem. I am not fully convinced we have made all the right optimizations (so to speak) hence I am writing this series.

Coming up: More on versioning interfaces with real code examples and tradeoffs. More on why breaking changes suck. ;)

Versioning Issues With Abstract Base Classes and Interfaces

Eilon Lipton recently wrote a bit about context objects in ASP.NET MVC and in an “Oh by the way” moment, tossed out the fact that we changed the IHttpContext interface to the HttpContextBase abstract base class (ABC for short).

Not long after, this spurred debate among the Twitterati. Why did you choose an Abstract Base Class in this case? The full detailed answer would probably break my keyboard in length, so I thought I would try to address it in a series of posts.

In the end, I hope to convince the critiques that the real point of contention is about maintaining backwards compatibility, not about choosing an abstract base class in this one instance.

Our Constraints

All engineering problems are about optimizing for constraints. As I’ve written before, there is no perfect design, partly because we’re all optimizing for different constraints. The constraints you have in your job are probably different than the constraints that I have in my job.

For better or worse, these are the constraints my team is dealing with in the long run. You may disagree with these constraints, so be it. We can have that discussion later. I only ask that for the time being, you evaluate this discussion in light of these constraints. In logical terms, these are the premises on which my argument rests.

• Avoid Breaking Changes at all costs
• Allow for future changes

Specifically I mean breaking changes in our public API once we RTM. We can make breaking changes while we’re in the CTP/Beta phase.

You Can’t Change An Interface

The first problem we run into is that you cannot change an interface.

Now some might state, “Of course you can change an interface. Watch me! Changing an interface only means some clients will break, but I still changed it.”

The misunderstanding here is that after you ship an assembly with an interface, any changes to that interface result in a new interface. Eric Lippert points this out in this old Joel On Software forum thread...

The key thing to understand regarding "changing interfaces" is that an interface is a _type_.  A type is logically bound to an assembly, and an assembly can have a strong name.

This means that if you correctly version and strong-name your assemblies, there is no "you can’t change this interface" problem.  An interface updated in a new version of an assembly is a _different_ interface from the old one.

(This is of course yet another good reason to get in the habit of strong-naming assemblies.)

Thus trying to make even one tiny change to an interface violates our first constraint. It is a breaking change. You can however add a new virtual method to an abstract base class without breaking existing clients of the class. Hey, it’s not pretty, but it works.

Why Not Use An Interface And an Abstract Base Class?

Why not have a corresponding interface for every abstract base class? This assumes that the purpose of the ABC is simply to provide the default implementation of an interface. This isn’t always the case. Sometimes we may want to use an ABC in the same way we use an interface (all methods are abstract...revised versions of the class may add virtual methods which throw a NotImplementedException).

The reason that having a corresponding interface doesn’t necessarily buy us anything in terms of versioning, is that you can’t expose the interface. Let me explain with a totally contrived example.

Suppose you have an abstract base class we’ll randomly call HttpContextBase. Let’s also suppose that HttpContextBase implements an IHttpContext interface. Now we want to expose an instance of HttpContextBase via a property of another class, say RequestContext. The question is, what is the type of that property?

Is it...

public IHttpContext HttpContext {get; set;}

? Or is it...

public HttpContextBase HttpContext {get; set;}

If you choose the first option, then we’re back to square one with the versioning issue. If you choose the second option, we don’t gain much by having the interface.

What Is This Versioning Issue You Speak Of?

The versioning issue I speak of relates to clients of the property. Suppose we wish to add a new method or property to IHttpContext. We’ve effectively created a new interface and now all clients need to recompile. Not only that, but any components you might be using that refer to IHttpContext need to be recompiled. This can get ugly.

You could decide to add the new method to the ABC and not change the interface. What this means is that new clients of this class need to perform an interface check when they want to call this method every time.

public void SomeMethod(IHttpContext context)
{
  HttpContextBase contextAbs = context as HttpContextBase;
  if(contextAbs != null)
  {
    contextAbs.NewMethod();
  }
    
  context.Response.Write("Score!");
}

In the second case with the ABC, you can add the method as a virtual method and throw NotImplementedException. You don’t get compile time checking with this approach when implementing this ABC, but hey, thems the breaks. Remember, no perfect design.

Adding this method doesn’t break older clients. Newer clients who might need to call this method can recompile and now call this new method if they wish. This is where we get the versioning benefits.

So Why Not Keep Interfaces Small?

It’s not being small that makes an interface resilient to change. What you really want is an interface that is small and cohesive with very little reason to change. This is probably the best strategy with interfaces and versioning, but even this can run into problems. I’ll address this in more detail in an upcoming post, but for now will provide just one brief argument.

Many times, you want to divide a wide API surface into a group of distinct smaller interfaces. The problem arises when a method needs functionality of several of those interfaces. Now, a change in any one of those interfaces would break the client. In fact, all you’ve really done is spread out one large interface with many reasons to change into many interfaces each with few reasons to change. Overall, it adds up to the same thing in terms of risk of change.

Alternative Approaches

These issues are one of the trade-offs of using statically typed languages. One reason you don’t hear much about this in the Ruby community, for example, is there really aren’t interfaces in Ruby, though some have proposed approaches to provide something similar. Dynamic typing is really great for resilience to versioning.

One thing I̻’d love to hear more feedback from others is why, in .NET land, are we so tied to interfaces? If the general rule of thumb is to keep interfaces small (I’ve even heard some suggest interfaces should only have one method), why aren’t we using delegates more instead of interfaces? That would provide for even looser coupling than interfaces.

The proposed dynamic keyword and duck-typing features in future versions of C# might provide more resilience. As with dynamically typed languages such as Ruby, the trade-off in these cases is that you forego compile time checking for run-time checking. Personally, I think the evidence is mounting that this may be a worthwhile tradeoff in many cases.

ASP.NET MVC Update

By the way my blogging frequency has declined, you can guess I’ve been quite busy here at Microsoft preparing for the next release of ASP.NET MVC.

It’s not just working on specs, design meetings, etc... that keep me busy. It’s preparing for several talks, various spec reviews, building hands on labs, demo and and app building, etc...that keeps me busy. All the while I am still learning the ropes and dealing with selling a house in L.A. and buying a house up here. There’s a lot that goes into being a PM I naively didn’t expect, on top of just how much work goes into simply moving.

Not that I’m complaining. It’s still a lot of fun. ScottGu posted an entry on his blog about the fun we’re having in preparing for the upcoming ASP.NET MVC Mix Preview.

Here’s a screenshot that shows a tooling feature I’m particularly excited about.

I’ve written about the challenges Microsoft faces with bundling Open Source software in the past and what I thought they should do about it...

What I would have liked to have seen is for Team System to provide extensibility points which make it extremely easy to swap out MS Test for another testing framework. MS Test isn’t the money maker for Microsoft, it’s the whole integrated suite that brings in the moolah, so being able to replace it doesn’t hurt the bottom line.

What we have here is similar in spirit to what I hoped for. AFAIK it is not going to be integrated to the level that Visual Studio Team Test is integrated, but it also won’t require VSTS. This is a feature of the ASP.NET MVC project template.

I’m also excited about some of the community feedback we were able to incorporate such as removing the ControllerActionAttribute among other things. A hat or two might be eaten over that one ;).

In any case, there are still some areas I’m not yet happy with (there always will be, won’t there?), so we are not done by any measure. I’ll reserve talking about that until after Mix when you have the code in your hands and can follow along.

Blocking Direct Access To Views in ASP.NET MVC

UPDATE: I improved this based on some feedback in my comments.

With ASP.NET MVC it is possible for someone to try and navigate directly to a .aspx view. In general, this only leaves them with an ugly error message as Views typically need ViewData in order to work.

However, one approach that I think will easily work is to create a Web.config file in the root of your Views directory that contains the following.

We need to do more testing on our side to make sure there's no pathological case in doing this, but so far in my personal testing, it seems to work.

<?xml version="1.0"?>
<configuration>
  <system.web>
    <httpHandlers>
      <remove verb="*" path="*.aspx"/>
      <add path="*.aspx" verb="*" 
          type="System.Web.HttpNotFoundHandler"/>
    </httpHandlers>
  </system.web>

  <system.webServer>
    <validation validateIntegratedModeConfiguration="false"/>
    <handlers>
      <remove name="PageHandlerFactory-ISAPI-2.0"/>
      <remove name="PageHandlerFactory-ISAPI-1.1"/>
      <remove name="PageHandlerFactory-Integrated"/>
      <add name="BlockViewHandler" path="*.aspx" verb="*" 
        preCondition="integratedMode" 
        type="System.Web.HttpNotFoundHandler"/>
    </handlers>
  </system.webServer>
</configuration>

Let me know if you run into problems with this.

Tell Me Your Unit Testing Pains

If you know me, you know I go through great pains to write automated unit tests for my code. Some might even call me anal about it. Those people know me too well.

For example, in the active branch of Subtext, we have 882 unit tests, of which I estimate I wrote around 800 of those. Yep, if you’re browsing the Subtext unit test code and something smells bad, chances are I probably dealt it.

Unfortunately, by most definitions of Unit Test, most of these tests are really integration tests. Partly because I was testing legacy code that and partly because I was blocked by the framework, not every method under test could be easily tested in isolation.

Whether you’re a TDD fan or not, I think most of us can agree that unit testing your own code is a necessary practice, whether it is manually or automated.

I still think it’s worthwhile to take that one step further and automate your unit tests whenever possible and where it makes most sense (for large values of make sense).

When writing an automated unit test, the key is to try and isolate the unit under test (typically a method) by both controlling the external dependencies for the method and being able to capture any side-effects of the method.

Sometimes though, external code that your code makes calls into can sometimes be written in such a way that makes it challenging to test your own code. Often, you have to resort to building all sorts of Bridge or Adapter classes to abstract away the thing you’re calling. Sometimes you are plain stuck.

What “external code” might exhibit this characteristic of making your own code hard to test? I don’t have anything in particular in mind but...oh...off the top of my head if you made me pick one totally spontaneously, I might mention my way of example one little piece of code called the .NET Framework.

For the most part, I’ve had very few problems with the Base Class Libraries or other parts of the Framework. Most of my testing woes came when writing code against ASP.NET. The ASP.NET MVC framework hopes to help address some of that.

I’ve been in a lot of internal discussions recently talking with various people and teams about testable code. In order to contribute more value to these discussions, I am trying to gather specific cases and scenarios where testing your code is really painful.

What I am not looking for is feedback such as

“It’s hard to write unit tests when writing a Foo application/control/part”.

or

“Class Queezle should really make method Bloozle public.”

Perhaps Bloozle should be public, but I am not interested in theoretical pains. If the inaccessibility of Bloozle caused a real problem in a real unit test, that's what I want to hear.

What I am looking for is specifics! Concrete scenarios that are blocked or extremely painful.Including the actual unit test is even better! For example...

“When writing ASP.NET code, I want to have helper methods that accept an HttpContext instance and get some values from its various properties and then perform a calculation. Because HttpContext is sealed and is tightly coupled to the ASP.NET stack, I can't mock it out or replace it with a test specific subclass. This means I always have to write some sort of bridge or adapter that wraps the context which gets tedious.”

Only, you can’t use that one, because I already wrote it. Ideally, I’d love to hear feedback from across the board, not just ASP.NET. Got issues with WPF, WCF, Sharepoint Web Parts, etc... tell us about it. Please post them in my comments or on your blog and link to this post. Your input is very valuable and could help shape the future of the Framework, or at least help me to sound like I am clued into customer needs next time I talk to someone internal about this. ;)