October 2011 Blog Posts
If you’re in the Reykjavik area on November 7th, come join me for a beer-up. A Beer-Up is basically a meet-up, but with lots of beer!
- When: November 7th, 2011 at 8:00 PM
- Where: The English Pub (yes, I went all the way to Iceland for an English pub)
- Why: To talk about ASP.NET, ASP.NET MVC, NuGet, Software Development whatever geeky topics you want. And if we do our jobs right, by the end of the night we’ll discuss life, philosophy, and which direction is my hotel?
Blue Lagoon in Iceland - Photo from sxc.hu
I’ll be stopping overnight in Reykjavik on my way to Oredev 2011. I’m pretty excited as I’ve always been fascinated by the natural beauty of such a geologically active place. I definitely plan to see the Blue Lagoon geothermal spa (pictured above) during my stay.
If you’re in the area and love to talk about coding, technology, whatever, do join us!
We made a recent change to make it easy to update the NuGet documentation. In this post, I’ll cover what the change was, why we made it, and how it makes it easier to contribute to our documentation.
Our docs run as a simple ASP.NET Web Pages application that renders documentation written in the Markdown format. The Markdown text is not stored in a database, but live as files that are part of the application source code. That allows us to use source control to version our docs.
We used to host the source for the docs site in Mercurial (hg) on CodePlex.com. Under the old system, it took the following to contribute docs.
- Install Mercurial (TortoiseHG for example) if you didn’t already have it.
- Fork our repository and clone it to your local machine.
- Open it up in Visual Studio.
- Make and commit your changes.
- Push your changes.
- Send a pull request.
It’s no surprise that we don’t get a lot of pull requests for our documentation. Oh, and I didn’t even mention all the steps once we received such a pull request.
As anyone who’s ever run an open source product knows, it’s hard enough to get folks to contribute to documentation in the first place. Why add more roadblocks?
To improve this situation, we moved our documentation repository to Github for three reasons:
- In-browser editing of files with MarkDown preview.
- Pull requests can be merged at the click of a button.
- Support for deploying to AppHarbor (which CodePlex also has)
With this in place, it’s now easy to be a “drive-by” contributor to our docs. Let’s walk through an example to see what I mean. In this example, I’m posing as a guy named “FakeHaacked” with no commit access to the NuGetDocs repository.
Here’s a sample page from our docs (click for larger). The words at the end of the first paragraph should be links! Doh! I should fix that.
First, I’ll visit the NuGet Docs repository (TODO: Add a link to each page with the path to the Github repository page).
Cool, I’m logged in as FakeHaacked. Now I just need to navigate to the page that needs the correction. All of the documentation pages are in the site folder.
Pro tip, type the letter “t” while on this page to use incremental search to search for the page you want to edit.
Here’s the page I want to edit.
Since this file is a Markdown file, I can see a view of the file that’s a nice approximation of what it will look like when it’s deployed. It’s not exactly the same since we have different CSS styles on the production site.
See that blue button just above the content and to the right? That allows me to “fork” this page and edit the file. Forking it, for those not familiar with distributed version control, means it will create a clone of the main repository. I’m free to work and do whatever I want in that clone without worry that I will affect the real thing.
Let’s click that button and see what happens.
Cool, I get a nice editor with preview for editing the page right here in the browser. I’ll go ahead and make those last two references into Markdown formatted links.
When I’m done, I can scroll down, type in a commit message describing the change, and click the blue Propose File Change button.
Once you’re happy with the set of changes you’ve made, click the button to send a pull request. This lets the folks who maintain the documentation to know you have changes that are ready for them to pull in.
And that’s it. You’ve done your part. Thank you for your contribution to our docs! Now let’s look at what happens on the other side. I’ll put on my project maintainer hat and visit the site. Notice I’m logged in as Haacked now and I see there’s an outstanding pull request.
Cool, I can take a look at it, quickly see a diff, and comment on it. Notice that Github was able to determine that this file is safe to automatically merge into the master branch.
All I have to do is click the big blue button, enter a message, and I’m done!
It’s that easy for me to merge in your changes.
Summary
You might ask why we don’t use the Github Pages feature (or even Git-backed wikis). We started the docs site before we were on Github and didn’t know about the pages feature.
If I were to start over, I’d probably just use that. Maybe we’ll migrate in the future. One benefit of our current implementation is we get that nice Table of Contents widget generated for us dynamically (which we can probably do with Github Pages and Jekyll) and we can use Razor for our layout template.
The downside of our current approach is that we can’t create new doc pages this way, but I’ll submit a feature request to the Github team and see what happens.
So if you are reading the NuGet docs, and see something that makes you think, “That ain’t right!”, please go fix it! It’s easy and contributing to open source documentation makes you a good person. It’s how I got started in open source.
Oh, and if you happen to be experienced with Git, you can always use the traditional method of cloning the repository to your local machine and making changes. That gives you the benefit of running the site to look at your change.
Recently, a group of covert ninjas within my organization started to investigate what it would take to change our internal build and continuous integration systems (CI) to take advantage of NuGet for many of our products, and I need your input!
Hmm, off by one error slays me again. -Image from Ask A Ninja. Click on the image to visit.
Ok, they’re not really covert ninjas, that just sounds much cooler than a team of slightly pudgy software developers. Ok, they’ve told me to speak for myself, they’re in great shape!
In response to popular demand, we changed our minds and decided to support Semantic Versioning (SemVer) as the means to specify pre-release packages for the next version of NuGet (1.6).
In part, this is the cause of the delay for this release as it required extensive changes to NuGet and the NuGet.org gallery. I will write a blog post later that covers this in more detail, but for now, you can read our spec on it which is mostly up to date. I hope.
I’m really excited to change our own build to use NuGet because it will force us to eat our own dogfood and feel the pain that many of you feel with NuGet in such scenarios. Until we feel that pain, we won’t have a great solution to the pain.
A really brief intro to Semver
You can read the SemVer spec here, but in case you’re lazy, I’ll provide a brief summary.
SemVer is a convention for versioning your public APIs that gives meaning to the version number. Each version has three parts, Major.Minor.Patch.
In brief, these correspond to:
- Major: Breaking changes.
- Minor: New features, but backwards compatible.
- Patch: Backwards compatible bug fixes only.
Additionally, pre-release versions of your API can be denoted by appending an arbitrary string to the Patch number. For example:
- 1.0.1alpha
- 1.0.1beta
- 1.0.1Fizzlebutt
When you’re ready to release, you just remove the string and that version is considered “higher” than all the pre-release versions. The pre-release versions are given precedence in alphabetical order (well technically lexicographic ASCII sort order).
Therefore, the following is an example from lowest to highest versions of a package.
- 1.0.1alpha
- 1.0.1alpha2
- 1.0.1beta
- 1.0.1rc
- 1.0.1zeeistalmostdone
- 1.0.1
How NuGet uses SemVer
As I mentioned before, I’ll write up a longer blog post about how SemVer figures into your package. For now, I just want to make it clear that if you’re using 4-part version numbers today, your packages will still work and behave as before.
It’s only when you specify a 3-part version with a version string that NuGet gets strict about SemVer. For example, NuGet allows 1.0.1beta but does not allow 1.0.1beta.234 as a version.
How to deal with nightly builds?
So the question I have is, how do we deal with nightly (or continous) builds?
For example, suppose I start work on what will become 1.0.1beta. Internally, I may post nightly builds of 1.0.1beta for others in my team to use. Then at some point, I’ll stamp a release as the official 1.0.1beta for public consumption.
The problem is, each of those builds need to have the package version incremented. This ensures that folks can revert back to a last-known-good nightly build if a problem comes up. SemVer doesn’t seem to address how to handle internal nightly (or continuous) builds. It’s really focused on public releases.
Note, we’re thinking about this for our internal setup, not for the public gallery. I’ll address that question later.
We had a few ideas in mind.
Stick with the previous version number and change labels just before release
The idea here is that when we’re working on 1.0.1beta, we version the packages using the alpha label and increment it with a build number.
- 1.0.1alpha (public release)
- 1.0.1alpha-0001 (internal build)
- 1.0.1alpha-0002 (internal build)
A variant of this approach is to append the date (and counter) in number format.
- 1.0.1alpha (public release)
- 1.0.1alpha-20101025001 (internal build)
- 1.0.1alpha-20101026001 (internal build on the next day)
- 1.0.1alpha-20101026002 (another internal build on the same day)
With this approach, when we’re ready to cut the release, we simply change the package to be 1.0.1beta and release it.
The downside of this approach is that it’s not completely clear that these are internal nightly builds of what will be 1.0.1beta. They could be builds of 1.0.1alpha2.
Yet another variant of this approach is to name our public releases with an even Minor or Patch number and our internal releases with an odd one. So when we’re ready to work on 1.0.2beta, we’d version the package as 1.0.1beta. When we’re ready to release, we change it to 1.0.2beta.
Have a separate feed with its own versioning scheme.
Another thought was to simply have a completely separate feed with its own versioning scheme. So you can choose to grab packages from the stable feed, or the nightly feed.
In the nightly feed, the package version might just be the date.
- 2010.10.25001
- 2010.10.25002
- 2010.10.25003
The downside of this approach is that it’s not clear at all what release version these will apply to. Also, when you’re ready to promote one to the stable feed, you have to move it in there and completely change the version number.
Support an optional Build number for Semantic Versions
For NuGet 1.6, you can still use a four-part version number. But NuGet is strict if the version is clearly a SemVer version. For example, if you specify a pre-release string, such as 1.0.1alpha, NuGet will not allow a fourth version part.
But we had the idea that we could extend SemVer to support this concept of a build number. This might be off by default in the public NuGet.org gallery, but could be something you could turn on individually. What it would allow you to do is continue to push new builds of 1.0.1alpha with an incrementing build number. For example:
- 1.0.1beta.0001 (nightly)
- 1.0.1beta.0002 (nightly)
- 1.0.1beta.0003 (nightly)
- 1.0.1beta (public)
Note that unlike a standard 4-part version, 1.0.1beta is a higher version than 1.0.1beta.0003.
While I’m hesitant to suggest a custom extension to SemVer, it makes a lot of sense to me. After all, this would be a convention applied to internal builds and not for public releases.
Question for you
So, do you like any of these approaches? Do you have a better approach?
While I love comments on my blog, I would like to direct discussion to the NuGet Discussions page. I look forward to hearing your advice on how to handle this situation. Whatever we decide, we want to bake in first-class support into NuGet to make this sort of thing easier.
If you’re not familiar with WCF Web API, it’s a framework with nice HTTP abstractions used to expose simple HTTP services over the web. Its focus is targeted at applications that provide HTTP services for various clients such as mobile devices, browsers, desktop applications.
In some ways, it’s similar to ASP.NET MVC as it was developed with testability and extensibility in mind. There are some concepts that are similar to ASP.NET MVC, but with a twist. For example, where ASP.NET MVC has filters, WCF has operation handlers.
One question that comes up often with Web API is how do you authenticate requests? Well, you run Web API on ASP.NET (Web API also supports a self-host model), one approach you could take is to write an operation handler and attach it to a set of operations (an operation is analogous to an ASP.NET MVC action).
However, some folks like the ASP.NET MVC approach of slapping on an AuthorizeAttribute. In this blog post, I’ll show you how to write an attribute, RequireAuthorizationAttribute, for WCF Web API that does something similar.
One difference is that in the WCF Web API case, the attribute simply provides metadata, but not the the behavior, for authorization. If you wanted to use the existing ASP.NET MVC AuthorizeAttribute in the same way, you could do that as well, but I leave that as an exercise for the reader.
I’ll start with the easiest part, the attribute.
[AttributeUsage(AttributeTargets.Method)]
public class RequireAuthorizationAttribute : Attribute
{
public string Roles { get; set; }
}
For now, it only applies to methods (operations). Later, we can update it to apply to classes as well if we so choose. I’m still learning the framework so I didn’t want to go bite off too much all at once.
The next step is to write an operation handler. When properly configured, the operation handler runs on every request for the operation that it applies to.
public class AuthOperationHandler
: HttpOperationHandler<HttpRequestMessage, HttpRequestMessage>
{
RequireAuthorizationAttribute _authorizeAttribute;
public AuthOperationHandler(RequireAuthorizationAttribute authorizeAttribute)
: base("response")
{
_authorizeAttribute = authorizeAttribute;
}
protected override HttpRequestMessage OnHandle(HttpRequestMessage input)
{
IPrincipal user = Thread.CurrentPrincipal;
if (!user.Identity.IsAuthenticated)
{
throw new HttpResponseException(HttpStatusCode.Unauthorized);
}
if (_authorizeAttribute.Roles == null)
{
return input;
}
var roles = _authorizeAttribute.Roles.Split(new[] { " " },
StringSplitOptions.RemoveEmptyEntries);
if (roles.Any(role => user.IsInRole(role)))
{
return input;
}
throw new HttpResponseException(HttpStatusCode.Unauthorized);
}
}
Notice that the code accesses HttpContext.Current. This restricts this operation handler to only work within ASP.NET applications. Hey, I write what I know!
Many folks replied to me that I should use Thread.CurrentPrincipal. My brain must have been off when I wrote this to not think of it. :)
Then all we do is ensure that the user is authenticated and in one of the specified roles if any role is specified. Very simple straightforward code at this point.
The final step is to associate this operation handler with some operations. In general, when you build a Web API application, the application author writes a configuration class that derives from WebApiConfiguration and either sets it as the default configuration, or passes it to a service route.
Within that configuration class, the author can specify an action that gets called on every request and gives the configuration class a chance to map a set of operation handlers to an operation.
For example, in a sample Web API app, I added the following configuration class.
public class CommentsConfiguration : WebApiConfiguration
{
public CommentsConfiguration()
{
EnableTestClient = true;
RequestHandlers = (c, e, od) =>
{
// TODO: Configure request operation handlers
};
this.AppendAuthorizationRequestHandlers();
}
}
The RequestHandlers is a property of type Action<Collection<HttpOperationHandler>, ServiceEndpoint, HttpOperationDescription>
In general, it would be up to the application author to wire up the authentication operation handler I wrote to the appropriate actions. But I wanted to provide a method that helps with that. That’s the AppendAuthorizationRequestHandlers method in there, which is an extension method I wrote.
public static void AppendAuthorizationRequestHandlers(
this WebApiConfiguration config)
{
var requestHandlers = config.RequestHandlers;
config.RequestHandlers = (c, e, od) =>
{
if (requestHandlers != null)
{
requestHandlers(c, e, od); // Original request handler
}
var authorizeAttribute = od.Attributes.OfType<RequireAuthorizationAttribute>()
.FirstOrDefault();
if (authorizeAttribute != null)
{
c.Add(new AuthOperationHandler(authorizeAttribute));
}
};
}
Since I didn’t want to stomp on the existing request handlers, I set the RequestHandlers property to a new action that calls the existing action (if any) and then does my custom registration logic.
I’ll admit, I couldn’t help thinking that if RequestHandlers was an event, rather than an action, that sort of logic could be handled for me. 
Have events fallen out of favor? They do work well to decouple code in this sort of scenario, but I digress.
The interesting part here is that the action’s third parameter, od, is an HttpOperationDescription. This is a description of the operation that includes access to such things as the attributes applied to the method! I simply look to see if the operation has the RequireAuthorizationAttribute applied and if so, I add the AuthOperationHandler I wrote earlier to the operation’s collection of operation handlers.
With this in place, I can now write a service that looks like this:
[ServiceContract]
public class CommentsApi
{
[WebGet]
public IQueryable<Comment> Get()
{
return new[] { new Comment
{
Title = "This is neato",
Body = "Ok, not as neat as I originally thought." }
}.AsQueryable();
}
[WebGet(UriTemplate = "auth"), RequireAuthorization]
public IQueryable<Comment> GetAuth()
{
return new[] { new Comment
{
Title = "This is secured neato",
Body = "Ok, a bit neater than I originally thought." }
}.AsQueryable();
}
}
And route to the Web API service like so:
public class Global : HttpApplication
{
protected void Application_Start(object sender, EventArgs e)
{
RouteTable.Routes.MapServiceRoute<CommentsApi>("comments",
new CommentsConfiguration());
}
}
With this in place, a request for /comments allows anonymous, but a request for /comments/auth requires authentication.
If you’re interested in checking this code out, I pushed it to my CodeHaacks Github repository as a sample. I won’t make this into a NuGet package until it’s been thoroughly vetted by the WCF Web API team because it’s very likely I have no idea what I’m doing. I’d rather one of those folks make a NuGet package for this. 
And if you’re wondering why I’m writing about Web API, we’re all part of the same larger team now, so I figured it’s good to take a peek at what my friends are up to.
I like to live life on the wild side. No, I don’t base jump off of buildings or invest in speculative tranches made up of junk stock derivatives. What I do is attempt to run recurring background tasks within an ASP.NET application.
Writing code is totally just like this - Photo by DVIDSHUB – CC BY 2.0
But before I do anything wild with ASP.NET, I always talk to my colleague, Levi (sadly, no blog). As a developer on the internals of ASP.NET, he knows a huge amount about it, especially the potential pitfalls. He’s also quite the security guru. As you read this sentence, he just guessed your passwords. All of them.
When he got wind of my plan, he let me know it was evil, unsupported by ASP.NET and just might kill a cat. Good thing I’m a dog person. I persisted in my foolhardiness and suggested maybe it’s not evil, just risky. If so, how can I do it as safely as possible? What are the risks?
There are three main risks, one of which I’ll focus on in this blog post.
- An unhandled exception in a thread not associated with a request will take down the process. This occurs even if you have a handler setup via the Application_Error method. I’ll try and explain why in a follow-up blog post, but this is easy to deal with.
- If you run your site in a Web Farm, you could end up with multiple instances of your app that all attempt to run the same task at the same time. A little more challenging to deal with than the first item, but still not too hard. One typical approach is to use a resource common to all the servers, such as the database, as a synchronization mechanism to coordinate tasks.
- The AppDomain your site runs in can go down for a number of reasons and take down your background task with it. This could corrupt data if it happens in the middle of your code execution.
It’s this last risk that is the focus of this blog post.
Bye Bye App Domain
There are several things that can cause ASP.NET to tear down your AppDomain.
- When you modify web.config, ASP.NET will recycle the AppDomain, though the w3wp.exe process (the IIS web server process) stays alive.
- IIS will itself recycle the entire w3wp.exe process every 29 hours. It’ll just outright put a cap in the w3wp.exe process and bring down all of the app domains with it.
- In a shared hosting environment, many web servers are configured to tear down the application pools after some period of inactivity. For example, if there are no requests to the application within a 20 minute period, it may take down the app domain.
If any of these happen in the middle of your code execution, your application/data could be left in a pretty bad state as it’s shut down without warning.
So why isn’t this a problem for your typical per request ASP.NET code? When ASP.NET tears down the AppDomain, it will attempt to flush the existing requests and give them time to complete before it takes down the App Domain. ASP.NET and IIS are considerate to code that they know is running, such as code that runs as part of a request.
Problem is, ASP.NET doesn’t know about work done on a background thread spawned using a timer or similar mechanism. It only knows about work associated with a request.
So tell ASP.NET, “Hey, I’m working here!”
The good news is there’s an easy way to tell ASP.NET about the work you’re doing! In the System.Web.Hosting namespace, there’s an important class, HostingEnvironment. According to the MSDN docs, this class…
Provides application-management functions and application services to a managed application within its application domain
This class has an important static method, RegisterObject. The MSDN description here isn’t super helpful.
Places an object in the list of registered objects for the application.
For us, what this means is that the RegisterObject method tells ASP.NET that, “Hey! Pay attention to this code here!” Important! This method requires full trust!
This method takes in a single object that implements the IRegisteredObject interface. That interface has a single method:
public interface IRegisteredObject
{
void Stop(bool immediate);
}
When ASP.NET tears down the AppDomain, it will first attempt to call Stop method on all registered objects.
In most cases, it’ll call this method twice, once with immediate set to false. This gives your code a bit of time to finish what it is doing. ASP.NET gives all instances of IRegisteredObject a total of 30 seconds to complete their work, not 30 seconds each. After that time span, if there are any registered objects left, it will call them again with immediate set to true. This lets you know it means business and you really need to finish up pronto! I modeled my parenting technique after this method when trying to get my kids ready for school.
When ASP.NET calls into this method, your code needs to prevent this method from returning until your work is done. Levi showed me one easy way to do this by simply using a lock. Once the work is done, the code needs to unregister the object.
For example, here’s a simple generic implementation of IRegisteredObject. In this implementation, I simply ignored the immediate flag and try to prevent the method from returning until the work is done. The intent here is I won’t pass in any work that’ll take too long. Hopefully.
public class JobHost : IRegisteredObject
{
private readonly object _lock = new object();
private bool _shuttingDown;
public JobHost()
{
HostingEnvironment.RegisterObject(this);
}
public void Stop(bool immediate)
{
lock (_lock)
{
_shuttingDown = true;
}
HostingEnvironment.UnregisterObject(this);
}
public void DoWork(Action work)
{
lock (_lock)
{
if (_shuttingDown)
{
return;
}
work();
}
}
}
I wanted to get the simplest thing possible working. Note, that when ASP.NET is about to shut down the AppDomain, it will attempt to call the Stop method. That method will try to acquire a lock on the _lock instance. The DoWork method also acquires that same lock. That way, when the DoWork method is doing the work you give it (passed in as a lambda) the Stop method has to wait until the work is done before it can acquire the lock. Nifty.
Later on, I plan to make this more sophisticated by taking advantage of using a Task to represent the work rather than an Action. This would allow me to take advantage of task cancellation instead of the brute force approach with locks.
With this class in place, you can create a timer on Application_Start (I generally use WebActivator to register code that runs on app start) and when it elapses, you call into the DoWork method here. Remember, the timer must be referenced or it could be garbage collected.
Here’s a small example of this:
using System;
using System.Threading;
using WebBackgrounder;
[assembly: WebActivator.PreApplicationStartMethod(
typeof(SampleAspNetTimer), "Start")]
public static class SampleAspNetTimer
{
private static readonly Timer _timer = new Timer(OnTimerElapsed);
private static readonly JobHost _jobHost = new JobHost();
public static void Start()
{
_timer.Change(TimeSpan.Zero, TimeSpan.FromMilliseconds(1000));
}
private static void OnTimerElapsed(object sender)
{
_jobHost.DoWork(() => { /* What is it that you do around here */ });
}
}
Recommendation
This technique can make your background tasks within ASP.NET much more robust. There’s still a chance of problems occurring though. Sometimes, the AppDomain goes down in a more abrupt manner. For example, you might have a blue screen, someone might trip on the plug, or a hard-drive might fail. These catastrophic failures can take down your app in such a way that leaves data in a bad state. But hopefully, these situations occur much less frequently than an AppDomain shutdown.
Many of you might be scratching your head thinking it seems weird to use a web server to perform recurring background tasks. That’s not really what a web server is for. You’re absolutely right. My recommendation is to do one of the following instead:
- Write a simple console app and schedule it using Windows task schedule.
- Write a Windows Service to manage your recurring tasks.
- Use an Azure worker or something similar.
Given that those are my recommendations, why am I still working on a system for scheduling recurring tasks within ASP.NET that handles web farms and AppDomain shutdowns I call WebBackgrounder (NuGet package coming later)?
I mean, besides the fact that I’m thick-headed? Well, for two reasons.
The first is to make development easier. When you get latest from our source code, I just want everything to work. I don’t want you to have to set up a scheduled task, or an Azure worker, or a Windows server on your development box. A development environment can tolerate the issues I described.
The second reason is for simplicity. If you’re ok with the limitations I mentioned, this approach has one less moving part to worry about when setting up a website. There’s no need to configure an external recurring task. It just works.
But mostly, it’s because I like to live life on the edge.
Today, October 15 2011, marks four years of being a Microsoft employee for me. As such, it’s time for a little introspection, but in many ways, Tim Heuer already introspected for me. Much of what he writes echoes my own experience, thus leaving me with less to write about. 
It’s the Microsoft way, or the highway. Which is conveniently located near Microsoft Way. - Photo by Todd Bishop, CC BY 2.0
Looking back in my archives, I realized I haven’t written a whole lot about what it’s like to work here. I do have a few posts such as…
Regarding the second post, the funny thing is you never stop drinking from the fire hose here. At least I haven’t yet. And that’s both a good thing, but it can be wearying at times. I’ve been taking a lot of mini-vacations lately to keep my sanity.
So far, the past four years have been a real blast.
I’ve had a passion for open source for a very long time. When I started, ASP.NET MVC 1.0 was just getting going as a proprietary project. Very few teams at that time released their source code, much less under a proper OSS license. Although it involved sitting in a lot of meetings with lawyers, reviewing lots of dry legal documents, I loved the opportunity to help drive the process to get ASP.NET MVC licensed under the Ms-PL, a liberal OSI certified open source license. Announcing that release was a happy day for me.
Since that time, we’ve shipped four RTM releases of ASP.NET MVC (recall that we released ASP.NET MVC 3 twice), each incorporating more and more third-party open source software, another milestone. As they say, shipping is a feature! And the ASP.NET team, and the MVC team in particular, are all really great people to work with. Hat tip to them all!
In this time, I’ve also had the great pleasure to work on NuGet from its inception. NuGet goes one step further in that it’s not only an open source project under the Apache v2 license, but it accepts contributions. Microsoft contributed NuGet to the Outercurve Foundation (an independent open source foundation not unlike the Apache and Eclipse foundation that recently received its non-profit status!) early in its life allowing it to flourish as an open source project.
Microsoft has a team of employees who are allowed to spend work-time contributing to the NuGet project. All development, issue tracking, and discussion occurs in a public location, the NuGet CodePlex site, with Mercurial (hg) as the source code repository.
The NuGet team is a dedicated group of folks and it’s really a joy to work with them. The community involvement and feedback has been tremendous. I used to tell folks that I wanted to work on open source as my day job. That wish came true. It’s a real joy.
The past four years have also had their fair share of challenges behind the scenes. When you work with the intensely smart folks that I have the pleasure to work with, it’s hard not to feel the effects of the Impostor Syndrome, as Hanselman writes. And with everything I work on, I have a keen eye for all the shortcomings and faults in what we produce. I find it hard to tout our releases as much as I could because I always wish we could have done better. Perhaps because I’m a perfectionist, but more likely due to my half-Asian upbringing. I’m forced to find fault in everything.
Despite all that, I do take pride in the great work that the teams I work with have done. Their efforts have been tremendous and they deserve all the credit. I only hope my contributions were indeed contributions and not just overhead.
And I want to thank many of you, who’ve offered us encouragement and constructive criticism via Twitter, your blogs, my blog, our forums, StackOverflow, and so on. All of it is very much appreciated! Without your help, I don’t think I would have done as well in the past year. Don’t be surprised when I come knocking on your door asking for more help. Did I mention already that NuGet accepts contributions?
Technorati Tags:
microsoft,
life,
work
A long while ago I wrote about the potential dangers of Cross-site Request Forgery attacks, also known as CSRF or XSRF. These exploits are a form of confused deputy attack.
Screen grab from The Police Academy movie.In that post, I covered how ASP.NET MVC includes a set of anti-forgery helpers to help mitigate such exploits. The helpers include an HTML helper meant to be called in the form that renders a hidden input, and an attribute applied to the controller action to protect. These helpers work great when in a typical HTML form post to an action method scenario.
But what if your HTML page posts JSON data to an action instead of posting a form? How do these helpers help in that case?
You can try to apply the ValidateAntiForgeryTokenAttribute attribute to an action method, but it will fail every time if you try to post JSON encoded data to the action method. On one hand, the most secure action possible is one that rejects every request. On the other hand, that’s a lousy user experience.
The problem lies in the fact that the under the hood, deep within the call stack, the attribute peeks into the Request.Form collection to grab the anti-forgery token. But when you post JSON encoded data, there is no form collection to speak of. We hope to fix this at some point and with a more flexible set of anti-forgery helpers. But for the moment, we’re stuck with this.
This problem became evident to me after I wrote a proof-of-concept library to ASP.NET MVC action methods from JavaScript in an easy manner. The JavaScript helpers I wrote post JSON to action methods in order to call the actions. So I set out to fix this in my CodeHaacks project.
There are two parts we need to tackle this problem. The first part is on the client-side where we need to generate and send the token to the server. To generate the token, I just use the existing @Html.AntiForgeryToken helper in the view. A little bit of jQuery code grabs the value of that token.
var token = $('input[name=""__RequestVerificationToken""]').val();
That’s easy. Now that I have the value, I just need a way to post it to the server. I choose to add it to the request headers. In vanilla jQuery (mmmm, vanilla), that looks similar to:
var headers = {};
// other headers omitted
headers['__RequestVerificationToken'] = token;
$.ajax({
cache: false,
dataType: 'json',
type: 'POST',
headers: headers,
data: window.JSON.stringify(obj),
contentType: 'application/json; charset=utf-8',
url: '/some-url'
});
Ok, so far so good. This will generate the token in the browser and send it to the server, but we have a problem here. As I mentioned earlier, the existing attribute which validates the token on the server won’t look in the header. It only looks in the form collection. Uh oh! It’s Haacking time! I’ll write a custom attribute called ValidateJsonAntiForgeryTokenAttribute.
This attribute will call into the underlying anti-forgery code, but we need to get around that form collection issue I mentioned earlier.
Peeking into Reflector, I looked at the implementation of the regular attribute and followed its call stack. It took me deep into the bowels of the System.Web.WebPages.dll assembly, which contains a method with the following signature that does the actual work to validate the token:
public void Validate(HttpContextBase context, string salt);
Score! The method takes in an instance of type HttpContextBase, which is an abstract base class. That means we can can intercept that call and provide our own instance of HttpContextBase to validate the anti-forgery token. Yes, I provide a forgery of the request to enable the anti-forgery helper to work. Ironic, eh?
Here’s the custom implementation of the HttpContextBase class. I wrote it as a private inner class to the attribute.
private class JsonAntiForgeryHttpContextWrapper : HttpContextWrapper {
readonly HttpRequestBase _request;
public JsonAntiForgeryHttpContextWrapper(HttpContext httpContext)
: base(httpContext) {
_request = new JsonAntiForgeryHttpRequestWrapper(httpContext.Request);
}
public override HttpRequestBase Request {
get {
return _request;
}
}
}
private class JsonAntiForgeryHttpRequestWrapper : HttpRequestWrapper {
readonly NameValueCollection _form;
public JsonAntiForgeryHttpRequestWrapper(HttpRequest request)
: base(request) {
_form = new NameValueCollection(request.Form);
if (request.Headers["__RequestVerificationToken"] != null) {
_form["__RequestVerificationToken"]
= request.Headers["__RequestVerificationToken"];
}
}
public override NameValueCollection Form {
get {
return _form;
}
}
}
In general, you can get into all sorts of trouble when you hack around with the http context. But in this case, I’ve implemented a wrapper for a tightly constrained scenario that defers to default implementation for most things. The only thing I override is the request form. As you can see, I copy the form into a new NameValueCollection instance and if there is a request verification token in the header, I copy that value in the form too. I then use this modified collection as the Form collection.
Simple, but effective.
The custom attribute follows the basic implementation pattern of the regular attribute, but uses these new wrappers.
[AttributeUsage(AttributeTargets.Method | AttributeTargets.Class,
AllowMultiple = false, Inherited = true)]
public class ValidateJsonAntiForgeryTokenAttribute :
FilterAttribute, IAuthorizationFilter {
public void OnAuthorization(AuthorizationContext filterContext) {
if (filterContext == null) {
throw new ArgumentNullException("filterContext");
}
var httpContext = new JsonAntiForgeryHttpContextWrapper(HttpContext.Current);
AntiForgery.Validate(httpContext, Salt ?? string.Empty);
}
public string Salt {
get;
set;
}
// The private context classes go here
}
With that in place, I can now decorate action methods with this new attribute and it will work in both scenarios, whether I post a form or post JSON data. I updated the client script library for calling action methods to accept a second parameter, includeAntiForgeryToken, which causes it to add the anti-forgery token to the headers.
As always, the source code is up on Github with a sample application that demonstrates usage of this technique and the assembly is in NuGet with the package id “MvcHaack.Ajax”.
Go that way instead - Photo by JacobEnos CC some rights reserved
In an ASP.NET web application, it’s very common to write some jQuery code that makes an HTTP request to some URL (a lightweight service) in order to retrieve some data. That URL might be handled by an ASP.NET MVC controller action, a Web API operation, or even an ASP.NET Web Page or Web Form. If it can return curly brackets, it can be respond to a JavaScript request for JSON.
One pain point when hosting lightweight HTTP services on ASP.NET is making a request to a URL that requires authentication. Let’s look at a snippet of jQuery to illustrate what I mean. The following code makes a request to /admin/secret/data. Let’s assume that URL points to an ASP.NET MVC action with the AuthorizeAttribute applied, which requires that the request must be authenticated.
$.ajax({
url: '/admin/secret/data',
type: 'POST',
contentType: 'application/json; charset=utf-8',
statusCode: {
200: function (data) {
alert('200: Authenticated');
// Bind the JSON data to the UI
},
401: function (data) {
alert('401: Unauthenticated');
// Handle the 401 error here.
}
}
});
If the user is not logged in when this code executes, you would expect that the 401 status code function would get called. But if forms authentication (often called FormsAuth for short) is configured, that isn’t what actually happens. Instead, you get a 200 with the contents of the login page (or a 404 if you don’t have a login page). What gives?
If you crack open Fiddler, it’s easy to see the problem. Instead of the request returning an HTTP 401 Unauthorized status code, it instead returns a 302 pointing to a login page. This causes jQuery (well actually, the XmlHttpRequest object) to automatically follow the redirect and issue another request to the login page. The login page handles this new request and return its contents with a 200 status code. This is not the desired result as the code expects JSON data to be returned in response to a 200 code, not HTML for the login page.
This “helpful” behavior when requesting a URL that requires authentication is a consequence of having the FormsAuthenticationModule enabled, which is the default in most ASP.NET applications. Under the hood, the FormsAuthenticationModule hooks into the request pipeline and changes any request that returns a 401 status code into a redirect to the login page.
Possible Solutions
I’m going to cover a few possible solutions I’ve seen around the web and then present the one that I prefer. It’s not that these other solutions are wrong, but they are only correct in some cases.
Remove Forms Authentication
If you don’t need FormsAuth, one simple solution is to remove the forms authentication module as this post suggests. This is a great solution if you’re sole purpose is to use ASP.NET to host a Web API service and you don’t need forms authentication. But it’s not a great solution if your app is both a web application and a web service.
Register an HttpModule to convert Redirects to 401
This blog post suggests registering an HTTP Module that converts any 302 request to a 401. There are two problems with this approach. The first is that it breaks the case where the redirect is legitimate and not the result of FormsAuth. The second is that it requires manual configuration of an HttpModule.
Install-Package MembershipService.Mvc
My colleague, Steve Sanderson, has an even better approach with his MembershipService.Mvc and MembershipService.WebForms NuGet packages. These packages expose ASP.NET Membership as a service that you can call from multiple devices.
For example, if you want your Windows Phone application to use an ASP.NET website’s membership system to authenticate users of the application, you’d use his package. He provides the MembershipClient.WP7 and MembershipClient.JavaScript packages for writing clients that call into these services.
These packages deserve a blog post in their own right, but I’m going to just focus on the DoNotRedirectLoginModule he wrote. His module takes a similar approach to the previous one I mentioned, but he checks for a special value in HttpContext.Items, a dictionary for storing data related to the current request, before reverting a redirect back to a 401.
To prevent a FormsAuth redirect, an action method (or ASP.NET page or Web API operation) would simply call the helpful method DoNotRedirectToLoginModule.ApplyForRequest. This sets the special token in HttpContext.Items and the module will rewrite a 302 that’s redirecting to the login page back to a 401.
My Solution
Steve’s solution is a very good one. But I’m particularly lazy and didn’t want to have to call that method on every action when I’m writing an Ajax heavy application. So what I did was write a module that hooks in two events of the request.
The first event, PostReleaseRequestState, occurs after authentication, but before the FormsAuthenticationModule converts the status to a 302. In the event handler for this event, I check to see if the request is an Ajax request by checking that the X-Requested-With request header is “XMLHttpRequest”.
If so, I store away a token in the HttpContext.Items like Steve does. Then in the EndRequest event handler, I check for that token, just like Steve does. Inspired by Steve’s approach, I added a method to allow explicitly opting into this behavior, SuppressAuthenticationRedirect.
Here’s the code for this module. Warning: Consider this “proof-of-concept” code. I haven’t tested this thoroughly in a wide range of environments.
public class SuppressFormsAuthenticationRedirectModule : IHttpModule {
private static readonly object SuppressAuthenticationKey = new Object();
public static void SuppressAuthenticationRedirect(HttpContext context) {
context.Items[SuppressAuthenticationKey] = true;
}
public static void SuppressAuthenticationRedirect(HttpContextBase context) {
context.Items[SuppressAuthenticationKey] = true;
}
public void Init(HttpApplication context) {
context.PostReleaseRequestState += OnPostReleaseRequestState;
context.EndRequest += OnEndRequest;
}
private void OnPostReleaseRequestState(object source, EventArgs args) {
var context = (HttpApplication)source;
var response = context.Response;
var request = context.Request;
if (response.StatusCode == 401 && request.Headers["X-Requested-With"] ==
"XMLHttpRequest") {
SuppressAuthenticationRedirect(context.Context);
}
}
private void OnEndRequest(object source, EventArgs args) {
var context = (HttpApplication)source;
var response = context.Response;
if (context.Context.Items.Contains(SuppressAuthenticationKey)) {
response.TrySkipIisCustomErrors = true;
response.ClearContent();
response.StatusCode = 401;
response.RedirectLocation = null;
}
}
public void Dispose() {
}
public static void Register() {
DynamicModuleUtility.RegisterModule(
typeof(SuppressFormsAuthenticationRedirectModule));
}
}
There’s a package for that
Warning: The following is proof-of-concept code I’ve written. I haven’t tested it thoroughly in a production environment and I don’t provide any warranties or promises that it works and won’t kill your favorite pet. You’ve been warmed.
Naturally, I’ve written a NuGet package for this. Simply install the package and all Ajax requests that set that header (if you’re using jQuery, you’re all set) will not be redirected in the case of a 401.
Install-Package AspNetHaack
Note that the package adds a source code file in App_Start that wires up the http module that suppresses redirect. If you want to turn off this behavior temporarily, you can comment out that file and you’ll be back to the old behavior.
The source code for this is in Github as part of my broader CodeHaacks project.
Why don’t you just fix the FormsAuthenticationModule?
We realize this is a deficiency with the forms authentication module and we’re looking into hopefully fixing this for the next version of the Framework.