asp.net mvc, asp.net comments suggest edit

UPDATE: I updated this post slightly to address some good criticsm from Rob Mensching by removing the MsiZap section. Note that this post does not blame the Windows Installer for the few problems we have with ASP.NET MVC installations. The major culprit tends to be either devenv /setup or ngen.exe.

The point in this post is to provide tools for those who are very careful, know what they are doing, and really want to take a peek at how the installer works. These tools can help provide insight to whether the two culprits I just mentioned really are the issue causing the installation to fail in your particular case. Everything I point out here IS AT YOUR OWN RISK! :)

For the most part, the installations for ASP.NET MVC have been quite stable and we’ve tested the installer at every release and 99% of the problems we’ve run into fall into those two buckets. I did not intend to give the impression that we saved installation testing for the last moment, because that’s just not true. We merely had a last minute requirement change for our installer. RC 2 was a response to that change, not to a bug found. Now back to the post. :)

Having an installer crap out on you when you’re excited to try out a new piece of software is an incredibly frustrating experience. Frustrating both for the user as well as for the product team. Every time I read on Twitter that someone is having trouble installing ASP.NET MVC, a new ulcer throbs in my side.

A big part of the frustration is the lack of control and insight into what’s going wrong. Jacques (no blog), a developer on the ASP.NET team has put together a troubleshooting guide based on his research into what makes installers tick to help alleviate some of this lack of control. While his focus was primarily on the ASP.NET MVC installer, much of this information is useful for all MSI installers.

Windows Installer Log Files

Log files are not automatically generated by an MSI. To generate a log file, you will need to use msiexec from a command prompt to install the MSI, for example

msiexec /i AspNetMVC1-RC2.msi /l*v .\install.log

Alternatively you can enable logging by default by configuring a registry key for this. See this article on enabling Windows Installer Logging or if you are using Windows XP, take a look at this article on the same topic specific to Windows XP. Please note that enabling this feature can have an impact on system performance.

The MSI logs are probably the best place to start when trying to figure out why an installation failed. At the very least you will be able to identify the step in the installer that caused the problem.

For example, one common source of trouble is when a required service is not started such as:

  • .NET Optimization Service
  • ASP.Net State Service

Visual Studio Log Files

The ASP.NET MVC specific context menus in Visual Studio are installed by executing devenv.exe with the /setup switch. When the installer executes this command it automatically generates a log file at the location

%TEMP%\ MvcTemplates.log

The log file can be consulted for more details in case this step of the installation failed.

Orca

Orca is an editor for Windows install packages (MSI files) and is part of the Windows Installer SDK (the Orca MSI is located at %Program Files%\Microsoft SDKs\Windows\v6.0A\bin). While we don’t generally encourage people to edit the ASP.NET MVC installer, for those willing to take matters into their own hands (at your own risk!) Orca can be used to disable some of the steps executed by the installer. When you open the MSI you will see all the tables used by the installer. The feature table is probably the most interesting since it shows everything that will be installed.

clip_image002

Notice that by default, the Level attribute of all the features are set to 1. This means that the feature is enabled to be installed. We have some logic that will disable some of the features based on the information that we gather about the system on which were are installing. For example, the VWDExpress_Feature will be disabled if a user does not have Visual Web Developer installed. Similarly we will disable one of the MvcGacNgenXX_Features depending on whether the underlying OS is a 64-bit or 32-bit version.

Let’s say that you’ve tried to install MVC, but for some reason the installer fails to create a native image for System.Web.Mvc.dll using NGEN. Using Orca you can disable the MvcGACNgen32_Feature and MvcGACNgenAll_Feature by deleting these two rows from the MSI and saving it under a different name. Of course the best thing to do is to investigate why NGEN is failing in the first place.

Editing the MSI is not really encouraged since it may cause other problems such as failing to uninstall properly.

NGEN failures

We automatically GAC and NGEN System.Web.Mvc.dll during the setup process. Installing into the GAC (Global Assembly Cache) is a relatively painless operation. NGEN on the other hand is more prone to causing the installer to fail.

When installing via double clicknig the MSI, you will see an error message before the installation is rolled back if NGEN fails. The error message itself is not that useful as it will most likely just show a cryptic message and an HRESULT value (You can do a web search for the HRESULT and see whether there are workarounds available). To investigate the failure, take a look at the log generated by the MSI (assuming you had logging turned on when you made the install). Open the log file and search for ExecNetFx (this is the name of the WiX component the installer uses to run NGEN).

ExecNetFx:  Microsoft (R) CLR Native Image Generator 
  - Version 2.0.50727.3053
ExecNetFx:  Copyright (c) Microsoft Corporation.  All rights reserved.
ExecNetFx:  Installing assembly System.Web.Mvc, Version=1.0.0.0, 
  Culture=neutral, PublicKeyToken=31BF3856AD364E35
ExecNetFx:  Catastrophic failure (Exception from HRESULT: 0x8000FFFF 
  (E_UNEXPECTED))
ExecNetFx:  Error 0x8007006d: failed to allocate output string
ExecNetFx:  Error 0xffffffff: Command line returned an error.
ExecNetFx:  Error 0xffffffff: failed to execute Ngen command: 
  C:\Windows\Microsoft.NET\Framework\v2.0.50727\ngen.exe install 
  "System.Web.Mvc, Version=1.0.0.0, Culture=neutral, 
  PublicKeyToken=31BF3856AD364E35"

The example above is a very difficult problem to troubleshoot, but there are a few things you can try:

  • Open a command prompt, go to %WINDIR%\Microsoft.NET\Framework\v2.0.50727 and execute the following commands and then try to install MVC again
    • ngen ExecuteQueuedItems
    • ngen Update
  • Examine the NGEN log files located in %WINDIR%\Microsoft.NET\Framework\v2.0.50727. There are two files, ngen.log and ngen_service.log. The logs may contain information about other problems in the system that caused the failure, for example a missing DLL that may cause an update operation to fail when NGEN attempts to update invalid native images.
  • Make sure that the .NET Runtime Optimization service is running clip_image004

Windows Event Viewer

The installer executes a number of external tasks during the installation process.

  • devenv.exe /installvstemplates: This step is used to register the MVC project templates in Visual Studio. (For Visual Web Developer the executable is called VWDExpress.exe).
  • devenv.exe /setup: This step is executed to merge all the metadata in Visual Studio and is used by the MVC packages to register the context menus.
  • ngen.exe install: Used to generate native images of the System.Web.Mvc assembly. See this article for more details.
  • ngen.exe update: Updates any native images that have become invalid.

You will most likely see multiple warning events such as the example below. Visual Studio will log warning events for unknown attributes it encounters in templates and usually just moves past them. These warnings can be safely ignored.

clip_image006

But you may see other events that may cause the install to fail such as the example below:

clip_image008

Solving this problem is easy enough; restart the Cryptographic Service and run the installer again.

Visual Web Developer Express

A few users have reported that after installing MVC, they don’t see an option to create an MVC Web Application Project in Visual Web Developer. If you run into this problem, try the following:

  • Open VWD.
  • Go to Tools and select Import and Export Settings.
  • Select Reset all settings and click Next to complete the wizard.

Third Party Utilities

A number of users have reported problems during the installation of MVC or when trying to use some of the shortcuts inside Visual Studio. In many cases this was resolved by removing certain add-ins:

  • PowerCommands
  • Clone Detective
  • Spec# and F#

If you review the MSI logs you will most likely see something like the following:

MSI (s) (5C:70) [13:27:26:106]: Executing op: CustomActionSchedule(Action=VisualStudio_VSSetup_Command,ActionType=3122,Source=C:\Program Files\Microsoft Visual Studio 9.0\Common7\IDE\devenv.exe,Target=/setup,)

MSI (s) (5C:70) [13:28:41:075]: Note: 1: 1722 2: VisualStudio_VSSetup_Command 3: C:\Program Files\Microsoft Visual Studio 9.0\Common7\IDE\devenv.exe 4: /setup 
Error 1722. There is a problem with this Windows Installer package. A program run as part of the setup did not finish as expected. Contact your support personnel or package vendor. Action VisualStudio_VSSetup_Command, location: C:\Program Files\Microsoft Visual Studio 9.0\Common7\IDE\devenv.exe, command: /setup 

MSI (s) (5C:70) [13:28:42:716]: Product: Microsoft ASP.NET MVC RC 2 -- Error 1722. There is a problem with this Windows Installer package. A program run as part of the setup did not finish as expected. Contact your support personnel or package vendor. Action VisualStudio_VSSetup_Command, location: C:\Program Files\Microsoft Visual Studio 9.0\Common7\IDE\devenv.exe, command: /setup

If installing MVC fails in this manner, try the following:

  • Create a log file, for example, devenv.log.
  • Open a Visual Studio command prompt and execute devenv /setup /log devenv.log. Make sure that you have administrator rights since that’s required by the /setup switch.
  • Scan the log file and look for any <description> elements that contain errors. This may point to problematic packages that are causing devenv.exe to fail when MVC is trying to configure its context menus.

Another option to try is the following:

  • Open a Visual Studio command prompt.
  • Start Visual Studio by executing the command devenv /ResetSkipPkgs.
  • Close Visual Studio.
  • Execute devenv /setup.

Tags: aspnetmvc , msizap , installation

asp.net mvc, asp.net comments suggest edit

UPDATE: This post is outdated. ASP.NET MVC 1.0 has been released already!

Today we’ve made the Release Candidate 2 for ASP.NET MVC available for download.

This post will cover some of the changes with ASP.NET MVC we made in response to internal and external feedback since our last Release Candidate.

Let me provide the quick and dirty summary, and then fill in the details.

  • Setup will now require .NET 3.5 SP1
  • Bin deployment to 3.5 host without SP1still possible
  • New server-only install mode

Now onto the details

Setup Requires .NET Framework 3.5 SP1

The new installer will require that .NET Framework 3.5 SP1 be installed on your machine. For your development environment, we recommend that you also install Visual Studio 2008 SP1, but this is not required.

The reason we made this change is that we were including the System.Web.Routing.dll and System.Web.Abstractions.dll assemblies with the MVC installer. However, it does not make sense for us to co-ship assemblies which are part of the Framework as this would negatively affect our ability to service these two assemblies.

Bin deployment to 3.5 host without SP1 Still Possible

We are not taking a runtime dependency on SP1 other than our existing dependency on System.Web.Routing.dll and System.Web.Abstractions.dll. Thus you can still bin deploy your application to a hosting provider who has .NET 3.5 installed without SP1 by following these instructions.

Note that in such a configuration, you take on the risk of servicing those assemblies. Should we release any important updates to any of these assemblies, you’ll have to manually patch your application. However, you will still enjoy full CSS (formerly PSS) support for this configuration.

New Server-Only Install Mode

We’re adding an option to the installer that enables installing on a server that does not have Visual Studio at all on the machine, which is useful for production servers and hosting providers.

The installer will no longer block on a machine that does not have Visual Studio installed. Instead, it will continue the standard MVC installation without installing the Visual Studio templates. The assemblies will still be installed into the GAC and native images will also be generated.

Certain other requirements have also been relaxed. For example, if the machine on which the installation is performed contains Visual Web Developer Express Edition 2008 without SP1, the installation will still proceed, but with a warning prompt. You can also automate this installation by invoking the installer using the command line (all on one line):

msiexec /i AspNetMvc-setup.msi /q /l*v .\mvc.log MVC_SERVER_INSTALL="YES"

Summary

As a result of these changes, we realized it would be prudent to have one more public release candidate. As I mentioned, there are very few runtime and tooling changes. Most of the changes are in the installer and we want to make sure that the installer is rock solid before we call it an RTM.

Based on all your feedback from the first Release Candidate, as well as our own investigations and testing, we are confident that the Release Candidate 2 will be solid and lead to a strong RTM.

In case you missed it above, here’s the link to the download page. You can find out what else has changed in the RC 2 release notes.

And before I forget, as usual, we published the source code and our MvcFutures assembly on CodePlex.

asp.net mvc, asp.net comments suggest edit

MIX09_SpeakerBling_VegasStrip_136x186 I finally have my registration worked out and I’m officially going to Mix09 this year! Woohoo! Not only am I going, but I’ll be speaking at Mix for the first time, so be sure to come by and say hi.

I’ve mentioned this before, but Mix is easily one of my favorite Microsoft conferences. Jeff Atwood said it well:

MIX is by far my favorite Microsoft conference after attending the ’06 and ’07 iterations…

What I love about Mix is that it …

  • is relatively small and intimate, at around 2,000 attendees.
  • seamlessly merges software engineering and design.
  • includes a lot of non-Microsoft folks, even those that are traditionally hostile to Microsoft, so there’s plenty of perspective.

I’ve been to every one so far:

I’ll be giving the following two sessions at Mix this year:

MIX09-T50F ASP.NET MVC: America’s Next Top Model View Controller Framework

See how to build a simple application that highlights some of the key capabilities of ASP.NET MVC. Also, learn how to improve productivity using some of the cool new tooling features

MIX09-T44F Microsoft ASP.NET Model View Controller (MVC): Ninja on Fire Black Belt Tips

See a walkthrough of several tips and tricks for getting the most out of the ASP.NET MVC framework.

The first talk is an introductory overview and the second one allows me to dig in a bit more.

I’ve been working hard on putting together some demos so I will hopefully have something interesting to show off. In any case, if you’re going to the conference, leave a comment and when you’re there, be sure to come by and say hello.

Also, let me know if there’s anything specific you’d like me to cover. This should be a fun one.

personal comments suggest edit

Day two of ALT.NET is over and I’m already pooped (for you non-English speakers, that means tired, not something else that might come to mind).

Once again, photos by our Chronicler, Brad Wilson. As a testament to how engaging the sessions were, there are a lot fewer photos from day two in his photostream.

Encouraging Open Source on .NET by Brad
Wilson.

The first session I went to was on the topic of Encouraging Open Source in the .NET Space as seen above, which veered all over the place. Many felt the industry is shifting towards more and more Open Source software so those who can leverage that will be better off than those who can.

One interesting idea that came out of it was there’s a need for more education regarding Open Source. For example, understanding licensing is very challenging. It’d be great to have simple tools like a compatibility matrix or a license chooser (ala Creative Commons license generator).

In the meanwhile, I’ve tried to do my best to explain what little I know of software licensing in the past.

Another interesting point tied into our TDD discussion later on in the day related to the fact that for many shops, it doesn’t exist unless Microsoft endorses it.

.NET on the Mac, Linux, and iPhone by Brad
Wilson.

The next session I attended was “.NET/Mono on the Mac, Linux, and iPhone” facilitated by the always entertaining Miguel De Icaza and Joseph Hill. I’m quite certain that everyone was there because they wanted to pull in $10,000 a day writing the next Fart app for the iPhone using C#.

TDD and Microsoft by Brad
Wilson.

Later in the day, Karen Liu (a PM on the Managed Languages/IDE team), Euan Garden (PM for Visual Studio Test), and I (photo above) gave a wide ranging session on TDD and Microsoft, which covered investments we’re making across Developer Division to help make the experience for writing proper unit tests better for developers such as ASP.NET MVC, WPF’s Model-View-ViewModel, Visual Studio improvements, Silverlight, etc….

The focus of our efforts has been addressing the need for our tools and frameworks to support all developers who write automated tests. But the bar set by the expectations of a TDD developer is typically very high, and by striving to meet those expectations, we feel all developers benefit.

I showed off a few slides to set the context for what groups we’ve talked to and what improvements we’re seeing start to happen for the next wave of products.

Karen Liu gave a few demos of how Visual Studio 2010 greatly improves the workflow for a test-first developer and Euan Garden led a discussion about the Visual Studio unit testing framework.

We discussed where we’d all like to see Visual Studio take unit testing in the future. There were quite a few voices who said they’d like to see Visual Studio include something like xUnit.NET, much in the same way that ASP.NET MVC includes jQuery, because for the places where they work, for better or worse, it doesn’t exist unless it’s part of Visual Studio. This is a common theme I’ve heard when it comes to people wanting to promote a development tool at their workplace for which Microsoft does not have an alternative.

To illustrate this point, one person came up to me and told me he had never used jQuery (and didn’t write a lot of JavaScript) until it was included in ASP.NET MVC and now he’s very happily using jQuery with ASP.NET MVC.

To me, that’s a great testimonial for how leveraging Open Source in smart ways can make customers happier.

personal comments suggest edit

The third and last day of ALT.NET Seattle was a short one for me. My poor extremely patient wife was not feeling well today so I had to leave right at lunch time. But before I left, I did manage to attend a great session by John Lam and Jimmy Schementi on “Adding Scripting Support to .NET Applications”. In fact, you can watch the session here via Kyte.tv.

John and Jimmy covered the topic of hosting IronRuby to provide “end-user” ability to script an application. The classic example is that many 3-D games, Half-Life for example, write their core 3-D engines etc… in C++. However, they often provide a scripting language such as LUA which allows others to script the behavior of objects in the game, as these types of things change often and you want the flexibility to do so in a lightweight language.

The same may be true in an enterprise application where you want to write the whole app in C#, but perhaps allow configuring the rules engine using IronRuby.

I showed up to the session late and asked if John and Jimmy were going to show a demo. Unfortunately, they didn’t have one prepared, so I gave everyone a sneak preview of a demo where I have an ASP.NET MVC app hosting IronRuby in a manner where I think some C# developers will see the value. It was just a little something I’ve been hacking away at in my copious free time and hope to present at the MVP Summit and Mix. I won’t talk about it yet as I have a bit more work I want to do before I put it on my blog.

Before the session Aaron Jensen showed me some really cool stuff he’s doing with Spark View Engine and ASP.NET MVC. After the session, Eric Hexter showed me some neat stuff his company has been doing with T4 templates and ASP.NET MVC. This really got me fired me up with ideas for the next version of ASP.NET MVC to help support such efforts.

One of the coolest things at this conference was the usage of Kyte.tv. I honestly hadn’t heard of it until last week, but everyone was recording sessions live via Kyte.tv. After going home early, I was able to get back online and watch a couple more sessions recorded by Scott Hanselman when my son and wife took a nap. How cool was that.

personal comments suggest edit

 

ALT.NET Opening Ceremonies by Brad
Wilson

Day one of the ALT.NET Seattle conference is over and I’m looking forward to tomorrow’s sessions.

As an Open Spaces event, the first order of business was for us, the attendees, to set the session agenda for the rest of the conference. In the above photo, you can see Scott Hanselman proposing a topic in one of my favorite conference photos taken by Brad Wilson. This process took about two hours after which many of us headed out to Red Robin for a nerd dinner.

This is my second ALT.NET Open Spaces event and I really like the principles set forth:

  • Whoever comes is the right people
  • Whatever happens is the only thing that could have happened
  • Whenever it starts is the right time
  • When it’s over, it’s over

In some ways, it reminds me of the 10 Principles of BurningMan, specifically these two:

Radical Inclusion

Anyone may be a part of Burning Man. We welcome and respect the stranger. No prerequisites exist for participation in our community.

Participation \ Our community is committed to a radically participatory ethic. We believe that transformative change, whether in the individual or in society, can occur only through the medium of deeply personal participation. We achieve being through doing. Everyone is invited to work. Everyone is invited to play. We make the world real through actions that open the heart.

There’s no hard line drawn between attendees and speakers. Everyone is welcome to be both attendee and speaker as everyone generally has something interesting to share, teach, and learn.

The other part I love about this conference is seeing familiar faces as well as meeting collaborators from around the world I’ve never met in person until now. For example, Steve Harman and I finally meet Simone Chiaretta in person, the third Musketeer of Subtext, who apparently can’t keep his eyes open for a photo after being up for over 30 hours due to flying in from Italy.

3315077517_855eae566b

I look forward to seeing what tomorrow brings.

Technorati Tags: alt.net,altnetseattle

asp.net mvc, asp.net, dlr comments suggest edit

Last July, I blogged about an IronRuby ASP.NET MVC prototype Levi and I put together with John Lam and Jimmy Schementi of the DLR team. It was really rough around the edges (and still is!)

IronRuby on ASP.NET MVC
DemoOne of the benefits of doing that prototype was that it inspired all the work around action and controller descriptors in ASP.NET MVC (something I need to write more about later) which decoupled us from exposing reflection in our public API and improved the overall design of ASP.NET MVC greatly. This had the nice side-effect of making the implementation of IronRuby on top of ASP.NET MVC much cleaner.

In this updated prototype, I’ve now implemented support for ASP.NET MVC filters. You can define action filters and authentication filters (I need to test the other filter types). Keep in mind, this is a very rough prototype code still. I’ve just been swamped up to my neck lately and this is a spare-time labor of love.

I’ve only implemented one type of filter so far. You can specify a class to apply to an action method and the class implements a specific filter interface. I haven’t done anything like the more rails-y filter_before and filter_after thing.

Here’s an example of an action filter in IronRuby. This one simply writes something to the response in the before method, and does nothing in the after method.

class MyFilter < IronRubyMvc::Controllers::RubyActionFilter
    def on_action_executing(context)
      context.http_context.response.write 'MyFilter '
    end
    
    def on_action_executed(context)
      # noop
    end
    
    def method_missing(name, *args)
        show_action_info name, args
    end
end

(Gee, I wish I had a ruby syntax highlighter plug-in for WLW)

And here’s the use of that filter within a controller.

require 'HomeModel'
require 'MyFilter'

class HomeController < Controller
  filter :index, MyFilter

  def index
    view nil, 'layout', HomeModel.new
  end  
end

Notice that the way you define a filter on the index action is:

filter :action_name, FilterClassName

In the sample code I uploaded, you can see the effects of the filter at the top of the page. :) Hopefully I’ll find more time to update this, but as I said, it’s a labor of love, but time is in short supply.

In the meanwhile, I also need to look into whether there’s enough interest to make this a CodePlex project. There’s a bit of due diligence I have to do before I put code up on CodePlex, which is why I haven’t done it already because I’ve been busy.

And before I forget, here’s the download location for the sample.

Ivan Porto Carerra has taken this prototype and is running with it. To download the latest, check out his IronRubyMVC GitHub project.

code comments suggest edit

Alternate Title: Linq, it’s not just for SQL.

I admit, I’m not very proficient with functional programming. It almost feels like a gang war at times - on one side of the tracks is Turing’s crew, sporting their imperative ways. On the other side is the Church group, luring wayward souls onto their turf with the promise of code salvation in the form of functional language.

Matt Podwysocki is one of those Church evangelists, constantly reaching out to me, a lost soul, with the promises of eternal code salvation in the form of F#. I keep meaning to check it out, but you know how that goes.

What I’ve slowly come to realize though is that the more I use and understand the Linq extensions in C#, the more functional my programming has become in certain cases.

450px-Native_American_tobacco_flowerIt turns out that C# is the ultimate gateway drug for functional programming^1^. It has just enough functional elements to give you a taste for functional, but with enough friction at times (for example, the type inference is not as good as F#) that it may slowly push me over.

Let me give you a recent concrete example using Subtext. One of the features Subtext has is the ability to take the title of a blog post, and automatically generate a URL slug from the title.

Slugs have to be unique, so we want to make sure that the slug we generate doesn’t conflict. For fun, I implemented it in such a way that we could handle up to 6 conflicts.

For example, suppose you wrote a blog post entitled “Hello World”. The first time you published it, you would have the slug “hello-world”. When you wrote and published a new blog post with the same title, we’d append “again” to the end. Here’s what would happen if you kept repeating this process.

  • hello-world
  • hello-world-again
  • hello-world-yet-again
  • hello-world-and-again
  • hello-world-once-again
  • hello-world-to-beat-a-dead-horse
  • We throw an exception

Yeah, it’s kind of a stupid feature. I doubt anyone would ever post more than two blog posts with the same title. In a way, it’s a bit of an easter egg. The original code for this was very imperative. Here’s the gist of the code:

string EnsureUniqueSlug(string slug, string separator) {
  Entry currentEntry = Repository.GetEntry(slug);
  int tryCount = 0;
  string newSlug;
  while (currentEntry != null) {
    switch (tryCount) {
      case 0:
        newSlug = slug + separator + "Again";
        break;
      case 1:
        newSlug = slug + separator + "Yet" + separator + "Again";
        break;
      case 2:
        newSlug = slug + separator + "And" + separator + "Again";
        break;
      case 3:
        newSlug = slug + separator + "Once" + separator + "More";
        break;
      case 4:
        newSlug = slug + separator + "To" + separator + "Beat" 
          + separator + "A" + separator + "Dead" 
          + separator + "Horse";
        break;
      case 5:
        throw new InvalidOperationException();
    }
    tryCount++;
    currentEntry = Repository.GetEntry(newslug);
  }
  return newSlug;
}

I was revisiting this code today, and I realized I could write this more succinctly using Linq extensions.

When you step back a moment, what I have to start with is an enumeration of suffixes. What I want to do is transform them into potential slugs, and then find the first slug where there is no matching slug in the database.

Functional languages are great for working with sets and doing transformations over sets. At least that’s what Matt tells me. Here’s what I ended up doing.

string EnsureUniqueness(string originalSlug, string separator) {
  string[] suffixFormats = new[] { 
    string.Empty, "{0}Again", "{0}Yet{0}Again", "{0}And{0}Again"
      , "{0}Once{0}Again", "{0}Once{0}More"
      , "{0}To{0}Beat{0}A{0}Dead{0}Horse" };
  var slugs = suffixFormats.Select(
    s => originalSlug + String.Format(s, separator));
  return slugs.First(slug => Repository.GetEntry(slug) == null);
}

The first line is a static array of the potential suffixes. At this point, I should really move this line outside of this method and perhaps even have this list be configurable. But for this blog post, let’s leave it here.

The second line converts the suffixes into an enumeration of slugs. I then simply call the First method on that list of slugs passing in a lambda which specifies a condition. The First method will return the first element in the enumeration where the lambda returns true.

In other words, it’ll return the first slug where the repository tells me there is no blog post with a matching slug. If there is no match, the First method throws an InvalidOperationException.

For those who are not familiar with lambdas and and the extension methods I used, the second code might be a bit confusing. But once you know what’s going on, I think it’s much more readable, simple, and shows my intent better.

It reads how I think about the problem.

  1. Convert the list of suffixes into a list of potential slugs
  2. Grab the first slug where there is no matching entry in the database

What would be really cool is if I could somehow switch to F# inline with a C# file. Kind of crazy, but it would be the thing that would probably get me to actually use it in a project.

For those of you who have been doing functional programming for a long time, you’ll probably scoff at this simple example, but for an old imperative programmer like me, it feels like a new world opening up.

^1^After I wrote this, I realized that Ruby might actually be the ultimate gateway drug for functional programming. But I’m kind of focusing on static typed language afficionados, so forgive me. ;)

Tags: functional , linq

asp.net mvc, asp.net, code comments suggest edit

Today I read something where someone was comparing Web Forms to ASP.NET MVC and suggested that Web Forms does a lot more than ASP.NET MVC to protect your site from malicious attacks.

One example cited was that Server controls automatically handled HTML encoding so you don’t have to really think about it. The idea here is that Web Forms automatically protects you from XSS attacks.

My friends, I’m afraid this is just not true. Take a look at the following page code.

<%@ Page Language="C#" Inherits="System.Web.UI.Page" %>
<%
//For demo purposes, we have inline code here.
// Pretend the following userInput came from the database
string userInput = "<script>alert('You’ve been Haacked!');</script>";
label1.Text = userInput;
literal1.Text = userInput;
%>

<html>
<head>
    <title></title>
</head>
<body>
    <form id="form1" runat="server">
    <div>
        <asp:Label ID="label1" runat="server" />
        <asp:Literal ID="literal1" runat="server" />
    </div>
    </form>
</body>
</html>

In this page, we simulate taking some some user input we got from somewhere, whether it’s from a form post or from a database, and we place it in a Label control, and in a Literal control.

These are two of the most common controls used to display user input. I’ll save you the suspense of having to actually try it out, and just show you what happens when run this page.

Message from
webpage

Contrary to popular belief, these controls do not automatically HTML encode their output. I don’t see this as some gaping security flaw because it may well be that the intention of these controls, and general usage, is to display HTML markup the developer specifies, not what the user specifies. The potential security flaw lies in using these controls without understanding what they actually do.

The lesson here is that you always have to think about security. There’s no silver bullet. There’s no panacea. Damien Guard has a great post where he lists other signs your ASP.NET application might be succeptible to injection attacks, pointing out various ways that protection is not automatic.

The best approach is to take a more holistic approach. Create a threat model for your website and start attacking your own site as if you were a hacker, looking for flaws. Conduct security reviews and use any automated tools you can find for finding potential flaws. I recommend taking a look at CAT.NET combined with the AntiXss library.

In this particular case, I don’t think Web Forms provides any more automatic security than ASP.NET MVC. With MVC, we’ve swapped server controls with our helper methods, which properly encode output. If you don’t use our helpers, it’s roughly equivalent to not using the server controls.

Interestingly enough, in order to get that particular user input to the page in the first place is tricky. If you were to create a Web Form with a text input and a button, and type that script tag into text box and click the button, you’d be greeted by the following yellow screen of death.

request-validation

By default, ASP.NET has Request Validation turned on, which prevents requests with suspicious looking data such as the one I tried. Note that ASP.NET MVC also has Request Validation turned on by default too. You can turn it off per Controller or Action via the ValidateRequestAttribute like so.

[ValidateInput(false)]
public ActionResult SomeAction(string someInput) {
}

This is not to say that I think ASP.NET MVC provides just as much automatic protection that Web Forms does. This is not exactly the case. There are some cases where Web Forms does provide more automatic protection that ASP.NET MVC leaves to you, the developer.

For example, ASP.NET MVC does not have an automatic equivalent of Event Validation which was introduced in ASP.NET 2.0. Note that event validation is very different from request validation and is very specific in to server controls. For example, as the blog post I linked to mentions, if you add a DropDownList control with three options, and a user posts a non-existent option, you will get an exception. ASP.NET MVC doesn’t have such automatic validation. In some cases, this is a good thing because it makes AJAX scenarios simpler.

What ASP.NET MVC does have is a set of Anti Forgery (CSRF) helpers which require a bit of manual intervention.

To recap, while I do agree that Web Forms does provide a bit more automatic security than ASP.NET MVC, the gap is not as wide as you might think. Server controls are no more nor less secure than using the Helpers with ASP.NET MVC. And all of that is irrelevant because it is still up to the developer’s to take responsibility for the security of his or her site. I’ve heard of many developers who had to turn off Request and Event Validation for various reasons. In those cases, they examined the attack vectors opened up by these changes and provided alternate protections to replace the ones they turned off.

asp.net, asp.net mvc comments suggest edit

Here’s a little known fact about T4 Templates…well besides that they exist in the first place. Visual Studio 2005 had support for T3 templates. T4 is the natural successor to T3. T3 is also the abbreviation used when marketing the movie Terminator 3: Rise of the Machines. I heard that the names are related, but I haven’t confirmed it beyond a doubt, which seems appropriate given the title of the movie. But you didn’t hear that from me, deal?

terminator3 Dude looks ready to write some code!

Although they’ve been around forever, I haven’t really dived into T4 templates until recently as I’ve been using them in Subtext. I’m excited that we’re now incorporating them into ASP.NET MVC.

The Visual Web Developer Team Blog just posted an in-depth look at T4 templates in ASP.NET MVC entitled, T4 Templates: A Quick-Start Guide for ASP.NET MVC Developers.

Be sure to check out their other post covering What’s New for MVC Tools in ASP.NET MVC 1.0 Release Candidate.

asp.net, asp.net mvc comments suggest edit

UPDATE: I have a followup to this post that works for down-level browsers.

In a recent post, Stephen Walther pointed out the dangers of using a link to delete data. Go read it as it provides very good coverage of the issues. The problem is not restricted to delete operations. Any time you allow a GET request to modify data, you’re asking for trouble. Read this story about something that happened to BackPack way back in the day to see what I mean.

The reason that delete operations deserve special attention is that it’s the most common case where you would use a link to change information. If you were editing a product record, for example, you would use a form. But a delete operation typically only needs one piece of information (the id) which is easy to encode in the URL of a GET request.

If you are using jQuery, one simple way to turn any link into a POST link is to add the following onclick attribute value:

$.post(this.href); return false;

For example

<a href="/go/delete/1" onclick="$.post(this.href); return false;">Delete</a>

Will now make a POST request to /go/delete/1 rather than a GET. Of course, you need to enforce this on the server side. This is pretty easy with ASP.NET MVC.

[AcceptVerbs(HttpVerbs.Post)]
public ActionResult Delete(int id) {
  //Delete that stuff!
}

The AcceptVerbs attribute specifies that this action method only responds to POST requests, not GET requests.

At this point, you could easily write helpers specifically for delete links. I usually write very specific helper methods such as Html.DeleteProduct or Html.DeleteQuestion. Here’s an example of one I wrote for a sample app I’m building.

public static string DeleteAnswerLink(this HtmlHelper html, string linkText
  , Answer answer) {
    return html.RouteLink(linkText, "answer",
        new { answerId = answer.Id, action = "delete" }, 
        new { onclick="$.post(this.href); return false;" });
}

The nice thing about this approach is that you can leverage the existing helper methods by adding a minimal amount of extra information via the onclick attribute.

I hope the combination of Stephen’s post and this post will lead you to safer deleting.

asp.net mvc, asp.net comments suggest edit

Earlier this morning, I posted on making a simple jQuery delete link which makes it easy to create a delete link that does a form post to a delete action. Commenters pointed out that my solution won’t work for down-level browsers such as some mobile phones, and they were right. I wasn’t really concerned about down-level browsers.

One solution for down-level browsers is to render a proper form with a submit button, and then hide the form with JavaScript. Of course this takes a bit more work. Here’s what I did. I made sure I had the following script in my master template.

<script type="text/javascript">
 $("form.delete-link").css("display", "none");
 $("a.delete-link").show();
 $("a.delete-link").live('click', function(ev) {
    ev.preventDefault(); 
    $("form.delete-link").submit(); 
 });
</script>

When the following HTML is rendered in the page…

<form method="post" action="/go/delete/1" 
  class="delete-link">
  <input type="submit" value="delete" />
  <input name="__RequestVerificationToken" type="hidden" 
  value="Jrcn83M7T...8Z6RkdIfMZIJ5mVb" />
</form>
<a class="delete-link" href="/go/delete/1" 
  style="display:none;">delete</a>

… the jQuery code shown above will hide the form, but display the link (notice the link is hidden by default). When the link is clicked, it posts the form. However, in cases where there is no JavaScript, the form will be displayed, but the link will not be because the JavaScript is the thing that hides the form.

To make this easier to use, I wrote the following helper:

public static string DeleteLink(this HtmlHelper html
  , string linkText
  , string routeName
  , object routeValues) {
  var urlHelper = new UrlHelper(html.ViewContext.RequestContext);
  string url = urlHelper.RouteUrl(routeName, routeValues);

  string format = @"<form method=""post"" action=""{0}"" 
  class=""delete-link"">
<input type=""submit"" value=""{1}"" />
{2}
</form>";

  string form = string.Format(format, html.AttributeEncode(url)
    , html.AttributeEncode(linkText)
    , html.AntiForgeryToken());
  return form + html.RouteLink(linkText, routeName, routeValues
  , new { @class = "delete-link", style = "display:none;" });
}

Notice that we’re using the AntiForgery helpers included with ASP.NET MVC. What this means is that I need to make one small change to my delete method on my controller. I need to add the ValidateAntiForgeryToken attribute to the method.

[ValidateAntiForgeryToken]
[AcceptVerbs(HttpVerbs.Post)]
public ActionResult Delete(int id) {
  //Delete it
}

I’ve left out a bit. For example, I didn’t specify a callback to the jQuery code. So what should happen when this action method returns? I leave that as an exercise to the reader. I may address it in a future follow-up to this blog post. In my code, I’m just being cheesy and doing a full redirect, which works fine.

asp.net mvc, asp.net comments suggest edit

Hello there. :)

On Tuesday, we announced the release candidate for ASP.NET MVC. While there is much new in there to be excited about and many many bug fixes, there were two changes introduced in the RC that broke some scenarios which previously worked in the Beta, as reported by customers.

We’ve updated the Release Candidate with a refresh that addresses these two issues. You can use the recently released Microsoft Web Platform Installer 1.0 to install ASP.NET MVC RC Refresh. It happens to be a handy tool for installing not just ASP.NET MVC, but everything you might need to use ASP.NET MVC such as Visual Web Developer Express 2008 SP1.

The link on the official download page appears to be updated with the new MSI (we pushed it out yesterday), but we’ve experienced some odd proxy caching issues etc where some people were still getting the old MSI.

In order to be safe, you can get the download directly from this download link.I don’t anticipate any problems with that link, but being paranoid, the way to fully ensure you have the refresh is to right click on the downloaded file, select the Digital Signatures tab, and make sure the Time Stamp says Wednesday, January 28 and not Friday, January 23.

msi-props

Don’t forget to fully uninstall the previous RC before installing this one.

Also note that this refresh does not address the Controls Collection Cannot Be Modified issue I reported recently. We will address that soon. The following describes the issues that this refresh does fix.

The first change was that we changed our helper methods that generate URLs and links to now generate relative paths instead of absolute paths. This caused problems with several AJAX scenarios.

The fix to this in the refresh was to roll back that check-in so the behavior went back to the way it was in the Beta. We will not use relative paths, and we have no intention of re-introducing the generation of relative paths. Because we rolled back this change to a known good state, we feel very confident in the current behavior. URL generation should work the same way it did back in the Beta.

The other regression is that in some cases, the RouteUrl (and thus RouteLink) methods return an empty string when you specify a route name, but the route has default parameters which are not parameters in the URL.

For example, if you have the following route:

routes.MapRoute("route-name", 
  "foo/bar", 
  new {controller="Home", action="index"});

Notice that controller has default value, but is not part of the URL. If you then specify:

<%= Url.RouteUrl("route-name") %>

You might expect that it would use that route to render the URL, but it doesn’t. This bug was introduced when we refactored all our url generating helpers to call into a common method. It turns out, however, that our RouteUrl methods (aka non-MVC specific) should have subtly different behavior than the MVC specific methods (such as Action). We added a flag to the common method so that this difference is taken into consideration. This was a fix that did not have a large surface area.

Another thing we did was update all our test cases (both unit tests and automated functional tests) with scenarios reported to us by customers. So now, we have these types of cases well covered.

If you downloaded the Release Candidate before today, you should check the digital signature timestamp as I described earlier and if it’s the old one, I recommend you go and download the refresh via the Web Platform Installer or directly at the URL I mentioned earlier.

As I mentioned before, we are very excited about this release and hope that you are enjoying writing code with it. :)

asp.net mvc, asp.net comments suggest edit

In my last post, I announced the happy news that the Release Candidate for ASP.NET MVC is available. In this post, I say mea culpa for a known bug within this release.

This bug is a consequence of a change we made in our default template. We know have a content placeholder in the <head> section of the Site.master page.

<head runat="server">
    <asp:ContentPlaceHolder ID="head" runat="server">
        <title></title>
    </asp:ContentPlaceHolder>
    <link href="../../Content/Site.css" rel="stylesheet" type="text/css" />
</head>

The benefit here is that it makes it easy to specify view specific scripts, style sheets, and title tags from the specific view, like so.

<asp:Content ID="indexHead" ContentPlaceHolderID="head" runat="server">
    <title>Home Page</title>
    <style type="text/css">
        /* Some style specific to this view */
    </style>
</asp:Content>

Long story short, if the Header control (aka <head runat="server" />) doesn’t see a title tag as a direct child of itself, it renders an empty title tag in order to ensure the page has valid HTML. This results in having two title tags, the one you intended and an empty one.

We put in a fix for this whereby we modify the controls collection of the header control, but the fix itself causes a problem when you have code nuggets within the <head> section. For example, the following causes an exception.

<head runat="server">
    <script src="<%= Url.Content("~/foo.js") %>" type="text/javascript">
    </script>
    <asp:ContentPlaceHolder ID="head" runat="server">
        <title></title>
    </asp:ContentPlaceHolder>
    <link href="../../Content/Site.css" rel="stylesheet" type="text/css" />
</head>

The exception this causes is:

The Controls collection cannot be modified because the control contains code blocks (i.e. <% … %>).

We unfortunately didn’t find this until very recently. The current workaround is simple. Place the script tag within a normal PlaceHolder control.

<head runat="server">
    <asp:PlaceHolder runat="server" id="mainScripts">
        <script src="<%= Url.Content("~/foo.js") %>" type="text/javascript">
        </script>
    </asp:PlaceHolder>
    <asp:ContentPlaceHolder ID="head" runat="server">
        <title></title>
    </asp:ContentPlaceHolder>
    <link href="../../Content/Site.css" rel="stylesheet" type="text/css" />
</head>

We have one simple solution to this we are bouncing around, and are investigating alternative solutions. We apologize for any inconveniences this may cause.

asp.net mvc, asp.net comments suggest edit

At long last I am happy, relieved, excited to announce the release candidate for ASP.NET MVC. Feel free to go download it now. I’ll wait right here patiently.

There have been a lot of improvements made since the Beta release so be sure to read the release notes. I’ve tried very hard to be thorough in the notes so do let me know if anything is lacking. We are also pushing new tutorials up to the ASP.NET MVC Website as I write this.

Also, don’t miss ScottGu’s usual epic blog post describing the many improvements. There’s also a post on the Web Tools team blog covering tooling aspects of this release in detail. As I mentioned when we released the Beta, we didn’t have plans for many new features in the runtime, but we did have a lot of tooling improvements to add. I’ve already described some of these changes in a previous blog post, as did ScottGu in his detailed look.

Our goal with this release was to fix all outstanding bugs which we felt were real showstoppers and try to address specific customer concerns. We worked hard to add a bit of spit and polish to this release.

Unfortunately, a few minor bugs did crop up at the last second, but we decided we could continue with this RC and fix the bugs afterwards as the impact appear to be relatively small and they all have workarounds. I wrote about one particular bug so that you’re aware of it.

For now, I want to share a few highlights. This is not an exhaustive list at all. For that, check out Scott’s post and the release notes.

Ajax

Yes, I do know that jQuery released a new version (1.3.1), and no, it is not in this release. :) We just didn’t have time to include it due to the timing of its release. However, we are performing due diligence right now and plan to include it with the RTM.

We did make some changes to our Ajax helpers to recognize the standard X-Requested-With HTTP header used by the major JavaScript libraries such as Prototype.js, jQuery, and Dojo. Thus the IsMvcAjaxRequest method was renamed to IsAjaxRequest and looks for this header rather than our custom one.

ControllerContext

ControllerContext no longer inherits from RequestContext, which will improve testing and extensibility scenarios. We would have liked to make changes to RequestContext, but it was introduced as part of the .NET Framework in ASP.NET 3.5 SP1. Thus we can’t change it in our out-of-band release.

Anti Forgery Helpers

These helpers were previously released in our “futures” assembly, but we’ve fixed a few bugs and moved them into the core ASP.NET MVC assembly.

These are helpers which help mitigate Cross-Site Request Forgery (CSRF) attacks. For a great description of these helpers, check out Steve Sanderson’s blog post on the topic.

MVC Futures

I added a couple of expression based helpers to the ASP.NET MVC futures assembly, Microsoft.Web.Mvc.dll. These are just samples to demonstrate how one could write such helpers. I’ll add a few more by the time we ship the RTM. Note, if you’re using the old futures assembly, it won’t work with the new RC. You’ll need to update to the new Futures assembly.

In case you missed it the first time,click here for the Download.

asp.net comments suggest edit

A while back on a lark, I posted a prototype demonstrating how one could use Routing within Web Forms. This is something you can do today with ASP.NET 3.5 SP1, because of the work we did to separate Routing from ASP.NET MVC. I would have liked to include Web Form Routing as part of the Routing feature when we were working on SP1, but we didn’t have the time to do so in a robust manner before SP1 was locked down.

Since then, Scott Galloway, who just happens to be my office mate, has taken the reigns and is the PM in change of guiding the Web Form Routing feature to be included in ASP.NET 4.0 in a more integrated fashion.

He wrote a blog post earlier today describing some of the ways that routing will be more deeply integrated into ASP.NET such as new properties of the Page class, cool uses of Expression Builders, etc… That’s just a teaser as he promises to go a little more in depth soon.

Technorati Tags: Routing,ASP.NET

code comments suggest edit

UPDATE: Be sure to read Peli’s post in which he explores all of these implementations using PEX. Apparently I have a lot more unit tests to write in order to define the expected behavior of the code.

I recently wrote a post in which I examined some existing implementations of named format methods and then described the fun I had writing a routine for implementing named formats.

In response to that post, I received two new implementations of the method that are worth calling out.

The first was sent to me by Atif Aziz, who I’ve corresponded with via email regarding Jayrock. He noticed that the slowest format method by James had one obvious performance flaw and fixed it up. He then sent me another version that passed all my unit tests. Here it is in full, and it’s much faster than before.

public static class JamesFormatter
{
  public static string JamesFormat(this string format, object source)
  {
    return FormatWith(format, null, source);
  }

  public static string FormatWith(this string format
      , IFormatProvider provider, object source)
  {
    if (format == null)
      throw new ArgumentNullException("format");

    List<object> values = new List<object>();
    string rewrittenFormat = Regex.Replace(format,
      @"(?<start>\{)+(?<property>[\w\.\[\]]+)(?<format>:[^}]+)?(?<end>\})+",
      delegate(Match m)
      {
        Group startGroup = m.Groups["start"];
        Group propertyGroup = m.Groups["property"];
        Group formatGroup = m.Groups["format"];
        Group endGroup = m.Groups["end"];

        values.Add((propertyGroup.Value == "0")
          ? source
          : Eval(source, propertyGroup.Value));

        int openings = startGroup.Captures.Count;
        int closings = endGroup.Captures.Count;

        return openings > closings || openings % 2 == 0
           ? m.Value
           : new string('{', openings) + (values.Count - 1) 
             + formatGroup.Value
             + new string('}', closings);
      },
      RegexOptions.Compiled 
      | RegexOptions.CultureInvariant 
      | RegexOptions.IgnoreCase);

    return string.Format(provider, rewrittenFormat, values.ToArray());
  }

  private static object Eval(object source, string expression)
  {
    try
    {
      return DataBinder.Eval(source, expression);
    }
    catch (HttpException e)
    {
      throw new FormatException(null, e);
    }
  }
}

The other version I got was from another Microsoftie named Henri Wiechers (no blog) who pointed out that a state machine of the sort that a parser or scanner would use, is well suited for this task. It also makes the code easier to understand if you’re accustomed to state machines.

public static class HenriFormatter
{
  private static string OutExpression(object source, string expression)
  {
    string format = "";

    int colonIndex = expression.IndexOf(':');
    if (colonIndex > 0)
    {
      format = expression.Substring(colonIndex + 1);
      expression = expression.Substring(0, colonIndex);
    }

    try
    {
      if (String.IsNullOrEmpty(format))
      {
        return (DataBinder.Eval(source, expression) ?? "").ToString();
      }
      return DataBinder.Eval(source, expression, "{0:" + format + "}") 
          ?? "";
    }
    catch (HttpException)
    {
      throw new FormatException();
    }
  }

  public static string HenriFormat(this string format, object source)
  {
    if (format == null)
    {
      throw new ArgumentNullException("format");
    }

    StringBuilder result = new StringBuilder(format.Length * 2);      

    using (var reader = new StringReader(format))
    {
      StringBuilder expression = new StringBuilder();
      int @char = -1;

      State state = State.OutsideExpression;
      do
      {
        switch (state)
        {
          case State.OutsideExpression:
            @char = reader.Read();
            switch (@char)
            {
              case -1:
                state = State.End;
                break;
              case '{':
                state = State.OnOpenBracket;
                break;
              case '}':
                state = State.OnCloseBracket;
                break;
              default:
                result.Append((char)@char);
                break;
            }
            break;
          case State.OnOpenBracket:
            @char = reader.Read();
            switch (@char)
            {
              case -1:
                throw new FormatException();
              case '{':
                result.Append('{');
                state = State.OutsideExpression;
                break;
              default:
                expression.Append((char)@char);
                state = State.InsideExpression;
                break;
            }
            break;
          case State.InsideExpression:
            @char = reader.Read();
            switch (@char)
            {
              case -1:
                throw new FormatException();
              case '}':
                result.Append(OutExpression(source, expression.ToString()));
                expression.Length = 0;
                state = State.OutsideExpression;
                break;
              default:
                expression.Append((char)@char);
                break;
            }
            break;
          case State.OnCloseBracket:
            @char = reader.Read();
            switch (@char)
            {
              case '}':
                result.Append('}');
                state = State.OutsideExpression;
                break;
              default:
                throw new FormatException();
            }
            break;
          default:
            throw new InvalidOperationException("Invalid state.");
        }
      } while (state != State.End);
    }

    return result.ToString();
  }

  private enum State
  {
    OutsideExpression,
    OnOpenBracket,
    InsideExpression,
    OnCloseBracket,
    End
  }
}

As before, I wanted to emphasize that this was not a performance contest. I was interested in correctness with reasonable performance.

Now that the JamesFormatter is up to speed, I was able to crank up the number of iterations to 50,000 without testing my patience and tried it again. I also made the format string slightly longer. Here are the new results.

named format perf take
2As you can see, they are all very close, though Henri’s method is the fastest.

I went ahead and updated the solution I had uploaded last time so you can try it yourself by downloading it now.

Tags: format strings , named formats

code comments suggest edit

I’ve been having trouble getting to sleep lately, so I thought last night that I would put that to use and hack on Subtext a bit. While doing so, I ran into an old Asynchronous Fire and Forget helper method written way back by Mike Woodring which allows you to easily call a delegate asynchronously.

On the face of it, it seems like you could simply call BeginInvoke on the delegate and be done with it, Mike’s code addresses a concern with that approach:

Starting with the 1.1 release of the .NET Framework, the SDK docs now carry a caution that mandates calling EndInvoke on delegates you’ve called BeginInvoke on in order to avoid potential leaks. This means you cannot simply “fire-and-forget” a call to BeginInvoke without the risk of running the risk of causing problems.

The mandate he’s referring to, I believe, is this clause in the MSDN docs:

No matter which technique you use, always call EndInvoke to complete your asynchronous call.

Note that it doesn’t explicitly say “or you will get a memory leak”. But a little digging turns up the following comment in the MSDN forums.

The reason that you should call EndInvoke is because the results of the invocation (even if there is no return value) must be cached by .NET until EndInvoke is called.  For example if the invoked code throws an exception then the exception is cached in the invocation data.  Until you call EndInvoke it remains in memory.  After you call EndInvoke the memory can be released.  For this particular case it is possible the memory will remain until the process shuts down because the data is maintained internally by the invocation code.  I guess the GC might eventually collect it but I don’t know how the GC would know that you have abandoned the data vs. just taking a really long time to retrieve it.  I doubt it does.  Hence a memory leak can occur.

The thread continues to have some back and forth and doesn’t appear to be conclusive either way, but this post by Don Box gives a very pragmatic argument.

…the reality is that some implementations rely on the EndXXX call to clean up resources.  Sometimes you can get away with it, but in the general case you can’t.

In other words, why take the chance? In any case, much of this discussion is made redundant with the C# 3.0 Action class combined with ThreadPool.QueueUserWorkItem aka (QUWI)

Here is the code in Subtext for sending email, more or less. I have to define a delegate and then pass that to FireAndForget

// declaration
delegate bool SendEmailDelegate(string to, 
      string from, 
      string subject, 
      string body);

//... in a method body
SendEmailDelegate sendEmail = im.Send;
AsyncHelper.FireAndForget(sendEmail, to, from, subject, body);

This code relies on the FireAndForget method which I show here. Note I am not showing the full code. I just wanted to point out that the arguments to the delegate are not strongly typed. They are simply an array of objects which provide no guidance to how many arguments you need to pass.

public static void FireAndForget(Delegate d, params object[] args)
{
  ThreadPool.QueueUserWorkItem(dynamicInvokeShim, 
    new TargetInfo(d, args));
}

Also notice that this implementation uses QUWI under the hood.

With C# 3.0, there is no need to abstract away the call to QUWI. Just pass in a lambda, which provides the benefit that you’re calling the actual method directly so you get Intellisense for the argumennts etc… So all that code gets replaced with:

ThreadPool.QueueUserWorkItem(callback => im.Send(to, from, subject, body));

Much cleaner and I get to get rid of more code! As I’ve said before, the only thing better than writing code is getting rid of code!

asp.net, asp.net mvc comments suggest edit

Rob pinged me today asking about how to respond to requests using different formats based on the extension in the URL. More specifically, he’d like to respond with HTML if there is no file extension, but with JSON if the URL ended with .json etc…

/home/index –> HTML

/home/index.json –> JSON

The first thing I wanted to tackle was writing a custom action invoker that would decide based on what’s in the route data, how to format the response.

This would allow the developer to simply return an object (the model) from an action method and the invoker would look for the format in the route data and figure out what format to send.

So I wrote a custom action invoker:

public class FormatHandlingActionInvoker : ControllerActionInvoker {
  protected override ActionResult CreateActionResult(
      ControllerContext controllerContext, 
      ActionDescriptor actionDescriptor, 
      object actionReturnValue) {
    if (actionReturnValue == null) {
      return new EmptyResult();
    }

    ActionResult actionResult = actionReturnValue as ActionResult;
    if (actionResult != null) {
      return actionResult;
    }

    string format = (controllerContext.RouteData.Values["format"] 
        ?? "") as string;
      switch (format) {
        case "":
        case "html":
          var result = new ViewResult { 
            ViewData = controllerContext.Controller.ViewData, 
            ViewName = actionDescriptor.ActionName 
          };
          result.ViewData.Model = actionReturnValue;
          return result;
          
        case "rss":
          //TODO: RSS Result
          break;
        case "json":
          return new JsonResult { Data = actionReturnValue };
    }

    return new ContentResult { 
      Content = Convert.ToString(actionReturnValue, 
       CultureInfo.InvariantCulture) 
    };
  }
}

The key thing to note is that I overrode the method CreateActionResult. This method is responsible for examining the result returned from an action method (which can be any type) and figuring out what to do with it. In this case, if the result is already an ActionResult, we just use it. However, if it’s something else, we look at the format in the route data to figure out what to return.

For reference, here’s the code for the HomeController which simply returns an object.

[HandleError]
public class HomeController : Controller {
  public object Index() {
    return new {Title = "HomePage", Message = "Welcome to ASP.NET MVC" };
  }
}

In order to make sure that all my controllers replaced the default invoker with this invoker, I wrote a controller factory that would set this invoker. I won’t show the code here, but I will include it in the download.

So at this point, we have everything in place, except for the fact that I haven’t dealt with how we get the format in the route data in the first place. Unfortunately, it ends up that this isn’t quite so straightforward. Consider the default route:

routes.MapRoute(
    "Default",
    "{controller}/{action}/{id}",
    new { controller = "Home", action = "Index", id = "" }
);

Since this route allows for default values at each segment, this single route matches all the following URLs:

  • /home/index/123
  • /home/index
  • /home

So the question becomes, if we want to support optional format extensions in the URL, would we have to support it for every segment? Making up a fictional syntax, maybe it would look like this:

routes.MapRoute(
    "Default",
    "{controller{.format}}/{action{.format}}/{id{.format}}",
    new { controller = "Home", action = "Index", id = "", format = ""}
);

Where the {.format} part would be optional. Of course, we don’t have such a syntax available, so I needed to put on my dirty ugly hacking hat and see what I could come up with. I decided to do something we strongly warn people not to do, inheriting HttpRequestWrapper with my own HttpRequestBase implementation and stripping off the extension before I try and match the routes.

Warning! Don’t do this at home! This is merely experimentation while I mull over a better approach. This approach relies on implementation details I should not be relying upon

public class HttpRequestWithFormat : HttpRequestWrapper
{
  public HttpRequestWithFormat(HttpRequest request) : base(request) { 
  }

  public override string AppRelativeCurrentExecutionFilePath {
    get
    {
      string filePath = base.AppRelativeCurrentExecutionFilePath;
      string extension = System.IO.Path.GetExtension(filePath);
      if (String.IsNullOrEmpty(extension)) {
        return filePath;
      }
      else {
        Format = extension.Substring(1);
        filePath = filePath.Substring(0, filePath.LastIndexOf(extension));
      }
      return filePath;
    }
  }

  public string Format {
    get;
    private set;
  }
}

I also had to write a custom route.

public class FormatRoute : Route
{
  public FormatRoute(Route route) : 
    base(route.Url + ".{format}", route.RouteHandler) {
    _originalRoute = route;
  }

  public override RouteData GetRouteData(HttpContextBase httpContext)
  {
    //HACK! 
    var context = new HttpContextWithFormat(HttpContext.Current);

    var routeData = _originalRoute.GetRouteData(context);
    var request = context.Request as HttpRequestWithFormat;
    if (!string.IsNullOrEmpty(request.Format)) {
      routeData.Values.Add("format", request.Format);
    }

    return routeData;
  }

  public override VirtualPathData GetVirtualPath(
    RequestContext requestContext, RouteValueDictionary values)
  {
    var vpd = base.GetVirtualPath(requestContext, values);
    if (vpd == null) {
      return _originalRoute.GetVirtualPath(requestContext, values);
    }
    
    // Hack! Let's fix up the URL. Since "id" can be empty string,  
    // we want to check for this condition.
    string format = values["format"] as string;
    string funkyEnding = "/." + format as string;
    
    if (vpd.VirtualPath.EndsWith(funkyEnding)) { 
      string virtualPath = vpd.VirtualPath;
      int lastIndex = virtualPath.LastIndexOf(funkyEnding);
      virtualPath = virtualPath.Substring(0, lastIndex) + "." + format;
      vpd.VirtualPath = virtualPath;
    }

    return vpd;
  }

  private Route _originalRoute;
}

When matching incoming requests, this route replaces the HttpContextBase with my faked up one before calling GetRouteData. My faked up context returns a faked up request which strips the format extension from the URL.

Also, when generating URLs, this route deals with the cosmetic issue that the last segment of the URL has a default value of empty string. This makes it so that the URL might end up looking like /home/index/.jsonwhen I really wanted it to look like /home/index.json.

I’ve omitted some code from this blog post, but you can download the project here and try it out. Just navigate to /home/index and then try /home/index.json and you should notice the response format changes.

This is just experimental work. There’d be much more to do to make this useful. For example, would be nice if an action could specify which formats it would respond to. Likewise, it might be nice to respond based on accept headers rather than formats. I just wanted to see how automatic I could make it.

In any case, I was just having fun and didn’t have much time to put this together. The takeaway from this is really the CreateActionResult method of ControllerActionInvoker. That makes it very easy to create interesting default behavior so that your action methods can return whatever they want and you can implement your own conventions by overriding that method.

Download the hacky experiment here**and play with it at your own risk. :)

code, format comments suggest edit

TRIPLE UPDATE! C# now has string interpolation which pretty much makes this post unnecessary and only interesting as a fun coding exercise.

DOUBLE UPDATE! Be sure to read Peli’s post in which he explores all of these implementations using PEX. Apparently I have a lot more unit tests to write in order to define the expected behavior of the code.

UPDATE: By the way, after you read this post, check out the post in which I revisit this topic and add two more implementations to check out.

Recently I found myself in a situation where I wanted to format a string using a named format string, rather than a positional one. Ignore for the moment the issue on whether this is a good idea or not, just trust me that I’ll be responsible with it.

The existing String.Format method, for example, formats values according to position.

string s = string.Format("{0} first, {1} second", 3.14, DateTime.Now);

But what I wanted was to be able to use the name of properties/fields, rather than position like so:

var someObj = new {pi = 3.14, date = DateTime.Now};
string s = NamedFormat("{pi} first, {date} second", someObj);

Looking around the internet, I quickly found three implementations mentioned in this StackOverflow question.

All three implementations are fairly similar in that they all use regular expressions for the parsing. Hanselman’s approach is to write an extension method of object (note that this won’t work in VB.NET until they allow extending object). James and Oskar wrote extension methods of the string class. James takes it a bit further by using DataBinder.Eval on each token, which allows you to have formats such as {foo.bar.baz} where baz is a property of bar which is a property of foo. This is something else I wanted, which the others do not provide.

He also makes good use of the MatchEvaluator delegate argument to the Regex.Replace method, perhaps one of the most underused yet powerful features of the Regex class. This ends up making the code for his method very succinct.

Handling Brace Escaping

I hade a chat about this sort of string parsing with Eilon recently and he mentioned that many developers tend to ignore or get escaping wrong. So I thought I would see how these methods handle a simple test of escaping the braces.

String.Format with the following:

Console.WriteLine(String.Format("{{{0}}}", 123));

produces the output (sans quotes) of “{123}”

So I would expect with each of these methods, that:

Console.WriteLine(NamedFormat("{{{foo}}}", new {foo = 123}));

Would produce the exact same output, “{123}”. However, only James’s method passed this test. But when I expanded the test to the following format, “{{{{{foo}}}}}”, all three failed. That should have produced “{{123}}”.

Certainly this is not such a big deal as this really is an edge case, but you never know when an edge case might bite you as Zune owners learned recently. More importantly, it poses an interesting problem - how do you handle this correctly? I thought it would be fun to try.

This is possible to handle correctly using regular expressions, but it’s challenging. Not only are you dealing with balanced matching, but the matching depends on whether the number of consecutive braces are odd or even.

For example, the following “{0}}” is not valid because the right end brace is escaped. However, “{0}}}” is valid. The expression is closed with the leftmost end brace, which is followed by an even number of consecutive braces, which means they are all escaped sequences.

Performance

As I mentioned earlier, only James’s method handles evaluation of sub-properties/fields via the use of the DataBinder.Eval method. Critics of his blog post point out that this is a performance killer.

Personally, until I’ve measured it in the scenarios in which I plan to use it, I doubt that the performance will really be an issue compared to everything else going on. But I thought I would check it out anyways, writing a simple console app which runs each method over 1000 iterations, and then divides by 1000 to get the number of milliseconds each method takes. Here’s the result:

format
perf

Notice that James’s method is 43 times slower than Hanselman’s. Even so, it only takes 4.4 milliseconds. So if you don’t use it in a tight loop with a lot of iterations, it’s not horrible, but it could be better.

My Implementation

At this point, I thought it would be fun to write my own implementation using manual string parsing rather than regular expressions. I’m not sure my regex-fu is capable of handling the challenges I mentioned before. After implementing my own version, I ran the performance test and saw the following result.

haackformat
perf

Nice! by removing the overhead of using a regular expression in this particular case, my implementation is faster than the other implementations, despite my use of DataBinder.Eval. Hopefully my implementation is correct, because fast and wrong is even worse than slow and right.

One drawback to not using regular expressions is that the code for my implementation is a bit long. I include the entire source here. I’ve also zipped up the code for this solution which includes unit tests as well as the implementations of the other methods I tested, so you can see which tests they pass and which they don’t pass.

The core of the code is in two parts. One is a private method which parses and splits the string into an enumeration of segments represented by the ITextExpression interface. The method you call joins these segments together, evaluating any expressions against a supplied object, and returning the resulting string.

I think we could optimize the code even more by joining these operations into a single method, but I really liked the separation between the parsing and joining logic as it helped me wrap my head around it. Initially, I hoped that I could cache the parsed representation of the format string since strings are immutable thus I could re-use it. But it didn’t end up giving me any real performance gain when I measured it.

public static class HaackFormatter
{
  public static string HaackFormat(this string format, object source)
  {

    if (format == null) {
        throw new ArgumentNullException("format");
    }

    var formattedStrings = (from expression in SplitFormat(format)
                 select expression.Eval(source)).ToArray();
    return String.Join("", formattedStrings);
  }

  private static IEnumerable<ITextExpression> SplitFormat(string format)
  {
    int exprEndIndex = -1;
    int expStartIndex;

    do
    {
      expStartIndex = format.IndexOfExpressionStart(exprEndIndex + 1);
      if (expStartIndex < 0)
      {
        //everything after last end brace index.
        if (exprEndIndex + 1 < format.Length)
        {
          yield return new LiteralFormat(
              format.Substring(exprEndIndex + 1));
        }
        break;
      }

      if (expStartIndex - exprEndIndex - 1 > 0)
      {
        //everything up to next start brace index
        yield return new LiteralFormat(format.Substring(exprEndIndex + 1
          , expStartIndex - exprEndIndex - 1));
      }

      int endBraceIndex = format.IndexOfExpressionEnd(expStartIndex + 1);
      if (endBraceIndex < 0)
      {
        //rest of string, no end brace (could be invalid expression)
        yield return new FormatExpression(format.Substring(expStartIndex));
      }
      else
      {
        exprEndIndex = endBraceIndex;
        //everything from start to end brace.
        yield return new FormatExpression(format.Substring(expStartIndex
          , endBraceIndex - expStartIndex + 1));

      }
    } while (expStartIndex > -1);
  }

  static int IndexOfExpressionStart(this string format, int startIndex) {
    int index = format.IndexOf('{', startIndex);
    if (index == -1) {
      return index;
    }

    //peek ahead.
    if (index + 1 < format.Length) {
      char nextChar = format[index + 1];
      if (nextChar == '{') {
        return IndexOfExpressionStart(format, index + 2);
      }
    }

    return index;
  }

  static int IndexOfExpressionEnd(this string format, int startIndex)
  {
    int endBraceIndex = format.IndexOf('}', startIndex);
    if (endBraceIndex == -1) {
      return endBraceIndex;
    }
    //start peeking ahead until there are no more braces...
    // }}}}
    int braceCount = 0;
    for (int i = endBraceIndex + 1; i < format.Length; i++) {
      if (format[i] == '}') {
        braceCount++;
      }
      else {
        break;
      }
    }
    if (braceCount % 2 == 1) {
      return IndexOfExpressionEnd(format, endBraceIndex + braceCount + 1);
    }

    return endBraceIndex;
  }
}

And the code for the supporting classes

public class FormatExpression : ITextExpression
{
  bool _invalidExpression = false;

  public FormatExpression(string expression) {
    if (!expression.StartsWith("{") || !expression.EndsWith("}")) {
      _invalidExpression = true;
      Expression = expression;
      return;
    }

    string expressionWithoutBraces = expression.Substring(1
        , expression.Length - 2);
    int colonIndex = expressionWithoutBraces.IndexOf(':');
    if (colonIndex < 0) {
      Expression = expressionWithoutBraces;
    }
    else {
      Expression = expressionWithoutBraces.Substring(0, colonIndex);
      Format = expressionWithoutBraces.Substring(colonIndex + 1);
    }
  }

  public string Expression { 
    get; 
    private set; 
  }

  public string Format
  {
    get;
    private set;
  }

  public string Eval(object o) {
    if (_invalidExpression) {
      throw new FormatException("Invalid expression");
    }
    try
    {
      if (String.IsNullOrEmpty(Format))
      {
        return (DataBinder.Eval(o, Expression) ?? string.Empty).ToString();
      }
      return (DataBinder.Eval(o, Expression, "{0:" + Format + "}") ?? 
        string.Empty).ToString();
    }
    catch (ArgumentException) {
      throw new FormatException();
    }
    catch (HttpException) {
      throw new FormatException();
    }
  }
}

public class LiteralFormat : ITextExpression
{
  public LiteralFormat(string literalText) {
    LiteralText = literalText;
  }

  public string LiteralText { 
    get; 
    private set; 
  }

  public string Eval(object o) {
    string literalText = LiteralText
        .Replace("{{", "{")
        .Replace("}}", "}");
    return literalText;
  }
}

I mainly did this for fun, though I plan to use this method in Subtext for email fomatting.

Let me know if you find any situations or edge cases in which my version fails. I’ll probably be adding more test cases as I integrate this into Subtext. As far as I can tell, it handles normal formatting and brace escaping correctly.