code, asp.net mvc, dlr comments edit

Say you’re building a web application and you want, against your better judgment perhaps, to allow end users to easily customize the look and feel – a common scenario within a blog engine or any hosted application.

With ASP.NET, view code tends to be some complex declarative markup stuck in a file on disk which gets compiled by ASP.NET into an assembly. Most system administrators would first pluck out their own toenail rather than allow an end user permission to modify such files.

It’s possible to store such files in the database and use a VirtualPathProvider to load them, but that requires your application (and thus their views) to run in full trust. Is there a way you could safely store such views in the database in an application running in medium trust where the code in the view is approachable?

At the ALT.NET conference a little while back, Jimmy Schementi and John Lam gave a talk about the pattern of hosting a scripting language within a larger application. For example, many modern 3-D Games have their high performance core engine written in C++ and Assembly. However, these games often use a scripting language, such as Lua, to write the scripts for the behaviors of characters and objects.

An example that might be more familiar to more people is the use of VBA to write macros for Excel. In both of these cases, the larger application hosts a scripting environment that allow end users to customize the application using a simpler lighter weight language than the one the core app is written in.

A long while back, I wrote a blog post about defining ASP.NET MVC Views in IronRuby followed by a full IronRuby ASP.NET MVC stack. While there was some passionate interest by a few, in general, I was met with the thunderous sound of crickets. Why the huge lack of interest? Probably because I didn’t really sell the benefit and the explain the pain it solves. I’m sure many of you were asking, Why bother? What’s in it for me?

After thinking about it some more, I realized that my prototypes appeared to suggest that if you want to take advantage of IronRuby, you would need to make some sort of wholesale switch to a new foreign language, not something to be undertaken lightly.

This is why I really like Jimmy and John’s recent efforts to focus on showing the benefits of hosting the DLR for scripting scenarios like the ones mentioned above. It makes total sense to me when I look at it in this perspective. The way I see it, most developers spend a huge bulk of their time in a single core language, typically their “language of choice”. For me, I spend the bulk of my time writing C# code.

However, I don’t think twice about the fact that I also write tons of JavaScript when I do web development, and I’ll write the occasional VB code when I need a new Macro for Visual Studio or Excel. I also write SQL when I need to. I’m happy to pick up and use a new language when it will enable me to do the job at hand more efficiently and naturally than C# does. I imagine many developers feel this way. The occasional use of a scripting languages is fine when it gets the job done and I can still spend most of my time in my favorite language.

So I started thinking about how that might work in a web application. What if you could write all your business logic and controller logic in your language of choice, but have your views written in a light weight scripting language. If my web application were to host a scripting engine, I could actually store the code in any medium I want, such as the database. Having them in the database makes it very easy for end users to modify it since it wouldn’t require file upload permissions into the web root.

This is where hosting the DLR is a nice fit. I put together a proof of concept for these ideas. This is just a prototype intended to show how such a workflow might work. In this prototype, you go about creating your models and controllers the way you normally would.

For example, here’s a controller that returns some structured data to the view in the form of an anonymous type.

public ActionResult FunWithScripting()
{
  var someData = new { 
    salutation = "Are you having fun with scripting yet?", 
    theDate = DateTime.Now,
      numbers = new int[] { 1, 2, 3, 4 } 
  };

  return View(someData);
}

Once you write your controller, but before you create your view, you compile the app and then go visit the URL.View does not exist
view

We haven’t created the view yet, so let’s follow the instructions and login. Afterwards, we this:

view
editor

Since the view doesn’t exist, I hooked in and provided a temporary view for the controller action which contains a view editor. Notice that at the bottom of the screen, you can see the current property names and values being passed to the view. For example, there’s an enumeration of integers as one property, so I was able to use the Ruby each method to print them out in the view.

The sweet little browser based source code editor is named Edit Area created by Christophe Dolivet. Unfortunately, at the time I write this, it doesn’t yet have support for ERB style syntax highlighting schemes. That’s why the <% and %> aren’t highlighted in yellow.

When I click Create View, I get taken back to the request for the same action, but now I can see the view I just created (click to enlarge).

Fun with scripting
view

In the future, I should be able to host C# views in this way. Mono already has a tool for dynamically compiling C# code passed in as a string which I could try and incorporate.

I’m seriously thinking of making this the approach for building skins in a future version of Subtext. That would make skin installation drop dead simple and not require any file directory access. Let me know if you make use of this technique in your applications.

If you try and run this prototype, please note that there are some quirky caching issues with editing existing views in the prototype. It’ll seem like your view is not being edited, but it’s a result of how views are being cached. It might take a bit of time before your edits show up. I’m sure there are other bugs I’m still in the process of fixing. But for the most part, the general principle is sound.

You can download the prototype here.

comments edit

Because of all the travel I did last year as well as the impending new addition to the family this year, I drastically cut down on my travel this year. There are only two conferences outside of Redmond I planned to speak at, one was Mix (see the links to videos of my talks) and the next one is the Norwegian Developer Conference also known as the NDC.

NDC_logo-2009Hanselman spoke at this conference last year and tells me it’s a good one. Besides, it’s in Norway! I’ve travelled through Norway once during college, taking a train from Oslo to Bergen, riding a boat on the fjords, and enjoying the profound natural beauty of the country. I guess I have a thing for cold places. :)

I’m pretty excited about the speaker line-up which includes a lot of great .NET and ALT.NET speakers you know and love. But what’s really got me perked up are the speakers outside of the typical .NET conference lineup.

One of my favorite conferences last year was Google IO which was a refreshing change of pace for me. For the NDC, I requested to stay an extra day so I could make sure to catch sessions by Mary Poppendieck, Robert “Uncle Bob” Martin, and Michael Feathers among others.

It looks like all my talks are on Day 1 of the conference. I’ll be upda’ting my Black Belt Ninja Tips ASP.NET MVC talk, talking about Ajax in the context of ASP.NET MVC, and giving a joint talk with Scott Hanselman which we’re still figuring out the exact details on.

My only concern is whether I need to worry if Jeremy Miller is going to try and express his man love for me while there. ;) Kidding aside, I’m approaching with a mind ready to absorb knowledge. If you’re in the area, definitely consider this as a conference to check out. It should be fun!

comments edit

What responsibility do we have as software professionals when we post code out there for public consumption?

I don’t have a clear cut answer in my mind, but maybe you can help me formulate one. :)

For example, I recently posted a sample on my blog intended to show how to use jQuery Grid with ASP.NET MVC.

The point of the sample was to demonstrate shaping a JSON result for the jQuery grid’s consumption. For the sake of illustration, I wanted the action method to be relatively self contained so that a reader would quickly understand what’s going on in the code without having to jump around a lot.

Thus the code takes some shortcuts with data access, lack of exception handling, and lack of input validation. It’s pretty horrific!

Now before we grab the pitchforks (and I did say “we” intentionally as I’ll join you) to skewer me, I did preface the code with a big “warning, DEMO CODE AHEAD” disclaimer and so far, nobody’s beaten me up too bad about it, though maybe by writing this I’m putting myself in the crosshairs.

Even so, it did give me pause to post the code the way I did. Was I making the right trade-off in sacrificing code quality for the sake of blog post demo clarity and brevity?

In this particular case, I felt it was worth it as I tend to categorize code into several categories. I’m not saying these are absolutely correct, just opening up my cranium and giving you a peek in my head about how I think about this:

  • Prototype Code – Code used to hash out an idea to see if it’s feasible or as a means of learning a new technology. Often very ugly throwaway code with little attention paid to good design.
  • Demo Code – Code used to illustrate a concept, especially in a public setting. Like prototype code, solid design is sometimes sacrificed for clarity, but these sacrifices are deliberateand intentional, which is very important. My jQuery Grid demo above is an example of what I mean.
  • Sample Code – Very similar to demo code, the difference being that good design principles should be demonstrated for the code relevant to the concept the sample is demonstrating. Code irrelevant to the core concept might be fine to leave out or have lower quality. For example, if the sample is showing a data access technique, you might still leave out exception handling, caching, etc… since it’s not the goal of the sample to demonstrate those concepts.
  • Production Code – Code you’re running your business on, or selling. Should be as high quality as possible given your constraints. Sometimes, shortcuts are taken in the short run (incurring technical debt) with the intention of paying down the debt ASAP.
  • Reference Code – This is code that is intended to demonstrate the correct way to build an application and should be almost idealized in its embracement of good design practices.

As you might expect, the quality the audience might expect from these characterizations is not hard and fast, but dependent on context. For example, for the Space Shuttle software, I expect the Production Code to be much higher quality than production code for some intranet application.

Likewise, I think where the code is posted and by whom is can affect perception. We might expect much less from some blowhard posting code to his personal blog, ummm, like this one.

Then again, if the person claims that his example is a best practice, which is a dubious claim in the first place, we may tend to hold it to much higher standards.

Now if instead of a person, the sample is posted on an official website of a large company, say Microsoft, the audience may expect a lot more than from a personal blog post. In fact, the audience may not make the distinction between sample and reference application. This appears to be the case recently with Kobe and in the past with Oxite.

Again, this is my perspective on these things. But my views have been challenged recently via internal and external discussions with many people. So I went to the font of all knowledge where all your wildest questions are answered: Twitter. I posed the following two questions:

Do you have different quality expectations for a sample app vs a reference app?

What if the app is released by MS? Does that change your expectations?

The answers varied widely. Here’s a small sampling that represents the general tone of the responses I received.

Yes. A sample app should be quick and dirty. A reference app should exhibit best practices (error checking, logging, etc)

No, same expectations… Even I ignore what is the difference between both.

Regardless of who releases the app, my expectations don’t change.

Yes being from MS raises the bar of necessary quality, because it carries with it the weight of a software development authority.

I don’t think I have ever thought about what the difference in the two is, isn’t a sample app basically a reference app?

I don’t think most people discriminate substantively betw the words “sample” and “reference.”

Everyone, Microsoft included, should expect to be judged by everything the produce, sample or otherwise.

yes, samples do not showcase scalability or security, but ref apps do… i.e ref apps are more “enterprisey”

IMHO, sample implies a quick attempt; mostly throw-away. Ref. implies a proposed best practice; inherently higher quality.

No. We as a community should understand the difference.However MS needs to apply this notion consistently to it’s examples.

Whatever you release as sample code, is *guaranteed* to be copy-pasted everywhere - ask Windows AppCompat if you don’t believe me

Note that this is a very unscientific sampling, but there is a lot of diversity in the views being expressed here. Some people make no distinction between sample and reference while others do. Some hold Microsoft to higher standards while others hold everybody to the same standard.

I found this feedback to be very helpful because I think we tend to operate under one assumption about how our audience sees our samples, but your audience might have a completely different view. This might explain why there may be miscommunication and confusion about the community reaction to a sample.

I highlighted the last two responses because they make up the core dichotomy in my head regarding releasing samples.

On the one hand, I have tended to lean towards the first viewpoint. If code has the proper disclaimer, shouldn’t we take personal responsibility in understanding the difference?

Ever since starting work on ASP.NET MVC, we’ve been approached by more and more teams at Microsoft who are interested in sharing yet more code on CodePlex (or otherwise) and want to hear about our experiences and challenges in doing so.

When you think about it, this is a great change in what has been an otherwise closed culture. There are a lot of teams at Microsoft and the quality of the code and intent of the code will vary from team to team and project to project. I would hate to slow down that stream of sample code flowing out because some people will misunderstand its purpose and intend and cut and paste it. Yes, some of the code will be very bad, but some of it will still be worth putting out there. After all, I tend to think that if we stop giving the bad programmers bad code to cut and paste, they’ll simply write the bad code themselves. Yes, posting good code is even better, but I think that will be a byproduct of getting more code out there.

On the other hand, there’s the macro view of things to consider. People should also know not to use a hair dryer in the shower, yet they still have these funny warning labels for a reason. The fact that people shouldn’t do something sometimes doesn’t change the fact that may still do it. We can’t simply ignore that fact and the impact it may have. No matter how many disclaimers we put on our code, people will cut and paste it. It’s not so bad that a bad programmer uses bad code, but that as it propagates, the code gets confused with the right way and spreads to many programmers.

Furthermore, the story is complicated even more by the inconsistent labels applied to all this sample code, not to mention the inconsistent quality.

So What’s the Solution?

Stop shipping samples.

Nah, I’m just kidding. ;)

Some responses were along the lines of Microsoft should just post good code. I agree, I would really love it if every sample was of superb quality. I’d also like to play in a World Cup and fly without wings, but I don’t live in that world.

Obviously, this is what we should be striving for, but what do we do in the meantime? Stop shipping samples? I hope not.

Again, I don’t claim to have the answers, but I think there are a few things that could help. One twitter response made a great point:

a reference app is going to be grilled. Even more if it comes from the mothership. Get the community involved *before* it gets pub

Getting the community involved is a great means of having your code reviewed to make sure you’re not doing anything obviously stupid. Of course, even in this, there’s a challenge. Jeremy Miller made this great point recently:

We don’t have our own story straight yet.  We’re still advancing our craft.  By no means have we reached some sort of omega point in our own development efforts. 

In other words, even with community involvement, you’re probably going to piss someone off. But avoiding piss is not really the point anyways (though it’s much preferred to the alternative). The point is to be a participant in advancing the craft alongside the larger community. Others might disagree with some of your design decisions, but hopefully they can see that your code is well considered via your involvement with the community in the design process.

This also helps in avoiding the perception of arrogance, a fault that some feel is the root cause of why some of our sample apps are of poor quality. Any involvement with the community will help make it very clear that there’s much to learn from the community just as there is much to teach.

While I think getting community involved is important, I’m still on the fence on whether it must happen before its published. After all, isn’t publishing code a means of getting community involvement in the first place? As Dare says:

getting real feedback from customers by shipping is more valuable than any amount of talking to or about them beforehand

Personally, I would love for there to be a way for teams to feel free to post samples (using the definition I wrote), without fear of misconstrued intent and bad usage. Ideally in a manner where it’s clear that the code is not meant for cut and paste into real apps.

Can we figure out a way to post code samples that are not yet the embodiment of good practices in a responsible manner with the intent to improve the code quality based on community feedback? Is this even a worthy goal or should Microsoft samples just get it right the first time, as mentioned before, or don’t post at all?

Perhaps both of those are pipe dreams. I’m definitely interested in hearing your thoughts. :)

Another question I struggle with is what causes people to not distinguish between reference apps and sample apps? Is there no distinction to make? Or is this a perception problem that can be corrected with a concerted effort to make such labels consistently applied, perhaps? Or via some other means.

As you can see, I have my own preconceived notions about those things, but I’m putting them out there and challenging them based on what I’ve read recently. Please do comment and let me know your thoughts.

code, asp.net mvc comments edit

Tim Davis posted an updated version of this solution on his blog. His includes the following:

  • jqGrid 3.8.2
  • .NET 4.0 Updates
  • VS2010
  • jQuery 1.4.4
  • jQuery UI 1.8.7

Continuing in my pseudo-series of posts based on my ASP.NET MVC Ninjas on Fire Black Belt Tips Presentation at Mix (go watch it!), this post covers a demo I did not show because I ran out of time. It was a demo I held in my back pocket just in case I went too fast and needed one more demo.

A common scenario when building web user interfaces is providing a pageable and sortable grid of data. Even better if it uses AJAX to make it more responsive and snazzy. Since ASP.NET MVC includes jQuery, I figured it’d be fun to use a jQuery plugin for this demo, so I chose jQuery Grid.

After creating a standard ASP.NET MVC project, the first step was to download the plugin and to unzip the contents to my scripts directory per the Installation instructions.

jquery-grid-scripts

For the purposes of this demo, I’ll just implement this using the Index controller action and view within the HomeController.

With the scripts in place, go to the Index view and add the proper call to initialize the jQuery grid. There are three parts to this:

First, make sure to add the required script and CSS declarations.

<link rel="stylesheet" type="text/css" href="/scripts/themes/coffee/grid.css" 
  title="coffee" media="screen" />
<script src="/Scripts/jquery-1.3.2.js" type="text/javascript"></script>
<script src="/Scripts/jquery.jqGrid.js" type="text/javascript"></script>
<script src="/Scripts/js/jqModal.js" type="text/javascript"></script>
<script src="/Scripts/js/jqDnR.js" type="text/javascript"></script>

Notice that the first line contains a reference to the “coffee” CSS file. There are multiple themes included and when you choose a theme, you need to be sure to include the theme’s CSS file. I chose coffee, because I drink a lot of it.

The Second step is to initialize the grid with a bit of JavaScript. This looks a bit funky if you’re not used to jQuery, but I assure you, it’s pretty straightforward.

<script type="text/javascript">
    jQuery(document).ready(function(){ 
      jQuery("#list").jqGrid({
        url:'/Home/GridData/',
        datatype: 'json',
        mtype: 'GET',
        colNames:['Id','Votes','Title'],
        colModel :[
          {name:'Id', index:'Id', width:40, align:'left' },
          {name:'Votes', index:'Votes', width:40, align:'left' },
          {name:'Title', index:'Title', width:200, align:'left'}],
        pager: jQuery('#pager'),
        rowNum:10,
        rowList:[5,10,20,50],
        sortname: 'Id',
        sortorder: "desc",
        viewrecords: true,
        imgpath: '/scripts/themes/coffee/images',
        caption: 'My first grid'
      }); 
    }); 
</script>

There are a few things you’ll have to be sure to configure here. First is the url property which points to the URL that will provide the JSON data. Notice that the value is /Home/GridData which means we’ll be implementing an action method named GridData soon. During the course of this post, we’ll change that property to point to different action methods.

The colNames property contains the display names for each column separated by columns. Ideally it should match up with the items in the colModel property.

The colModel property is an array that is used to configure each column of the grid, allowing you to specify the width, alignment, and sortability of a column. The index property of a column is an important one as that is the value that is sent to the server when sorting on a column.

See the documentation for more details on the HTML and JavaScript used to configure the grid.

The Third step is to add a bit of HTML to the page which will house the grid.

<h2>My Grid Data</h2>
<table id="list" class="scroll" cellpadding="0" cellspacing="0"></table>
<div id="pager" class="scroll" style="text-align:center;"></div>

With this in place, it’s time to implement the GridData action method to return the JSON in the proper format.

But first, let’s take a look at the JSON format expected by the grid. From the documentation, you can see it will look something like:

{ 
  total: "xxx", 
  page: "yyy", 
  records: "zzz",
  rows : [
    {id:"1", cell:["cell11", "cell12", "cell13"]},
    {id:"2", cell:["cell21", "cell22", "cell23"]},
      ...
  ]
}

The documentation I linked to also provides some gnarly looking PHP code you can use to generate the JSON data. Fortunately, you won’t have to deal with that. By using the Json helper method with an anonymous object, we can write some relatively clean looking code which looks almost just like the spec. Here’s my first cut of the action method, just to get it to display some fake data.

public ActionResult GridData(string sidx, string sord, int page, int rows) {
  var jsonData = new {
    total = 1, // we'll implement later 
    page = page,
    records = 3, // implement later 
    rows = new[]{
      new {id = 1, cell = new[] {"1", "-7", "Is this a good question?"}},
      new {id = 2, cell = new[] {"2", "15", "Is this a blatant ripoff?"}},
      new {id = 3, cell = new[] {"3", "23", "Why is the sky blue?"}}
    }
  };
  return Json(jsonData, JsonRequestBehavior.AllowGet);
}

A couple of things to point out. The arguments to the action methods are named according to the query string parameter names that jQuery grid sends via the Ajax request. I didn’t choose those names.

By naming the arguments to the action method exactly the same as what is in the query string, we have a very convenient way to retrieve these values. Remember, arguments passed to an action method should be treated with care.Never trust user input!

In this example, we statically create some JSON data and use the Json helper method to return the data back to the grid and Voila! It works!

jquery-grid-demo

Yeah, this is great for a simple demo, but I use a real database to store my data! Understood. It’s time to hook this up to a real database. As you might guess, I’ll use the HaackOverflow database for this demo as well as LinqToSql.

I’ll assume you know how to add a database and create a LinqToSql model already. If not, look at the source code I’ve included. Once you’ve done that, it’s pretty easy to transformat the data we get back into the proper JSON format.

public ActionResult LinqGridData    (string sidx, string sord, int page, int rows) {
  varnew HaackOverflowDataContext();

  var jsonData = new {
    total = 1, //todo: calculate
    page = page,
    records = context.Questions.Count(),
    rows = (
      from question in context.Questions
      select new {
        id = question.Id,
        cell = new string[] { 
          question.Id.ToString(), question.Votes.ToString(), question.Title 
        }
      }).ToArray()
  };
  return Json(jsonData, JsonRequestBehavior.AllowGet);
}

Note that the method is a tiny bit busier, but it follows the same basic structure as the JSON data. After changing the JavaScript code in the view to point to this action instead of the other, we can now see the first ten records from the database in the grid.

But we’re not done yet. At this point, we want to implement paging and sorting. Paging is pretty easy, but sorting is a bit tricky. After all, what we get passed into the action method is the name of the sort column. At that point, we want to dynamically create a LINQ expression that sorts by that column.

One easy way to do this is to use the Dynamic Linq Query library which ScottGu wrote about a while back. This library adds extension methods which make it easy to create more dynamic Linq queries based on strings. Of course, with great power comes great responsibility. Make sure to validate the strings before you pass them into the methods. With this in place, we rewrite the action method to be (warning, DEMO CODE AHEAD!):

public ActionResult DynamicGridData
    (string sidx, string sord, int page, int rows) {
  var context = new HaackOverflowDataContext();
  int pageIndex = Convert.ToInt32(page) - 1;
  int pageSize = rows;
  int totalRecords = context.Questions.Count();
  int totalPages = (int)Math.Ceiling((float)totalRecords / (float)pageSize);

  var questions = context.Questions
    .OrderBy(sidx + " " + sord)
    .Skip(pageIndex * pageSize)
    .Take(pageSize);

  var jsonData = new {
    total = totalPages,
    page = page,
    records = totalRecords,
    rows = (
      from question in questions
      select new {
        id = question.Id,
        cell = new string[] {
          question.Id.ToString(), question.Votes.ToString(), question.Title 
        }
    }).ToArray()
  };
  return Json(jsonData, JsonRequestBehavior.AllowGet);
}

Some things to note: The first part of this method does some initial calculations to figure out the number of pages we’re dealing with based on the page size (passed in) and the total record count.

Then given that info, we use the Dynamic Linq extension methods to do the actual paging and sorting via the line:

var questions = context.Questions.OrderBy(…).Skip(…).Take(…);

Once we have that, we can simply transform that into the array that jQuery grid expects and place that in the larger JSON payload represented by the jsonData variable.

With all this in place, you now have a pretty snazzy approach to paging and sorting data using AJAX. Now go forth and wow your customers. ;)

And before I forget, here’s the sample project that uses all three approaches.

personal comments edit

Every good developer knows to always have a backup. For example, over two years ago, I announced my world domination plans. But there was a single point of failure in me putting all my world domination plans on the tiny shoulders of just one progeny. My boy needs a partner in crime.

mia So my wife and I conspired together and we’re happy to announce that baby #2 is on the way. Together, the two of them will be unstoppable!

My wife is past her first trimester and we expect the baby to RTF (Release To Family) around October.

This second time around has been a bit more challenging. My poor wife, bless her heart, has had to deal with much more sever nausea this time around.

Notice the crinkle in the ultrasound photo. My son did that. ;) He’s trying to destroy the evidence.

Many of you who have more than one children might be able to relate to this, but I really considered not writing a blog announcement for my second child. There was the feeling that it was such a novel thing the first time, but now it’s becoming old hat (not really!).

But then I mentally fast forwarded 16 years later and pictured my future daughter finding the firstborn announcement and not finding her own blog announcement. Try explaining that to a kid. I can deal without the drama.

Well I won’t have to! Hi honey, here’s your announcement. Now you be good while daddy goes shopping for a shotgun and a shovel. :)

asp.net, asp.net mvc comments edit

There are a couple of peculiarities worth understanding when dealing with title tags and master pages within Web Forms and ASP.NET MVC. These assume you are using the HtmlHead control, aka <head runat="server" />.

The first peculiarity involves a common approach where one puts a ContentPlaceHolder inside of a title tag like we do with the default template in ASP.NET MVC:

<%@ Master ... %>
<html>
<head runat="server">
  <title>
    <asp:ContentPlaceHolder ID="titleContent" runat="server" />
  </title>
</head>
...

What’s nice about this approach is you can set the title tag from within any content page.

<asp:Content ContentPlaceHolderID="titleContent" runat="server">
  Home
</asp:Content>

But what happens if you want to set part of the title from within the master page. For example, you might want the title of every page to end with a suffix, “ – MySite”.

If you try this (notice the – MySite tacked on):

<%@ Master ... %>
<html>
<head runat="server">
  <title>
    <asp:ContentPlaceHolder ID="titleContent" runat="server" /> - MySite
  </title>
</head>
...

And run the page, you’ll find that the – MySite is not rendered. This appears to be a quirk of the HtmlHead control. This is because the title tag within the HtmlHead control is now itself a control. This will be familiar to those who understand how the AddParsedSubObject method works. Effectively, the only content allowed within the body of the HtmlHead control are other controls.

The fix is pretty simple. Add your text to a LiteralControl like so.

<%@ Master ... %>
<html>
<head runat="server">
  <title>
    <asp:ContentPlaceHolder ID="titleContent" runat="server" /> 
    <asp:LiteralControl runat="server" Text=" - MySite" />
  </title>
</head>
...

The second peculiarityhas to do with how the HeaderControl really wants to produce valid HTML markup.

If you leave the <head runat="server"></head> tag empty, and then view source at the rendered output, you’ll notice that it renders an empty <title> tag for you. It looked at its child controls collection and saw that it didn’t contain an HtmlTitle control so it rendered one for you.

This can cause problems when attempting to use a ContentPlaceHolder to render the title tag for you. For example, a common layout I’ve seen is the following.

<%@ Master ... %>
<html>
<head runat="server">
  <asp:ContentPlaceHolder ID="headContent" runat="server"> 
    <title>Testing</title>  
  </asp:ContentPlaceHolder>
</head>
...

This approach is neat because it allows you to not only set the title tag from within any content page, but any other content you want within the <head> tag.

However, if you view source on the rendered output, you’ll see two <title> tags, one that you specified and one that’s empty.

Going back to what wrote earlier, the reason becomes apparent. The HtmlHead control checks to see if it contains a child title control. When it doesn’t find one, it renders an empty one. However, it doesn’t look within the content placeholders defined within it to see if they’ve rendered a title tag.

This makes sense when you consider how the HtmlHead tag works. It only allows placing controls inside of it. However, a ContentPlaceHolder allows adding literal text in there. So while it looks the same, the title tag within the ContentPlaceHolder is not an HtmlTitle control. It’s just some text, and the HtmlHead control doesn’t want to parse all the rendered text from its children.

This is why I tend to take the following approach with my own master pages.

<%@ Master ... %>
<html>
<head runat="server">
  <title><asp:ContentPlaceHolder ID="titleContent" runat="server" /></title>
  <asp:ContentPlaceHolder ID="headContent" runat="server"> 
  </asp:ContentPlaceHolder>
</head>
...

Happy Titling!

asp.net comments edit

In my last blog post, I walked step by step through a Cross-site request forgery (CSRF) attack against an ASP.NET MVC web application. This attack is the result of how browsers handle cookies and cross domain form posts and is not specific to any one web platform. Many web platforms thus include their own mitigations to the problem.

It might seem that if you’re using Web Forms, you’re automatically safe from this attack. While Web Forms has many mitigations turned on by default, it turns out that it does not automatically protect your site against this specific form of attack.

In the same sample bank transfer application I provided in the last post, I also included an example written using Web Forms which demonstrates the CSRF attack. After you log in to the site, you can navigate to /BankWebForm/default.aspx to try out the Web Form version of the transfer money page. it works just like the MVC version.

To simulate the attack, make sure you are running the sample application locally and make sure you are logged in and then click on https://haacked.com/demos/csrf-webform.html.

Here’s the code for that page:

<html xmlns="http://www.w3.org/1999/xhtml" >
<head>
    <title></title>
</head>
<body>
  <form name="badform" method="post"
    action="http://localhost:54607/BankWebForm/Default.aspx">
    <input type="hidden" name="ctl00$MainContent$amountTextBox"
      value="1000" />
    <input type="hidden" name="ctl00$MainContent$destinationAccountDropDown"
      value="2" />
    <input type="hidden" name="ctl00$MainContent$submitButton"
      value="Transfer" />
    <input type="hidden" name="__EVENTTARGET" id="__EVENTTARGET"
      value="" />
    <input type="hidden" name="__EVENTARGUMENT" id="__EVENTARGUMENT"
      value="" />
    <input type="hidden" name="__VIEWSTATE" id="__VIEWSTATE"
      value="/wEP...0ws8kIw=" />
    <input type="hidden" name="__EVENTVALIDATION" id="__EVENTVALIDATION"
      value="/wEWBwK...+FaB85Nc" />
    </form>
    <script type="text/javascript">
        document.badform.submit();
    </script>
</body>
</html>

It’s a bit more involved, but it does the trick. It mocks up all the proper hidden fields required to execute a bank transfer on my silly demo site.

The mitigation for this attack is pretty simple and described thoroughly in this this article by Dino Esposito as well as this post by Scott Hanselman. The change I made to my code behind based on Dino’s recommendation is the following:

protected override void OnInit(EventArgs e) {
  ViewStateUserKey = Session.SessionID;
  base.OnInit(e);
}

With this change in place, the CSRF attack I put in place no longer works.

When you go to a real bank site, you’ll learn they have all sorts of protections in place above and beyond what I described here. Hopefully this post and the previous one provided some insight into why they do all the things they do. :)

Technorati Tags: asp.net,security

asp.net, code, asp.net mvc comments edit

A Cross-site request forgery attack, also known as CSRF or XSRF (pronounced sea-surf) is the less well known, but equally dangerous, cousin of the Cross Site Scripting (XSS) attack. Yeah, they come from a rough family.

CSRF is a form of confused deputy attack. Imagine you’re a malcontent who wants to harm another person in a maximum security jail. You’re probably going to have a tough time reaching that person due to your lack of proper credentials. A potentially easier approach to accomplish your misdeed is to confuse a deputy to misuse his authority to commit the dastardly act on your behalf. That’s a much more effective strategy for causing mayhem!

In the case of a CSRF attack, the confused deputy is your browser. After logging into a typical website, the website will issue your browser an authentication token within a cookie. Each subsequent request to sends the cookie back to the site to let the site know that you are authorized to take whatever action you’re taking.

Suppose you visit a malicious website soon after visiting your bank website. Your session on the previous site might still be valid (though most bank websites guard against this carefully). Thus, visiting a carefully crafted malicious website (perhaps you clicked on a spam link) could cause a form post to the previous website. Your browser would send the authentication cookie back to that site and appear to be making a request on your behalf, even though you did not intend to do so.

Let’s take a look at a concrete example to make this clear. This example is the same one I demonstrated as part of my ASP.NET MVC Ninjas on Fire Black Belt Tips talk at Mix in Las Vegas. Feel free to download the source for this sample and follow along.

Here’s a simple banking website I wrote. If your banking site looks like this one, I recommend running away.

banking-login-pageThe site properly blocks anonymous users from taking any action. You can see that in the code for the controller:

[Authorize]
public class HomeController : Controller
{
  //...
}

Notice that we use the AuthorizeAttribute on the controller (without specifying any roles) to specify that all actions of this controller require the user to be authentication.

After logging in, we get a simple form that allows us to transfer money to another account in the bank. Note that for the sake of the demo, I’ve included an information disclosure vulnerability by allowing you to see the balance for other bank members. ;)

bank-transfer-screen

To transfer money to my Bookie, for example, I can enter an amount of $1000, select the Bookie account, and then click Transfer. The following shows the HTTP POST that is sent to the website (slightly edited for brevity):

POST /Home/Transfer HTTP/1.1
Referer: http://localhost:54607/csrf-mvc.html
User-Agent: ...
Content-Type: application/x-www-form-urlencoded
Host: 127.0.0.1:54607
Content-Length: 34
Cookie: .ASPXAUTH=98A250...03BB37

Amount=1000&destinationAccountId=3

There are three important things to notice here. We are posting to a well known URL, /Home/Transfer, we are sending a cookie, .ASPXAUTH, which lets the site know we are already logged in, and we are posting some data (Amount=1000&destinationAccountId=3), namely the amount we want to transfer and the account id we want to transfer to. Let’s briefly look at the code that executes the transfer.

[AcceptVerbs(HttpVerbs.Post)]
public ActionResult Transfer(int destinationAccountId, double amount) {
  string username = User.Identity.Name;
  Account source = _context.Accounts.First(a => a.Username == username);
  Account destination = _context.Accounts.FirstOrDefault(
    a => a.Id == destinationAccountId);
            
  source.Balance -= amount;
  destination.Balance += amount;
  _context.SubmitChanges();
  return RedirectToAction("Index");
}

Disclaimer: Do not write code like this. This code is for demonstration purposes only. For example, I don’t ensure that amount non-negative, which means you can enter a negative value to transfer money from another account. Like I said, if you see a bank website like this, run!

The code is straightforward. We simply transfer money from one account to another. At this point, everything looks fine. We’re making sure the user is logged in before we transfer money. And we are making sure that this method can only be called from a POST request and not a GET request (this last point is important. Never allow changes to data via a GET request).So what could go wrong?

Well BadGuy, another bank user has an idea. He sets up a website that has a page with the following code:

<html>
<head>
    <title></title>
</head>
<body>
    <form name="badform" method="post"
     action="http://localhost:54607/Home/Transfer">
        <input type="hidden" name="destinationAccountId" value="2" />
        <input type="hidden" name="amount" value="1000" />
    </form>
    <script type="text/javascript">
        document.badform.submit();
    </script>
</body>
</html>

What he’s done here is create an HTML page that replicates the fields in bank transfer form as hidden inputs and then runs some JavaScript to submit the form. The form has its action set to post to the bank’s URL.

When you visit this page it makes a form post back to the bank site. If you want to try this out, I am hosting this HTML here. You have to make sure the website sample code is running on your machine before you click that link to see it working.

Let’s look at the contents of that form post.

POST /Home/Transfer HTTP/1.1
Referer: https://haacked.com/demos/csrf-mvc.html
User-Agent: ...
Content-Type: application/x-www-form-urlencoded
Host: 127.0.0.1:54607
Content-Length: 34
Cookie: .ASPXAUTH=98A250...03BB37

Amount=1000&destinationAccountId=2

It looks exactly the same as the one before, except the Referer is different. When the unsuspecting bank user visited the bad guy’s website, it recreated a form post to transfer funds, and the browser unwittingly sent the still active session cookie containing the user’s authentication information.

The end result is that I’m out of $1000 and BadGuy has his bank account increased by $1000. Drat!

It might seem that you could rely on the checking the Referer to prevent this attack, but some proxy servers etc… will strip out the Referer field in order to maintain privacy. Also, there may be ways to spoof the Referer field. Another mitigation is to constantly change the URL used for performing sensitive operations like this.

In general, the standard approach to mitigating CSRF attacks is to render a “canary” in the form (typically a hidden input) that the attacker couldn’t know or compute. When the form is submitted, the server validates that the submitted canary is correct. Now this assumes that the browser is trusted since the point of the attack is to get the general public to misuse their own browser’s authority.

It turns out this is mostly a reasonable assumption since browsers do not allow using XmlHttp to make a cross-domain GET request. This makes it difficult for the attacker to obtain the canary using the current user’s credentials. However, a bug in an older browser, or in a browser plugin, might allow alternate means for the bad guy’s site to grab the current user’s canary.

The mitigation in ASP.NET MVC is to use the AntiForgery helpers. Steve Sanderson has a great post detailing their usage.

The first step is to add the ValidateAntiForgeryTokenAttribute to the action method. This will validate the “canary”.

[ValidateAntiForgeryToken]
public ActionResult Transfer(int destinationAccountId, double amount) {
  ///... code you've already seen ...
}

The next step is to add the canary to the form in your view via the Html.AntiForgeryToken() method.

The following shows the relevant section of the view.

<% using (Html.BeginForm("Transfer", "Home")) { %>
<p>
    <label for="Amount">Amount:</legend>
    <%= Html.TextBox("Amount")%>
</p>
<p>
    <label for="destinationAccountId">
      Destination Account:
    </legend>
    <%= Html.DropDownList("destinationAccountId", "Select an Account") %>
</p>
<p>
    <%= Html.AntiForgeryToken() %>
    <input type="submit" value="transfer" />
</p>
<% } %>

When you view source, you’ll see the following hidden input.

<input name="__RequestVerificationToken" 
  type="hidden" 
  value="WaE634+3jjeuJFgcVB7FMKNzOxKrPq/WwQmU7iqD7PxyTtf8H8M3hre+VUZY1Hxf" />

At the same time, we also issue a cookie with that value encrypted. When the form post is submitted, we compare the cookie value to the submitted verification token and ensure that they match.

Should you be worried?

The point of this post is not to be alarmist, but to raise awareness. Most sites will never really have to worry about this attack in the first place. If your site is not well known or doesn’t manage valuable resources that can be transferred to others, then it’s not as likely to be targeted by a mass phishing attack by those looking to make a buck.

Of course, financial gain is not the only motivation for a CSRF attack. Some people are just a-holes and like to grief large popular sites. For example, a bad guy might use this attack to try and post stories on a popular link aggregator site like Digg.

One point I would like to stress is that it is very important to never allow any changes to data via GET requests. To understand why, check out this post as well as this story about the Google Web Accelerator.

What about Web Forms?

It turns out Web Forms are not immune to this attack by default. I have a follow-up post that talks about this and the mitigation.

If you missed the link to the sample code before, you can download the source here (compiled against ASP.NET MVC 2).

Technorati Tags: aspnetmvc,asp.net,csrf,security

code, humor comments edit

I’ve been relatively quiet on my blog lately in part because of all the work on ASP.NET MVC. However, the ASP.NET team is a relatively small team so we often are required to work on multiple features at the same time. So part of the reason I’ve been so busy is that while we were wrapping up ASP.NET MVC, I was also busy working on a core .NET Framework feature we plan to get into the next version (it was a feature that originated with our team, but we realized it belongs in the BCL).

The goal of the feature is to help deal with the very common task of handling string input. In many cases, the point is to convert the input into another type, such as an int or float. However, how do you deal with the fact that the string might not be convertible to the other type.

We realized we needed a type to handle this situation. A type that would represent the situation after the user has submitted input, but before you attempt the conversion. At this point, you have a string or another type.

clip_image004_thumb

For more details on the StringOr<T> Community Technology Preview (CTP), please see details on lead developer Eilon Lipton’s Blog (he’s a big fan of cats as you can see). He provides source code and unit tests for download. As always, please do provide feedback as your feedback is extremely important in helping shape this nascent technology.

Tags: framework , .net

asp.net mvc, asp.net comments edit

First let me begin by assuring you, this is not an April Fool’s joke.

2871423645_2f690a0c61Exciting news! Scott Guthrie announced today that we have released the source code for ASP.NET MVC 1.0 under the Ms-PL license, an OSI approved Open Source license with all the rights that license entails.

You can download the Ms-PL licensed source package from the download details page here. Just scroll down and look for the file named AspNetMvc1.Ms-PL.source.zip. My baby is growing up!

A big thanks must go out to everyone involved in making this happen and to those who approved it. It’s truly a team effort. When I joined Microsoft, I remember walking into ScottGu’s office to try and plant the seed for releasing ASP.NET MVC under the Ms-PL license. I came in armed with reasons why we should, but found him to be immediately receptive, if not already thinking along those lines. In fact, a lot of people such as Brian Goldfarb, my management chain, our LCA contact, etc… were completely on board, which was unexpected (though maybe it should not have been) and encouraging to me.

However, there’s agreement to do something and the actual doing. It still a lot of people to do the leg-work to make it happen. I personally was kept me very busy in the days leading up to the official RTM release. Let’s just say I feel like I’m one course away from getting a law degree.

I know one of the first questions some of you will ask is will we accept source code contributions (I’ve already seen the question on Twitter :). Unfortunately, at this time the answer is no, we do not accept patches. Please don’t let that stop you from contributing in other ways. The terms of the license do mean we need to stay on our toes to keep putting out compelling releases and we will work hard not to disappoint.

Personally (and this is totally my own opinion), I’d like to reach the point where we could accept patches. There are many hurdles in the way, but if you went back in time several years and told people that Microsoft would release several open source projects (Ajax Control Toolkit, MEF, DLR, IronPython and IronRuby, etc….) you’d have been laughed back to the present. Perhaps if we could travel to the future a few years, we’ll see a completely different landscape from today.

However, it is a complex issue and I don’t want to downplay that, but there are many of us who are looking for novel solutions and trying to push things forward. I really think in the long run, it is good for us and four our customers, otherwise we wouldn’t care.

But coming back to the present, I’m extremely pleased with where we are now and look forward to what will happen in the future. Someone once expressed disappointment that my involvement in open source projects seriously declined after joining Microsoft. It was my hope at the time that by the time it released, it would be clear that technically, I had been working on OSS. :)

subtext comments edit

Simo beat me to the punch in writing about this, After many long years being hosted on SourceForge, the Subtext submarine is moving into a new project hosting port.

We’ve finally moved off of SourceForge and onto Google Code’s project hosting. Our main site (primarily for end users) is still at http://subtextproject.com/ and I’ve hopefully updated every place it points to SourceForge to now point to Google Code.

Subtext-moves Image stolen from Simo’s blog. ;)

This was a very tough decision between CodePlex and Google Code. CodePlex is a great platform and I really like what they’ve done with being able to vote on issues etc… They seem to be innovating and adding new features at a rapid clip. I host Subkismet, a smaller project, on CodePlex and probably would choose it for a brand new project.

My one big complaint with CodePlex is that we really want native Subversion access, not a subversion bridge to TFS. For example, I was able to run the svnsync command to get the entire SVN history for Subtext into Google Code. That’s not something I could do today with CodePlex.

One other thing I really like with Google Code is that it’s fast. When you go to our project page, and click on the tabs, notice how fast the transition is. Click on an issue and see how fast you get there. Make a change and save it and it just snaps back. I spend a lot of time triaging and organizing issues etc… so this snappiness is really important to me.

Another great feature I love is how well code review is integrated into Google Code. For example, you can use the web interface to look at any revision in our repository. Take a look at r3406 for example.

Click on the diff link next to each file that was changed. For example, the diff for AkismetClient.cs. You get a nice side-by-side diff of the changes. You can double click on any line of code to leave a comment. Scroll down to line 160 and take a look at a comment. Don’t worry, I was the original author of that file so I’m not offending anybody but myself with that comment.

So there’s a lot I’d love to see improved with Google Code, but I’m pretty happy with the usability of the site overall. It’s a far improvement over SourceForge where I started to viscerally hate managing issues and doing any sort of administrative tasks over there.

We’re now using Google Groups for our Subtext discussion list and we have a separate group for notification emails such as commit emails. I’m going to leave the tracker and file releases on at SourceForge for a while longer until we’ve moved everything over. Unfortunately, there’s no automatic import from SourceForge for bug reports. But if you’re interested in keeping tabs on the progress Subtext is making, feel free to join the groups.

code comments edit

Recently, I tried to accomplish a simple task on a website which frustrated me because what should have been simple, was not. All I wanted to do was go to the Mix website and quickly find links to my sessions so I could post them here. Even I should be able to figure this out.

As a note, I’m using the Mix site as my illustration here, but I do it out of love and not mean spiritedness. Mix is my favorite conference, but its website…leaves something to be desired.

It seems particularly interesting that I’d run into this with the Mix site because the whole conference caters to a web design audience as well as web development. It just goes to show that even the best designers sometimes lose sight of what makes a user interface usable in the pursuit of cool flashy design. After all, designers often are trying to impress their designer friends when they should be creating a site that allows its audience to accomplish something?

Let’s take a look at my experience. Here’s the front page. Where do I start?

mix-site

Notice that there’s no search feature (I already tried Google). Fine, so I click all sessions and get this timeline view. There’s a search button there but when I type “MVC” and hit GO, nothing happens. I also try my last name. No go.

Sessions - Windows Internet
Explorer Near the bottom, I click on SPEAKERS and find my name. I then try clicking on one of my sessions which is highlighted as a link, and nothing happens (it’s since been fixed). When I highlight the session, a pop-up which appears to have links shows up, but I can’t seem to move my mouse over it because it disappears before my mouse gets there.

Speakers - Windows Internet
Explorer

After a moment of pondering, it occurs to me that maybe the video of my talk was not yet available at the time (it is now as I write this). It would have been nice if the site clearly indicated this, but instead, they were providing a link that went nowhere, giving me the impression that the video was available. Very confusing!

Now contrast that to this great post by Greg Duncan.

Mix 09 Quick Video Link List - Gregs Cool [Insert Clever Name] of the
Day - Windows Internet
Explorer

Notice that it’s the height of simplicity. It’s a list of all sessions (after all, the list of sessions wasn’t all that long) with headings showing you which video format the session is available in. I would have loved to see something like this on the Mix site.

A quick F3 to bring up my browser’s search feature, type in MVC, and boom goes the dynamite, I’ve found my sessions and am happily dissecting my speaking performance in disgust.

Kudos to Greg for putting this together. The fact that he even needed to put this together is a major indication that there’s a problem over at the Mix site.

The problem here is that in the pursuit of cool design, designers sometimes confuse ornamentation with good design. The Mix site has all sorts of flashy elements showing off a mastery of jQuery. Whoopee! And typically, I love those elements because when done well, they can make a site more usable. But in this case, I much prefer Greg’s simpler approach.

There’s not that much data to display, so rather than chop it up into pages, he just puts it all in one page which allows me, the reader, to make use of the familiar tools built into my browser such as F3 to search. Bam! Done! I can move on now.

Keep that in mind next time as you put together your next great user interface. How can you make it simpler and leverage the tools already in the client?

asp.net mvc comments edit

After my critical post of the Mix website, I found this other site, http://sessions.visitmix.com/, which should have been prominently linked to from the main site because it has a working search bar and is fairly usable and flashy!

I gave two sessions on ASP.NET MVC at Mix.

As you can see, we tried to have a bit of fun with the session titles. If you’re not tired of hearing me talk about MVC, you can also see an interview I did with Adam Kinney on Channel 9.

I will be posting my demos on my blog sometime soon hopefully. I want to give a few of them the full blog treatment. Partly because all of my demos are built live and I don’t keep the final product around, so I need to redo the steps. And I’m lazy. There. I said it.

comments edit

subtext200x200One of the cool products that I’m personally excited about announced at Mix is the updated Web Platform Installer.

I’m not going to lie. Part of the reason I’m excited about it is that it includes the latest version Subtext! The Web PI tool is a really nice way of installing and trying out various free and open source applications out there. It installs everything you need to get Subtext up and running on your local machine.

All you have to do is go to the Web App Gallery, find an application and click the Install button and it will install the application (if you have Web PI already installed). If you don’t have Web PI installed, it will prompt you to install Web PI and then install the app.

In this screenshot, you can see the dependencies needed to run Subtext are already listed.

Install
(2)

Subtext 2.1.1 is a very minor update to Subtext with a few bug fixes. The major fix is that the “forgot password” feature now works properly.

This was a little last minute surprise for the Subtext team as I literally put the required install package together in the last minutes leading up to Mix. In the meanwhile, major refactoring work is ongoing in our subversion repository. For example, the trunk now uses a custom Routing implementation (not yet built on ASP.NET MVC, but moving that way) Feel free to join in and help fix bugs and test. :)

asp.net, asp.net mvc comments edit

newdotnetlogo_2_thumb Today I’m happy to write that ASP.NET MVC 1.0 RTW (Release To Web) is now officially released.

This was one of several announcements ScottGu made at the Mix 09 conference today, which I unfortunately missed because I was on a plane to Vegas enroute to Mix 09. I was busy  back at the mother ship making sure everything was in order for the release.

Woohoo!

It’s been nearly a year and a half since I joined Microsoft and started working on it and what a ride it’s been.

Some highlights during that time:

With ASP.NET MVC, we wanted to release early and often providing a lot of transparency with our design process. We made a lot of changes based on user feedback. All in all, I counted 10 releases:

  1. December 2007 CTP
  2. Preview 2
  3. Preview 2.5 (April CodePlex release)
  4. Preview 3
  5. CodePlex Preview 4
  6. Preview 5
  7. Beta
  8. RC 1
  9. RC 2
  10. RTM!

A great way to learn about ASP.NET MVC is to go to the website, http://asp.net/mvc/. Also be sure to check out the free eBook which contains Chapter 1 of Professional ASP.NET MVC. This chapter contains an end-to-end walkthrough of building an application with ASP.NET MVC.

We also have an updated set of documents on MSDN worth checking out.

A great way to install ASP.NET MVC is via the new Web Platform Installer. I highly recommend it. Not only can you install ASP.NET MVC using it, but many other free applications such as one of my favorites, Subtext!

Technorati Tags: aspnetmvc,asp.net,subtext

code, asp.net mvc comments edit

I don’t know about you, but a great way to learn a new technology is to start using it. But to even start using it, it helps to be able to look at a real-world running application built on that technology. Combine that with source code and a walkthrough, and I think you have a winning combination.

That’s where NerdDinner comes in.

NerdDinner.com is the brainchild (and a big child at that) of Scott Hanselman. The concept is simple, let’s get nerds together over dinner so great ideas can flourish. The site is a great way to organize nerd dinners all over the place. The site is live and built using ASP.NET MVC, so go check it out.

But don’t stop there. Now that you’ve seen the site, go check out the source code on CodePlex. What better way to understand how the pieces fit together than to browse the source code for NerdDinner.

And I hope you don’t stop there either. There’s often a tendency to impart too much authority and expertise to an author. But as I’ve learned from writing this book among others, being an author doesn’t require any special expertise, gift, or ability. It just requires a love for self-inflicted pain. My personal pain threshold has been surpassed, so if you hear me talking about writing another, smack me…please.

In the spirit of Nerd Dinners, let’s get the ideas rolling! Create issues and let us know what you like and don’t like and what you think works and doesn’t work. We’re all in this together learning and refining our craft.

book-coverLooking at the site and source will show you what NerdDinner.com is, but to learn how it was built, I highly recommend checking out ScottGu’s blog post which presents a brisk end-to-end walkthrough of building NerdDinner.com.

His post is a preview of Chapter 1 of our book, Professional ASP.NET MVC 1.0. This chapter gets you started, while the rest of the book (still in production) dives deeper into each of the areas and concepts of the framework. And by the way, the eBook (aka Chapter 1) is licensed under a Creative Commons Attribution No Derivatives license, so feel free to redistribute.

So if you’ve been wondering why my blogging frequency declined this past year, now you know. Getting a release ready for RTM is a lot of work. Writing a book on that release at the same time is pure insanity.

Fortunately, I was in very good company with my esteemed co-authors, Rob “I Feel your pain” Conery, Scott “Five-head” Hanselman, and Scott “The GU” Guthrie, all of whom are completely insane! You heard it here first.

asp.net mvc comments edit

Yesterday, I wrote about troubleshooting Windows MSI Installers and talked about the pain we here feel when an installation fails. Turns out, it’s not always our fault. ;) It appears there’s a hotfix released for Visual Studio which addresses a problem with installing ASP.NET MVC when you have a third party add-in installed. I mentioned the three above because they are among the most commonly used add-ins which run into problems.

You can read about the Hotfix on the Visual Web Developer blog or simply go here to download it.

Note, that this doesn’t solve the problem for all add-ins. For example, someone reported an installation failure when they had Clone Detective installed. Uninstalling Clone Detective, installing ASP.NET MVC, and then reinstalling Clone Detective solved the problem. We’re still looking into why our installers are having problems with these add-ins.

asp.net mvc, asp.net comments edit

UPDATE: I updated this post slightly to address some good criticsm from Rob Mensching by removing the MsiZap section. Note that this post does not blame the Windows Installer for the few problems we have with ASP.NET MVC installations. The major culprit tends to be either devenv /setup or ngen.exe.

The point in this post is to provide tools for those who are very careful, know what they are doing, and really want to take a peek at how the installer works. These tools can help provide insight to whether the two culprits I just mentioned really are the issue causing the installation to fail in your particular case. Everything I point out here IS AT YOUR OWN RISK! :)

For the most part, the installations for ASP.NET MVC have been quite stable and we’ve tested the installer at every release and 99% of the problems we’ve run into fall into those two buckets. I did not intend to give the impression that we saved installation testing for the last moment, because that’s just not true. We merely had a last minute requirement change for our installer. RC 2 was a response to that change, not to a bug found. Now back to the post. :)

Having an installer crap out on you when you’re excited to try out a new piece of software is an incredibly frustrating experience. Frustrating both for the user as well as for the product team. Every time I read on Twitter that someone is having trouble installing ASP.NET MVC, a new ulcer throbs in my side.

A big part of the frustration is the lack of control and insight into what’s going wrong. Jacques (no blog), a developer on the ASP.NET team has put together a troubleshooting guide based on his research into what makes installers tick to help alleviate some of this lack of control. While his focus was primarily on the ASP.NET MVC installer, much of this information is useful for all MSI installers.

Windows Installer Log Files

Log files are not automatically generated by an MSI. To generate a log file, you will need to use msiexec from a command prompt to install the MSI, for example

msiexec /i AspNetMVC1-RC2.msi /l*v .\install.log

Alternatively you can enable logging by default by configuring a registry key for this. See this article on enabling Windows Installer Logging or if you are using Windows XP, take a look at this article on the same topic specific to Windows XP. Please note that enabling this feature can have an impact on system performance.

The MSI logs are probably the best place to start when trying to figure out why an installation failed. At the very least you will be able to identify the step in the installer that caused the problem.

For example, one common source of trouble is when a required service is not started such as:

  • .NET Optimization Service
  • ASP.Net State Service

Visual Studio Log Files

The ASP.NET MVC specific context menus in Visual Studio are installed by executing devenv.exe with the /setup switch. When the installer executes this command it automatically generates a log file at the location

%TEMP%\ MvcTemplates.log

The log file can be consulted for more details in case this step of the installation failed.

Orca

Orca is an editor for Windows install packages (MSI files) and is part of the Windows Installer SDK (the Orca MSI is located at %Program Files%\Microsoft SDKs\Windows\v6.0A\bin). While we don’t generally encourage people to edit the ASP.NET MVC installer, for those willing to take matters into their own hands (at your own risk!) Orca can be used to disable some of the steps executed by the installer. When you open the MSI you will see all the tables used by the installer. The feature table is probably the most interesting since it shows everything that will be installed.

clip_image002

Notice that by default, the Level attribute of all the features are set to 1. This means that the feature is enabled to be installed. We have some logic that will disable some of the features based on the information that we gather about the system on which were are installing. For example, the VWDExpress_Feature will be disabled if a user does not have Visual Web Developer installed. Similarly we will disable one of the MvcGacNgenXX_Features depending on whether the underlying OS is a 64-bit or 32-bit version.

Let’s say that you’ve tried to install MVC, but for some reason the installer fails to create a native image for System.Web.Mvc.dll using NGEN. Using Orca you can disable the MvcGACNgen32_Feature and MvcGACNgenAll_Feature by deleting these two rows from the MSI and saving it under a different name. Of course the best thing to do is to investigate why NGEN is failing in the first place.

Editing the MSI is not really encouraged since it may cause other problems such as failing to uninstall properly.

NGEN failures

We automatically GAC and NGEN System.Web.Mvc.dll during the setup process. Installing into the GAC (Global Assembly Cache) is a relatively painless operation. NGEN on the other hand is more prone to causing the installer to fail.

When installing via double clicknig the MSI, you will see an error message before the installation is rolled back if NGEN fails. The error message itself is not that useful as it will most likely just show a cryptic message and an HRESULT value (You can do a web search for the HRESULT and see whether there are workarounds available). To investigate the failure, take a look at the log generated by the MSI (assuming you had logging turned on when you made the install). Open the log file and search for ExecNetFx (this is the name of the WiX component the installer uses to run NGEN).

ExecNetFx:  Microsoft (R) CLR Native Image Generator 
  - Version 2.0.50727.3053
ExecNetFx:  Copyright (c) Microsoft Corporation.  All rights reserved.
ExecNetFx:  Installing assembly System.Web.Mvc, Version=1.0.0.0, 
  Culture=neutral, PublicKeyToken=31BF3856AD364E35
ExecNetFx:  Catastrophic failure (Exception from HRESULT: 0x8000FFFF 
  (E_UNEXPECTED))
ExecNetFx:  Error 0x8007006d: failed to allocate output string
ExecNetFx:  Error 0xffffffff: Command line returned an error.
ExecNetFx:  Error 0xffffffff: failed to execute Ngen command: 
  C:\Windows\Microsoft.NET\Framework\v2.0.50727\ngen.exe install 
  "System.Web.Mvc, Version=1.0.0.0, Culture=neutral, 
  PublicKeyToken=31BF3856AD364E35"

The example above is a very difficult problem to troubleshoot, but there are a few things you can try:

  • Open a command prompt, go to %WINDIR%\Microsoft.NET\Framework\v2.0.50727 and execute the following commands and then try to install MVC again
    • ngen ExecuteQueuedItems
    • ngen Update
  • Examine the NGEN log files located in %WINDIR%\Microsoft.NET\Framework\v2.0.50727. There are two files, ngen.log and ngen_service.log. The logs may contain information about other problems in the system that caused the failure, for example a missing DLL that may cause an update operation to fail when NGEN attempts to update invalid native images.
  • Make sure that the .NET Runtime Optimization service is running clip_image004

Windows Event Viewer

The installer executes a number of external tasks during the installation process.

  • devenv.exe /installvstemplates: This step is used to register the MVC project templates in Visual Studio. (For Visual Web Developer the executable is called VWDExpress.exe).
  • devenv.exe /setup: This step is executed to merge all the metadata in Visual Studio and is used by the MVC packages to register the context menus.
  • ngen.exe install: Used to generate native images of the System.Web.Mvc assembly. See this article for more details.
  • ngen.exe update: Updates any native images that have become invalid.

You will most likely see multiple warning events such as the example below. Visual Studio will log warning events for unknown attributes it encounters in templates and usually just moves past them. These warnings can be safely ignored.

clip_image006

But you may see other events that may cause the install to fail such as the example below:

clip_image008

Solving this problem is easy enough; restart the Cryptographic Service and run the installer again.

Visual Web Developer Express

A few users have reported that after installing MVC, they don’t see an option to create an MVC Web Application Project in Visual Web Developer. If you run into this problem, try the following:

  • Open VWD.
  • Go to Tools and select Import and Export Settings.
  • Select Reset all settings and click Next to complete the wizard.

Third Party Utilities

A number of users have reported problems during the installation of MVC or when trying to use some of the shortcuts inside Visual Studio. In many cases this was resolved by removing certain add-ins:

  • PowerCommands
  • Clone Detective
  • Spec# and F#

If you review the MSI logs you will most likely see something like the following:

MSI (s) (5C:70) [13:27:26:106]: Executing op: CustomActionSchedule(Action=VisualStudio_VSSetup_Command,ActionType=3122,Source=C:\Program Files\Microsoft Visual Studio 9.0\Common7\IDE\devenv.exe,Target=/setup,)

MSI (s) (5C:70) [13:28:41:075]: Note: 1: 1722 2: VisualStudio_VSSetup_Command 3: C:\Program Files\Microsoft Visual Studio 9.0\Common7\IDE\devenv.exe 4: /setup 
Error 1722. There is a problem with this Windows Installer package. A program run as part of the setup did not finish as expected. Contact your support personnel or package vendor. Action VisualStudio_VSSetup_Command, location: C:\Program Files\Microsoft Visual Studio 9.0\Common7\IDE\devenv.exe, command: /setup 

MSI (s) (5C:70) [13:28:42:716]: Product: Microsoft ASP.NET MVC RC 2 -- Error 1722. There is a problem with this Windows Installer package. A program run as part of the setup did not finish as expected. Contact your support personnel or package vendor. Action VisualStudio_VSSetup_Command, location: C:\Program Files\Microsoft Visual Studio 9.0\Common7\IDE\devenv.exe, command: /setup

If installing MVC fails in this manner, try the following:

  • Create a log file, for example, devenv.log.
  • Open a Visual Studio command prompt and execute devenv /setup /log devenv.log. Make sure that you have administrator rights since that’s required by the /setup switch.
  • Scan the log file and look for any <description> elements that contain errors. This may point to problematic packages that are causing devenv.exe to fail when MVC is trying to configure its context menus.

Another option to try is the following:

  • Open a Visual Studio command prompt.
  • Start Visual Studio by executing the command devenv /ResetSkipPkgs.
  • Close Visual Studio.
  • Execute devenv /setup.

Tags: aspnetmvc , msizap , installation

asp.net mvc, asp.net comments edit

UPDATE: This post is outdated. ASP.NET MVC 1.0 has been released already!

Today we’ve made the Release Candidate 2 for ASP.NET MVC available for download.

This post will cover some of the changes with ASP.NET MVC we made in response to internal and external feedback since our last Release Candidate.

Let me provide the quick and dirty summary, and then fill in the details.

  • Setup will now require .NET 3.5 SP1
  • Bin deployment to 3.5 host without SP1still possible
  • New server-only install mode

Now onto the details

Setup Requires .NET Framework 3.5 SP1

The new installer will require that .NET Framework 3.5 SP1 be installed on your machine. For your development environment, we recommend that you also install Visual Studio 2008 SP1, but this is not required.

The reason we made this change is that we were including the System.Web.Routing.dll and System.Web.Abstractions.dll assemblies with the MVC installer. However, it does not make sense for us to co-ship assemblies which are part of the Framework as this would negatively affect our ability to service these two assemblies.

Bin deployment to 3.5 host without SP1 Still Possible

We are not taking a runtime dependency on SP1 other than our existing dependency on System.Web.Routing.dll and System.Web.Abstractions.dll. Thus you can still bin deploy your application to a hosting provider who has .NET 3.5 installed without SP1 by following these instructions.

Note that in such a configuration, you take on the risk of servicing those assemblies. Should we release any important updates to any of these assemblies, you’ll have to manually patch your application. However, you will still enjoy full CSS (formerly PSS) support for this configuration.

New Server-Only Install Mode

We’re adding an option to the installer that enables installing on a server that does not have Visual Studio at all on the machine, which is useful for production servers and hosting providers.

The installer will no longer block on a machine that does not have Visual Studio installed. Instead, it will continue the standard MVC installation without installing the Visual Studio templates. The assemblies will still be installed into the GAC and native images will also be generated.

Certain other requirements have also been relaxed. For example, if the machine on which the installation is performed contains Visual Web Developer Express Edition 2008 without SP1, the installation will still proceed, but with a warning prompt. You can also automate this installation by invoking the installer using the command line (all on one line):

msiexec /i AspNetMvc-setup.msi /q /l*v .\mvc.log MVC_SERVER_INSTALL="YES"

Summary

As a result of these changes, we realized it would be prudent to have one more public release candidate. As I mentioned, there are very few runtime and tooling changes. Most of the changes are in the installer and we want to make sure that the installer is rock solid before we call it an RTM.

Based on all your feedback from the first Release Candidate, as well as our own investigations and testing, we are confident that the Release Candidate 2 will be solid and lead to a strong RTM.

In case you missed it above, here’s the link to the download page. You can find out what else has changed in the RC 2 release notes.

And before I forget, as usual, we published the source code and our MvcFutures assembly on CodePlex.

asp.net mvc, asp.net comments edit

MIX09_SpeakerBling_VegasStrip_136x186 I finally have my registration worked out and I’m officially going to Mix09 this year! Woohoo! Not only am I going, but I’ll be speaking at Mix for the first time, so be sure to come by and say hi.

I’ve mentioned this before, but Mix is easily one of my favorite Microsoft conferences. Jeff Atwood said it well:

MIX is by far my favorite Microsoft conference after attending the ’06 and ’07 iterations…

What I love about Mix is that it …

  • is relatively small and intimate, at around 2,000 attendees.
  • seamlessly merges software engineering and design.
  • includes a lot of non-Microsoft folks, even those that are traditionally hostile to Microsoft, so there’s plenty of perspective.

I’ve been to every one so far:

I’ll be giving the following two sessions at Mix this year:

MIX09-T50F ASP.NET MVC: America’s Next Top Model View Controller Framework

See how to build a simple application that highlights some of the key capabilities of ASP.NET MVC. Also, learn how to improve productivity using some of the cool new tooling features

MIX09-T44F Microsoft ASP.NET Model View Controller (MVC): Ninja on Fire Black Belt Tips

See a walkthrough of several tips and tricks for getting the most out of the ASP.NET MVC framework.

The first talk is an introductory overview and the second one allows me to dig in a bit more.

I’ve been working hard on putting together some demos so I will hopefully have something interesting to show off. In any case, if you’re going to the conference, leave a comment and when you’re there, be sure to come by and say hello.

Also, let me know if there’s anything specific you’d like me to cover. This should be a fun one.