Scripting ASP.NET MVC Views Stored In The Database

Say you’re building a web application and you want, against your better judgment perhaps, to allow end users to easily customize the look and feel – a common scenario within a blog engine or any hosted application.

With ASP.NET, view code tends to be some complex declarative markup stuck in a file on disk which gets compiled by ASP.NET into an assembly. Most system administrators would first pluck out their own toenail rather than allow an end user permission to modify such files.

It’s possible to store such files in the database and use a VirtualPathProvider to load them, but that requires your application (and thus their views) to run in full trust. Is there a way you could safely store such views in the database in an application running in medium trust where the code in the view is approachable?

At the ALT.NET conference a little while back, Jimmy Schementi and John Lam gave a talk about the pattern of hosting a scripting language within a larger application. For example, many modern 3-D Games have their high performance core engine written in C++ and Assembly. However, these games often use a scripting language, such as Lua, to write the scripts for the behaviors of characters and objects.

An example that might be more familiar to more people is the use of VBA to write macros for Excel. In both of these cases, the larger application hosts a scripting environment that allow end users to customize the application using a simpler lighter weight language than the one the core app is written in.

A long while back, I wrote a blog post about defining ASP.NET MVC Views in IronRuby followed by a full IronRuby ASP.NET MVC stack. While there was some passionate interest by a few, in general, I was met with the thunderous sound of crickets. Why the huge lack of interest? Probably because I didn’t really sell the benefit and the explain the pain it solves. I’m sure many of you were asking, Why bother? What’s in it for me?

After thinking about it some more, I realized that my prototypes appeared to suggest that if you want to take advantage of IronRuby, you would need to make some sort of wholesale switch to a new foreign language, not something to be undertaken lightly.

This is why I really like Jimmy and John’s recent efforts to focus on showing the benefits of hosting the DLR for scripting scenarios like the ones mentioned above. It makes total sense to me when I look at it in this perspective. The way I see it, most developers spend a huge bulk of their time in a single core language, typically their “language of choice”. For me, I spend the bulk of my time writing C# code.

However, I don’t think twice about the fact that I also write tons of JavaScript when I do web development, and I’ll write the occasional VB code when I need a new Macro for Visual Studio or Excel. I also write SQL when I need to. I’m happy to pick up and use a new language when it will enable me to do the job at hand more efficiently and naturally than C# does. I imagine many developers feel this way. The occasional use of a scripting languages is fine when it gets the job done and I can still spend most of my time in my favorite language.

So I started thinking about how that might work in a web application. What if you could write all your business logic and controller logic in your language of choice, but have your views written in a light weight scripting language. If my web application were to host a scripting engine, I could actually store the code in any medium I want, such as the database. Having them in the database makes it very easy for end users to modify it since it wouldn’t require file upload permissions into the web root.

This is where hosting the DLR is a nice fit. I put together a proof of concept for these ideas. This is just a prototype intended to show how such a workflow might work. In this prototype, you go about creating your models and controllers the way you normally would.

For example, here’s a controller that returns some structured data to the view in the form of an anonymous type.

public ActionResult FunWithScripting()
{
  var someData = new { 
    salutation = "Are you having fun with scripting yet?", 
    theDate = DateTime.Now,
      numbers = new int[] { 1, 2, 3, 4 } 
  };

  return View(someData);
}

Once you write your controller, but before you create your view, you compile the app and then go visit the URL.View does not exist view

We haven’t created the view yet, so let’s follow the instructions and login. Afterwards, we this:

view editor

Since the view doesn’t exist, I hooked in and provided a temporary view for the controller action which contains a view editor. Notice that at the bottom of the screen, you can see the current property names and values being passed to the view. For example, there’s an enumeration of integers as one property, so I was able to use the Ruby each method to print them out in the view.

The sweet little browser based source code editor is named Edit Area created by Christophe Dolivet. Unfortunately, at the time I write this, it doesn’t yet have support for ERB style syntax highlighting schemes. That’s why the <% and %> aren’t highlighted in yellow.

When I click Create View, I get taken back to the request for the same action, but now I can see the view I just created (click to enlarge).

Fun with scripting view

In the future, I should be able to host C# views in this way. Mono already has a tool for dynamically compiling C# code passed in as a string which I could try and incorporate.

I’m seriously thinking of making this the approach for building skins in a future version of Subtext. That would make skin installation drop dead simple and not require any file directory access. Let me know if you make use of this technique in your applications.

If you try and run this prototype, please note that there are some quirky caching issues with editing existing views in the prototype. It’ll seem like your view is not being edited, but it’s a result of how views are being cached. It might take a bit of time before your edits show up. I’m sure there are other bugs I’m still in the process of fixing. But for the most part, the general principle is sound.

You can download the prototype here.

What others have said

Requesting Gravatar... Andrea Balducci Apr 23, 2009 12:27 AM
# re: Scripting ASP.NET MVC Views Stored In The Database
Great! I'm currently using VirtualPathProvider to embed views and controllers in separate assemblies and dynamically load them at application startup (so i can can mix mvc apps in a portal web app just by xcopy deployment) and I was thinking about adding dynamic view creation in a future version of my framework. I'll try your prototype as soon as i can, just loving this approach!

Thanks for sharing!
Requesting Gravatar... Shiju Varghese Apr 23, 2009 12:47 AM
# re: Scripting ASP.NET MVC Views Stored In The Database
Phill, Awesome post.
Requesting Gravatar... Vijay Santhanam Apr 23, 2009 12:57 AM
# re: Scripting ASP.NET MVC Views Stored In The Database
Cool! Your zip is missing MemberPropertyInfo.cs
Requesting Gravatar... Haacked Apr 23, 2009 1:11 AM
# re: Scripting ASP.NET MVC Views Stored In The Database
@Vijay thanks for the catch! I forgot to svn add that file before I svn exported the project. In other words, my bad!
Requesting Gravatar... Jonathan Adams Apr 23, 2009 1:20 AM
# re: Scripting ASP.NET MVC Views Stored In The Database
I have been doing something similar with a standard web forms using nvelocity. This would be a much better fit especially now I making the hike over to MVC , excellent post thanks phil !
Requesting Gravatar... Dan Elliott Apr 23, 2009 1:29 AM
# re: Scripting ASP.NET MVC Views Stored In The Database
Fantastic article, Phil! Thank you :)

I look forward to giving it a whirl.

Kindness,

Dan
Requesting Gravatar... Maarten Balliauw Apr 23, 2009 2:00 AM
# re: Scripting ASP.NET MVC Views Stored In The Database
Sounds like a great way to embed view inside my plugin assemblies (blog.maartenballiauw.be/.../ASPNET-MVC-and-the-...(MEF).aspx)

Do you have any idea what this means to performance?
Requesting Gravatar... Charlie M Apr 23, 2009 2:38 AM
# re: Scripting ASP.NET MVC Views Stored In The Database
Fantastic! Excellent post and exactly what I was looking for.

ASP.NET MVC + a dynamic language = a pretty decent CMS offering!
Requesting Gravatar... Vijay Santhanam Apr 23, 2009 2:52 AM
# re: Scripting ASP.NET MVC Views Stored In The Database
ModelProperties.ascx is missing too
Requesting Gravatar... Andy Apr 23, 2009 3:41 AM
# re: Scripting ASP.NET MVC Views Stored In The Database
Am I right that in ASP.NET 4.0 that VirtualPathProvider will work in Medium Trust (I'm sure this was mentioned a month back on one of the ASP.NET team member's blogs, but have lost the link!)? Of course, your approach will help for hosts with older .NET installs.
Requesting Gravatar... huey Apr 23, 2009 5:09 AM
# re: Scripting ASP.NET MVC Views Stored In The Database
I really hate having to make ViewModels (I try and just pass Business Object but it doesn't always fit). It can become annoying when you start to get a ton of ViewModels that offer no benefit as far as I can tell -- just clutter.

I'm not really dynamic language smart, and maybe you can do this with C#, but passing an anonymous type to the View seems awesome. No need to create a model just for the hand off from the controller to the view.

Is this only available with dynamic languages? I wouldn't want to use a dynamic language for my business logic (I'm comfortable with C#) but for the view it seems to have benefits.
Requesting Gravatar... Andy Apr 23, 2009 5:11 AM
# re: Scripting ASP.NET MVC Views Stored In The Database
Update - the post that suggests VirtualPathProviders and BuildProviders will work in Medium Trust in ASP.NET 4.0 is weblogs.asp.net/.../...t-compiler-black-magic.aspx.
Requesting Gravatar... Chris Cyvas Apr 23, 2009 5:25 AM
# re: Scripting ASP.NET MVC Views Stored In The Database
I was just looking for something like this last week - excellent!
Requesting Gravatar... Page Brooks Apr 23, 2009 5:34 AM
# re: Scripting ASP.NET MVC Views Stored In The Database
I think Graffiti CMS does something similar to this using Chalk.

Requesting Gravatar... HB Apr 23, 2009 6:28 AM
# re: Scripting ASP.NET MVC Views Stored In The Database
@huey

I hear what you're saying - I've written a class called AnonymousType that hides all the complexities of Reflection and lets me use simpler methods like something.Get<int>("Total").

You could probably write your own pretty easily, or at least until the new dynamic comes out.
Requesting Gravatar... Dody Gunawinata Apr 23, 2009 8:17 AM
# re: Scripting ASP.NET MVC Views Stored In The Database
Override the caching lookup in VirtualFile to zero so you will not have a cache issue. The problem is that you are pretty much recompiling the view on the fly for every request.
Requesting Gravatar... Haacked Apr 23, 2009 8:51 AM
# re: Scripting ASP.NET MVC Views Stored In The Database
Yep, in .NET 4.0, VirtualPathProvider will run in medium trust. Once that happens, I may be able to swap out my approach to use VPP combined with DLR support for ASP.NET. Or I could use Spark in that case, which also supports IronRuby and IronPython.

As for performance, it's probably not faster than a fully compiled view, but with proper caching, it should hopefully not be a problem on a real site.
Requesting Gravatar... john Apr 23, 2009 10:18 AM
# re: Scripting ASP.NET MVC Views Stored In The Database
is there any chance to get a run through of the code. I downloaded it, i might be new to this whole thing but it looks like you are using a virtualpathprovider still. Although I can't open all your projects with my version of visual studio.
Requesting Gravatar... Guillaume Apr 23, 2009 3:06 PM
# re: Scripting ASP.NET MVC Views Stored In The Database
As far as I know BRail is using this kind of approach using Boo as a scripting language. I don't know if someone tryed a DB implementation of it yet.

Does anyone have some interesting links about BRail ?
Requesting Gravatar... james Apr 23, 2009 3:46 PM
# re: Scripting ASP.NET MVC Views Stored In The Database
Quote from "http://www.lua.org/about.html"

'Please do not write it as "LUA", which is both ugly and confusing, because then it becomes an acronym with different meanings for different people. So, please, write "Lua" right!'

;-)
Requesting Gravatar... Farrio Apr 23, 2009 7:09 PM
# re: Scripting ASP.NET MVC Views Stored In The Database
OMG! COOL!
Requesting Gravatar... Haacked Apr 23, 2009 9:20 PM
# re: Scripting ASP.NET MVC Views Stored In The Database
@Huey all our built in helpers work with anonymous types. If you call <%= Html.Eval("PropertyName") %> we'll do reflection on Model object to grab that value.
Requesting Gravatar... Koistya `Navin Apr 24, 2009 1:10 AM
# re: Scripting ASP.NET MVC Views Stored In The Database
Great finding. Going to use this approach in my apps.
Requesting Gravatar... Daniel Plaisted Apr 24, 2009 3:30 PM
# re: Scripting ASP.NET MVC Views Stored In The Database
"Mono already has a tool for dynamically compiling C# code passed in as a string which I could try and incorporate."

So does vanilla .NET. See http://support.microsoft.com/kb/304655, or the CSharpCodeProvider class.
Requesting Gravatar... zihotki Apr 25, 2009 1:49 PM
# re: Scripting ASP.NET MVC Views Stored In The Database
Daniel,

CSharpCodeProvider can be used only in FullTrust environment. But the main goal of this article is to show to make this working in MediumTrust too. So CSharpCodeProvider can not be used here.
Requesting Gravatar... bradvin Apr 30, 2009 12:38 AM
# re: Scripting ASP.NET MVC Views Stored In The Database
awesome post!

"Mono already has a tool for dynamically compiling C# code"

how would you go about using this tool from your code?
Requesting Gravatar... Haacked May 04, 2009 9:10 AM
# re: Scripting ASP.NET MVC Views Stored In The Database
@bradvin I think it's just a library you reference. You can go to the Mono site and look around. Maybe ask someone in the forums. I've never used it. I was just told about it.
Requesting Gravatar... Eric Sims May 14, 2009 7:43 AM
# re: Scripting ASP.NET MVC Views Stored In The Database
Phil, did you ever get the caching bug figured out? It seems that you have to override the CacheDependency methond (like Dody said) (at least in the standard VPP implementation).

I think the default is that it creates a cache in the 'Temporary ASP.NET Files' folder and keeps it there for a default of 20 minutes.

My problem is that I'm having difficulty invalidating that cache based on a trigger of some sort (file dependency, etc).

I can, of course, delete those files manually, but that's no use in the real world.

Please let me know once you have this figured out.
Requesting Gravatar... Bret (RunXc) May 19, 2009 1:13 PM
# re: Scripting ASP.NET MVC Views Stored In The Database
I am curious. Why did you use IronRuby instead of IronPython?? I have been thinking of learning a bit about the DLR but I am not sure which Language to look at first.
Requesting Gravatar... Eric Sims May 19, 2009 1:16 PM
# re: Scripting ASP.NET MVC Views Stored In The Database
i figured out in that edit_area editor how to make the syntax highlight. it's pretty easy actually.

you just create a new .js file (copy a similar existing one like html.js) and add your own highlight logic and instead of doing a 'color: #ff0000', you just do a 'background-color: #ff0000'.

hope this helps somebody.
Requesting Gravatar... Haacked May 19, 2009 7:22 PM
# re: Scripting ASP.NET MVC Views Stored In The Database
@Bret I've used Ruby in the past (with Rails for example) and I'm more familiar with it than Python. That's all.
Requesting Gravatar... David Robbins May 27, 2009 4:41 AM
# re: Scripting ASP.NET MVC Views Stored In The Database
Very cool. In a sense it is similar to microtemplates but processed server side.
Requesting Gravatar... Ian Mckay Jan 24, 2010 1:41 PM
# re: Scripting ASP.NET MVC Views Stored In The Database
Hi Phil. Great post! This is exactly what I've been looking for.

When I try to download the source I'm getting a corrupted zip file error in win 7 explorer and winrar.
Requesting Gravatar... Lars Jan 25, 2010 12:15 PM
# re: Scripting ASP.NET MVC Views Stored In The Database
Hello!

I am getting the same Zip file error. Any chance of building a new zip file?
Requesting Gravatar... Ed Feb 17, 2010 8:55 AM
# re: Scripting ASP.NET MVC Views Stored In The Database
Anxious to see it...but file is corrupt
Requesting Gravatar... Kieron Lanning Mar 01, 2010 5:45 AM
# re: Scripting ASP.NET MVC Views Stored In The Database
Hi Phil, as some people have reported, the zip file is corrupt, think you could fix/ post an update?

Thanks,
Kieron
Requesting Gravatar... silky Mar 06, 2010 2:18 AM
# re: Scripting ASP.NET MVC Views Stored In The Database
Wow. This has got to be one of the worst ideas I've read about in a while. I'll be honest, I've read your site before (a long time ago) and decided that it wasn't to my liking, but promoting such a dynamic strategy in our language, and the numerous resulting issues that could come from this implementation (that you haven't raised) it really disturbs me. I know that won't mean anything, but I'm just fairly shocked. And it is even more disturbing that so many people seem to like it. I think someone such as yourself, who numerous people seem to respect, should *at the very least* comment on the downsides of such an implementation, because they are significant.

I realise this was posted a while ago, so maybe you've commented on it since then; I hope so. Regardless, amongst the claims of joy and brilliance about this system, I'd like to express my disgust.
Requesting Gravatar... Tad Mar 13, 2010 1:30 PM
# re: Scripting ASP.NET MVC Views Stored In The Database
Phil , could you re upload that project ?:) please :):)
Requesting Gravatar... Adam Mar 16, 2010 11:50 AM
# re: Scripting ASP.NET MVC Views Stored In The Database
It's been almost a year since this was posted and I'm interested in doing this for an upcoming MVC 2.0 project, so I was wondering if anyone has tried using this technique for a real application. I posted a question on StackOverflow, so any help would be appreciated. Thanks!
Requesting Gravatar... Fred Apr 07, 2010 2:10 PM
# re: Scripting ASP.NET MVC Views Stored In The Database
the download link is broken.
Requesting Gravatar... mary Jul 04, 2010 11:41 PM
# DVD Ripper
I'd like you. looking forward to see more of your wonderful article, refueling.
Requesting Gravatar... DVD Ripper Jul 04, 2010 11:44 PM
# blu-ray
wehhde
Requesting Gravatar... Andrew Siemer Jul 28, 2010 12:11 PM
# re: Scripting ASP.NET MVC Views Stored In The Database
Phil, this is exactly what I was hoping to find (without actually looking for it). I am under way creating a new multi-tenant CMS app (don't ask why) and was going to use NVelocity for dynamic view content in my views across multiple sites. Given that NVelocity hasn't been updated in years...I was really hoping for another language - and Ruby is super popular! Now I need to figure out a way to pull routes and controllers from the db and dynamically compile/load controllers... Thoughts on this last piece?
Requesting Gravatar... Matt Aug 30, 2010 12:00 PM
# re: Scripting ASP.NET MVC Views Stored In The Database
Any way you can put the code back up? Getting a file not found.

What do you have to say?

(will show your gravatar)
Please add 1 and 5 and type the answer here: