TDD
All things TDD, including Mocking, etc...
Developers take pride in speaking their mind and not shying away from touchy subjects. Yet there is one subject makes many developers uncomfortable. Testing. I’m not talking about drug testing, unit testing, or any form of automated testing. After all, while there are still some holdouts, at least these types of tests involve writing code. And we know how much developers love to write code (even though that’s not what we’re really paid to do). No, I’m talking about the kind of testing where you get your hands dirty actually trying the application. Where you attempt to...
Suppose you have a test that needs to compare strings. Most test frameworks do a fine job with their default equality assertion. But once in a while, you get a case like this: [Fact]
public void SomeTest()
{
Assert.Equal("Hard \tto\ncompare\r\n", "Hard to\r\ncompare\n");
}
Let’s pretend the first value in the above test is the expected value and the second value is the value you obtained by calling some method.
Clearly, this test fails. So you look at the output and this is what you see:
It’s pretty hard to compare those...
In the past, I’ve tried various schemes to structure my unit tests but never fell into a consistent approach. Pretty much the only rule I had (which I broke all the time) was to write a test class for each class I tested. I would then fill that class with a ton of haphazard test methods. That was until I saw the approach that Drew Miller took with NuGet.org. The way he structured the unit tests struck me as odd at first, but quickly won me over. Drew tells me he can’t take all the credit for this approach....
Given how central JavaScript is to many modern web applications, it is important to use unit tests to drive the design and quality of that JavaScript. But I’ve noticed that there are a lot of developers that don’t know where to start. There are many test frameworks out there, but the one I love is QUnit, the jQuery unit test framework. Most of my experience with QUnit is writing tests for a client script library such as a jQuery plugin. Here’s an example of one QUnit test file I wrote a while ago (so you know...
A while back I wrote about mocking successive calls to the same method which returns a sequence of objects. Read that post for more context. In that post, I had written up an implementation, but quickly was won over by a better extension method implementation from Fredrik Kalseth. public static class MoqExtensions
{
public static void ReturnsInOrder<T, TResult>(this ISetup<T, TResult> setup,
params TResult[] results) where T : class {
setup.Returns(new Queue<TResult>(results).Dequeue);
}
}
As good as this extension method is, I...
I admit, up until now I’ve largely ignored the BDD (Behavior Driven Development) Context/Specification style of writing unit tests. It’s been touted as a more approachable way to learn TDD (Test Driven Development) and as a more natural transition from user stories to the actual code design. I guess my hesitation to give it a second thought was that I felt I didn’t need a more approachable form of TDD. Recently, my Subtext partner in crime, Steve Harman, urged me to take another fresh look at BDD Context/Specification style tests. I trust Steve’s opinion, so I took another look...
A while back I talked about how testable code helps manage complexity. In that post, I mentioned one common rebuttal to certain design decisions made in code in order to make it more testable. Why would I want to do XYZ just do improve testability? Recently, I heard one variation of this comment in the comments to my post on unit test boundaries. Several people suggested that it’s fine to have unit tests access the database, after all, the code relies on data from the database, it should be tested. Implicit...
One principle to follow when writing a unit test is that a unit test should ideally not cross boundaries. Michael Feathers takes a harder stance in saying… A test is not a unit test if: It talks to the database It communicates across the network It touches the file system It can’t run at the same time as any of your other unit tests ...
Way down the road, it would be nice to be able to build ASP.NET MVC applications using a DLR language such as IronRuby. However, enabling DLR language support isn’t free.
There are going to be places in our design that are specific to statically typed languages (such as Attribute based filters) that just wouldn’t work (or would be too unnatural) with a dynamic language.
Ideally we can minimize those cases, and for the ones we can’t, we need to make sure the extensibility of the framework allows for extending the system in such a way that we can provide a DLR friendly...
When I build applications, I personally like to have my unit tests in a separate class library project than the application I am testing.
That’s just how I roll.
I just assumed this is how everyone structures their unit tests, but I’ve talked with some people who have a deep history in TDD who put their code in the same project.
So I wanted to create a simple poll to find out how people are actually structuring their unit tests. I’ve been involved in various internal discussions looking at
how Microsoft design, tools, code can better support unit testing across the board.
Note...
UPDATE: I should have entitled this “Comparing Rhino Mocks and MoQ for State Based Testing”. I tend to prefer state-based testing over interaction based testing except in the few cases where it is absolutely necessary or makes the test much cleaner. When it is necessary, it is nice to have interaction based testing available. So my comparison is incomplete as I didn’t compare interaction based testing between the two frameworks. For the longest time I’ve been a big fan of Rhino Mocks and have often written about it glowingly. When Moq came on the scene, I remained blissfully ignorant...
In a recent post, Frans Bouma laments the lack of computer science or reference to research papers by agile pundits in various mailing lists (I bet this really applies to nearly all mailing lists, not just the ones he mentions). What surprised me to no end was the total lack of any reference/debate about computer science research, papers etc. ... in the form of CS research. Almost all the debates focused on tools and their direct techniques, not the computer science behind them. In general asking ’Why’ wasn’t answered with: "research has shown that..." but with...
UPDATE: Completely ignore the contents of this post. All of this is out-dated. Test specific subclasses are no longer necessary with ASP.NET MVC since our April CodePlex refresh Just a brief note on writing unit tests for controller actions. When your action has a call to RedirectToAction or RenderView (yeah, pretty much every action) be aware that these methods have dependencies on various context objects. If you attempt to mock these objects, you sometimes also have to mock their dependencies and their dependencies' dependencies and so on, depending on what you are trying to test. This is why...
One of the guiding principles in the design of the new ASP.NET MVC Framework is enabling TDD (Test Driven Development) when building a web application. If you want to follow along, this post makes use of ASP.NET MVC CodePlex Preview 4 which you’ll need to install from CodePlex. I’ll try and keep this post up to date with the latest releases, but it may take me time to get around to it. This post will provide a somewhat detailed walk-through of building a web application in a Test Driven manner while also demonstrating one way to integrate a Dependency...
Sometimes when writing unit tests, you run into the case where you want to override the behavior of a specific method.
Here’s a totally contrived example I just pulled from my head to demonstrate this idea. Any similarity to specific real world scenarios is coincidental ;). Suppose we have this class we want to test.
public class MyController
{
public void MyAction()
{
RenderView("it matches?");
}
public virtual void RenderView(string s)
{
throw new NotImplementedException("To ensure this method is overridden.");
}
}
What we have here is a class...
Note that in the same vein as Pele, Ronaldinho and Ronaldo, Joel has reach that Brazillian Soccer player level of stardom in the geek community and can pretty much go by just his first name. Admit it, you knew who I was referring to in the title. Admit it! Please indulge me in a brief moment of hubris. I was reading part 1 of Joel Spolsky’s talk at Yale that he gave on November 28 and came upon this quote on code provability... The problem, here, is very fundamental. In order to...
My friend (and former boss and business partner) Micah found this gem of a quote from Donald Knuth addressing code proofs. Beware of bugs in the above code; I have only proved it correct, not tried it. Micah writes more on the topic and reminds me of why I enjoyed working with him so much. He’s always been quite thoughtful in his approach to problems. And I’m not just saying that because he agrees with me. ;) On another note, several commenters pointed out that one thing I didn’t mention before, but should...
I’m currently doing some app building with ASP.NET MVC in which I try to cover a bunch of different scenarios. One scenario in particular I wanted to cover is approaching an application using a Test Driven Development approach. I especially wanted to cover using various Dependency Injection frameworks, to make sure everything plays nice. Since I’ve already seen demos with Castle Windsor and Spring.NET, I wanted to give StructureMap a try. Here is the problem I’ve run into. Say I have a class like so: public class MyController :...
Frans Bouma wrote an interesting response to my last post, Writing Testable Code Is About Managing Complexity entitled Correctness Provability should be the goal, not Testability. He states in his post: When focusing on testability, one can fall into the trap of believing that the tests prove that your code is correct. God I hope not. Perhaps someone in theory could fall into that trap, but a person could also fall into the trap and buy a modestly priced bridge I have to sell to them in the bay area? This seems like...
When discussing the upcoming ASP.NET MVC framework, one of the key benefits I like to tout is how this framework will improve testability of your web applications.
The response I often get is the same question I get when mention patterns such as Dependency Injection, IoC, etc...
Why would I want to do XYZ just to improve testability?
I think to myself in response
Just to improve testability? Isn’t that enough of a reason!
That’s how excited I am about test driven development. Testing seems enough of a reason for me!
Of course, when I’m done un-bunching my knickers, I realize that despite all the benefits...
UPDATE: This content is a bit outdated as these interfaces have changed in ASP.NET MVC since the writing of this post.
One task that I relish as a PM on the ASP.NET MVC project is to build code samples and sample applications to put the platform through its paces and try to suss out any problems with the design or usability of the API.
Since testability is a key goal of this framework, I’ve been trying to apply a Test Driven Development (TDD) approach as I build out the sample applications. This has led to some fun discoveries in terms of using...
It is a sad fact of life that, in this day and age, arguments are not won with sound logic and reasoning. Instead, applying the principle of framing an argument is much more effective at swaying public opinion. So the next time you try to make headway introducing Test Driven Development (or even simply introducing writing automated unit tests at all) into an organization and are rebuffed with... Don’t bring your fancy schmancy flavor of the week agile manifesto infested “methodology” here kiddo. I’ve been writing software my way for a loooong time... You can reply with... ...
This is a quick follow-up to my last post. That seemed like such a common test situation I figured I’d write a quick generic method for encapsulating those two tests. I’ll start with usage. [Test]
public void FileBrowserSecureCreationTests()
{
AssertSecureCreation<FileBrowserConnector>(new string[] {"Admins"});
}
And here’s the method.
/// <summary>
/// Helper method. Makes sure you can create an instance
/// of a type if you have the correct role.</summary>
/// <typeparam name="T"></typeparam>
/// <param name="allowedRoles"></param>
public static void AssertSecureCreation<T>(string[] allowedRoles
, params object[] constructorArguments)
{
try
{
Activator.CreateInstance(typeof (T),...
This is a simple little demonstration of how to write unit tests to test out a specific role based permission issue using NUnit/MbUnit and Rhino Mocks.
In Subtext, we have a class named FileBrowserConnector that really should only ever be constructed by a member of the Admins role. Because this class can write to the file system, we want to take extra precautions other than simply restricting access to the URL in which this object is created.
Here are two tests I wrote to begin with.
[Test]
[ExpectedException(typeof(SecurityException))]
public void NonAdminCannotCreateFileConnector()
{
new FileBrowserConnector();
}
[Test]
public void AdminCanCreateFileConnector()
{
MockRepository mocks = new MockRepository();
IPrincipal principal;
using (mocks.Record())
...
Testing code written for the web is challenging. Especially code that makes use of the ASP.NET intrinsic objects such as the HttpRequest object. My goal is to make testing such code easier. A while ago, I wrote some code to simulate the HttpContext in order to make writing such unit tests easier. My goal wasn’t to replace web testing frameworks such as Selenium, Watin, or AspUnit. Instead, I’m a fan of the Pareto principle and I hoped to help people easily reach the 80 of the 80/20 rule before reaching out to one of these tools to cover...
Most of the time when I’m testing my code, I only test it using the en-US culture since, ...well..., I speak English and I live in the U.S. Isn’t the U.S. the only country that matters anyway? ;) Fortunately, there are Subtext team members living in other countries ready to smack such nonsensical thoughts from my head and keep me honest about Localization and Internationalization issues. Simone, who is an Italian living in New Zealand, pointed out that a particular unit test that works on my machine always fails on his machine. Here’s the test. [RowTest]
[Row("4/12/2006", "04/12/2006 00:00:00 AM")]
[Row("20070123T120102", "01/23/2007 12:01:02...
If you’ve worked with unit test frameworks like NUnit or MbUnit for a while, you are probably all too familiar with the set of assertion methods that come built into these frameworks. For example: Assert.AreEqual(expected, actual);
Assert.Between(actual, left, right);
Assert.Greater(value1, value2);
Assert.IsAssignableFrom(expectedType, actualType);
// and so on...
While the list of methods on the Assert class is impressive, it leaves much to be desired. For example, I needed to assert that a string value was a member of an array. Here’s the test I wrote.
[Test]
public void CanFindRole()
{
string[] roles = Roles.GetRolesForUser("pikachu");
bool found = false;
foreach (string role in roles)
...
Although I am a big fan of Rhino Mocks, I typically favor State-Based over Interaction-Based unit testing, though I am not totally against Interaction Based testing.
I often use Rhino Mocks to dynamically create Dummy objects and Fake objects rather than true Mocks, based on this definition given by Martin Fowler.
Dummy objects are passed around but never actually used. Usually they are just used to fill parameter lists.
Fake objects actually have working implementations, but usually take some shortcut which makes them not suitable for production (an in memory database...
Ayende just announced the release of Rhino Mocks 3.0. The downloads are located here. If you aren’t subscribed to Ayende’s blog, I highly recommend it. This guy never sleeps and churns out code like a tornado.
Ever since I discovered mocking frameworks in general, and especially Rhino Mocks, mocking has become an essential part of my unit testing toolkit.
A while ago I wrote a short intro demonstrating how to write unit tests for events defined by an interface. This small example shows the usefulness of something like Rhino Mocks.
If you’re wondering what the difference between a mocks, stubs, and fakes, be...
Leon Bambrick (aka SecretGeek) has started a series on Agile methodologies and Test Driven Development (TDD) in which he brings up his own various hidden objections to TDD in order to see if his prejudices can be overcome. One of the questions he asks is an age old argument against TDD. Who Tests the Tests? Leon sees potential for a stack overflow since, given that the tests are code, and that according to TDD, code should be tested, shouldn’t there be tests for the tests? The short answer is that the code tests the tests, and the tests test the...
Here we are already looking ahead to learn about the language features of C# 3.0 and I am still finding new ways to make my code better with good “old fashioned” C# 2.0.
Like many people, I tend to fall into certain habits of writing code. For example, today I was writing a unit test to test the source of a particular event. I wanted to make sure that the event is raised and that the event arguments were set properly. Here’s the test I started off with (some details changed for brevity) which reflects how I would do this in the old days.
private bool eventRaised...
Last night a unit test saved my life (with apologies). Ok, maybe not my life, but the act of writing some unit tests did save me the embarrasment of an obscure bug which was sure to hit when I least expected it. It is cases like this that made me into such a big fan of writing automated unit tests.
Not too long ago I wrote a C# Akismet API for Subtext. In writing the code, I followed design principles focused on making the API as testable as possible. For example, I applied Inversion of Control (IOC) by having the AkismetClient constructor take...
UPDATE: I’ve since supplemented this with another approach.
Jeremy Miller asks the question, “How do you organize your NUnit test code?”. My answer? I don’t, I organize my MbUnit test code. Bad jokes aside, I do understand that his question is more focused on the structure of unit testing code and not the structure of any particular unit testing framework. I pretty much follow the same structure that Jeremy does in that I have a test fixture per class (sometimes more than one per class for special cases). I experimented with having a test fixture per method, but gave up on that as it became a maintenance headache. Too...
I am working on some code using the Model View Presenter pattern for an article I am writing. I am using an event based approach based on the work that Fowler did. For the sake of this discussion, here is an example of a simplified view interface.
public interface IView
{
event EventHandler Load;
}
In the spirit of TDD I follow this up with the shell of my Presenter class
public class Presenter
{
public Presenter(IView view)
{
throw new NotImplementedException("Not implemented.");
}
}
And...
Although I agree in spirit with most of Joel’s discussion of methodologies and rock star programmers, I’m in a bit of disagreement over the quote from Tamir he posts.
For instance, in software development, we like to have people unit-test their code. However, a good, experienced developer is about 100 times less likely to write bugs that will be uncovered during unit tests than a beginner. It is therefore practically useless for the former to write these...
I disagree with this, but in part because I think this is based on a flawed assumption over the purpose of unit testing....
One of the holy grails for unit testing is to get 100% code coverage from your tests. However, you can’t sit back and smoke a cigar when you reach that point and assume your code is invulnerable. Code coverage just is not enough.
One obvious reason is that Code Coverage cannot help you find errors of omission. That is, even if you had 100% code coverage from your tests, if you forget to implement a feature (and a test for that feature), then you’re shit out of luck.
However, apart from errors of omission, there’s the case presented here. Imagine...
Jonathan de Halleux, aka Peli, never ceases to impress me with his innovations within MbUnit. In case you're not familiar with MbUnit, it's a unit testing framework similar to NUnit.
The difference is that while NUnit seems to have stagnated, Jonathan is constantly innovating new features, test fixtures, etc... for a complete unit testing solution. In fact, he's even made it so that you can run your NUnit tests within MbUnit without a recompile.
His latest feature is not necessarily a mind blower, but it's definitely will save me a lot of time writing the same type of code over...
Test First Development, the process of writing unit tests to test the code you are about to write, is one of my favorite software practices that has an impact on producing better written code. However, it's no a panacea. It is true that I use the debugger much less often because of TDD, but there are still occasions where it's important to manually step through code line by line.
Personally, I use NCover as my first line of defense. It highlights the lines of code that haven't been executed by my unit tests. A lot of...