Comparing Strings in Unit Tests

tdd, code 0 comments suggest edit

Suppose you have a test that needs to compare strings. Most test frameworks do a fine job with their default equality assertion. But once in a while, you get a case like this:

[Fact]
public void SomeTest()
{
    Assert.Equal("Hard \tto\ncompare\r\n", "Hard  to\r\ncompare\n");
}

Let’s pretend the first value in the above test is the expected value and the second value is the value you obtained by calling some method.

Clearly, this test fails. So you look at the output and this is what you see:

test-output

It’s pretty hard to compare those strings by looking at them. Especially if they are two huge strings.

This is why I typically write an extension method against string used to better output a string comparison. Here’s an example of a test using my helper.

[Fact]
public void Fact()
{
    "Hard  to\rcompare\n".ShouldEqualWithDiff("Hard \tto\ncompare\r\n");
}

And here’s an example of the output.

test-compare-output

At the very top, the assert message is the same as before. I deferred to the existing Assert.Equal method in xUnit (typically Assert.AreEqual in other test frameworks) to output the error message.

Underneath the existing message are headings for three columns: the character index, the expected character, and the actual character. For each character I print out the int value and the actual character.

Of course in some cases, I don’t print out the actual value. If I were to do that for new line characters and tab characters, it’d screw up the formatting. So instead, I special case those characters and print out the escape sequence in C# for those characters.

This makes it easy to compare two strings and see every difference when a test fails. Even the hidden ones.

This is a simple quick and dirty implementation available in a Gist. For example, it doesn’t do any real DIFF comparisons and try to line up similarities. That’d be a nice improvement to make at some point. If you can improve this, feel free to fork the gist and send me a pull request.

Found a typo or error? Suggest an edit! If accepted, your contribution is listed automatically here.

Comments

avatar

10 responses

  1. Avatar for Jon Galloway
    Jon Galloway January 14th, 2012

    If you're doing diff comparisons, I'd recommend checking out Approval Tests: http://approvaltests.sourceforge.net/

  2. Avatar for Royston Shufflebotham
    Royston Shufflebotham January 14th, 2012

    Often string differences like this occur a long way into a string (e.g. 250 chars into a 300 char string), so many string diff displays (e.g. NUnit's) only show the bit that actually differs (with a bit of context) rather than the whole string.
    I usually find that I just need to know the character code of the first differing character (usually CR instead of LF or vice versa!) to find the problem.

  3. Avatar for haacked
    haacked January 14th, 2012

    @Royston great point! I wanted to show a significant enough chunk of the string because in certain cases (such as when you're applying a normalization to the string), you want to see the pattern of differences to provide more information about what wen't wrong.
    One improvement I could make to my method is to put a max # of characters I'll show and show the segment near the first difference like NUnit does.
    Going even further, the differences could be spaced apart. So maybe show up to X differences and Y characters around each difference. That'd be a nice improvement. :)

  4. Avatar for Nik Molnar
    Nik Molnar January 15th, 2012

    Great idea Phil!
    This reminds me of the Power Assert library, take a look if you haven't seen it...

  5. Avatar for Siderite
    Siderite January 15th, 2012

    I've had a similar problem with comparing HTML (only there I had to ignore whitespace) and I've created a small class that receives a string in the constructor and has overridden Equals and ToString methods as well as an implicit cast to string.
    All you have to do is change Assert.Equal(s1,s2) with Assert.Equal(new Stringer(s1),new Stringer(s2)), changing the code to a minimum level and achieving your desired result with an encapsulated class.
    In your example you could transform tabs and carriage returns back into slash notation when outputing.

  6. Avatar for Jim Cooper
    Jim Cooper January 16th, 2012

    Or if you were to use a more sophisticated unit test framework (NUnit) :-) you'd have this test:

    [Test]
    public void CheckStringAsserts()
    {
    Assert.That("Hard \tto\ncompare\r\n", Is.EqualTo("Hard to\r\ncompare\n"));
    }

    and this output:

    String lengths are both 18. Strings differ at index 5.
    Expected: "Hard to\r\ncompare\n"
    But was: "Hard \tto\ncompare\r\n"
    ----------------^

    Both the test and the output are much nicer

  7. Avatar for Andrei Volkov
    Andrei Volkov January 16th, 2012

    Re: extension methods for assertion, check out Shouldy at http://shouldly.github.com/

  8. Avatar for Llewellyn Falco
    Llewellyn Falco April 13th, 2012

    As Jon said, approvaltests deals with this often & fairly well. Mainly by harnessing other tools (like a diff tool).
    You can see an example of a fairly complex string comparison here: http://youtu.be/vKLUycNLhgc

  9. Avatar for Jim Liddell
    Jim Liddell July 4th, 2012

    Nice one - this helped me track down some unescaped special characters in a failing unit test.
    Thanks!

  10. Avatar for Bartlomiej Filipek
    Bartlomiej Filipek February 12th, 2014

    great and simple to use post. Strings are a bit more complicated to compare that two ints.