Tip Jar: Concatenating A Delimited String

Update: I also wrote a more generic version using anonymous delegates for .NET 2.0 as a followup to this post.

Here’s one for the tip jar. Every now and then I find myself concatening a bunch of values together to create a delimited string.  In fact, I find myself in that very position on a current project. In my case, I am looping through a collection of objects concatenating together three separate strings, each for a different property of the object (long story).

Usually when building such a string, I will append the delimiter to the end of the string I am building during each loop.  But after the looping is complete, I have to remember to peel off that last delimiter.  Let’s look at some code, simplified for the sake of this discussion.

The first thing we’ll define is a fake class for demonstration purposes. It only has one property.

internal class Fake
{
    public Fake(string propValue)
    {
        this.SomeProp = propValue;
    }

    public string SomeProp;
    
    public static Fake[] GetFakes()
    {
        return new Fake[] {new Fake("one")
                , new Fake("two")
                , new Fake("three")
            };
    }
}

Now let’s look at one way to create a pipe delimited string from this array of Fake instances.

Fake[] fakes = Fake.GetFakes();

string delimited = string.Empty;
foreach(Fake fake in fakes)
{
    delimited += fake.SomeProp + "|";
}

delimited = delimited.Substring(0, delimited.Length - 1);
Console.WriteLine(delimited);

I never liked this approach because it is error prone. Do you see the problem? Yep, I forgot to make sure that delimited wasn’t empty when I called substring. I should correct it like so.

if(delimited.Length > 0)
    delimited = delimited.Substring(0, delimited.Length - 1);

When I write code like this, I almost always add a little disclaimer in the comments because I know someone down the line is going to call me an idiot for not using the StringBuilder class to concatenate the string. However, if I know that the size of the strings to concatenate and the number of concatenations will be small, there is no point to using the StringBuilder.  String Concatenations will win out. It all depends on the usage pattern.

But for the sake of completeness, let’s look at the StringBuilder version.

Fake[] fakes = Fake.GetFakes();

StringBuilder builder = new StringBuilder();
foreach(Fake fake in fakes)
{
    builder.Append(fake.SomeProp);
    builder.Append("|");
}

string delimited = builder.ToString();
if(delimited.Length > 0)
    delimited = delimited.Substring(0, delimited.Length - 1);
Console.WriteLine(delimited);

Aesthetically speaking, this code is even uglier because it requires more code. And as I pointed out, depending on the usage pattern, it might not provide a performance benefit. Today, a better approach from a stylistic point of view came to mind. I don’t know why I didn’t think of it earlier.

Fake[] fakes = Fake.GetFakes();

string[] delimited = new string[fakes.Length];
for(int i = 0; i < fakes.Length; i++)
{
    delimited[i] = fakes[i].SomeProp;
}

string delimitedText = String.Join("|", delimited);
Console.WriteLine(delimitedText);

Since I know in advance how many items I am concatenating together (namely fakes.Length number of items), I can fully allocate a string array in advance, populate it with the property values, and then call the static String.Join method.

From a perf perspective, this is probably somewhere between string concatenation and StringBuilder, depending on the usage pattern. But for the most part, String.Join is quite fast, especially in .NET 2.0 (though my current project is on .NET 1.1.  Boohoo!).

Performance issues aside, this approach just feels cleaner to me.  It gets rid of that extra check to remove the trailing delimiter.  String.Join handles that for me.  To me, this is easier to understand.  What do you think?

What others have said

Requesting Gravatar... ShoeLace Nov 21, 2006 9:23 PM
# re: Tip Jar: Concatenating A Delimited String
I like it..

I use a very similar construct in Perl.
but i start with a delimited string..
use split to create the array.
manipulate teh arraneg as needed
the rejoin the sting using join.

i like it.. its neat and consice.
Requesting Gravatar... Steve Harman Nov 21, 2006 9:24 PM
# re: Tip Jar: Concatenating A Delimited String
I feel your pain!

I've actually used Join in both C# and Javascript for some code I wrote for manipulating the CSS Styles of a DOM element. It's nice to deal with the styles as an array, but when added back to the element, they need to be a single string with each Style separated by a space (" ").

Anyhow, good Tip!
Requesting Gravatar... Rob Conery Nov 21, 2006 10:02 PM
# re: Tip Jar: Concatenating A Delimited String
Another append method...

Fake[] fakes = Fake.GetFakes();
string delimited=string.empty;
string delimiter="|";

for(int i = 0; i < fakes.Length; i++)
{
i<fakes.Length ? delimiter="|",delimiter="";
delimited+=fakes[i]+delimiter;

}


... but, since you're constructing the array in the first place, can't you just build a "GetPipedList()"?
Requesting Gravatar... Haacked Nov 21, 2006 10:13 PM
# re: Tip Jar: Concatenating A Delimited String
The problem with your code is I'm concatenating a specific property of Fake instances together.

Also, I think you meant this:

delimited += (i < (fakes.Length - 1)) ? "|" : "");

I'm not a fan of that because of the check to see if you're at index length - 1. Just rubs me the wrong way.
Requesting Gravatar... Haacked Nov 21, 2006 10:18 PM
# re: Tip Jar: Concatenating A Delimited String
Doh! I meant:

delimited += fakes[i].SomeProp + (i < (fakes.Length - 1)) ? "|" : "");

Never code in a browser!
Requesting Gravatar... Rob Conery Nov 21, 2006 10:43 PM
# re: Tip Jar: Concatenating A Delimited String
Doesn't SubText come with a "post-time" compiler? Jeez!

I think I need another beer. I tell ya Vista is kickin my #$$ right now and here I am trying to write code. You can slap me with a big WTF :).
Requesting Gravatar... Dave Frank Nov 21, 2006 10:44 PM
# re: Tip Jar: Concatenating A Delimited String
I've been doing += and &= to concat strings since the dawn of time. Pretty coincidental that someone changed all my patch code to use stringbuilder and then the same day you blog about it. Eh? EH?? Lol, its all good. I guess subconciously I've always felt that creating a new instance of something to join text together must naturally be slower than just manipulating a string variable with built in functions, but I know stringbuilder has been designed with performance in mind, especially for large strings. Anyway you do bring up a good point that just because we're used to doing things one way doesn't mean that a different way isn't better (or at least equally valid), and it does help, especially when working with open source projects, to be familiar with different ways of doing things (or have an open mind to learn) :p.
Requesting Gravatar... Jon Galloway Nov 21, 2006 11:10 PM
# re: Tip Jar: Concatenating A Delimited String
When I'm not able to use the String.Join() trick, I've been using a minor variation of what you've got with TrimEnd():

foreach(Fake fake in fakes)
delimited += fake.SomeProp + "|";

if (!string.IsNullOrEmpty(delimited))
delimited.TrimEnd('|');
Requesting Gravatar... Michal Chaniewski Nov 21, 2006 11:23 PM
# re: Tip Jar: Concatenating A Delimited String
I use this technique almost every time I have to create a delimited string.

Also, if I don't know the size of the array so I can't initialize it beforehand, I use a List<string> collection to add items to, then:

string.Join("|", myList.ToArray());

It's a small performance hit, but in most cases it can be ignored, and I always prefer code readability and maintainability over premature optimization.
Requesting Gravatar... Simon Nov 22, 2006 3:15 AM
# re: Tip Jar: Concatenating A Delimited String
I tend to add the first item to the final string, and then add the others:


string result = items[0];
for (int loop = 1; loop < items.Count; items ++)
{
result += "|" + items[loop];
}


Obviously with some validation. I think I prefer your Join() though, thank you.
Requesting Gravatar... mgroves Nov 22, 2006 6:57 AM
# re: Tip Jar: Concatenating A Delimited String
I was just going to suggest the same thing that Simon does: 'prime the pump', I think I used to call it. Of course, having a TrimEnd and/or Join methods is always nice, but if I'm using a language that doesn't have them, I'd much rather prime the pump than try to shave off the trailing delimeter.
Requesting Gravatar... Gabe Nov 22, 2006 9:10 AM
# re: Tip Jar: Concatenating A Delimited String
I suppose it depends on the implementation of join, but it's pretty much always going to be faster than StringBuilder and concatentation (at least for large arrays).

Concatenation is O(n^2) because it has to copy the whole string it's created thus far every time something is added to it. StringBuilder is O(n*lg(n)) because it has to copy the whole string every time its size doubles. However, String.Join should be O(n) because it can walk the array to determine the size of string it needs to create, allocate it once, and copy each string into its correct location exactly once.
Requesting Gravatar... Sergio Pereira Nov 22, 2006 10:48 AM
# re: Tip Jar: Concatenating A Delimited String
Inspired by Ruby and prototype.js I wrote a utility class for my projects and I can end up with something like this:
string list = string.Join("|", Util.Pluck(fakes, "SomeProp") );

Util.Pluck returns a System.Array where each element is of the same type as the property told in the 2nd parameter.

I use it all the time. Yes it uses reflection but it does a good job of avoiding repeated metadata queries for property types.

I wish there was something simialr in .net 2.0... maybe there is and I've never been introduced to.
Requesting Gravatar... Brandon Nov 22, 2006 11:01 AM
# re: Tip Jar: Concatenating A Delimited String
Stupid question, and I apologize but why not check what the final string size is. Something like below, btw sorry if my syntax are off extremely use to writing VB code.

public Fake(string propValue)
{
if this.somProp = ""
{
this.someProp = propValue;
}
else
{
this.SomeProp = "|" & propValue;
}
}
Requesting Gravatar... Haacked Nov 22, 2006 11:03 AM
# re: Tip Jar: Concatenating A Delimited String
Because the Fake class is probably an existing class. It doesn't know that sometime in the future I'll want to concatenate it with the "|" character. It would be a mistake to put that in the logic of the Fake class.

What if I later want to concatenate it using the "," character. Or concatenate another property of that class?
Requesting Gravatar... Brandon Nov 22, 2006 11:10 AM
# re: Tip Jar: Concatenating A Delimited String
In that case then why not make a second parameter that sets what you want to send in as your delimiter, you could even make it an optional parameter that way if you want to use the default you can, or you can use a different delimeter. The code that would be needed isn't a huge change that way.
Requesting Gravatar... Haacked Nov 22, 2006 11:15 AM
# re: Tip Jar: Concatenating A Delimited String
Ok, forget "Fake". What if I gave you an array of Systew.Web.HttpRequest, and told you to get me a pipe delimited list of the "ContentLength" property. Remember, ContentLength is an integer. Not to mention, you can't change the constructor of that class.
Requesting Gravatar... CougerAC Nov 22, 2006 12:16 PM
# re: Tip Jar: Concatenating A Delimited String
Why not just import the Microsoft.VisualBasic library and use the function Join defined in the Strings Class? The Microsoft.VisualBasic library is installed on the target anyways, why not use it? (Oh yes, this exists in .Net 1.0,1.1,2.0 for VB.)
Requesting Gravatar... Tom Nov 22, 2006 12:26 PM
# re: Tip Jar: Concatenating A Delimited String
I don't like the approach, using join. I prefer the following which I feel is simpler and will perform better (uses less memory), especially with more items:

Fake[] fakes = Fake.GetFakes();
StringBuilder builder = new StringBuilder();
foreach(Fake fake in fakes)
{
if (builder.length > 0)
builder.Append('|'); // char, not string!
builder.Append(fake.SomeProp);
}

This doesn't require allocation of the string[] and also eliminates having to remember the fixup code at the end to remove the extra delimiter. It is also easy to look at code like this and be pretty confident that it is correct.

I would only use the join method if I already had a string[] around for another reason - I don't see any performance advantage possible in allocating this intermediate representation just for the purpose of passing it into join.
Requesting Gravatar... Haacked Nov 22, 2006 12:41 PM
# re: Tip Jar: Concatenating A Delimited String
Stylistically, I didn't like that check for the length on every pass. It's like you're checking it a bunch of times for that one time it's needed. I'd prefer priming the pump.

Also, you mentioned performance, have you tested it?
Requesting Gravatar... Tom Nov 22, 2006 1:17 PM
# re: Tip Jar: Concatenating A Delimited String
Yes, I tested the performance. They both run in about the same amount of time, up to a certain number of items where the join approach starts to take significantly longer to execute. Memory usage is about 2x on the join because of the intermediate string[] (just based on inspection of the code, not from the performance testing).
Requesting Gravatar... Tom Nov 22, 2006 1:34 PM
# re: Tip Jar: Concatenating A Delimited String
Also, priming the pump is fine, but it's not so good when your using an iterator like in the code that I've shown.
Requesting Gravatar... Haacked Nov 22, 2006 1:45 PM
# re: Tip Jar: Concatenating A Delimited String
Good point on the iterator and pump priming.
Requesting Gravatar... Kevin Dente Nov 22, 2006 2:16 PM
# re: Tip Jar: Concatenating A Delimited String
Come, come, we're all good test drivers here, right? Clearly if this kind of iteration/concatenation is happening more than once, it needs to be factored out into it's own class. :P

On a recent project where I did a bunch of similar concatenation, I created a DelimitedStringGenerator class which walked an IEnumerable<T> and concatenated the strings together. That way, you only have one implementation of your looping logic, and you can tweak the implementation all you want, as well as reuse it.

Thanks for the reminder about Join. The BCL is so massive its easy to forget about handy little things like that.

Requesting Gravatar... Marc C. Brooks Nov 22, 2006 5:09 PM
# re: Tip Jar: Concatenating A Delimited String
Ahem... StringBuilder's .Length parameters is NOT readonly... You can set the value or adjust as desired. So, the correct way (when Join doesn't really apply) is:
Fake[] fakes = Fake.GetFakes();

StringBuilder builder = new StringBuilder();
foreach(Fake fake in fakes)
{
builder.Append(fake.SomeProp);
builder.Append("|");
}

if (builder.Length > 0) builder.Length--;
string delimited = builder.ToString();
Console.WriteLine(delimited);


Requesting Gravatar... Truong Hong Thi Nov 22, 2006 9:27 PM
# re: Tip Jar: Concatenating A Delimited String
In your StringBuilder version, you could use StringBuilder.ToString(startIndex, length) instead of String.SubString.

I also had to write similar stuff recently. If I had an array of strings, String.Join then is the natural choice. However, I also had to concatenate Fake.SomeProp, and my method is similar to this:

string delimited = String.Empty;
Fake[] fakes = Fake.GetFakes();
if (fakes.Length > 0)
{
StringBuilder builder = new StringBuilder(fakes[0]);
for (int i = 1, n = fakes.Length; i < n; i++)
{
builder.Append("|");
builder.Append(fakes[i].SomeProp);
}

delimited = builder.ToString();
}
Requesting Gravatar... Wiennat Nov 23, 2006 12:50 AM
# re: Tip Jar: Concatenating A Delimited String
You can use string.TrimRight(char delimiter) method to tirm the delimiter.
Requesting Gravatar... Tom Nov 23, 2006 8:53 PM
# re: Tip Jar: Concatenating A Delimited String
Truong Hong, why do you do this?

for (int i = 1, n = fakes.Length; i < n; i++)

This is no different than the more simple version:

for (int i = 1; i < fakes.Length; i++)

These two statements cause the compiler to generate the same code in the loop, so your optimized version is not any faster.
Requesting Gravatar... Truong Hong Thi Nov 24, 2006 3:01 AM
# re: Tip Jar: Concatenating A Delimited String
Hi Tom, it is just my habit.
However, I think the compiler would generate the same code only if some optimization switch is turned on. Because, C# does not complain if you change fakes inside the loop, and, as the result, fakes.Length might vary.
I don't believe the compiler could examine the code and determine whether fakes.Length could change during the loop. If it does not know for sure fakes.Length won't change, it will have to invoke fakes.Length again and again.
Requesting Gravatar... Tom Nov 24, 2006 2:18 PM
# re: Tip Jar: Concatenating A Delimited String
If Length was a complicated property accessor or function, that would be true. But Length actually resolves to just a single read of a memory location. So it takes just as long to access fakes.Length in the loop as it does to access your 'n' variable.
Requesting Gravatar... you've been HAACKED Nov 24, 2006 8:36 PM
# Concatenating Delimited Strings With Generic Delegates
Concatenating Delimited Strings With Generic Delegates
Requesting Gravatar... Joe Cheng Nov 26, 2006 6:34 PM
# re: Tip Jar: Concatenating A Delimited String
I do this when I'm feeling lazy and performance is not critical (which is the vast majority of the time).

Fake[] fakes = Fake.GetFakes();

string delimited = string.Empty;
string delimiter = string.Empty;
foreach(Fake fake in fakes)
{
delimited += delimiter + fake.SomeProp;
delimiter = "|";
}

Console.WriteLine(delimited);
Requesting Gravatar... Victor Robinson Nov 27, 2006 7:04 AM
# re: Tip Jar: Concatenating A Delimited String
I recommend you check out this article from MSDN Magazine:

http://msdn.microsoft.com/msdnmag/issues/06/01/CLRInsideOut/

If I know the join will be 3 items or less (the article actually suggests 4, but 3 "feels" better in code), I use +. Otherwise when the number of items is indeterminate (in a loop), I use sb.Append().

Also, if there are more than 2 items in a single join I like use of string.Format() or sb.AppendFormat().

That said, I really like the idea of using array joins, but would like to see a performance comparison of all three.
Requesting Gravatar... Christopher Steen Nov 28, 2006 8:24 PM
# Link Listing - November 28, 2006
Common Gotcha: Don't forget to when adding providers [Via: ScottGu ] Mounting ISO Images with Vista...
Requesting Gravatar... Evan Dec 03, 2006 2:12 PM
# re: Tip Jar: Concatenating A Delimited String
Nice tip! I can't believe I've overlooked the use of Join for concatenating strings. It's simple and effective. That's a win/win in my book, and it saves me from writing the concatenation loop.
Requesting Gravatar... ct Dec 05, 2007 9:26 AM
# re: Tip Jar: Concatenating A Delimited String
Do you have an article that issue the problem of delimiter appearing in the tokens? I think that is a much more difficult problem
Requesting Gravatar... Haacked Dec 06, 2007 11:36 AM
# re: Tip Jar: Concatenating A Delimited String
@ct Sorry, I don't. There's a great library for doing this, but I can't remember the name.

What do you have to say?

(will show your gravatar)
Please add 2 and 1 and type the answer here: