comments edit

UPDATE: Whoops! I should have read the SLA for my free hosting provider more carefully. There’s a size limit to the files I may post on my ftp server. I will work to rectify this situation by finding a new FTP site. In the meantime, email me and I will send you the documents referenced below.

This is a work in progress. Pick up the Word doc here. Please make updates and changes with change tracking on and I’ll be sure to incorporate them into the master document.

Also, just for kicks, I generated MSDN style documentation of the RssBandit class library using NDoc. You can pick that up here

If the files are unavailable, let me know. Sometimes files get deleted if I go over my quota.

comments edit

My latest task is to produce reports useful for fraud detection. We’ve recently had incidents in which users from Vietnam opened up several accounts via stolen credit cards, deposited money via Paypal, then commit a charge back.

I won’t go into details about the specifics of these reports, but they simply identify common patterns in fraudulent activity and alert us to the potential mischief.

csharp code datetime comments edit

What is the proper way to add three hours to a DateTime for the PST timezone? Is it this?

var d = DateTime.Parse("Oct 26, 2003 12:00:00 AM");
var later = d.AddHours(3.0);
Console.WriteLine(later); // displays 10/26/2003 03:00:00 AM which is NOT correct!

A valiant attempt, but wrong. Instead try this:

var d = DateTime.Parse("Oct 26, 2003 12:00:00 AM");
var later = d.ToUniversalTime().AddHours(3.0).ToLocalTime();
Console.WriteLine(later); // displays 10/26/2003 02:00:00 AM which is correct!

Or better yet, use DateTimeOffset.

const string dateFormat = "MMM dd yyyy h:mm tt zzz";
var d = DateTimeOffset.ParseExact("Oct 26 2003 12:00 AM -07:00", dateFormat, CultureInfo.InvariantCulture);
var later = d.AddHours(3.0);

Why all the rigamarole of converting to universal time and back to local time? A little thing we like to call Daylight savings time. Without the conversion, adding three hours would have set the time to 3:00 AM which would be wrong.

Realizing this probably would have saved me from many hours of intense debugging sessions. I remember one particular case with a system of scripts and tools I set up to analyze the very large log files for a big client. It hummed along nicely until one fine spring day when it failed miserably. After an entire day of tracking down the source of the problem, I finally nailed it down to a script’s mishandling of Daylight Savings.

To find out more about proper DateTime handling, read the following article concerning best practices with manipulating date times.

comments edit

agentLately it seems that everybody is playing “six-degrees of separation” by sprouting a social networking service. Some of the most prominent are,,, and Google’s Orkut. The promise these services make is that by joining, you’ll be where all your friends are and in the process make new friends and business contacts, though in reality they tend to look like glorified dating services. Look around the blogosphere and you’ll find several opinions concerning the problems and weaknesses of these services. For example, Dare’s take on social networking services, Warren Ellis’s evaluation of Orkut, and Don Park’s suggestion for improving existing services. Robert Scoble, a popular blogger, points out in this blog entry, that none of these networks interoperate. A person ends up entering his data into each site over and over again.

A better model for social networking is the blogging community itself. Most blogs these days contain a list of other blogs called a blogroll. Typically a blogroll is a list of other blogs that the blogger finds interesting. Click on a link within a blogroll and you’re instantly transported to another blog with its own blogroll. Continue navigating blogrolls and you may cover the entire blogging community, or just end up with a headache and a strong desire never to hear about another person’s opinion of Janet Jackson’s Super Bowl boob.

These blogrolls create a de-facto network, though I hesitate to call it a “friendship” network as the social networking services do, but it is a network nonetheless. Many call the larger network of blogs, the “blogosphere”.

It wouldn’t be too difficult to create a software component that could index your blogging network and provide features similar to Friendster: “Haacked is in your network, would you like to kick him out?”, or “You have 13,237 bloggers in your network. You spend too much time online.” Perhaps this would be a web control hosted by a blog, or a new feature for an RSS aggregator.

Many of these blogs are hosted by some sort of blogging back-end software whether it is,, or my personal favorite Subtext. With some of these tools, there are more powerful ways to establish networks than simply following links in a blogroll. For example, after reading a blog entry about the definition of a track back, I realized that social networking occurs both ways. Not only are the owners of the blogs you link to (and those they link to and so on and so on) members of your network, but those that link to you (referrals) and those that comment on your site and create trackbacks to your site are also members.

Trackbacks are particularly interesting in that they are a form of machine communication between your blog and the blog of another author. “Hi, I’m Haacked’s blog. He recently wrote a post that referred to an entry in your blog and trashes you and your family. Here’s the URL. Have a nice day!” This communication provides more information than a simple link. Not only does the recipient of this trackback know that I read her blog, but she also knows that I was motivated enough to refer to her site in a post on my own blog.

In such an exchange, your blog is acting as a software agent on your behalf. Think about that for a second. In Subtext for example, the simple act of posting a blog entry with a link to another person’s blog grants your blog permission to contact her blog (on your behalf) and communicate your acknowledgment of her blog entry.

Now imagine if the next generation of these blogging tools contained new features focused on this idea of a blog as social networking tool and personal agent. Your blog could have many capabilities to act as an agent on your behalf. It’s nice to know that your blog is looking out for you.

For example, you might give your blog several pieces of personal information and create some rules about who is allowed to gain access this information in sum or in part. Taking the concept of the blogroll one step further and creating various new “rolls”, a rule might be, “Anyone that is in my “close-friend-roll” may have access to all of this information. Anyone in my “colleague-roll” may have access to my business information. And anyone who trackbacks to me may have my email address.” Now, the next time I link to your blog entry, your blog might volunteer some information about your blog to my blog, at which point my blog may refuse or accept this information (depending on my preferences).

Taken further, blogs may move past being a simple journal of your everyday thoughts, but become your representative on the web. Tired of filling out your billing and shipping information every time you shop online? Add the selected merchant sites to your “secured-merchant-roll” and the next time you purchase something from, just give them your URL and let them get your billing and shipping information.

This rich communication between blogs would occur through XML standards built into the next generation of blogging software. The current crop of blogging tools already supports a variety of XML standards such as the MetaWeblog API, TrackBack API, Comment API, and Blogger API. Perhaps these new communication abilities would be built on top of these existing APIs or require new APIs altogether.

The beauty of this approach is the fact that these APIs are open and not proprietary. Not to beat a TLA to death, but this architecture is an example of Microsoft’s favorite TLA these days, SOA or Service Oriented Architecture. It doesn’t matter which platform your blog is implemented on. As long as it supports these various API’s and can make an http request, it can participate in this rich communication.

personal comments edit

SnowboardingYesterday Akumi, Laura, Judy, and I drove to Mountain High (not a reference to smoking out) for a day of snowboarding. This would be my second time snowboarding since the big Conversion from skiing. Right now, I would rate myself as an intermediate snowboarder and advanced intermediate skiier.

Mountain High is only a two hour drive away from Los Angeles, which makes it ideal for a day trip. The only problem is that it’s only two hours away, meaning anybody and everybody shows up. To start the day, we took a lift up the beginner hill to get warmed up, and the hill looked like the aftermath of a battle scene from Lord Of The Rings. I’m talking Pellenor fields here, with bodies strewn all over the place. The beginners hill is in reality an obstacle course that would test even Hillary Lindh (Alaskan reference).

In any case, we had a great day of boarding. Akumi and I even tried a few jumps, though she fared much better than I did. When approaching the jumps, I need to learn to stay on my front edge. Rather than a “Need For Speed”, I have an intense “Fear Of Speed” when on a snowboard approaching a ramp. I’ll start my approach, gain some speed, and as I get close, start riding on my back edge where I feel very stable on the approach, but very unstable on the landing.

I welcome any snowboarding tips in my comments section.

comments edit

I just finished reading the Log4Net Introduction and FAQ and realize I’ve been missing out all these years. I won’t retrofit my current App (just yet), but I definitely need to give Log4Net a try with future projects.

In particular, I like the hierarchical logging that it enables, both with log levels and with the equivalent of logging namespaces. For exampl, you can create a log named “Security” and another log with “Security.Access”. If you set a log level on Security, the Security.Access will inherit that log level (unless you also specify a log level for it). Very cool.

comments edit

Logs Scott Hanselman asks the question Should one go to Production with a Debug build? Although I find that an intriguing question, I’m more intrigued by the fact that his company exclusively uses Log4Net exclusively.

In my current project, I am using the built in Tracing framework that is part of the System.Diagnostics namespace. It supports multiple logging levels and you can add custom Trace listeners. So my question is, what does a framework like Log4Net offer that the Tracing framework doesn’t?

csharp dotnet comments edit

Ok, I admit it. I’m a unit-tester-aholic. I’m compulsive about it. Sometimes going overboard:

string s = "hello world";
Assert.AreEqual(s, "hello world", "s changed. How about that?");

MbUnit is my friend. I feel a great sense of accomplishment when I can write a fully automated end to end unit test of an email Newsletter mailer. Previously, testing such an app required me to run the code, open up Outlook, check my email, and “uh huh, looks good.&8221; Now, by using a custom SMTP server class running on a separate thread (and on a custom port), my unit test can send a batch of emails and then ask the server if everything looks as its supposed to. “Aye aye. Everything checks out capt’n.”

However, there are certain cases that threaten to cure my obsessive compulsive behavior. For example, unit testing the pathological cases. Now ideally, these cases should happen infrequently enough that perhaps you can let them go. Sweep them under the carpet. Nobody has to know. Besides, who runs out of memory these days?

raising hand Well I did today. And I didn’t have a unit test for it. I could probably be forgiven for this one. I’m writing a Windows service that reads data into 10 DataSets. As each DataSet is read, I asynchronously open an SMTP connection and begin performing a mail-merge on that data set. I made sure to cap the number of concurrent connections to 10.

Now this app passed all my unit tests, but when I deployed it to the production server, it ran out of memory. After some analysis with a memory profiler, I discovered that I had violated Performance Rule #6, “90% of performance problems are designed in, not coded in.”

Each chunk of data I read in from the database is wrapped in a RecipientBundle class. The service then gives this class to an SmtpPool class which then schedules it to be sent on an available thread. The SmtpPool class keeps an ArrayList of the scheduled bundles to avoid sending duplicate bundles.

Hopefully you see the problem with this approach. I had a complete brain shutdown when I wrote the code to add the bundle to the ArrayList. Since the ArrayList is only used to avoid duplicate bundles, all that is necessary is to store the ID of the bundle. By adding the bundle, the ArrayList is holding a reference to the bundle, thus making sure it never gets garbage collected until the entire mailing is complete. Bad move. These bundles should be generation 0 objects. Created, sent, garbage collected.

So in any case, I’m thinking about whether or not I should write a unit case for dealing with OutOfMemoryExceptions. There are a lot of difficulties in doing so. Typically, for a case like this, I will try to write a unit test that fakes it. I will talk about how I do that later.

I already have a unit test for dealing with the ThreadAbortException. That’s also a story for another time as there are several difficulties with deaing with the ThreadAbortException. For example, it doesn’t seem to get raised in my methods that are running asynchronously. They just seem to die without a whimper. Also, there’s a slight issue with the CLR such that the finally block isn’t guaranteed to execute.

Till next time

comments edit

So I was off to a running start with my ambitious new goals of contributing to several open source projects and creating a blog worth reading. But as my wife so keenly pointed out, as soon as I get an inch better, I try to take it a yard.

You see, I’ve been going to Occupational Therapy and will soon get some Physical Therapy for my back and wrist problems related to RSI (Repetitive Stress Injury). Carpal Tunnel is a form of RSI, but it is not what I am afflicted with. I have ulnar neuritis with a dash of tendinitis and a sprinkle of cubital tunnel syndrom (carpal’s lesser known cousin from the Bronx) thrown in.

After reading Dare Obasanjo’s and Scott Watermasysk’s blogs, I was inspired to contribute to their respective open source projects. I started out with a mad flurry of documentation writing for RssBandit without taking frequent enough breaks. I know, bad move. My back and wrists have flared up and I’m in a bit of pain.

To add to my misery, I’ve caught a bad cold. Yippee Kiyay! Anyways. I’m going to turn things down a notch, but will try to write something worth reading every so often.

comments edit

I thought this was a joke until I realized that it was on Yahoo news. Bush and Cheney say we attacked Iraq because Saddam COULD HAVE made weapons of mass destruction. “Could have”. Gee, well North Korea “could have” made WMDs. Should we attack them? How about India who DO have. Heck, with a bit of time, I think I have enough intelligence and skill to make a WMD. Should I expect an airstrike on my apartment sometime soon? I hope not, my wife would not be pleased.

This turnaround is particularly amusing because didn’t Bush and his cronies attempt to educate the country about the concept of “Imminent Threat”. Saddam posed an “Imminent Threat”. I don’t know about you, but “Could Have” does not seem so immiment to me. It’s like locking me up because I could have made a baseball bat and beat you down. Certainly I could, but it will take me time to find the right piece of oak. - President Bush and Vice President Cheney yesterday said the war in Iraq was justified because Saddam Hussein could have made weapons of mass destruction.

[Via Yahoo! News - Top Stories]

comments edit

So I keep blabbing and blabbing about my new RSS Feed. I keep imploring you to get an RSS aggregator and subscribe to my feed. What the hell am I talking about?!

Well to clear things up, I wrote an article about how to get started with RssBandit, a very nice RSS Aggregator created by Dare Obasanjo et all. Please take the time to read it and follow its instructions. Especially you Dad. If I’ve done well, the article will clear thing up and you will have a subscription to my blog.

You can also download it as a word doc

comments edit

My latest tech toy is an HP Scanjet 5530 Photosmart Scanner (wow that’s a mouthful). The defining feature of this scanner is the photo feeder built in. Simply add a pile of photos (around 24 max at a time) into the feeder, press a button, and go watch your Tivo’d episode of 24 while your photos scan (it doesn’t take all that long).

The incessant creaking of my bookshelf as it strains under the load of my accumulated photos spurred me to purchase this scanner. Having shuttled around the planet while growing up, it’s important to me to archive the memories of the havoc I created.

To that end, I also spent a lot of time searching for the perfect photo organizational software. The ideal program would have a clean nice interface and allow me to tag photos, organize photos into albums, search with face and image recognition, and automatically fix bad hair-day pics. I ended up purchasing Adobe Photoshop Album 2.0, which I’m quite happy with, though it doesn’t have image recognition (nobody really does just yet) and won’t fix my bad hair (a lost cause).

However, this new memo from Microsoft Research describes several advances that Microsoft is working on that will make their way into future products. Examples include image recognition, improved ability to fix red-eye and cut people from one photo and paste into another. One neat feature that currently works in Asian versions of Office 2003 can convert a picture of a person into a cartoon character with pretty good resemblance. Currently it only works for asian faces. I have to admit, I’m very excited.

Read about it here to see the examples.