rx, software comments edit

What would you do if you could stop time for everyone but yourself?

When I was a kid, I watched a TV movie called The Girl, The Gold Watch, and Everything that explored this question. The main character, Kirby, inherits a very special gold watch from his Uncle that can stop time, but not for the bearer of the watch who is free to move around and troll people. Here's a clip from the movie where Kirby and his friend have a bit of fun with it.

The Girl, The Gold Watch, and Everything

This motif has been repeated in more recent movies as well. I often daydream about the shenanigans I could get into with such a device. If you had such a device, I'm sure you would do what I would do: use the device to write deterministic tests of asynchronous code of course!

Writing tests of asynchronous code can be very tricky. You often have to resort to calling Thread.Sleep or Task.Delay within an asynchronous callback so you can control the timing and assert what you need to assert.

For the most part, these are ugly hacks. What you really want is a way to control execution timing with fine grained control. You need a device like Kirby's golden watch.

Here's the good news. When you use Reactive Extensions (Rx), you have such a device at your disposal! Try not to get into too much trouble with it.

In the past, I've written how Rx can reduce the cognitive load of asynchronous code through a declarative model. Rather than attempt to orchestrate all the interactions that must happen asynchronously at the right time, you simply describe the operations that need to happen and Reactive Extensions orchestrates everything for you.

This nearly eliminates race conditions and deadlocks while also reducing the cognitive load and potential for mistakes when writing asynchronous code.

Those are all amazing benefits of this approach, yet those aren’t even my favorite thing about Reactive Extensions. My favorite thing is how the abstraction allows me to bend time itself to my will when writing unit tests. FEEL THE POWER!

Everything in Rx is scheduled using schedulers. Schedulers are classes that implement the IScheduler interface. This simple, but powerful, interface contains a Now property as well as three Schedule methods for scheduling actions to be run.

Control Time with the The TestScheduler

Rx provides the TestScheduler class (available in the Rx-Testing NuGet package) to give you absolute control over scheduling. This makes it possible to write deterministic repeatable unit tests.

Unfortunately, it's a bit of a pain to use as-is which is why Paul Betts took it upon himself to write some useful TestScheduler extension methods available in the reactiveui-testing NuGet package. This library provides the OnNextAt method. We'll use this to create an observable that provides values at specified times.

The following test demonstrates how we can use the TestScheduler.

[Fact]
public void SchedulerDemo()
{
    var sched = new TestScheduler();
    var subject = sched.CreateColdObservable(
        sched.OnNextAt(100, "m"), // Provides "m" at 100 ms
        sched.OnNextAt(200, "o"), // Provides "o" at 200 ms
        sched.OnNextAt(300, "r"), // Provides "r" at 300 ms
        sched.OnNextAt(400, "k")  // Provides "k" at 400 ms
    );

    string seenValue = null;
    subject.Subscribe(value => seenValue = value);

    sched.AdvanceByMs(100);
    Assert.Equal("m", seenValue);

    sched.AdvanceByMs(100);
    Assert.Equal("o", seenValue);

    sched.AdvanceByMs(100);
    Assert.Equal("r", seenValue);

    sched.AdvanceByMs(100);
    Assert.Equal("k", seenValue);
}

We start off by creating an instance of a TestScheduler. We then create an observable (subject) that provides four values at specific times. We subscribe to the observable and set the seenValue variable to whatever values the observable supplies us.

After we subscribe to the observable, we start to advance the scheduler's clock using the OnNextAt method. At this point, we are in control of time as far as the scheduler is concerned. Feel the power! The test scheduler is your gold watch.

Note that these are timings on a virtual clock. When you run this test, the code executes pretty much instantaneously. When you see AdvanceByMs(100), the scheduler's clock advances by that amount, but your computer's real clock does not have to wait 100 ms. You could call AdvanceByMs(99999999) and that statement would execute instantaneously.

Real World Example

Ok, that's neat. But let's see something that's a bit more real world. Suppose you want to kick off a search (as in an autocomplete scenario) when someone types in values into a text box. You probably don't want to kick off a search for every typed in value. Instead, you want to throttle it a bit. We'll write a method to do that that takes advantage of the Throttle method. From the MSDN documentation, the Throttle method:

Ignores the values from an observable sequence which are followed by another value before due time with the specified source, dueTime and scheduler.

Throttle is the type of method you might use with a text field that does incremental search while you’re typing. If you type a set of characters quickly one after the other, you don’t want a separate HTTP request for each character to be made. You’d rather wait till there’s a slight pause before searching because the old results are going to be discarded anyways. Here's a super simple Throttle example that throttles values coming from some subject. No matter how quickly the subject produces values, the Subscribe callback will only see values every 10 milliseconds.

  subject.Throttle(TimeSpan.FromMilliseconds(10))
      .Subscribe(value => seenValue = value);
public static IObservable<string> ThrottleTextBox(TextBox textBox, IScheduler scheduler)
{
    return Observable.FromEventPattern<TextChangedEventHandler, TextChangedEventArgs>(
        h => textBox.TextChanged += h,
        h => textBox.TextChanged -= h)
        .Throttle(TimeSpan.FromMilliseconds(400), scheduler)
        .Select(e => ((TextBox)e.Source).Text);
}

What we do here is use the Observable.FromEventPattern method create an observable from the TextChanged event. If you're not used to it, the FromEventPattern method is kind of gnarly.

Once again, Paul Betts has your back with the very useful ReactiveUI-Events package on NuGet. This package adds an Events extension method to most Windows controls that provides observable event properties. Here's the code rewritten using that. It's much easier to understand.

public static IObservable<string> ThrottleTextBox(TextBox textBox, IScheduler scheduler)
{
    return textBox
        .Events()
        .TextChanged // IObservable<TextChangedEventArgs>
        .Throttle(TimeSpan.FromMilliseconds(400), scheduler)
        .Select(e => ((TextBox)e.Source).Text);
}

What we're doing here is creating a method that signals us when the text of the TextBox changes, but only after there's been no change for 400 milliseconds. It will then give us the full text of the text box.

Here's a unit test to make sure we wrote this correctly.

[Fact]
public void TextBoxThrottlesCorrectly()
{
    var textBox = new TextBox();

    new TestScheduler().With(sched =>
    {
        string observed = null;
        ThrottleTextBox(textBox, sched)
            .Subscribe(value => observed = value);

        textBox.Text = "m";
        Assert.Null(observed);

        sched.AdvanceByMs(100);
        textBox.Text = "mo";
        Assert.Null(observed);

        textBox.Text = "mor";
        sched.AdvanceBy(399);  // Just about to kick off the throttle
        Assert.Null(observed);

        textBox.Text = "mork"; // But we changed it just in time.
        Assert.Null(observed);

        sched.AdvanceByMs(400); // Wait the throttle amount
        Assert.Equal("mork", observed);
    });
}

In this test, we're using the With extension method provided by reactiveui-testing package. This method takes in a lambda expression that provides us with a scheduler to pass into our Throttle method.

Within that lambda, I am once again in complete control of time. As you can see, I start advancing the clock here and there and changing the TextBox's Text values. As you'd expect, as long as I don't advance the clock more than 400 ms in between text changes, the ThrottleTextBox observable won't give us any values.

But at the end, I go ahead and advance the clock by 400 ms after a text change and we finally get a value from the observable.

Conclusion

The throttling of a TextBox (for autocomplete and search scenarios) is probably an overused and abused example for Rx, but there's a good reason for that. It's easy to grok and explain. But don't let that stop you from seeing the full power and potential of this technique.

It should be clear how this ability to control time makes it possible to write tests that can verify even the most complex asynchronous interactions in a deterministic manner (cue "mind blown").

Unfortunately, and this next point is important, the TestScheduler doesn't extend into real life, so your shenanigans are limited to your asynchronous Reactive code. Thus, if you call Thread.Sleep(1000) in your test, that thread will really be blocked for a second. But as far as the test scheduler is concerned, no time has passed.

The good news is, with the TestScheduler, you generally don't need to call Thread.Sleep in your tests. There are many methods in Reactive Extensions for converting asynchronous calling patterns into Observables.

So the TestScheduler might not be as much fun as Kirby's golden watch, it should make writing and testing asynchronous code a whole lot more fun than it was in the past.

personal blogging comments edit

Today Jeff Atwood commemorates 10 years of CodingHorror.com. Congratulations Jeff!

But as I read that a thought occurred to me. Haven't I been blogging as long as Jeff, albeit much less successfully? I mean, just look at the intro to his post and compare it to the intro to this post. His is way better. WAY BETTER!

And sure enough, I started this very blog you are reading (thank you for your patronage!) one day after CodingHorror posted his first blog post. I started on February 3, 2004 with this exciting post about, well, the blog itself.

Jeff and I didn't know each other back then but we became friends through blogging and at one point I tried to hire him to a company I co-founded but he wisely said no and went on to great things. This is why I love blogging. The community and serendipitous interactions that result have been a big part of my growth in the last decade.

Yes, there's nothing more boring than blogging about blogging (other than blogging about blogging about blogging). As I'm doing right now.

But this blog doesn't contain the full extent of my blogging history. I had a blog long before this one. I was reminded about it when Zach Holman wrote a great post entitled Only 90s Web Developers Remember This. I'm not yet 90 years old, but I do remember the practices he highlights. In fact, my previous blog perpetrated some of them.

I used to keep a blog at haack.org which no longer exists except in the great internet graveyard in the sky, The Internet Archive Wayback Machine.

Here's the homepage of my very first blog.

Please use IE 4.0 and above folks!

Digging this up and seeing that IE 4.0 disclaimer made me laugh out loud in light of what Zach writes about IE.

It's come to my attention that people today don't like Internet Explorer. I can only believe they hate Internet Explorer because it has devolved from its purest form, Internet Explorer 4.0.

Internet Explorer 4.0 was perfection incarnate in a browser. It had Active Desktop. It had Channels. It had motherfucking Channels, the coolest technology that never reached market adoption ever not even a little bit. IE4, in general, was so good that you were going to have it installed on your PC whether you liked it or not.

See, I knew perfection even back then.

And here's a shot of my very first blog post way back in November 10, 2000. Over 13 years ago! The header image wasn't broken back then.

-574

I'm still proud of that deep review of Charlie's Angels.

If all goes well, I hope to be blogging for the next 10 years. And hopefully there will be something in here that's interesting or useful to some of you in that time. Thanks for reading!

jekyll comments edit

Git does a pretty amazing job when it merges one branch into another. Most of the time, it merges without conflict. In a fairy tale world with rainbow skittles and peanut butter butterflies, every merge would be without conflict. But we live in the real world where it rains a lot and where merge conflicts are an inevitable fact of life.

Is this what an Octopus merge looks like? - Dhaka traffic by Ranveig Thatta license CC BY 2.0

Git can only do so much to resolve conflicts. If two developers change the same line of code in different ways, someone has to figure out what the end result should be. That can't be done automatically.

The impact of merge conflicts can be mitigated by doing work in small iterations and merging often. But even so, the occasional long running branch and gnarly merge conflict are unavoidable.

Often, we treat the work to resolve a merge conflict as trivial. Worse, merges often are not reviewed very carefully (I'll explain why later). A major merge conflict may contain a significant amount of work to resolve it. And any time there is significant work, others should probably review that code in a pull request (PR for short). After all, a bad merge conflict resolution could introduce or reintroduce subtle bugs that were presumed to be fixed already.

As a great example, take a look at the recent Apple Goto Fail bug. I'm not suggesting this was the result of a bad merge conflict resolution, but you could easily see how a bad merge conflict might produce such a bug and bypass careful code review.

if ((err = SSLHashSHA1.update(&hashCtx, &serverRandom)) != 0)
    goto fail;
if ((err = SSLHashSHA1.update(&hashCtx, @signedParams)) != 0)
    goto fail;
    goto fail;
if ((err = SSLHashSHA1.final(&hashCtx, &hashOut)) != 0)
    goto fail;

When my team is in this situation, we actually push the merge commit into its own branch and send a pull request.

For example, suppose I want to merge the master branch into a branch named long-running-branch. I'll create a new branch named something like merge-master-into-long-running-branch. I'll then perform the merge in that branch (when I am resolving a gnarly merge my outbursts can be rightly described as a performance). When I'm done and everything is working, I'll push that new branch to GitHub and create a pull request from it for others to review.

In git that looks like:

git checkout long-running-branch
git checkout -b merge-master-into-long-running-branch
git merge master
# Manually do a lot of work to resolve the conflicts and commit those changes
git push origin merge-master-into-long-running-branch

The first command just makes sure I'm in the long-running-branch. The second command uses the -b to create a new branch named merge-master-into-long-running-branch based off the current one. I then merge master into this branch. And finally I push it to GitHub.

That way, someone can do a quick review to make sure the merge doesn't break anything and merge it in.

However, this runs into some problems as articulated by my quotable co-worker Paul Betts. In a recent merge commit PR that I sent, he made the following comment just before he merged my PR.

I have no idea how to review a merge commit

The problem he alludes to is that when you merge one branch into another, the diff of that merge commit will show every change since the last merge. For the most part, that's all code that's already been reviewed and doesn't need to be reviewed again.

What you really want to look at is whether there were conflicts and what shenanigans did the person have to do to resolve those conflicts.

Well my hero Russell Belfer (no blog, but he's @arrbee on Twitter) to the rescue! He works on LibGit2 so as you'd expect, he knows a thing or two about how Git works.

Recall that when you merge one branch into another, a new merge commit is created that points to both branches. In fact, a merge commit may have two or more parents as it's possible to merge multiple branches into one at the same time. But in most cases a merge commit has exactly two parents.

Let's look at an example of such a merge commit from the SignalR project. This commit merges their release branch into their dev branch. The SHA for this commit is cc5b002a5140e2d60184de42554a8737981c846c which is pretty easy to remember but to be fair to those with drug addled brains, I'll use cc5b002a as a shorthand to reference this commit.

You can use git diff to look at either side of the merge. For example:

git diff cc5b002a^1 cc5b002a
git diff cc5b002a^2 cc5b002a

Recall that the ^ caret is used to denote which parent of a commit we want to look at. So ^1 is the first parent, ^2 is the second parent, and so on.

So how do we see only the lines that changed as part of the conflict resolution?

git diff-tree --cc cc5b002a

UPDATE: I just now learned from @jspahrsummers that git show cc5b002a works just as well and in my shell gives you the color coded diff. The merge commit generally won't contain any content except for the conflict resolution.

git show cc5b002a

As I'll show later, the --cc option is useful for finding interesting commits like this.

You can see the output of the gist show command in this gist. Notice how much less there is there compared to the full diff of the merge commit.

The git diff-tree command is a lower level command and if I had to guess, git show builds on top of it.

If we look at the git diff-tree documentation, we can see that the --cc flag is the one that's interesting to us.

--cc This flag changes the way a merge commit patch is displayed, in a similar way to the -c option. It implies the -c and -p options and further compresses the patch output by omitting uninteresting hunks whose the contents in the parents have only two variants and the merge result picks one of them without modification. When all hunks are uninteresting, the commit itself and the commit log message is not shown, just like in any other "empty diff" case.

Since the --cc option describes itself in terms of the -c option, let's look at that too.

-c This flag changes the way a merge commit is displayed (which means it is useful only when the command is given one , or --stdin). It shows the differences from each of the parents to the merge result simultaneously instead of showing pairwise diff between a parent and the result one at a time (which is what the -m option does). Furthermore, it lists only files which were modified from all parents.

The -p option mentioned generates a patch output rather than a normal diff output.

If you're not well versed in Git (and perhaps even if you are) that's a mouthful to read and a bit hard to fully understand what it's saying. But the outcome of the flag is simple. This option displays ONLY what is different in this commit from all of the parents of this commit. If there were no conflicts, this would be empty. But if there were conflicts, it shows us what changed in order to resolve the conflicts.

As I mentioned earlier, the work to resolve a merge conflict could itself introduce bugs. This technique provides a handy tool to help focus a code review on those changes and reduce the risk of bugs. Now go review some code!

If you're wondering how I found this example commit, I ran git log --min-parents=2 -p --cc and looked for a commit with a diff.

That filters the git log with commits that have at least two parents.

personal comments edit

Everyone is the protagonist of their own narrative. And in this narrative, it's only natural to see ourselves as the proverbial "good guy" of the story. We tend to rationalize our own actions as necessary or positive, much like Walter White until (spoiler alert) the end of Breaking Bad.

Here's an interesting thought experiment to try once in a while. Like Walter, turn this narrative on its head and imagine that you are the villain of your own narrative instead. What lessons would that provide?

heisenberg: hero or villain? By Alejandro Garcia CC BY-ND 3.0

Yesterday, as I drove my family to work and school, a driver cut me off as he drifted across two lanes to get into the left lane. This forced me to hit the brakes to avoid him. I pulled up next to his car and glared over, but he did not look back.

The running narrative in my head assimilated this into the story. This young punk was oblivious to what's going around him. He's a bad driver and a bad person.

But I had this sudden epiphany. What if I were the villain in this story? It did seem this guy had just pulled into the right lane from another street and he needed to get to the left lane so he could get on the freeway. Was I perhaps going a bit over the speed limit? (The answer to that is usually yes.)

In this alternative narrative, I was the aggressor. I sped along and did not allow this innocent person make it over to the turn lane. It made me realize that I've been in his shoes before. And many times I label the driver who approaches too fast as the bad guy when I try to change lanes. I couldn't have been in the right in every situation, could I?

This thought experiment mirrors the ideas I wrote about in my post Argue Well By Losing. The difference is the techniques I wrote about in that post apply when you're in the midst of a debate. This technique is for more contemplative moments.

When you change your narrative, you challenge your long held beliefs, biases, and prejudices. You realize that someone who holds a deep belief that's antithetical to your own, is a "good person" in their own mind. And in their narrative, you, who attempts to refute their belief, is "the bad guy."

This can lead to mutual understanding. Perhaps you still want to change their mind, but the tactics you take might change. Rather than go on the attack as your only option, you might look for ways to mitigate their concerns. You might help them see that those who oppose their beliefs are not the "bad guys." A great example of this approach is in this post The Distress of the Privileged.

Privileged distress. I’m not bringing this up just to discuss old movies. As the culture evolves, people who benefitted from the old ways invariably see themselves as victims of change. The world used to fit them like a glove, but it no longer does. Increasingly, they find themselves in unfamiliar situations that feel unfair or even unsafe. Their concerns used to take center stage, but now they must compete with the formerly invisible concerns of others.

If you are one of the newly-visible others, this all sounds whiny compared to the problems you face every day. It’s tempting to blast through such privileged resistance with anger and insult.

Tempting, but also, I think, a mistake. The privileged are still privileged enough to foment a counter-revolution, if their frustrated sense of entitlement hardens.

Note that this doesn't mean you excuse the marginalization of others by the privileged. It means you gain an understanding of what's going on in their heads. How they rationalize it with the narrative that they are a good person.

Unfortunately, this doesn't make for "good politics." Loud extremists create an environment where politicians feel pressure to form rigid views with no wiggle room. But the rest of us don't have to wallow in that bullshit. We can realize we're all the good guys of our own narratives just trying to live decent lives, even if we disagree on how to do that.

math comments edit

I'm kind of a fan of numbers. You might even say I'm a bit of a numberPHILe. You groan, but it's true. Numbers exhibit such interesting properties when you put them together.

Recently an "astounding result" from the Numberphile folks (see, I'm not the only one) on Youtube made its way around the internet. In this video, a couple of physicists "prove" that sum of all natural numbers results in -1/12 (or -0.08333... for you fraction haters). That is, if you take 1 + 2 + 3 + ... and keep adding the next number forever, the eventual sum is a negative fraction. Whatchoo talkin' 'bout, Willis?!

This result defies intuition. But as we'll see, most results when dealing with infinities defy intuition.

In this particular video, they seem to play a bit fast and loose with the math. But there is a kernel of truth to what they demonstrate. I thought it'd be fun (for non-zero values of fun) to explore this idea a bit and throw some code at it to learn a thing or two. But first, I should warn you I am not a good mathematician. I have a tiny bit of undergraduate background in math, but some of the concepts I'll explore are beyond my puny brain's capacity. Feel free to send me a pull request with corrections!

But first, a joke

There's another one later if you stick with it.

A mathematician will call an infinite series convergent if its terms go to zero. A physicist will call it convergent if the first term is finite.

This joke is funny to mathematicians because it pokes fun at the math ability of physicists. The joke is funny to me because it's wrong about the mathematician. Any decent mathematician knows that the terms going to zero is necessary, but not sufficient for the series to converge. For example, the terms of the harmonic series go to zero, but its sum is positive infinity.

harmonic series

For those who need a refresher, it might help if I define what a sequence is and what a series is.

Definitions

A sequence is an ordered list of elements. The elements can be anything that can be ordered such as a sequence of your family members ordered by how much you love them. For our purposes, we'll stick with numbers. A sequence can be finite or infinite. For infinite series, it's prudent to have some mathematical formula that tells you how to determine each member of the sequence rather than attempt to write them out by hand. I don't recommend this. Though it could come in handy if you ever need to disable an evil entity's control over your computer by keeping the computer busy.

A partial sum is the sum of a finite portion of the sequence starting at the beginning. In looking at the harmonic series, this is like plugging in a specific value for n. So in the case of n = 1, the partial sum is 1. For n = 2, it's 1.5. For n = 10 it's a lot of addition.

A series is itself a sequence where each value represents the partial sum from 0 to n. A finite series will have a last term that represents the finite value that the series converges to. What often is counter intuitive is that an infinite series can also converge to a finite number as n goes to infinity. There's no last term (it's infinite), but we can use mathematical tools to determine the limit (the value the series approaches) for the series.

Infinite series with finite sums

Let's look at an infinite series with a finite sum to prove that such a thing is possible. Zeno aside, intuition suggests that a series where each term approaches zero "fast enough" could result in a sum that is finite. The approach to zero just has to be "faster" than the growth via accumulation.

The following is known as the Basel Series where each member of the sequence is 1/n^2. In 1735, Leonhard Euler (pronounced "oiler", not "yuler"), at the tender young age of 28, famously gave a proof for the exact value of this series as n goes to infinity. If you have a math symbol fetish, you can go read Euler's proof here. There's enough Greek there to write a tragedy.

sum-of-reciprocal-squares

Since I lack the brilliance of Euler, I'll write some code to demonstrate this convergence because I have COMPUTERS! Eat it Euler!

The following method gets a partial sum of this series.

private static IEnumerable<double> GetEulerPartialSum(int n)
{
    double sum = 0.0;
    for (int i = 1; i <= n; i++)
    {
        sum += 1.0 / Math.Pow(i, 2);
        yield return sum;
    }
}

In a console program, we can iterate over this series.

static void Main(string[] args)
{
    foreach (var partialSum in GetEulerPartialSum(1000000))
    {
        Console.SetCursorPosition(0, Console.CursorTop);
        Console.Write(partialSum);
    }
    Console.ReadLine();
}

The reason for the SetCursorPosition call is to have each successive result overwrite the previous result. This way, you can watch the successive values change over time.

The first few decimal places converge in a near instant. It takes longer for the further out decimal places. As we get closer to 1 million iterations, we're adding tiny increments to the sum. If I could run this program with an infinite number of iterations, every decimal place would eventually converge to a value. When I say "every" I mean all infinity of them.

Graph it

With C#, it's not too hard to see a graph of this. Just add a reference to System.Windows.Forms.DataVisualization and add a Chart control to your application and add the following code:

First, we'll slightly modify our previous method to return data points.

private static IEnumerable<DataPoint> GetEulerPartialSum(int n)
{
    double sum = 0.0;
    for (int i = 1; i <= n; i++)
    {
        sum += 1.0 / Math.Pow(i, 2);
        yield return new DataPoint(i, (double)sum);
    }
}

Next, we'll add these data points to our chart.

chart.Series.Clear();
chart.Series.Add("euler");
chart.Series["euler"].ChartType = SeriesChartType.Line;
foreach (var point in GetEulerPartialSum(100))
{
    chart.Series["euler"].Points.Add(point);
}

Here's the result.

euler graph

Look at how pretty that is.

Whoops, time for another math and theoretical physics joke

Ok, it's been a while, let's get back to the topic at hand.

What is the result of 1 + 2 + 3 + 4 + ...?

The mathematician: I cannot respond if you do not say to me what it follows after the dots..

The physicist: it diverges!

Polchinski: = -1/12

Polchinksi was a string theorist.

So what's the problem with the proof in the video? Well as we've seen, in order for a summation of a series to converge, the terms need to go to 0. But the first series they cover, Grandi's Series, does not converge!

1 - 1 + 1 - 1 + 1 ...

Or more succinctly,

grandis-series

Here's some code for it and the graph that results.

private static IEnumerable<DataPoint> GetGrandiPartialSum(int n)
{
    double sum = 0.0;
    for (int i = 0; i < n; i++)
    {
        sum += Math.Pow(-1, i);
        yield return new DataPoint(i, sum);
    }
}

grandi's series

As you can see, that looks very different. There's no clear convergence to a single value. At every iteration, the partial sum is either 1 or 0!

So in the video, they mention that you could assign a value of 0.5 to this series. What they refer to is another way of assigning a sum to an infinite series. For example, a Cesàro summation is an approach where you take the average of the partial sums. This is slightly confusing so let me clarify. As we saw, the partial sum is either 0 or 1 at every iteration. If we sum up those partial sums at each iteration and divide it by the iteration, we get the average. That's the Cesàro partial sum.

n Sum Sum of Partial Sums Average = (Sum of Sums / n)
1 1 1 1
2 0 1 1/2 = 0.5
3 1 2 2/3 = 0.6666
4 0 2 2/4 = 0.5
5 1 3 3/5 = 0.6
6 0 3 3/6 = 0.5

As you can see, it appears to be converging to 0.5. So even though the Grandi series is divergent series and has an infinite sum, it is Cesàro summable and that sum is 0.5.

There's some code and a graph again to visualize it.

private static IEnumerable<DataPoint> GetGrandiCesaroPartialSum(int n)
{
    double sum = 0.0;
    double sumOfSums = 0.0;
    for (int i = 1; i <= n; i++)
    {
        sum += Math.Pow(-1, i);
        sumOfSums += sum;
        yield return new DataPoint(i, sumOfSums / i);
    }
}

grandi Cesàro summation

That's kind of neat.

But here's the thing, a Cesàro summation is not the same thing as a normal additive summation. You can think of its result as a different property of a series from the normal summation.

As an analogy, imagine that I show you two flat shapes and then provide a proof that the shapes cancel each other out by subtracting the perimeter of one from the area of the other. You'd probably call foul on that.

While not a perfect analogy, that's pretty close to what the video shows because they treat the Cesàro sum of the series as if it were a convergent series while they perform various operations that only apply to convergent series to get to their result.

In particular, the way they shift the series and start adding them triggered my Spidey sense. That might work for a convergent series, but not for divergent.

Here's a simple proof from Quora that shows how this leads to a contradiction. We'll let S represent the sum of the natural numbers. Now, let's subtract S from itself, but we'll shift things over by one like they did in the video.

 S =   1 + 2 + 3 + 4 + 5...
-S = -     1 + 2 + 3 + 4...
----------------------------
 0 =   1 + 1 + 1 + 1 + 1...

Let's subtract it from itself again shifted over by one.

 0 =   1 + 1 + 1 + 1 + 1...
-0 = -     1 + 1 + 1 + 1...
----------------------------
 0 =   1  

So here we've proven that 0 = 1. Mathematicians would call this a reductio ad absurdum which is latin for "that shit's cray!" In other words, the premise leads to an absurd result.

So how do you get -1/12?

While I disagree with the way they get to their result, these guys are not stupid. Also, they literally point to an equation that shows the sum of natural numbers is -1/12 in a book.

-508

IN A BOOK! Well that's all you need to know, right? Well if you look closely at the text, you'll see it mentions something about a "renormalization."

Riemann zeta function and analytic continuations.

As I mentioned earlier, much like a shape has different properties such as area, height, and weight, a summation can have different properties. Or more accurately, there are different approaches to summing a series.

As I was working on this blog post, I learned that the Numberphile folks, who created the original video, produced a follow-on video that shows an alternate proof that takes advantage of the Riemann Zeta function and analytic continuations.

riemann-zeta-function

It's worth a watch. They get into how this applies to string theory.

Analytic continuations are a technique to extend the domain of a function. Recall that a domain is the range of allowed inputs into a function. These are the inputs for which the function provides a valid output. Inputs outside of the domain might diverge to infinity, for example.

The Zeta function's domain is for values greater than 1. As you approach 1, the values of the Zeta function diverge to infinity. But we (well not "we", but mathematicians who have a clue what they're doing) can cry "YOLO!" and apply analytic continuation to extend the Zeta function below 1 to see what happens.

When you plug in -1 to the zeta function, you get -1/12. By definition of the zeta function, that also happens to equal 1 + 2 + 3 + 4 + .... So the zeta function is another way to assign a finite value to this divergent series.

To visualize this, look at this graph of the zeta function here, it diverges at x = 1. But if you continue to the left pass the divergence, you can see finite values again.

zeta function graph from http://planet.racket-lang.org/package-source/williams/science.plt/3/1/planet-docs/science/special-functions.html

In the original formulation of this function, Euler allowed for the exponents to be real numbers greater than 1. That crazy cat Riemann took it further and generalized it to complex exponents. If you recall, a complex number is in the form of a real part added to an imaginary part. Rather than worry about the implications of "imaginary" numbers, think of complex numbers as coordinates where the real part is the x-axis and the imaginary part is the y-axis. In that regard, it becomes easy to imagine a series that diverges in one dimension but converges in another.

Graphing the Zeta function in this way produces some beautiful 3-D graphs as seen on Wolfram Alpha and immortalized by XKCD.

XKCD on Riemann Zeta - Creative Commons BY-NC 2.5

Incomprehensible indeed! When you deal with infinite series, it's near impossible to have an intuitive handle on it. Our minds don't deal with infinity very well, but our math does.

I hope you enjoyed this exploration. I can't claim to know much about math, but I did find it fun to explore this idea with a bit of code and graphing. If you're interested in the code I used, check out my "infinite sums" GitHub Repository.

If you're looking for a practical use of this information, did you miss the part where I said it was math? LOL! But seriously, I thought of one practical use apart from describing 26 dimensional vibrating strings. The next time someone says you owe them money, just ask them if they'll accept an amount equal to the sum of all natural numbers. If they accept, send them a bill for $-0.083.

code comments edit

From the topic of this and my last post, you would be excused if you think I have some weird fascination with ducks. In fact, I'm starting to question it myself.

Is it a duck? - CC BY-ND 2.0 from http://www.flickr.com/photos/77043400@N00/224131630/

I just find the topic of duck typing (and structural typing) to be interesting and I'm not known to miss an opportunity to beat a dead horse (or duck for that matter).

Often, when we talk about duck typing, code examples only ask if the object quacks. But shouldn't it also ask if it quacks like a duck?

For example, here are some representative examples you might find if you Google the terms "duck typing". The first example is Python.

def func(arg):
    if hasattr(arg, 'quack'):
        arg.quack()
    elif hasattr(arg, 'woof'):
        arg.woof()

Here's an example in Ruby.

def func(arg)
  if arg.respond_to?(:quack)
    arg.quack
  end
end

Does this miss half the point of duck typing?

In other words, don't check whether it IS-a duck: check whether it QUACKS-like-a duck, WALKS-like-a duck, etc, etc,...

Note that this doesn't just suggest that we check whether the object quacks and stop there, as these examples do. It suggests we should go further and ask if it quacks like a duck.

Most discussions of duck typing tend to focus on whether the object has methods that match a given name. I haven't seen examples that also check the argument list. But yet another important aspect of a method is its return type. As far as I know, that's not something you can test in advance with Ruby or Python.

Suppose we have the following client code.

def func(arg)
  return unless arg.respond_to?(:quack)
  sound = arg.quack
  /quack quack/.match(sound)
end

And we have the following two classes.

class Duck
  def quack
    "quacketty quack quack"
  end
end

class Scientist
  def quack
    PseudoScience.new
  end
end

The Scientist certainly quacks, but it doesn't quack like a duck.

func(Duck.new) // returns a MatchData object
func(Scientist.new) // returns nil (match failed)

I think that's one reason why the example in my previous post actually tries and call the method to ensure that the argument indeed quacks like a duck.

My guess is that in practice, conflicts like this where a method has the same name, but different type, is rare enough that Ruby and Python developers don't worry about it too much.

Also, with such dynamic languages, it's possible to monkey patch an object to conform to the implicit contract if it doesn't match it exactly. Say if you have a RobotDuck you got from another library you didn't write and want to pass it in as a duck.

Thanks to GeekSam for reviewing my Ruby and Python idioms.

code comments edit

Eric Lippert writes one of my all time favorite tech blogs. Sadly, the purple font he was famous for is no longer, but the technical depth is still there.

In a recent post, he asks the question, "What is Duck Typing?" His post provides a thoughtful critique and deconstruction of the Wikipedia entry on the subject. Seriously, go read it, but please come back here afterwards!

For those of you too lazy to read it, I'll try and summarize crudely. He starts off with his definitions of "typing":

A compile-time type system provides three things: first, rules for defining types, like "array of pointers to integer". Second, rules for logically deducing what type is associated with every expression, variable, method, property, and so on, in a program. And third, rules for determining what programs are legal or illegal based on the type of each entity. These types are used by the compiler to determine the legality of the program.

A run-time type system provides three similar things: first, again, rules for defining types. Second, every object and storage location in the system is associated with a type. And third, rules for what sorts of objects can be stored in what sorts of storage locations. These types are used by the runtime to determine how the program executes, including perhaps halting it in the event of a violation of the rules.

He continues with a description of structural typing that sounds like what he always thought "duck typing" referred to, but notes that his idea differs from the Wikipedia definition. As far as he can tell, the Wikipedia definition sounds like it's just describing Late Binding.

But this is not even typing in the first place! We already have a name for this; this is late binding. "Binding" is the association of a particular method, property, variable, and so on, with a particular name2 in a particular context; if done by the compiler then it is "early binding", and if it is done at runtime then it is "late binding".3 Why would we even need to invent this misleadingly-named idea of "duck typing" in the first place??? If you mean "late binding" then just say "late binding"!

I agree that the Wikipedia definition is a bit unclear, but I think there's more to it than simple late binding. Also, I think some of the confusion lies in the fact that duck typing isn't so much a type system as it is a fuzzy approach to treating objects as if they are certain types (or close enough) based on their behavior rather than their declared type. This is a subtle distinction to late binding.

To back this up, I looked at the original Google Group post where the Alex Martelli first described this concept.

In other words, don't check whether it IS-a duck: check whether it QUACKS-like-a duck, WALKS-like-a duck, etc, etc, depending on exactly what subset of duck-like behaviour you need to play your language-games with.

This was a response to a question that asked the question (I'm paraphrasing), how do you handle method overloading with a single parameter in a dynamic language? Specifically, the question was in reference to the Python language.

To illustrate, in a static typed language like C#, you might have the following three methods of a class (forgive me if the example seems contrived. I lack imagination.):

public class PetOwner {
  public void TakeCareOf(Duck duck) {...}
  public void TakeCareOf(Robot robot) {...}
  public void TakeCareOf(Car car) {...}
}

In C#, the method that gets called is resolved at compile time depending on the type of the argument passed to it.

var petOwner = new PetOwner();
petOwner.TakeCareOf(new Duck()); // calls first method.
petOwner.TakeCareOf(new Robot()); // calls second method.
petOwner.TakeCareOf(new Car()); // calls third method.

But in a dynamic language, such as Python, you can't have three methods with the same name each with a single argument. Without a type declared for the method argument, there is no way to distinguish between the methods. Instead, you'd need a single method and do something else.

One approach is you could switch based on the runtime type of the argument passed in, but Alex points out that would be inappropriate in Python. I assume because it conflicts with Python's dynamic nature. Keep in mind that I'm not a Python programmer so I'm basing this on my best attempt to interpret Alex's words:

In other words, don't check whether it IS-a duck: check whether it QUACKS-like-a duck, WALKS-like-a duck, etc, etc, depending on exactly what subset of duck-like behaviour you need to play your language-games with.

As I said before, I don't know a lick of Python, so I'll pseuducode what this might look like.

class PetOwner:
    def take_care_of(arg):
        if behaves_like_duck(arg):
            #Pout lips and quack quack
        elif behaves_like_robot(arg):
            #Domo arigato Mr. Roboto
        elif behaves_like_car(arg):
            #Vroom vroom vroom farfegnugen

So rather than check if the arg IS A duck, you check if it behaves like a duck. The question is, how do you do that?

Alex notes this could be tricky.

On the other hand, this can be a considerable amount of work, depending on how you go about it (actually, it need not be that bad if you "just go ahead and try", of course catching the likely exceptions if the try does not succeed; but still, greater than 0).

One proposed approach is to simply treat it like a duck, and if it fails, start treating it like a fish. If that fails, try treating it like a dog.

I'd guess that code would look something like:

class PetOwner:
  def take_care_of(arg):
    try:
      arg.walk()
      arg.quack()
    except:
      try:
        arg.sense_danger_will_robinson()
        arg.dance_in_staccato_manner()
      except:
        arg.drive()
        arg.drift()

Note that this is not exactly the same as late binding as Eric proposes. Late binding is involved, but that's not the full picture. It's late binding combined with the branching based on the set of methods and properties that make up "duck typing."

What's interesting is that this was not the only possible solution that Alex proposed. In fact, he concludes it's not the optimal approach.

Besides, "explicit is better than implicit", goes one of Python's mantras. Just let the client-code explicitly TELL you which kind of argument they are passing you (and doing so through a named argument is simple and readable), and your work drops to zero, while removing no useful functionality whatever from the client.

He goes on to state that this implicit duck typing approach to method overloading seems to have dubious benefit.

The "royal-road" alternative route to overloading would, I think, be the use of suitable named-arguments. A rockier road, perhaps preferable in some cases, but more work for dubious benefit, would be the try/except approach to see if an argument supplies the functionalities you require.

The Python approach would be to pass in a discriminator. Even so, the object passed in would have to fulfill the set of requirements for the selected branch of code indicated by the discriminator. With the discriminator, it does feel more like we're just talking about late binding, but applied to a set of methods and properties, not just each one individually as you might do with late binding.

One observation I've heard is that "duck typing" sounds kind of like "duct taping." Not sure if there's anything to that, but if you forgive a bit of a stretch, I think there it may be an apt analogy.

On the Apollo 13 mission, the crew was faced with a situation where carbon dioxide levels were rising to dangerous levels in the Lunar Module. They had plenty of filters, but their square filters would not fit in the round barrels that housed the filters. In other words, their square filters were the wrong type (whether dynamic or static). Their solution was to use duct tape to cobble something together that would work. It wasn't the solution intended by the original design, but as long as the final contraption acts like an air filter (duck typing), they would survive. And they did. Like I said, the analogy is a bit of as stretch, but I think it embodies the duck typing approach.

Perhaps a better term is typing by usage. With explicit typing, you explicitly declare an object to be one type or another (whether at compile time or run time). With typing by usage, if it just happens to meet the needs of the consumer, then hey! It's a duck!

For static typed languages, I really like the idea of structural typing. It provides a nice combination of type safety and flexibility. Mark Rendle, in the comments to Eric's blog post provides this observation:

Structural Typing may be thought of as a kind of compiler-enforced subset of Duck Typing.

In other words, it's duck typing for static typed languages.

Also in the comments to Eric's post, someone linked to my blog post about duck typing. At the time I wrote that, "structural typing" wasn't in my vocabulary. If it had been, I could have been more precise in my post. For static languages, I find structural typing to be very compelling.

What do you think? Did I nail it? Or did I drop the ball and get something wrong or misrepresent an idea? Let me know in the comments.

UPDATE: Sam Livingston-Gray, also known as @geeksam notes another key difference between late binding that I completely missed:

@haacked method_missing illustrates the disconnect between binding and typing: an obj can choose how and whether to respond to a message

Recall that Eric defines "Late Binding" as:

"Binding" is the association of a particular method, property, variable, and so on, with a particular name in a particular context; if done by the compiler then it is "early binding", and if it is done at runtime then it is "late binding".

You could argue that method_missing is another form of late binding where the name is bound to method_missing because there is no other name to bind to. But conceptually, it feels very different to me. With binding, you usually think of the caller determining which method to call by name. And whether it's bound early or late is no matter, it's still the caller's choice. With method_missing it's the object in control of whethere it's going to respond to the method call (message).

oss comments edit

Today, I read a comment about a group of people who feel betrayed by the increase in code that Microsoft is releasing under an open source license.

I and my team are really troubled by MS's apparent policy of Open-sourcing code from under our feet.

We are in an industry with rather paranoid clients that contractually bar us from using Open Source software. We have immensely enjoyed using Rx but its open-sourcing threw us into turmoil.

I feel like we have been betrayed, and will think twice before adopting some new Microsoft framework again, wary of it being Open Sourced later on without prior warning.

And I understand this feeling. I really do. Feelings of betrayal are a natural consequence of progress.

By betrayal, I mean the following definition from Webster,

to hurt (someone who trusts you, such as a friend or relative) by not giving help or by doing something morally wrong

The Catholic Church at the time must have felt betrayed by Galileo when he lent his support to the heliocentric model of the universe because it deviated from their orthodoxy.

Factory owners who profited on cheap labor from children must have felt betrayed by the passage of child labor laws.

Racists who held onto the idea that other races were inferior to their own must have felt betrayed by the passage of the civil rights act.

These are grand examples. But often we experience much smaller and simpler betrayals as a result of little tiny foot steps of progress. Such as the betrayal some might feel when changing winds in the industry makes their antiquated business practices harder to sustain.

It's important to note, that betrayal does not imply progress. It can also result from regression. But doing the right thing always leads to some group feeling betrayed.

And what do we do about those who feel betrayed? It's easy to deride them as an anachronistic holdover from a past that no longer has a place in the present. That's the easy way to deal with it.

But for those who've only known the world to be one way all their life, feelings of betrayal are understandable when the world suddenly changes. Hopefully we can reach out and help those adapt to the new way. There's room for everyone. It's a painful path. But it can happen.

For the rest who would rather stew in the feelings of betrayal and hold onto their outdated ideas with an iron fist. These are the folks we should never, under any circumstances, let impede progress.

I'm sad to hear that these folks feel betrayed because their clients won't allow them to use Rx now that it is open source. Rx is a powerful and useful library. But when you put it in perspective, clients like these are going extinct. History has never been kind to businesses who cannot adapt to change. Best to try and educate the clients about how their policy is regressive and detrimental to their future health. And if that's not successful, then look for clients who are adaptable and wisely embrace open source as one of many means of ensuring their own survival in the long run.

jekyll comments edit

In my last 2013 recap blog post I wrote about the number of steps I recorded with Fitbit last year and the year prior. In case you missed it, they were:

  • 2012 - 3,115,606 steps (Note, I started recording in March)
  • 2013 - 4,577,481 steps

Someone asked me how I got those numbers because the Fitbit dashboard is confusing. Indeed it is. Here's how.

First, when you go to the dashboard, you have to mouse over the section to see the "more info" link.

Fitbit dashboard

Then, click on the "Year" tab. But you'll notice that you still don't see summary data. You have to click on the little back link.

Fitbit activity without totals

There's a brief pause, but the summary totals should show up on bottom.

Fitbit activity with totals

Unfortunately I can't get the same report for my sleep patterns without paying for the premium account. I might do that if I had tracked my sleep better last year.

Hopefully, if you have a Fitbit, this helps you.

blogging, personal comments edit

Another year comes to an end and tradition demands that I write a recap for the year. But it doesn't require that I write a very good one.

I wish I had the time and energy to write one of those recaps that captures the essence of the year in a thoughtful insightful manner. The best I can muster is "a lot of stuff happened."

Here, look at this picture of my tiny kids playing chess.

chess

Personal

This has been a great year for me. My son started first grade, and much to our relief, he loves it. At home, he started to learn to program. I even had my first conversation with him about refactoring and the DRY principle. Parents, it's never too early to talk to your kids about clean code!

My daughter just gets more and more interesting and fun to be around. She has a big personality and just wins over any room she's in. Sometimes we take walks together and she's now able to walk with me over a mile to the local frozen yogurt place. But she usually makes me carry her part of the way back.

And I finished my second year at GitHub. After a year and a half solely focused on GitHub for Windows, I've been able to bounce around a few other cool projects which keeps me excited every day. I still love working here.

I spoke at a few conferences, but I've certainly ramped that down as travel is tough on the family and I had a tiny bit more work travel this year.

Work

Contribution graphs are not a great way to determine the impact you've had in a year. They don't capture a lot of important work that happens outside of GitHub. Yes, it's true. Productive work does happen that's not captured by a Git commit.

Even so, I find them interesting to look at for some historical perspective. The gaps in a contribution graph tell as much a story as the areas that are filled in. For example, you can see when I go on vacation based on my graphs, though I'm not very good at staying away from the computer when I do.

Here's two of my contribution graphs. The first one is what I see as it shows contributions to both public and private repositories.

Haacked Contribution Graph

The second one shows what the public sees. This is perhaps a decent, though not perfect, representation of the work I've done with open source.

Haacked public Contribution Graph

As you can see, after shipping a major release of GitHub for Windows, I shifted my focus to some open source projects like choosealicense.com and octokit.net, making my public contribution graph much greener in the latter half of the year.

What I wrote, that people seemed to like

My three most popular posts written in 2013 according to Google Analytics are:

  1. Death to the if statement - more robust code with less control structures with 25,987 page views.
  2. Argue well by losing - You only learn something when you lose an argument with 21,264 views.
  3. Test Better - How developers should become better testers with 15,618 views

By the way, does anyone know how to easily do a report in Google Analytics for content created in a year? I'd find that useful.

What I've Shipped

This past year, I've had the pleasure to be involved in shipping the following:

  1. GitHub Enterprise support in GitHub for Windows
  2. Octokit.net
  3. ChooseALicense.com
  4. RestSharp - a few releases actually.
  5. According to FitBit, I had 4,577,481 steps this year. That's approximately 2,099miles. Compare this to the 3.1 million steps I took the year before. That's a huge improvement!

You People

Yeah, let's talk about you. You people are my favorite. Well, most of you.

  • Visitors 1,462,003 unique visitors made 2,091,606 visits. Those numbers are down 24.9% and 22.27% respectively from the previous year. I'd like to blame the death of blogging, but I suspect the quality of my writing has declined as I've focused more on other areas of my life.

  • RSS Subscribers According to FeedBurner, there are still 84,377 subscribers to my RSS feed which is surprising given the demise of Google Reader. I guess everybody found replacements. Or the stats are jacked.

Next Year

I'm looking forward to 2014. I've started learning F# by reading the Real-World Functional Programming book by Tomas Petricek and Jon Skeet. I'm hoping to incorporate more functional programming into my toolset. And I'm hoping to take even more steps.

Hopefully I can speak at a few conferences again this year. I'd love to speak in some new places. I'm really hoping to get a gig in South Korea this year. It'd be a chance to see how the industry is really growing there and to visit some of my family.

jekyll comments edit

Well this is a bit embarrassing.

I recently migrated my blog to Jekyll and subsequently wrote about my painstaking work to preserve my URLs.

But after the migration, despite all my efforts, I faced an onslaught of reports of broken URLs. So what happened?

Broken glass by Tiago Pádua CC-BY-2.0

Well it's silly. The program I wrote to migrate my posts to Jekyll had a subtle flaw. In order to verify that my URL would be correct, it made a web request to my old blog (which was still up at the time) using the generated file name.

This was how I verified that the Jekyll URL would be correct. The problem is that Subtext had this stupid feature where the date part of the URL didn't matter so much. It only cared about the slug at the end of the URL.

Thus requests for the following two URLs would receive the same content:

  • http://haacked.com/archive/0001/01/01/some-post.aspx
  • http://haacked.com/archive/2013/11/21/some-post.aspx

Picard Face Palm

This "feature" masked a timezone bug in my exporter that was causing many posts to generate the wrong date. Unfortunately, my export script had no idea these were bad URLs.

Fixing it!

So how'd I fix it? First, I updated my 404 page with information about the problem and where to report the missing file. You can set a 404 page by adding a 404.html file at the root of your Jekyll repository. GitHub pages will serve this file in the case of a 404 error.

I then panicked and started fixing errors by hand until my helpful colleagues Ben Balter and Joel Glovier reminded me to try Google Analytics and Google Webmaster Tools.

If you haven't set up Google Webmaster Tools for your website, you really should. There are some great tools in there including the ability to export a CSV file containing 404 errors.

So I did that and wrote a new program, Jekyll URL Fixer, to examine the 404s and look for the corresponding Jekyll post files. I then renamed the affected files and updated the YAML front matter with the correct date.

Hopefully this fixes most of my bad URLs. Of course, if anyone linked to the broken URL in the interim, they're kind of hosed in that regard.

I apologize for the inconvenience if you couldn't find the content you were looking for and am happy to refund anyone's subscription fees to Haacked.com (up to a maximum of $0.00 per person).

jekyll comments edit

In my last post, I wrote about preserving URLs when migrating to Jekyll. In this post, show how to preserve your Disqus comments.

This ended up being a little bit tricker. By default, disqus stores comments keyed by a URL. So if you people create Disqus comments at http://example.com/foo.aspx, you need to preserve that exact URL in order for those comments to keep showing up.

In my last post, I showed how to preserve such a URL, but it's not quite exact. With Jekyll, I can get a request to http://example.com/foo.aspx to redirect to http://example.com/foo.aspx/. Note that trailing slash. To Disqus, these are two different URLs and thus my comments for that page would not load anymore.

Fortunately, Disqus allows you to set a Disqus Identifier that it uses to look up a page's comment thread. For example, if you view source on a migrated post of mine, you'll see something like this:

<script type="text/javascript">
  var disqus_shortname = 'haacked';

  var disqus_identifier = '18902';
  var disqus_url = 'http://haacked.com/archive/2013/10/28/code-review-like-you-mean-it.aspx/';

  // ...omitted
</script>

The disqus_identifier can pretty much be any string. Subtext, my old blog engine, set this to the database generated ID of the blog post. So to keep my post comments, I just needed to preserve that as I migrated over to Jekyll.

So what I did was add my own field to my migrated Jekyll posts. You can see an example by clicking edit on one of the older posts. Here's the Yaml frontmatter for that post.

---
layout: post
title: "Code Review Like You Mean It"
date: 2013-10-28 -0800
comments: true
disqus_identifier: 18902
categories: [open source,github,code]
---

This adds a new disqus_identifier field that can be accessed in the Jekyll templates. Unfortunately, the default templates you'll find in the wild (such as the Octopress ones) won't know what to do with this. So I updated the disqus.html Jekyll template include that comes with most templates. You can see the full source in this gist.

But here's the gist of that gist:

var disqus_identifier = '{% if page.disqus_identifier %}{{ page.disqus_identifier}}{% else %}{{ site.url }}{{ page.url }}{% endif %}';
var disqus_url = '{{ site.url }}{{ page.url }}';

If your current blog engine doesn't explicitly set a disqus_identifier, the identifier is the exact URL where the comments are hosted. So you could set the disqus_identifier to that for your old posts and leave it empty for your new ones.

jekyll comments edit

In my last post I wrote about migrating my blog to Jekyll and GitHub Pages. Travis Illig, a long time Subtext user asked me the following question:

The only thing I haven't really figured out is how to nicely handle the redirect from old URLs (/archive/blah/something.aspx) to the new ones without extensions (/archive/blah/something/). I've seen some meta redirect stuff combined with JavaScript but... UGH.

UGH Indeed! I decided not to bother with changing my existing URLs to be extensionless. Instead, I focused on preserving my existing permalinks by structuring my posts such that they preserved their existing URLs.

How did I do this? My old URLs have an ASP.NET .aspx extension. Surely, GitHub Pages won't serve up ASPX files. This is true. But what it will serve up is a folder that just happens to have a name that ends with ".aspx".

The trick is in how I named the markdown files for my old posts. For example, check out a recent post: 2013-11-20-declare-dont-tell.aspx.markdown

Jekyll takes the part after the date and before the .markdown extension and uses that as the post's URL slug. In this case, the "slug" is declare-dont-tell.aspx.

The way it handles extensionless URLs is to create a folder with the slug name (in this case a folder named declare-dont-tell.aspx) and creates the blog post as a file named index.html in that folder. Simple.

Thus the URL for that blog post is http://haacked.com/archive/2013/11/20/declare-dont-tell.aspx/. But here's the beautiful part. GitHub Pages doesn't require that trailing slash. So if you make a request for http://haacked.com/archive/2013/11/20/declare-dont-tell.aspx, everything still works! GitHub simply redirects you to the version with the trailing slash.

Meanwhile, all my new posts from this point on will have a nice clean extensionless slug without breaking any permalinks for my old posts.

blogging, jekyll comments edit

The older I get, the less I want to worry about hosting my own website. Perhaps this is the real reason for the rise of cloud hosting. All of us old fogeys became too lazy to manage our own infrastructure.

For example, a while back my blog went down and as I frantically tried to fix it, I received this helpful piece of advice from Zach Holman.

@haacked the ops team gets paged when http://zachholman.com is down. You still have a lot to learn, buddy.

Indeed. Always be learning.

What Zach refers to is the fact that his blog is hosted as a GitHub Pages repository. So when his blog goes down (ostensibly because GitHub Pages is down), the amazing superheroes of the GitHub operations team jumps into action to save the day. These folks are amazing. Why not benefit from their expertise?

So I did.

One of the beautiful things about GitHub Pages is that it supports Jekyll, a simple blog aware static site generator.

If you can see this blog post, then the transition of my blog over to Jekyll is complete and (mostly) successful. The GitHub repository for this blog is located at https://github.com/haacked/haacked.com. Let me know if you find any issues. Or better yet, click that edit button and send me a pull request!

Screen grab from the 1931 movie Dr. Jekyll and Mr. Hide public domain

There are two main approaches you can take with Jekyll. In one approach, you can use something like Octopress to generate your site locally and then deploy the locally generated output to a gh-pages branch. Octopress has a nice set of themes (my new design is based off of the Greyshade theme) and plugins you can take advantage of with this approach. The downside of that approach is you can't publish a blog post solely through GitHub.com the website.

Another approach is to use raw Jekyll with GitHub pages and let GitHub Pages generate your site when your content changes. The downside of this approach is that for security reasons, you have a very limited set of Jekyll plugins at your disposal. Even so, there's quite a lot you can do. My blog is using this approach.

This allows me to create and edit blog posts directly from the web interface. For example, every blog post has an "edit" link. If you click on that, it'll fork my blog and take you to an edit page for that blog post. So if you're a kind soul, you could fix a typo and send me a pull request and I can update my blog simply by clicking the Merge button.

Local Jekyll

Even with this latter approach, I found it useful to have Jekyll running locally on my Windows machine in order to test things out. I just followed the helpful instructions on this GitHub Help page. If you are on Windows, you will inevitably run into some weird UTF Encoding issue. The solution is fortunately very easy.

Migrating from Subtext

Previously, I hosted my blog using Subtext, a database driven ASP.NET application. In migrating to Jekyll, I decided to go all out and convert all of my existing blog posts into Markdown. I wrote a hackish ugly console application, Subtext Jekyll Exporter, to grab all the blog post records from my existing blog database.

The app then shells out to Pandoc to convert the HTML for each post into Markdown. This isn't super fast, but it's a one time only operation.

If you have a blog stored in a database, you can probably modify the Subtext Jekyll Exporter to create the markdown post files for your Jekyll blog. I apologize for the ugliness of the code, but I have no plans to maintain it as it's done its job for me.

The Future of Subtext

It's with heavy heart that I admit publicly what everyone has known for a while. Subtext is done. None of the main contributors, myself included, have made a commit in a long while.

I don't say dead because the source code is available on GitHub under a permissive open source license. So anyone can take the code and continue to work on it if necessary. But the truth is, there are much better blog engines out there.

I started Subtext with high hopes eight years ago. Despite a valiant effort to tame the code, what I learned in that time was that I should have started from scratch.

I was heavily influenced by this blog post from Joel Spolksy, Things You Should Never Do.

Well, yes. They did. They did it by making the single worst strategic mistake that any software company can make:

They decided to rewrite the code from scratch.

Perhaps it is a strategic mistake for a software company, but I'm not so sure the same rules apply to an open source project done in your spare time.

So much time and effort was sacrificed at the altar of backwards compatibility as we moved mountains to make the migration from previous versions to next continue to work while trying to refactor as much as possible. All that time dealing with the past was time not spent on innovative new features. I was proud of the engineering we did to make migrations work as well as they did, but I'm sad I never got to implement some of the big ideas I had.

Despite the crap ton of hours I put into it, so much so that it strained my relationship at times, I don't regret the experience at all. Working on Subtext opened so many doors for me and sparked many lifelong friendships.

So long Subtext. I'll miss that little submarine.

code, rx comments edit

Judging by the reaction to my Death to the If statement where I talked about the benefits of declarative code and reducing control statements, not everyone is on board with this concept. That’s fine, I don’t lose sleep over people being wrong.

Photo by Grégoire Lannoy CC BY 2.0

My suspicion is that the reason people don’t have the “aha! moment” is because examples of “declarative” code are too simple. This is understandable because we’re trying to get a concept across, not write the War and Peace of code. A large example becomes unwieldy to describe.

A while back, I tried to tackle this with an example using Reactive Extensions. Imagine the code you would write to handle both the resize and relocation of a window, where you want to save the position to disk, but only after a certain interval has passed since the last of either event.

So you resize the window, then before the interval has passed you move the window. And only have you stop moving it and resizing it for this interval, does it save to disk.

Set aside your typical developer bravado and think about what that code looks like in a procedural or object oriented language. You functional reactive programmers can continue to smirk smugly.

The code is going to be a bit gnarly. You will have to write bookkeeping code such as saving the time of the last event so you can check that the duration has passed. This is because you’re telling the computer how to throttle.

With declarative code, you more or less declare what you want. “Hey! Give me a throttle please!” (Just because you are declaring, it doesn’t mean you can’t be polite. I like to add a Please suffix to all my methods). And declarations are much easier to compose together.

This example is one I wrote about in my post Make Async Your Buddy with Reactive Extensions. But I made a mistake in the post. Here’s the code I showed as the end result:

Observable.Merge(
    Observable.FromEventPattern
      <SizeChangedEventHandler, SizeChangedEventArgs>
        (h => SizeChanged += h, h => SizeChanged -= h)
        .Select(e => Unit.Default),
    Observable.FromEventPattern<EventHandler, EventArgs>
        (h => LocationChanged += h, h => LocationChanged -= h)
        .Select(e => Unit.Default)
).Throttle(TimeSpan.FromSeconds(5), RxApp.DeferredScheduler)
.Subscribe(_ => this.SavePlacement());

I’ll give you a second to recover from USBS (Ugly Syntax Blindness Syndrome).

The code isn’t incorrect, but there’s a lot of noise in here due to the boilerplate expressions used to convert an event into an observable sequence of events. I think this detracted from my point.

So today, I realized I should add a couple of really simple extension methods that describe what’s going on and hides the boilerplate.

// Returns an observable sequence of a framework element's
// SizeChanged events.
public static IObservable<EventPattern<SizeChangedEventArgs>> 
    ObserveResize(this FrameworkElement frameworkElement)
{
  return Observable.FromEventPattern
    <SizeChangedEventHandler, SizeChangedEventArgs>(
        h => frameworkElement.SizeChanged += h,
        h => frameworkElement.SizeChanged -= h)
      .Select(ep => ep.EventArgs);
}

// Returns an observable sequence of a window's 
// LocationChanged events.
public static IObservable<EventPattern<EventArgs>> 
    ObserveLocationChanged(this Window window)
{
  return Observable.FromEventPattern<EventHandler, EventArgs>(
      h => window.LocationChanged += h,
      h => window.LocationChanged -= h)
    .Select(ep => ep.EventArgs);
}

This then allows me to rewrite the original code like so:

this.ObserveResize()
  .Merge(this.ObserveLocationChanged())
  .Throttle(TimeSpan.FromSeconds(5), RxApp.MainThreadScheduler)
  .Subscribe(_ => SavePlacement());

That code is much easier to read and understand what’s going on and avoids the plague of USBS (unless you’re a Ruby developer in which case you have a high sensitivity to USBS).

The important part is we don’t have to maintain tricky bookkeeping code. There’s no code here that keeps track of the last time we saw one or the other event. Here, we just declare what we want and Reactive Extensions handles the rest.

This is what I mean by declare, don’t tell. We don’t tell the code how to do its job. We just declare what we need done.

UPDATE: ReactiveUI (RxUI) 5.0 has an assembly Reactive.Events that maps every event to an observable for you! For example:

control.Events()
  .Clicked
  .Subscribe(_ => Console.WriteLine("foo"));

That makes things much easier!

code comments edit

Not long ago I wrote a blog post about how platform restrictions harm .NET. This led to a lot of discussion online and on Twitter. At some point David Kean suggested a more productive approach would be to create a UserVoice issue. So I did and it quickly gathered a lot of votes.

I’m visiting Toronto right now so I’ve been off of the Internet all day and missed all the hubbub when it happened. I found out about it when I logged into Gmail and I saw I had an email that the user voice issue I created was closed. My initial angry knee-jerk reaction was “What?! How could they close this without addressing it?!” as I furiously clicked on the subject to read the email and follow the link to this post.

Bravo!

Serious Kudos to the .NET team for this. It looks like most of the interesting PCL packages are now licensed without platform restrictions. As an example of how this small change sends out ripples of goodness, we can now make Octokit.net depend on portable HttpClient and make Octokit.net itself more cross platform and portable without a huge amount of work.

I’m also excited about the partnership between Microsoft and Xamarin this represents. I do believe C# is a great language for cross-platform development and it’s good to see Microsoft jumping back on board with this. This is a marked change from the situation I wrote about in 2012.

code comments edit

Over the past few years I’ve become more and more interested in functional programming concepts and the power, expressiveness, and elegance they hold.

But you don’t have to abandon your language of choice and wander the desert eating moths and preaching the gospel of F#,  Haskell, or Clojure to enjoy these benefits today!

In his blog post, Unconditional Programming, Michael Feathers ponders how less control structures lead to better code,

Control structures have been around nearly as long as programming but it's hard for me to see them as more than an annoyance.  Over and over again, I find that better code has fewer if-statements, fewer switches, and fewer loops.  Often this happens because developers are using languages with better abstractions.  They aren't consciously trying to avoid control structures but they do.

We don’t need to try and kill every if statement, but perhaps the more we do, the better our code becomes.

Msi\_if\_coverPhoto from wikimedia: Cover of If by the artist Mindless Self Indulgence

He then provides an example in Ruby of a padded “take” method.

…I needed to write a 'take' function to take elements from the beginning of an array.  Ruby already has a take function on Enumerable, but I needed to special behavior.  If the number of elements I needed was larger than the number of elements in the array, I needed to pad the remaining space in the resulting array with zeros.

I recommend reading his post. It’s quite interesting. At the risk of spoiling the punch line, here’s the before code which makes use of a conditional...

  def padded_take ary, n
    if n <= ary.length
      ary.take(n)
    else
      ary + [0] * (n - ary.length)
  end
  end

… and here is the after code without the conditional. In this case, he pads the source array with just enough elements as needed and then does the take.

  def pad ary, n
    pad_length = [0, n - ary.length].max
    ary + [0] * pad_length
  end

  def padded_take ary, n
    pad(ary, n).take(n)
  end

I thought it would be interesting to translate the after code to C#. One thing to note about the Ruby code is that it always allocates a new array whether it’s needed or not.

Now, I haven’t done any benchmarks on it so I have no idea if that’s bad or not compared to how often the code is called etc. But it occurred to me that we could use lazy evaluation in C# and completely circumvent the need to allocate a new array while still being expressive and elegant.

I decided to write it as an extension method (I guess that’s similar to a Mixin for you Ruby folks?).

public static IEnumerable<T> PaddedTake<T>(
  this IEnumerable<T> source, int count)
{
  return source
    .Concat(Enumerable.Repeat(default(T), count))
    .Take(count);
}

This code takes advantage of some Linq methods. The important thing to note is that Concat and Repeat are lazily evaluated. That’s why I didn’t need to do any math to figure out the difference in length between the source array and the the take count.

I just passed the total count we want to take to Repeat. Since Repeat is lazy, we could pass in int.MaxValue if we wanted to get all crazy up in here. I just passed in count as it will always be enough and I like to play it safe.

Now my Ruby friends at work might scoff at all those angle brackets and parentheses in the code, but you have to admit that it’s an elegant solution to the original problem.

Here is a test to demonstrate usage and show it works.

var items = new[] {1, 2, 3};

var result = items.PaddedTake(5).ToArray();

Assert.Equal(5, result.Length);
Assert.Equal(1, result[0]);
Assert.Equal(2, result[1]);
Assert.Equal(3, result[2]);
Assert.Equal(0, result[3]);
Assert.Equal(0, result[4]);

I also ran some quick perf tests on it comparing PaddedTake to the built in Take . PaddedTake is a tiny bit slower, but the amount is like the extra light cast by a firefly at noon of a sunny day. The performance of this method is way more affected by the number of elements in the array and the number of elements you are taking. But in my tests, the performance of PaddedTake stays pretty close to Take as we grow the array and the take.

I think it’d be interesting to have a build task that reported back the number of `if` statements and other control structures per line of code and see if you can bring that down over time. In any case, I hope this helps you improve your own code!