Querying the Future With Reactive Extensions

UPDATE: After an email exchange with Eric Meijer, I learned that I was a bit imprecise in this treatment. Or, as the colloquial term goes, “wrong”. :) I’ve changed the title to reflect more accurately what Reactive extensions provide.

Iterating over a collection of items seems like a pretty straightforward mundane concept. I don’t know about you, but I don’t spend the typical day thinking about the mechanics of iteration, much like I don’t spend a lot of time thinking about how a roll of toilet paper is made. At least I didn’t until watching Elmo Potty Time with my son. Now I think about it all the time, but I digress.

the-futureHistorically, I’ve always thought of iteration as an action over a static set of items. You have this collection of elements, perhaps a snapshot of data, and you then proceed to grab a reference to each one in order and do something with that reference. What you do with it is your business. I’m not going to pry.

It wasn’t till the yield operator was introduced into C# that I realized this was a very limited view of iteration. For example, using the yield operator makes it easy to enumerate over computed sets, as demonstrated by iterating over the Fibonacci sequence. In this case, the set of elements being iterated is not a static set.

Reactive Extensions

Recently, Matt “is his middle name really not F#” Podwysocki swung by my office to show me yet another way of thinking about iterations via the Reactive Extensions to JavaScript. These extensions are based on the same concept applied in the Reactive Extensions for .NET which I’ve sadly ignored until now.

There’s a channel9 video where Eric Meijer describes these extensions as push collections, as contrasted with normal collections where you pull each item from the collection.

Unfortunately, when I first heard this analogy, it didn’t click in my head. That’s not terribly unusual as it often takes a few bat swings at my head for something to stick. It wasn’t till I understood the pattern of code that reactive extensions are a replacement for, did it click. By inverting the analogy that Eric used, these extensions made a lot more sense to me.

Typically, when you write code to handle user interactions, you write events and methods (event handlers) which handle the events. In my mind, this is a very “push” way to handle it. For example, as soon as a user moves the mouse over an element you’re interested in, a mouseover event gets pushed to your mouseover event handler method.

Reactive extensions inverts this model by taking what I would call a “pull” model of events. Using these extensions, you can treat the sequence of user events (such as the sequence of mouse over events) as if it were a normal collection (well actually, as an enumeration). Thus you can write LINQ queries over the collection which do things like filtering, grouping, composing, etc.

Your code really looks like it’s dealing with a fully “populated” collection, even though elements of that collection may not have occurred yet.

Effectively, you’re enumerating querying over the future.

The mental shift for me is to realize that we’re actually working with sequences being “pushed” into our query in this case and not queries running over already populated collections.

Speaking of keyboard presses, Matt Podwysocki took my Live Preview jQuery Plugin and ported it to use the Reactive Extensions for JavaScript. You can see a demo of it in action here (view source for the code).

The snippet that’s pretty cool to me is the following:

textarea
  .ToObservable("keyup")
  .Take(1)
  .SelectMany(function() {
  return Rx.Observable.Start(function() {
    return textarea.reloadPreview(); });
  }).Repeat()
.Subscribe(function() {});

As Matt told me, if you squint hard enough, it looks like you’re writing a LINQ query in JavaScript. :)

What others have said

Requesting Gravatar... Ryan Mar 26, 2010 1:50 AM
# re: Enumerating the Future With Reactive Extensions
Erik's talk at mix about this was fantastic:

http://live.visitmix.com/MIX10/Sessions/FTL01
Requesting Gravatar... Jay Kimble Mar 26, 2010 2:27 AM
# re: Enumerating the Future With Reactive Extensions
I was at the MIX2010 session where Erik Meijer went over this stuff. It was fabulous. Still wrapping my head around it, but I think there are some really cool applications of the library.

His presentation was both fun and informative (if you haven't seen it yet).. let's just say that he used poker chips to illustrate one of his points...
Requesting Gravatar... Steve Calvert Mar 26, 2010 2:28 AM
# re: Enumerating the Future With Reactive Extensions
I saw the Rx talk by Eric Meijer at MIX10. It was probably one of the coolest talks I saw there, apart from the MVC talks you gave :P

I was super excited about this, but also had a hard time understanding how it would fit into my existing code. I really want to look into refactoring the jQuery UI autocomplete plugin to use Rx, as I've experienced exactly what Eric shows when dealing with autocompletes.
Requesting Gravatar... fschwiet Mar 26, 2010 4:28 AM
# re: Enumerating the Future With Reactive Extensions
I've been watching some blogs on Reactive Framework, both for .Net and for Javascript. I've kind of wrapped my head around the 'what' but I still don't see the 'why'.
Requesting Gravatar... Hellfire Mar 26, 2010 4:45 AM
# re: Enumerating the Future With Reactive Extensions
I still don't get it and I'm finding that to be the case with more and more technologies these days. Might be time to retire.
Requesting Gravatar... Peter Kellner Mar 26, 2010 4:59 AM
# re: Enumerating the Future With Reactive Extensions
Somehow, I'm thinking somehow this will tie into MVC :)
Requesting Gravatar... Santosh Kulkarni Mar 26, 2010 5:10 AM
# re: Enumerating the Future With Reactive Extensions
There is a bug in your livepreview demo (http://demo.haacked.com/livepreview/).

Keep typing, and your content overflow out of the preview div instead of word wrapping.

Leaving my mark !! ;)
Requesting Gravatar... Santosh Kulkarni Mar 26, 2010 5:12 AM
# re: Enumerating the Future With Reactive Extensions
There is a bug in your livepreview demo, the original one.

Try typing incessantly, and the content overflows out of the preview div instead of word wrapping.

Also, it seems one can not post comments on your blog, with hyperlinks in the comments. The comments gets marked as spam.

Requesting Gravatar... haacked Mar 26, 2010 5:44 AM
# re: Enumerating the Future With Reactive Extensions
@Santosh regarding the word wrapping, you need to have spaces in what you're typing for it to word-wrap. It's not a bug in my live preview code but a styling issue. I could have used CSS to auto clip the content, but I didn't want to.
Requesting Gravatar... Santosh Kulkarni Mar 26, 2010 6:16 AM
# re: Enumerating the Future With Reactive Extensions
oh okay .. my bad .. wordwrap was prolly a wrong choice of word .. but its much more intuitive to expect for the text to flow on to the next line instead of overflowing out of bounds when you see a box .. that is what i was thinking.
Requesting Gravatar... haacked Mar 26, 2010 12:45 PM
# re: Enumerating the Future With Reactive Extensions
@Santosh well take that up with the CSS gods. :) My point is that the output of the code is HTML + CSS. If you take a DIV, give it a specific width, and then put a long unbroken string like so:

<div style="width:20px; border: solid 1px #000;">
BBBBBBBBLLLLLLLLLLLLAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAHHHHHHHHHHHH
</div>

The text is going to overflow the DIV.

What you can do is something like:

<div style="width: 20px; overflow: auto; border: solid 1px #000;">
BBBBBBBBLLLLLLLLLLLLAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAHHHHHHHHHHHH
</div>

Note, that's a change to the style, not the JavaScript code. In my demo, I chose not to do that because most people leave comments that are standard words broken with spaces. :)
Requesting Gravatar... taber loveless Mar 26, 2010 5:58 PM
# re: Enumerating the Future With Reactive Extensions
Thanks for this post! Although not the point of it, I now have a better understanding of 'yield return'. Now I have to run ILDASM and see what the difference is between 'x.Find(x => x.y == z)' and its foreach+yield-return counter THEN begin thinking about ways to apply Reactive Extensions!
This is what I get for not sleeping tonight?!? thanks Phil :P ;)
Requesting Gravatar... paul Mar 29, 2010 12:02 AM
# re: Querying the Future With Reactive Extensions
Phil, I'm kinda missing the "so what" from your post. Ok, so it inverts the model. Ok, so your code is using a collection-ish api, even though the "stuff" isn't there at compile time. Not really understanding why that matters.
Requesting Gravatar... Rodrigo Dellacqua Mar 29, 2010 11:51 AM
# re: Querying the Future With Reactive Extensions
Way to copy smalltalk/ruby.

Requesting Gravatar... Pete Mar 30, 2010 10:02 AM
# re: Querying the Future With Reactive Extensions
Phil,

When I first played with reactive in silverlight, and now that I see it in Javascript, I still don't see it as a major improvement over traditional event handling. What WOULD be interesting is a true functional reactive paradigm where we could do things like textarea.x = {mouse.x} and have the updates made automatically when mouse.x changes ("spreadsheet programming"). I'm not easily seeing how the reactive extensions are any more interesting than the equivalent jQuery:


textarea.keyup( function(e) {
textarea.reloadPreview();
});


I'm sure I am missing something or else Microsoft wouldn't be making such a big deal of this. I have seen the examples using Silverlight of reacting to responses from web services, for example, and still I'm not seeing the light.
Requesting Gravatar... Anthony Apr 06, 2010 8:21 PM
# re: Querying the Future With Reactive Extensions
Like others I get what is going on I don't get the why... With the example you have given, why would you want to handle events this way instead of in the traditional manner...
Requesting Gravatar... Jason Hurdlow Jul 13, 2010 5:08 AM
# re: Querying the Future With Reactive Extensions
For those who "don't get it", maybe I can help:
The real power of Rx comes not with simple one-event situations like in the given example. For that you're correct that the traditional way works just fine, and the additional complexity of Rx is just that. Where Rx comes into its own is when you have a more complex set of events that you need to coordinate. With the ability of Rx to compose events you can dramatically simplify complex event handling scenarios.
Example: I had a situation where I was writing a somewhat complex hand-off from one web-hosted silverlight app to another, and it did things like "send app2 a message, if it responds, then close yourself, if it doesn't open a browser window with that app, then wait 2 seconds and send another message, if that doesn't respond, wait 2 more seconds then...." you get the idea. I fussed with normal event handlers, flags, timers, etc... for the better part of a day and couldn't get it quite right. That was when I stumbled upon Rx. I had it working in Rx in about 20 minutes. The code was much smaller, MUCH easier to read, and as a bonus, it worked perfectly! :)
That was a hallelujah moment for me, and when you eventually find yourself in that sort of a situation the lights will come on and you'll "get it". :)
Another way to look at it is that you put your event filtering "up front" (in a clear, concise manner) and let Rx and LINQ do all the hard work for you, and you only get called when you really need to know. Kind of like a high-powered executive who's personal assistant screens his calls, compiles messages, filters out the riff-raff, coordinates meetings, and only brings items to his attention that he really needs to know about. That's Rx.
Requesting Gravatar... Kalid Aug 04, 2010 7:41 AM
# re: Querying the Future With Reactive Extensions
@Jason: Just wanted to say thanks for that example, it helped clarify the "why" of Rx. Simple examples like the live preview are easier to understand, but unfortunately don't show the power/advantage over the traditional approach.
Requesting Gravatar... Lee Campbell Aug 24, 2010 8:32 PM
# re: Querying the Future With Reactive Extensions
I totally get where other are coming from when they ask "So what?". This was my take on Rx in Dec'09 but now I am converted. It is nothing fantastically revolutionary, but a nice evolution. I felt the same with LINQ, becasue at the time I could already query DB/XML and I had Anon Delgates. However I would hate to go back and not have extension methods, Action/Func or deffered execution.
Having worked with the Rx Library for a fair few months I have put together what I hope to be a path to unfolding the "Why" (because who cares that IObservable is the math dual of IEnumerable) :-)
This series will walk through what and how and lead you to why.
leecampbell.blogspot.com/...

HTH
Lee

What do you have to say?

(will show your gravatar)
Please add 7 and 2 and type the answer here: