This site provides a variety of code, links, and articles for web programming, along with news items of interest — at least to me :)
Note, this site is a work in progress so "please pardon my dust!"
CodeBetter.Com - Stuff you need to Code Better!
Launching beta, or “How to decide when and where to cut corners”

Executive Summary: Introspection on how to cut corners for the initial launch of a public application with reference to Nate Kohari’s recent post on the subject.

As we amble forward on our little private venture, there are a number of applications and companies we watch closely. It’s hard to say whether we want to emulate them or if our philosophies naturally align and we’re attracted to like-minded people. Maybe a bit of both.

One of those applications/companies is AgileZen. From the measured approach to features to the deliberate way the user experience falls into the background, we not only use the product, we often look to it for inspiration. Sort of a “What would Nate and Niki do?” approach.

I read Nate’s account of the time before and after launch which a lot of nods in recognition. The almost universal message we’ve been given by people both successful and unsuccessful in building a public application is (and I’m quoting verbatim from one of them): “Just launch the &*%$ thing”.

This is what I thought about while I read the part in Nate’s tale about cutting corners. Initially, there was a lot of kludge and after launch, he rewrote almost every line of JavaScript.

He makes no apologies for this nor should he. It is, I think, a valid decision for a product like his (and ours). Provided you’re smart about it. Recognize where the crap is and isolate it and make a concerted effort to go back to it after you have money coming in. The point he makes is:

Before we could afford to spend a lot of time to make the software maintainable, we first had to prove that we would actually have to maintain it – we had to make sure that we actually had a market.

It’s something I’ve wrestled with a little, especially as we had a learning curve of using an unfamiliar technology. How far do we cut corners? Forego UI tests? Unit tests? Proper separation of concerns?

My partner is a little more battle-scarred in this area and he was pretty adamant that we keep this reasonably maintainable from the start, even if it takes longer to write code, which it inevitable would. Which means we would have to sacrifice something on the iron triangle (or the lesser known Pewter Scale). But that was an easy choice. We wanted relatively high quality and to deliver on time and we were constrained by budget. That left scope as our only real variable. And I love me some restricted scope. Features are, to my mind, the only aspect of the triangle worth arguing over.

But back to the quality aspect. I’m speculating but I suspect we didn’t cut as many corners as Nate did. Or maybe we just didn’t cut the same corners. Though I won’t claim anywhere near 100% code coverage, we have a great many unit tests. And as part of our process, no story can be marked complete unless it’s been code reviewed by at least one person. What’s missing so far is a formal QA person, which is a matter of not having the resources (or put another way, not placing a high enough priority on it based on the resources available to us), and a UI testing process, which is near the end of prototyping thanks to Winnipeg’s best kept secret.

So the question is, did we hurt ourselves but placing an undue focus on quality aspects of the process we didn’t need to?

I have no idea. And while it makes a nice, lengthy blog post, in the end, I haven’t been stressing about it much (at least not until I read the blog post; thanks for nothin’, Kohari). But here’s the position we’re in: We’ll be releasing beta next week and our first order of business is to start looking at new features, which we will release at regular intervals (at least bi-weekly, if not more often). We have no refactoring to do because our codebase is exactly where we want it to be architecturally.

The difference is we’ve lost a month-ish worth of income. And at the moment, that income is still a big unknown. AgileZen could have launched to a whole lot of nothing in which case, Nate would have chalked it up to experience and thanked his lucky stars he didn’t invest any more time than he did in it. We may do the same and realize that we should have launched sooner so we hadn’t sunk any more effort into it.

There’s an element of faith involved though. Maybe it won’t catch on but I really believe it will. And if I plan to be looking at this code for the next five to ten years or more, I like the fact that we put a little extra effort into it early on.

Another nice side effect of this is that it’s helped defined the culture of our development team to a large degree. It helped us to build the team we have and by setting the ground rules early, it’s helped us maintain a high standard so that it’s almost become second nature by now.

Man, this blog is saving me a ton of money in psychologist fees.

Kyle the Clear-headed

 
Join me at ADNUG on Sept. 13th to talk about Functional Programming for Everyday .Net

I'm presenting at ADNUG next Monday on the 13th.  By request I'm going to expand on one of my old MSDN articles:  Functional Programming for Everyday .Net.

It's not up on the ADNUG site, but here's the abstract:

Functional Programming for Everyday .Net Development

Forget the silly Fibonacci sequence examples and abstract math problems, how do functional programming techniques help me do my job with typical enterprise development projects?  In this talk I'll look at places where it's advantageous to compose code with first class functions instead of objects.  I'll show how to use Continuation Passing Style to achieve lower coupling in your code internals by getting closer to  the "Tell, Don't Ask" ideal.  Finally, I'll show how passing blocks can be a way to formulate API's that result in less code for the consumer to write, more readable code, and fewer errors.

 

I *think* I submitted this talk to CodeMash as well, so there's a chance I might give this talk again in January there.

 

Don't worry, I'm still on my side-project break so I haven't done anything whatsoever to prepare yet:)

 
Visual C++ 2010: What’s new for MFC library?

Issam Lahlali, the CppDepend development leader, just wrote about new interesting things in MFC 2010. The results are obtained by comparing with CppDepend the MFC 2010 C++ codebase with the previous version. Enjoy!

 
Reactive Extensions for .NET – Event-based Async Operations

With the many posts that I’ve done on the Reactive Extensions for both JavaScript and .NET, I’ve covered a wide variety of the basics as well as some of the deeper stuff.  This time, I’d like to get back to the basics that I do when I give a talk on Rx in person, and this time explaining some of the problems we face today with reactive programming in general, asynchronous and event-based.

Why Do We Need It?

Let’s go into an example of using the event-based asynchronous programming methods that became a bit more prevalent when .NET 2.0 came around, especially with the introduction of WCF in the .NET 3.0 timeframe.  This was an alternative to the other methods of asynchronous programming which included the Begin/End Pattern and using WaitHandles.  One distinct advantage it did have was for the ability to report progress, report incremental results or state changes that the others could not provide.  But with it, comes many downsides as well which we’ll go into and where the Reactive Extensions can help. 

Typically, we’d have some form of asynchronous method which has no return value and takes in any number of parameters, and a given user token which would allow us to correlate our token with what comes in the user state to make sure we’re looking at the right result. 

public void DoSomethingAsync(object token) 
{
    // Kick off something async
}

And we’d also have some form of EventArgs which would encompass our result and exception should one occur and optionally a cancelled flag should we need to do so and be notified that it happened.  Then we’d also have our EventHandler which is then called when our asynchronous method is complete.

public class DoSomethingCompletedEventArgs : EventArgs 
{
    public string Result { get; set; }
    public bool Cancelled { get; set; }
    public Exception Error { get; set; }
}

public event EventHandler<DoSomethingCompletedEventArgs>
    DoSomethingCompleted;

To give you an idea of some of the problems we face with this pattern, let’s go over an example using the System.Net.WebClient class and downloading a string asynchronously via the DownloadStringAsync method.  Let’s enumerate some of the challenges in the code below.

var uri = new Uri("http://search.twitter.com/search.json?q=4sq.com");
var client = new WebClient();

client.DownloadStringCompleted += (sender, args)
{
    // TODO: Get the only one we want?
    // TODO: Check for cancel?
    // TODO: Check for exception?
    // TODO: Check for result?
    
    // TODO: What about more composition?
};

client.DownloadStringAsync(uri);

// TODO: How to unsubscribe
client.DownloadStringCompleted -= ???

The first challenge is how to make sure we get the value we want.  After all, if multiple calls are made to this WebClient instance, we’ll need to correlate the user state with our token.  Next, we need to check for cancellation and do something when that happens.  After checking for cancellation, we might also want to check for exceptions and handle them appropriately as well.  And finally we get our result, now what do we do with it?  Better yet, now that I have the result, how can I compose it with another asynchronous call?  Finally, when using lambdas for subscribing to events, I can’t unsubscribe our inline lambda, so that could cause an issue as well.  Making some of those changes to handle our cases properly, we end up with code that looks like the following:

var uri = new Uri("http://search.twitter.com/search.json?q=4sq.com");
var client = new WebClient();
var token = new object();

DownloadStringCompletedEventHandler h = (sender, args)
{
    // Check if it's ours
    if (token != args.UserState) return;

    if (args.Cancelled)
    {
        // Do something to say we stopped
    }
    else if (args.Error != null)
    {
        // Do something with that error
    }
    else
    {
        // Well, now what?
    }
};

client.DownloadStringCompleted += h;
client.DownloadStringAsync(uri, token);

// When we're ready to clean up
client.DownloadStringCompleted -= h;

We solved a good majority of the problems with a lot of boilerplate.  But within all this boilerplate, we’re losing sight of our core goal here, which is to get our result and do something with it, especially when that next call happens to be to another asynchronous call.  For example, what if we want to download a string from somewhere, transform it into JSON, modify the results and then upload somewhere else?  Our code might look like the following should we use the Reactive Extensions for .NET to solve it.

var client = new WebClient();
var searchUri = new Uri("http://search.twitter.com/search.json?q=4sq.com");
var uploadUri = new Uri("http://localhost:8080/uploads");

IObservable<Unit> query =
    from result in client.DownloadStringAsObservable(searchUri, new object())
    let transformed = TransformResult(result)
    from upload in client.UploadStringAsObservable(
        uploadUri,
        "POST",
        transformed,
        new object())
    select upload;
    
var subscription = query.Subscribe(
    x => {}, // Nothing to do
    exn => 
    {
        // Do something with the exception
    }, 
    () => 
    {
        // Indicate we're finished
    });

What we’re able to do is take operations that would have required quite a bit of code in terms of boilerplate, and allow for composition through LINQ via the Reactive Extensions for .NET.  But, how do we get there? 

How Do We Get There?

What we need to do at this point is to transform our asynchronous call from above and turn it into an IObservable<T> instance.  Using the FromEvent which we’ve covered earlier, won’t cut it because ultimately, we want to combine the asynchronous call with the event handler which is invoked upon completion, which FromEvent cannot provider.  Instead, we want to focus our effort into the Observable.Create and Observable.CreateWithDisposable methods to help us.  This allows us to create IObservable<T> instances from the subscribe implementation.  We can either choose the CreateWithDisposable method which gives us an IObserver<T> instance and we return a concrete IDisposable instance, or we call Create in which are given the same IObserver<T> instance, but we return an Action instead of the IDisposable instance.  Let’s look at those signatures below:

public static IObservable<TSource> CreateWithDisposable<TSource>(
    Func<IObserver<TSource>, IDisposable> subscribe
)

public static IObservable<TSource> Create<TSource>(
    Func<IObserver<TSource>, Action> subscribe
)

Using the Observable.Create method, let’s create a method called DownloadStringAsObservable extension method to handle the asynchronous request.  Inside the Observable.Create, we’ll create our DownloadStringCompletedEventHandler where we can check for our user token, handle cancellation by calling observer.OnCompleted, handle exceptions via observer.OnError and finally handle our result by calling observer.OnNext and observer.OnCompleted to indicate we are finished.  We then take this handler and listen to the DownloadStringCompleted event.  We want to be able to catch any errors in our DownloadStringAsync should it throw an exception, so we put that in a try/catch block and send the Exception to observer.OnError.  Finally, Being the good citizens we are, we want to clean up after ourselves by removing our handler from the DownloadStringCompleted event when we dispose of our subscription.

public static IObservable<string> DownloadStringAsObservable(
    this WebClient client, 
    Uri address,
    object userToken)
{
    return Observable.Create<string>(observer =>
    {
        DownloadStringCompletedEventHandler handler = (sender, args) =>
        {
            if (args.UserState != userToken) return;
    
            if (args.Cancelled)
                observer.OnCompleted();
            else if(args.Error != null)
                observer.OnError(args.Error);
            else
            {
                observer.OnNext(args.Result);
                observer.OnCompleted();
            }
        };
    
        client.DownloadStringCompleted += handler;
        
        try
        {
            client.DownloadStringAsync(address, token);
        }
        catch (Exception ex)
        {
            observer.OnError(ex);
        }
        
        return () => client.DownloadStringCompleted -= handler;
    });
}

The sample logic applies to uploading a string as well by wrapping the UploadStringAsync method as we do below.

public static IObservable<Unit> UploadStringAsObservable(
    this WebClient client, 
    Uri address, 
    string method, 
    string data,
    object userToken)
{
    return Observable.Create<Unit>(observer =>
    {
        UploadStringCompletedEventHandler handler = (sender, args) =>
        {
            if (args.UserState != userToken) return;
    
            if (args.Cancelled)
                observer.OnCompleted();
            else if (args.Error != null)
                observer.OnError(args.Error);
            else
            {
                observer.OnNext(new Unit());
                observer.OnCompleted();
            }
        };
    
        client.UploadStringCompleted += handler;
        
        try
        {
            client.UploadStringAsync(address, data, method, token);
        }
        catch(Exception ex)
        {
            observer.OnError(ex);
        }
        
        return () => client.UploadStringCompleted -= handler;
    });          
}

By using these tokens as we do above, we could easily wrap the DownloadProgressChanged event as well filtering for our token as well.

public static IObservable<DownloadProgressChangedEventArgs> 
    DownloadProgressChangedAsObservable(
        this WebClient client,
        object userToken)
{
    return Observable.FromEvent<DownloadProgressChangedEventArgs>(
            client,
            "DownloadProgressChanged")
        .Where(x => x.EventArgs.UserState == userToken)
        .Select(x => x.EventArgs)
}

What we have done here is taken a less than optimal event-based asynchronous programming model and moved it to use Reactive Extensions for .NET so that we can take care of uniform cancellation checking, error handling and allow for composition.

Conclusion

Asynchronous programming is something that’s creeping up more and more to .NET developers, especially with Silverlight, Windows Phone and so forth.  Unfortunately, dealing with asynchronous programming has been and continues to be hard.  By using the Reactive Extensions for .NET, we have ways to build bridges from the old asynchronous programming models, to a more composable form where we can uniformly handle exceptions, cancellation and so forth.

So with that, download it, and give the team feedback!

 
Super Simple CQRS Example

Under 500 lines of code before the client

http://github.com/gregoryyoung/m-r

 

I will write a blog post about some of the stuff in it but its pretty straight forward.

 
Catching the iPhone in the .net

Smartphones and all other kinds of mobile devices are everywhere, almost everybody has one. Not just for entertainment but also as a client for all kinds of enterprise software. Making choices when developing desktop software was easy, the main discussion usually was restricted to deciding which Windows version to target. Making choices when developing software for mobile is harder. No platform is dominant and each platform has other tools and languages. The nice thing with the Windows Mobile platform is that you can use the same tools and languages (read C# and the .net platform) as used for desktop applications. The nice thing about Apples iPhone is that it has an incredible UI, is also widely used and has a good possibility for developing commercial applications.

Wouldn’t it be nice to have the best of both worlds ? The quality of the tools and language of Windows Mobile and the pleasure of use of an iPhone  ?

A small Windows Mobile rant

I have to get something of my chest first. This is an important reason, besides the iPhone just being sexier :), to  look further than WM.

Building applications for Windows Mobile is, thanks to the .net compact framework, no big deal. For a couple of years my Windows Smartphones were a nice and reliable target. But instead of getting better the Windows Mobile platform has become worse in the everyday use, despite all the “integration” in Windows. Let me sum up my complaints over the last year:

  • Since Vista many a smartphone is no longer recognized when connected over USB. Switch to bluetooth ?
  • After a bluetooth connection with my laptop I have to reboot the phone before it will connect to anything else (like my old a plain vanilla carkit)
  • Office 64 bit does not sync with any Windows Mobile device at all Switch back to Office 32 bit ?

All of these complaints are well known and acknowledged. And will not be fixed. Period. (None of them are a problem when trying to sync an iPhone with MS Outlook) So you are isolated more and more with Windows Mobile. Yes things will be better with Mobile 7. But that’s somewhere in the future and there is no way to migrate existing hard- and software.

The good thing with WM7 is that all software development is based on Silverlight. Which has proven to be a nice development environment with a very good subset of the .net framework which encapsulates the platform. You only have take care about the form factor of the hardware and can forget all about the dreaded OS. For now it’s time to look at the iPhone.

Wrapping up iOS

At first sight the tools for creating iOS (the OS for the iPhone and iPad) applications don’t look very appealing. The API is rich but the language (objective C) is not very friendly. Worst of all you have to do your own memory management. Memory is freed using ref-counting, just like in COM. I have had my share of that working with Delphi in COM and avoid it like the plague ever since. The main problem with refcounted objects is what happens with them when they are passed as a parameter to some other piece of code. Without a clue what will happen to their count while spending some time there. Memory leaks as well as null pointers are usually the result. Trying to avoid that leads to ugly changes in the software’s design, only trying to satisfy memory management.

Here the MonoTouch framework comes to the rescue. It wraps up the iOS framework in a .net style library and adds a C# compiler for iOS. All external resources are wrapped up in the IDisposable pattern. Over the years a garbage collector and IDisposable have proven to be, despite the initial skepticism, quite reliable. And have lead to code which is far simpler.

Building an iPhone app in C#

You need several things to do this, the first is a Mac with the newest version of OS X. It does not have to a monster, for me the smallest Macbook is by far powerful enough. Monotouch runs as a part of MonoDevelop, which looks like a simple Visual Studio for the Mac.

Monodevelop

Simple, but all the essentials are included. Monodevelop has integrated nUnit support and refactoring. Don’t expect too much of the latter, it pales compared to that of VS. Not to mention Resharper. Typing code in MonoDevelop is a new form of sports on itself. The difference in the navigation keys itself is enough to drive you nuts in the beginning. And, without Resharper, there is so much more which you have to type.

The good part is that running and debugging the app works like a charm. The iPhone emulator works well, including network support. Which can be another PITA developing for Windows mobile. Just for fun, try watching a Youtube movie in the emulator. The debugger is a way of exploring what’s inside an iPhone. The stability of the environment is good, it takes inspecting null pointers to non managed resources to hang up MonoDevelop. With “Force Quit” in the apple OS (comaparable to the Windows Task Manager) a quick restart of the tool is easy.

Apple support

The App Store is a way to sell your apps. Before an app is accepted for resale it is tested and scanned by Apple. A lot has been said about Apple banning apps not created with Apple’s own SDK. Monotouch apps compile to native iOS code and Apps built with Monotouch are reported to be accepted to the appstore. Working for the appstore takes some investments. Beside the mac hardware and an entrance fee to the store there is $500 for a Monotouch licence to deploy your apps to an iPhone or the store. Apps built with the free (trial) version only deploy to the emulator.

Taking the bait

Having explored Monotouch I have taken the bait. As with every new thing the first steps on the curve are hard. There are a lot of resources on the web, the quality varies. As a first step I started with refactoring the examples. By hand… Soon the result was astonishing. Most examples are in a very chatty syntax; refactoring the noise to auto-properties soon revealed the intention of most of the code. And quite soon I found my way. I’ve got the iPhone in my .net. For now I can only advise you to, in case you have a Mac, take a look at Monotouch. Enjoy! More on my experiences later.

 
Education

What if software development was taught like we teach a language? (This question led me to following thoughts, I suck at grammar and am not saying this is how we actually learn, these are just the thoughts that came to me while pondering that question.) That we are first taught the most fundamental of basics in a language, the ABCs, the sounds of our native tongue. Through this most primitive of understanding we start to grow our proficiency.

I didn't attend university courses on software development but I would guess that it doesn't start with learning about AND/OR/NOT constructs. Not that we need to spend a crap ton of time on this topic but do we spend any? Is there any value in it, when I read the Pattern on the Stone I found understanding the basics to be fascinating. It's understanding things at that low level that I feel has allowed me understand things at the higher levels.

As we progress in our understanding, we should move away from using primitive language structures, and start to develop a level of comprehension that goes beyond simple sentences, and one word responses. We need to learn how to compose prepositional sentences, and transitions. When I read Object Thinking I started to see the world as the designers of object oriented languages wanted me too. Now I just need to find the books on functional and other language styles.

Learning how to compose the chapters and novels of our software development can prove to be an even larger challenge for us in the software development community. It takes an awful long time to build up a program that has enough complexity to warrant being compared to a 300 or 400 page book. We study the architecture of our language from the time we are infants, how can we embed the concept of architecture in programming through out the whole of the education? When our projects are relatively small by comparison the large programs we will become responsible for when we move out to the real world?

Though I have studied many other languages than English (German, Spanish, Japanese, and Chinese) I am not sure its helped me really understand English. But my friends who actually are proficient in there non-native languages say that it helps. Are we asked to study multiple languages? This is an area I am sure is better addressed, but are we studying the differences in the languages. The values enshrined in .Net over Perl over Python over Ruby over Assembly etc? To see why some languages suck at some tasks and other languages are great for other tasks. Are we taught how to build a simple language, and then a simple language on top of another language, and then a DSL. To see the various amounts of effort required to achieve the simplest of things?

Where are the art and craft classes of software development, where the value is on the aesthetic of the program and on the code. Where is the poetry class of software design where we focus on writing that one-liner of pearl? Where is the class that teaches us the code smells and how to fix them.

What about the journalism class where we learn to communicate to the masses. Where we learn to speak to an audience larger than one passionate reader. To publish on a deadline, to not write above our readership. To realize that even though we may have an extended vocabulary and that is some thing that we should pride our selves on, that we must remember that people other than us are going to be reading it and that the goal is comprehension not on flexing our vocabulary might.

Ok, well I think I have pushed this metaphor about as far as I want to. Thank you for reading, I hope that it inspires you to think about what you will learn next.

-d

 
Validating Code Rules in Visual Studio

One popular aspect of the tool NDepend is the integration of the tool into the Continuous Integration build process. Every morning the team leader gets an up-to-date report telling if some coding rules written with CQL have been violated within the last 24h.

With NDepend v3 integrated in Visual Studio 2010, 2008 and 2005, we wanted to provide a smoother answer to the critical scenario of when a developer violates a rule. We think that waiting till the day after to discover that a rules get violated is too long, it is not enough agile-oriented. We wanted CQL rules violation being obviously warned as soon as possible, right within Visual Studio.

The idea of what we’ve done with NDepend v3 is that all CQL rules get checked into Visual Studio as soon as one or several VS projects have been (re)compiled.

The developer gets permanently informed of CQL rules validation status thanks to a progress circle we added in the bottom-right corner of the Visual Studio window.

  •   green, means that all CQL rules are validated,
  •   yellow, means some activated CQL rules are violated, 
  •   red, means some activated CQL rules don’t compile and (eventually) some activated CQL rules are violated, 
  •   blue + progressing, means that the code is currently analyzed by NDepend,
  •   grey, means that no NDepend project is currently loaded in VisualStudio. 

Hovering the circle with the mouse shows a form summarizing all CQL rules status + a pointer to Show CQL Explorer panel:


The CQL Explorer lists all rules grouped by categories…



…and by clicking a rule you can jump to the CQL edition Windows, where you can edit the rule and see culprit methods or classes that violate the rule.


Of course double clicking a culprit method or class lets jump to its source code declaration. And if the application analyzed by NDepend spawns several VS solutions, and these solutions are currently opened in several VS instances, the code element gets edited within the adequate VS instance, the one that contains the VS solution that contains the code element. This behavior at first glance might surprise and even look like a bug. But with the habits it becomes a powerful time-saver trick. And this multi-VS-instances communication trick is made possible thanks to the Application-Wide code analysis of NDepend.

 

CQL rules validation phase is fast. The performance challenge was to make this happens almost instantly to avoid slowing the developer machine. Hopefully for a large 100K Lines of Code application, code gets re-analyzed and 200 CQL rules can get checked, all within 3 seconds after the (re)compilation of one or several .NET assemblies. These fast performances were made possible thanks to the development of a new technology of incremental code analysis. With incremental code analysis, only modified code gets re-analyzed. I can attest that this was extremely challenging and complex development!

Finally, in the form shown on progress circle hovering with mouse, there are 2 links Analysis Execution and Analysis Refresh in VS.

  • The option panel Analysis Refresh in VS let’s customize when an NDepend analysis should occurs.

  • The option panel Analysis Execution let’s finally customize the analysis process (incremental/full analysis, in/out process, report generation…).

 

Something that we are thinking of implementing for the future would be an option to forbid check-in if not all CQL rules are green. The development complexity comes from the multitude of Source File Management technologies (including TFS) to support. Any comment on this potential feature is welcomed dear reader.

 
Introduction to the Reactive Extensions for JavaScript – Buffering

We’ve come a long way in the series on the Reactive Extensions for JavaScript.  After spending some time with Ruby and with extension points in other libraries, let’s step back to some of the basic operators again.  This time, let’s talk about how we can buffer our input, which is to say we can put our observable values into a buffer based upon either time or by count.  This means that instead of flooding our system with calls to OnNext, we instead fill up a buffer based upon the given criteria of time or count, and then call OnNext with that value.  Let’s look deeper into how we can use this in today’s post.

Before we get started, let’s get caught up to where we are today:

 

Buffering Output

In a traditional pull-based model of synchronous programming, you could have scenarios where yielding the next value could be a potentially expensive blocking operations.  Contrasting it to the reactive push based model, we could get flooded with requests as our systems are eagerly sending us data as fast as it can.  Potentially this can be a problem which could overload our system.  To compensate for this, we have with the Reactive Extensions (for both .NET and JavaScript), a notion of buffering, which is to say based upon some criteria, we can buffer the results and return them as an array.  We have the choice of either buffering by count threshold, or by a given time span.  Once the criteria has been exceeded (or we reach the end), we have the values yielded to us at once in array form.  Let’s first look at the buffering by count.

…With Count?

The first buffering mechanism we have with the Reactive Extensions is by count.  To make use of this functionality, we can call the BufferWithCount method which takes a count, and optionally a number to indicate how many you want values to skip in between buffers being yielded.  This method then returns to us an array of values inside an observable sequence.

// count: the size of the buffer
// skip (Optional): how many values to skip
// returns: Observable<Array<T>>

Rx.Observable.prototype.BufferWithCount =  function(
    count, 
    skip);

Let’s look at this in action.  We’ll take an array with ten numbers and buffer them every two values, so in all we should have yielded to us five arrays.  We’ll iterate through our buffer and indicate its value and position in the buffer.

Rx.Observable.FromArray([1, 2, 3, 4, 5, 6, 7, 8, 9, 10])
    .BufferWithCount(2)
    .Subscribe(
        function (next) {
            $.each(next, function (index, value) {
                $("<p/>").html("Buffered: " + index + "-" + value)
                    .appendTo(results);
            });
        });

Below is the result of the BufferWithCount specifying only the count and not the skip count.  I have highlighted how many arrays you will get.

Buffered: 0-1
Buffered: 1-2 // one
Buffered: 0-3
Buffered: 1-4 // two
Buffered: 0-5
Buffered: 1-6 // three
Buffered: 0-7
Buffered: 1-8 // four
Buffered: 0-9
Buffered: 1-10 // five

Next, we’ll try specifying the skip count, which is to say how many elements we should skip.  For example, if I have a buffer set at two and then I specify four as the skip value, then it will yield 1 and 2 and then skip 3 and 4 before then yielding to us 5 and 6.  Let’s show how that code might look:

Rx.Observable.FromArray([1, 2, 3, 4, 5, 6, 7, 8, 9, 10])
    .BufferWithCount(2, 4)
    .Subscribe(
        function (next) {
            $.each(next, function (index, value) {
                $("<p/>").html("Buffered: " + index + "-" + value)
                    .appendTo(results);
            });
        });

And now we can sure enough see our results where we yield only three arrays this time due to us skipping 3, 4, 7 and 8.

Buffered: 0-1
Buffered: 1-2 // one
Buffered: 0-5
Buffered: 1-6 // two
Buffered: 0-9
Buffered: 1-10 // three

But this of course isn’t the only mechanism we have in our arsenal.  Let’s now talk about buffering with time.

…With Time?

In addition to buffering with count, we have a notion of buffering with time, which we can achieve with the BufferWithTime method.  This method takes in a timespan in milliseconds, an optional time shift, which acts like the skip from BufferWithCount, and an optional Rx Scheduler.  This then returns to us, as before, an Observable of an array.

// timeSpan: time span in milliseconds to buffer
// timeShift (Optional): time to skip listening in milliseconds
// scheduler (Optional): a custom scheduler
// returns: Observable<Array<T>>

Rx.Observable.prototype.BufferWithTime = function(
    timeSpan, 
    timeShift, 
    scheduler);

To show this in action, we’ll hop back to some of our knowledge gained in our custom schedulers post.  We’ll create our custom DelayedScheduler based upon a given delay time.  This will help take an existing array and then space it out over time.

var delayedScheduler = Rx.DelayedScheduler = function (delay) {

    return new Rx.Scheduler(
        function (action) {
            var id = window.setTimeout(action, delay);
            return Rx.Disposable.Create(function () {
                window.clearTimeout(id);
            });
        },
        function (action, dueTime) {
            var id = window.setTimeout(action, dueTime);
            return Rx.Disposable.Create(function () {
                window.clearTimeout(id);
            });
        },
        function () {
            return new Date().getTime();
        });
};

With the scheduler now in place, we can take our array and delay each item by 100 milliseconds by calling the FromArray with our scheduler.  We can then buffer it with a given timeout of 300 milliseconds and then subscribe much as we have before.

Rx.Observable.FromArray(
        [1, 2, 3, 4, 5, 6, 7, 8, 9, 10], 
        new Rx.DelayedScheduler(100))
    .BufferWithTime(300)
    .Subscribe(
        function (next) {
            $.each(next, function (index, value) {
                $("<p/>").html("Buffered: " + index + "-" + value)
                    .appendTo(results);
            });
        });

This will yield to us some interesting results in that we have four arrays yielded to us, some with three elements and some with only two due to the timing.

Buffered: 0-1
Buffered: 1-2 // one
Buffered: 0-3
Buffered: 1-4 
Buffered: 2-5 // two
Buffered: 0-6
Buffered: 1-7
Buffered: 2-8 // three
Buffered: 0-9
Buffered: 1-10 // four

Now, let’s try with a time shift specified.  In this example, we’ll set our timeout once again as 300 milliseconds and our shift of 100 milliseconds, which should then allow us to skip the numbers 3 and 7 based upon our timing.

Rx.Observable.FromArray(
        [1, 2, 3, 4, 5, 6, 7, 8, 9, 10], 
        new Rx.DelayedScheduler(100))
    .BufferWithTime(300, 400)
    .Subscribe(
        function (next) {
            $.each(next, function (index, value) {
                $("<p/>").html("Buffered: " + index + "-" + value)
                    .appendTo(results);
            });
        });

And sure enough that turns out to be the case as we have three arrays yielded to us, missing both the 3 and 7.

Buffered: 0-1
Buffered: 1-2 // one
Buffered: 0-4
Buffered: 1-5
Buffered: 2-6 // two
Buffered: 0-8
Buffered: 1-9
Buffered: 2-10 // three

Now we shouldn’t be limited to an either or at this point, why not have the best of both worlds?

…With Both?

As we’ve already covered both buffering with count and time, let’s cover how we can get the best of both worlds and when either the timeout or the buffer size has been hit, then yield us the values in the buffer.  We can do that through the use of the BufferWithTimeOurCount which we specify the time in milliseconds, a count, and an optional Rx Scheduler.  This then yields to us our Observable of an array.

// timeSpan: time used to determine buffer size
// count: count used to determine buffer size
// scheduler (Optional): optional scheduler
// returns: Observable<Array<T>>

Rx.Observable.prototype.BufferWithTimeOrCount = function(
    timeSpan, 
    count, 
    scheduler);

To show this in action, I’m going to need another custom scheduler, but this time completely random as to allow me to show off both behaviors.  We’ll take much of the above code from our DelayedScheduler and instead create a random number up to the delay parameter.  Then, every time a new value is to be yielded, then we delay that value by the given random number.

var randomScheduler = Rx.RandomScheduler = function (delay) {

    return new Rx.Scheduler(
    function (action) {
        var randomnumber = Math.random() * (delay + 1) << 0;
        var id = window.setTimeout(action, randomnumber);
        return Rx.Disposable.Create(function () {
            window.clearTimeout(id);
        });
    },
    function (action, dueTime) {
        var id = window.setTimeout(action, dueTime);
        return Rx.Disposable.Create(function () {
            window.clearTimeout(id);
        });
    },
    function () {
        return new Date().getTime();
    });
};

Let’s show this off by taking our standard array that we’ve used so far and use our customer RandomScheduler of a maximum of 100 milliseconds.  We’ll then buffer it by either a timeout of 200 milliseconds or 4 items and then subscribe like normally.

Rx.Observable.FromArray(
        [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
        new Rx.RandomScheduler(100))
    .BufferWithTimeOrCount(200, 4)
    .Subscribe(
        function (next) {
            $.each(next, function (index, value) {
                $("<p/>").html("Buffered: " + index + "-" + value)
                    .appendTo(results);
            });
        });

We’ll see in our results that we’ll hit the timeout quite a few more times than we hit with the count, but it shows that we do indeed hit both the timeout and count buffers.

Buffered: 0-1
Buffered: 1-2 // timeout
Buffered: 0-3
Buffered: 1-4 // timeout
Buffered: 2-5
Buffered: 3-6 // count
Buffered: 0-7
Buffered: 1-8 // timeout
Buffered: 0-9
Buffered: 1-10 // timeout

And there you have it, a simple walkthrough of using buffers with the Reactive Extensions for JavaScript (and if you squint real hard, .NET as well).

Conclusion

Dealing with asynchronous programming has been in the forefront of many minds in the JavaScript community.  At JSConf, there were several examples of frameworks trying to get around the idea of callbacks and instead lean more towards composition.  By utilizing the Reactive Extensions for JavaScript, we’re able to compose together asynchronous and event-based operations together and transform the results in interesting ways.

In the reactive programming world, we could easily have situations where we have systems pushing values at us faster than we would like.  To compensate for this, we have a notion of buffering in the Reactive Extensions which allow for us to specify either count buffers, timeout buffers, or heck, even both.  This gives us some much needed flexibility on how we process the data that is coming at us.

So with that, download it, and give the team feedback!

 
Being fueled by developer passion

Tonite I came across this really nice post from Kelly (@kellabyte) on how passion drives her to continually learn and explore. She also talks about how a simple idea ends up igniting the fires and quickly spawns into putting in a lot of late nights :-) Kelly also talks about the challenge of finding an environment that lets you fulfill those desires as part of your day job rather than just in the off hours. I really appreciate Kelly’s honesty and passion, it shines through in her article.

As a side note, I can totally resonate with what Kelly’s saying. I was there for over 10 years. At each place I was trying to find my “groove”, but none of the places felt quite right. Flash forward and I can say that persistence, resilience and relentlessness finally did pay off (fortunately). Certainly there were many tradeoffs in getting where I wanted to be. And although your employer might not like this, for me it meant jumping around until I found the thing that made me happy.

It can happen….I am living proof!

Check out Kelly’s post. How does it resonate with you?

 
Weekend online course (USA edition)

I was going to do the last class on a Saturday in GMT http://dddcqrs3.eventbrite.com but a bunch of North Americaners were saying they would like a weekend version as well in their timezones. As such I put up one for Saturday that people can register for http://usacqrsddd.eventbrite.com/ 

 

The material is the same as previously just in a weekend time format.

 

Material covered includes:

  • CQRS
  • Event Sourcing
  • Building an Event Store
  • How the domain changes
  • Evolving an existing architecture
  • Testing
  • Eventual Consistency
  • Versioning

The format of this is meant to be open to the community. As such the only form of payment is donations.

 

 
Tools for disparate teams: AgileZen Edition

Still neck-deep in start-up mode these days. I still don’t feel any different than I did before we started this venture whole hog in January but that might be because I was prepared for it. Since the baby came in November, I was already familiar with the lack of sleep, persistent anxiety, and the constant mood swings between “this is the greatest feeling ever” and “what the &*%$ was I thinking?!?” so it was a smooth segue to entrepreneurship.

In any case, I’ve waxed somewhat comprehensible on the topic of working remotely fairly often. In most cases, I worked on a team where I was the only remote person. Nowadays, I’m part of a start-up where the entire dev team is scattered. And long-time readers will be horrified to hear that I’m the one in charge.

It’s been a learning process, to be sure, and continues to be. We’ve adopted procedures that I increasingly refer to as “spry” in that they help us work faster. But I’m scared to call it Agile or Lean because my knowledge of either of those processes is limited to what I read about it in the little-known Bazooka Joe comic series on the subject. So in order to avoid much “tut-tut”-ing from people on Twitter, I just say “this is what we do; if it follows a formal methodology, that’s not my fault.”

To that end, I’m going to fly directly in the face of the “favour individuals and interaction over processes and tools” part of Agile and post on a few of the tools we’ve been using to keep us organized. Or, more accurately, keep us from becoming too disorganized. In this episode:

AgileZen

When we first started our venture, my partner had everything in BaseCamp. It worked well enough. The Writeboard feature was useful and it delivers on its promise of keeping communication centralized. But we were using a task list to manage the features we wanted to put in and it very quickly became unwieldy for that. Not to knock BaseCamp. It is in the realm of software we want to emulate in that it’s great at what it does and makes no apologies if it doesn’t work outside of those boundaries.

The two obvious choices in my mind were LeanKitKanban and AgileZen. We took a look at both tools and decided on AgileZen because of its simplicity. We also took a very brief look at AxoSoft but it was deemed overkill for our needs. The pricing page alone doesn’t exactly inspire confidence in the whole “keep it simple” thing we were trying to strive for.

Likes

There’s plenty to like about AgileZen. The visual nature of it is well-suited for a remote team for daily stand-ups. Our process is generally to leave the board as-is for most of the day and move things around just at the stand-up. Helps give a sense of progress that was accomplished the previous day.

AgileZen follows a growing trend with software: less is more. This is an approach espoused by 37Signals and it’s one we’re very much on board with in our own product. So it gives us a nice warm fuzzy feeling to use a product that aligns with our own philosophy.

The overall user experience shows great attention to detail. It wasn’t until very recently, after having used the product for months, that I noticed the Add Story “form” had no labels. Instead, all the fields have watermarks indicating what to put in the field.

Because of this, the overall feature set doesn’t really give a decent picture of the application. Yes, you can colour code stories and add tags and, more recently, filter based on search criteria. None of these individually make AgileZen better than competitors. It’s more the way they all seem to work as a cohesive product.

Wish List

There is but one single feature that I think is the sole major gap in the product: a full API. There is a REST API to query things but nothing to write information back. (At least not yet; the feature is in the works.) This is something that would have *really* come in handy when we migrated all our tasks out of BaseCamp. But even now, stories come from other sources and almost all of the other products we use (e.g. BitBucket, ZenDesk) have an API so it would be pretty trivial to write a little utility to, say, take a ticket from ZenDesk and create a story from it in AgileZen. (And as I noted on Twitter, no prizes for what such a utility would be called.)

Also, as much as I marvel at the user experience, there are some minor quirks. The drag and drop isn’t infallible. There have been many cases when I just wish there was an arrow icon that I could click to move it to the next column without me having to fight with drag-n-drop, especially on my laptop where I’m forced to use the touchpad.

Couple of other annoyances are very likely low priority due to the potential number of people it affects. First, I use the Vimium Chrome extension which makes it easy to navigate a page with the keyboard. But some areas of the app aren’t “clickable” with this extension. Second, the fact that it kicks me out after I log in from another computer. I use AgileZen from at least two different virtual machines on any given day so I often have to re-login. This has been alleviated thanks to Vimium and AutoHotkey using a similar technique I used to migrate the stories out of Basecamp. Now, I just press Win+a to log in to AgileZen.

I hesitate to mention this possible feature but it’s something that would come in handy for us: threaded discussions attached to a story. The reason I hesitate is because, frankly, if I were on the AgileZen team and someone asked for this, my answer would be a flamboyant “suck it, user!”. It pretty much flies in the face of everything AgileZen is trying to achieve with its feature set. Internally, when we discuss a feature like this that would very obviously make the app overly complicated, we usually say discard the idea and say, “Yeah, there’s no need to Microsoft the app.” That’s how I predict the conversation would go with threaded discussions in AgileZen.

This list is kind of a double-edged sword because as I mentioned, any new features threatens the “keep it simple” vibe the app has at the moment. That said, Nate and Niki are smart people and I trust they will find a way to give me what I need, even if it’s not exactly what I asked for.

Wrap-up

Besides using the software, AgileZen has a certain quality that we are actively aspiring to achieve. It has the type of user experience that produces an emotional response. That response is almost universally positive so much so that even though it has a few quirks, I love using it. It’s something we use everyday and coupled with a decent screen-sharing tool (which is coming up next), it’s been a valuable tool for our remote team.

 
Advanced Code Diff from within Visual Studio

Code evolution and code maintenance are some of the most prominent characteristics of software engineering. Visual Studio doesn’t deal directly with code evolution. To explore code changes one needs to plug a Source Control Manager (like TFS) to Visual Studio. Source Control Manager can tell that a source file has been changed and it can trace text change history. But there is an impedance mismatch : Source Control Manager deals with text files while Visual Studio deals with code. Source Control Manager won’t make subtle difference between comments change, code in method refactored, type added, method visibility change or field removed.

NDepend comes with some advanced code evolution and code diff features since mid 2007. One of the coolest results of NDepend v3 integrated in Visual Studio 2010, 2008 and 2005, is that now, developer benefit from several fine-grained code diff features, one click away from their natural developing environment. Let‘s expose 3 typical code diff scenarios.

 

Code Diff from within Code Source

Since NDepend recognizes the code element currently edited in source, the user can, for instance, right click a method and see if it has been changed. If the method code has indeed been changed, several options appear on the right click NDepend menu. One of the option is Compare Older and Newer version of source file, which leads to opening a diff textual application to see changes



Here is for example,the  Araxis Merge textual diff tool opened from clicking Compare Older and newer version of source file on the NUnit v2.4.8 vs. v2.5.3 method IsPlatformSupported().



Look back in the first screenshot above, and you’ll see several other interesting options, including Open my declaration in older source code and Compare older and newer version disassembled with Reflector. With this last option (with Reflector) the language used for disassembling can be chosen (choosing IL is especially interesting). Also, comparing disassembled version can work, even from just 2 different versions of an assembly, the source code is not required.

Let’s precise that not only method changes are taken account but namespace, type and field changes as well. For example, one can right click a namespace source code declaration and ask for types or methods of the namespace that has been refactored or added.

At this point you might wonder how NDepend/Visual Studio knows about the previous version of the code to compare against to (what is called the Baseline for Comparison). This concept is explained later in this post, let’s jump to the second VS code diff scenario.

 

Code Diff from within Solution Exlorer or Solution Navigator

As we just saw, there is an NDepend menu in source code right-click menu, but there is also an NDepend menu in Solution Explorer (and in Solution Navigator) right-click menu. So for example one can right-click a project in the Solution Explorer and ask for Select Methods (of the project) where Code was Changed.





This will result in a CQL query generated that Select Methods (of the project) where Code was Changed. Naturally, for each method listed, one can jump straight to source code declaration or ask for diff.



One point interesting, is that NDepend comes with a heuristic to infer namespace from Solution Explorer folders. Indeed, it is a popular good practice to organize source code in a folders hierarchy that mirrors the namespace hierarchy. So right-clicking a folder can result in asking for code changes in a namespace, a narrowed scope than project.

 

Code Change Review through Global Code Diff

The third scenario to expose, is Global Code Diff. NDepend has its own global Visual Studio menu that comes with a sub-menu Compare.



This sub-menu Compare provides several options, including Search some particular Code Element changed. This option opens the NDepend Search panel with the option Search Change. The Search Change Panel is actually just a CQL query generator related to Code change. For example, in the screenshot below, we can see that asking for Method + Change + Code was Changed or was Added generates the CQL query:  SELECT METHODS WHERE CodeWasChanged OR WasAdded

The result shown is ideal to do efficient Code Change Review: not only all code changes are nicely organized at a glance, but for each methods refactored, the developer is just one click away to observe diff in source files.

The Search Change Panel’s options offers various possibilities to explore diff, including searching for code elements where code and only code (not comment) was changed or where comment and only comment was changed, where visibility was changed, where was added or removed etc….

A bonus option is to search diff in tier code, like asking for which library types is not used anymore for example.

Typically, developers like to write tests for testing automatically refactored and new code. Another cool bonus option is to search for diff coupled with code coverage by tests ratio, like asking for methods where code was changed (i.e refactored methods) and where it is not actually properly covered by tests.

 

Defining the Baseline for Comparison

Earlier in the post we introduced the concept of Baseline for Comparison.  This represents the previous snapshot version of the code base against which, the comparison is done. Typically, the Baseline for Comparison represents the latest version of the code in production.

The Baseline for Comparison can be specified through the menu: Visual Studio > NDepend > Compare > Define the project’s Baseline for Comparison. The dialog below appears and let chooses through different options to define the baseline. The Baseline for Comparison option is then persisted in the NDepend project file.



To harness all the diff capabilities we presented, it is important that the Baseline for Comparison points to a previous analysis result with .NET assemblies, PDB files and source code, all available somewhere on the local machine. If PDB files point to a different root folders (typically because .NET assemblies have been created on a different machine) NDepend comes with a source file rebasing option. If older version of source files is not available, it is still possible to query for what was changed, but of course, this won’t be possible to compare the missing older version of source files with the newer version. However, the option Compare older and newer version disassembled with Reflector will still be available as long as the older version of the .NET assemblies is available (no matter if PDB files and source files are absent).

Finally, any diff tools can be plugged through the menu Visual Studio > NDepend > Compare > Source Files Compare Tools (this is something explained in details in the post Diff tools).

Enjoy live code diffing in Visual Studio!

 

 
Improving Your Audio: Pop Filter Edition

In Improving Your Audio: Hardware Edition, I focused on the importance of good audio hardware. No amount of post-processing is going to turn poor raw audio into a listenable podcast/webcast/screencast. It would be like trying to print a high resolution image from a grainy scan. Sure you can interpolate pixels to clean up the graininess, but you’re not going to make detail magically appear that wasn’t in the original scan. The same is true with audio. You can clean up bad audio by removing pops and hisses, but you’re not going to make good sound magically appear from poor quality raw audio.

Pop FilterOne inexpensive piece of equipment that will save you a lot of retakes and clean-up is the pop filter. A pop filter is a thin screen of fabric that sits between you and your microphone and will set you back about $20. The pop filter is the black circle on the gooseneck in the image on right. (Image used with permission under Creative Common Attribution 2.0.) A pop filter prevents “p” and “t” sounds from making “popping” sounds in your audio. Without a pop filter, you end up with audio like this:

I’ve recorded the same phrase at the same distance from the microphone, but now with a pop filter between me and the mic:

The popping is caused by the rapid burst of air overloading the input capacity of the microphone, which results in clipping. You can see it if we look at the waveform of the raw audio.

Audio Clipping Due to Popping

Notice how the audio in the top recording is clipped where the microphone is overloaded. (+/- 1.0 is 100% input in Audacity.)

Most headset microphones like the LifeChat LX-3000 have a wind shield, which performs the same function as a pop filter. A wind shield is a fancy term for that piece of foam on the actual microphone. The main disadvantage of a wind shield over a pop filter is that wind shields “colour” the audio more. A good pop filter is acoustically neutral, which means that your audio sounds the same with and without the pop filter – it only eliminates the popping from “p” and “t” sounds. Also remember to attach your pop filter to your mic stand or boom and not directly to the microphone otherwise the microphone will pick up vibrations from the pop filter.

The moral of the story… If you’re going to spend money on good audio gear, don’t forget to buy a pop filter. The $20 it costs you will more than pay for itself in better audio quality and time saved in fewer edits and retakes.

Until next time, happy ‘casting!

 
I'm Taking a Break -- StoryTeller and StructureMap users please read

All,

I've come to realize this week that I'm in need of an extended mental holiday from the absurd level of side project work that I've been doing for the past several years.  I have some work on FubuMVC coming up as part of a Dovetail project, but otherwise I think I'll try to recharge the batteries for a while and shut down the computer at night for awhile.

 

StoryTeller

As of this moment, I'm shelving StoryTeller altogether.  The 1.0 release was admittedly rushed for an early adopter and has been problematic to get up and running out in the wild.   I know that StoryTeller needs some project automated "warmup" infrastructure and a *lot* more documentation and blogging to make it more approachable, but I'm stretched too thin to do that work right now and there frankly isn't enough interest to justify the effort.  The code is up on GitHub if anybody wants to play with it -- but frankly, don't use it unless you're willing to get into the code and contribute pull requests back in.  At this juncture, I'm recommending that folks look at other alternatives like Cucumber and its .Net clones.  I'm still saying that FitNesse is nothing but a high friction tool that you're better off without.

 

StructureMap

StructureMap is a different story because people actually use it and depend on it.  StructureMap has some known weaknesses with its nested container support, the obsolete method noise to remove, people wanting more and more convention support, and better lifecycle support that I'd like to address with the 3.0 version I've had on hold for 3 months while I concentrated on Storyteller -- plus there's the endless whining on Twitter about the documentation being out of date.  It's been on my TODO list for 6 months to convert the documentation over to the Spree guides like the ones for FubuMVC for easier maintenance, but haven't had the bandwidth.  If you have too much time on your hands, I would dearly love to have a volunteer or two to help me refresh the docs infrastructure and get more examples in place, especially for the technologies that I don't use regularly like ASP.Net MVC, WCF, EF, Linq to Sql, and WPF.  I will get back to StructureMap work before summer is over, but I'm prioritizing my own mental health for awhile first.

 

Oh, and I'm swearing off Twitter again.  Nothing but snark, pessimism, and negativity there, especially from the .Net side of the fence.  Should improve with me in a timeout anyway;)

 

I'll see y'all later when things seem like fun again,

Jeremy

 

 
Dojo Deferred and the Reactive Extensions for JavaScript

We’ve covered quite a bit in this series including how well the Reactive Extensions for JavaScript plays with other libraries and what integration points we have.  Our main thrust in providing this is that we don’t want to replace your library of choice, whether it be jQuery, Dojo, MooTools, YUI in addition to the server components of node.js.  Instead, we want to build bridges from them when you encounter the pains of asynchronous and event-based composition which frequently crop up in JavaScript applications both on the web and even on the server.  Case in point, let’s go over our latest integration with the Dojo Toolkit’s promises via the dojo.Deferred module.

Before we get started, let’s get caught up to where we are today:

 

Dojo Deferred

The Dojo dojo.Deferred module is a central piece of the Dojo Toolkit which serves as the basis for such things as AJAX calls like xhrGet among others.  At its core, it serves as a promises API which acts as a proxy for value which has not yet been calculated, and through this API, we have a clear separation of concerns which allows us to abstract the asynchronous behavior via callbacks into the promise itself, instead of the method signature.  The same could be said of the Reactive Extensions for JavaScript as well. 

Although dojo.Deferred has been around for a while, the newest release of the Dojo Toolkit has added another method to handle both the success and failure cases of a given computation, such as the following code.  Imagine we had an asynchronous function which returns to us a dojo.Deferred object, which does nothing more than delay by a second and return the value.

function someDeferredFunction() {
    var deferred = new dojo.Deferred();
    setTimeout(
        function() {
            deferred.callback(42); 
        }, 1000);
    return deferred;    
}

Then we can subscribe to both its success and failure cases by using the then function.

var deferred1 = someDeferredFunction();
deferred1.then(
    // Success
    function(value) {
        alert(value);
    },
    // Failure
    function(error) {
        alert(error);
    });

We can see a bit of overlap between the ideas here and the heart of the Reactive Extensions for JavaScript.  The difference between the two is that this API treats asynchronous calls as one shot resumable methods whereas RxJS treats these as observable sequences.  So, how could we take advantage of this API and build a bridge between the two libraries?

From Deferred to Observable

The first order of business is to see how we can take a given dojo.Deferred and turn it into an Observable sequence yielding a single value.   Using the Reactive Extensions for JavaScript with the Dojo support added via the “rx.dojo.js” script, we can seamless leap between the two by calling the asObservable prototype method we added.

var deferred2 = someDeferredFunction();
deferred2.asObservable()
    .Select(function(value) { return value * value; })
    .Subscribe(
        function(value) {
            alert(value);
        });

In this example, we took the someDeferredFunction from above and then called the asObservable method, then we’re free to do such things as projections via Select in which we square the returned value and then subscribe to the result via Subscribe.  To show you how we did it, let’s walk through the code below.  First, we create an AsyncSubject which represents and asynchronous operation which accepts only one value and then caches it for all future observations.  Then we can call the then function and in the success case, we call OnNext with the result and OnCompleted to indicate we are finished, and in the failure case, we send the error on via OnError.  FInally we return the subject, thus completing our bridge.

dojo.Deferred.prototype.asObservable = function () {
    var subject = new Rx.AsyncSubject();
    
    this.then(
        // Success
        function (value) {
            subject.OnNext(value);
            subject.OnCompleted();
        },
        // Failure
        function (error) {
            subject.OnError(error);
        });
        
    return subject;
};

But, what about the other way around?  Can we go from an Observable sequence to a dojo.Deferred?

From Observable to Deferred

Because we feel like building bridges, we want to give the ability for traffic to go both ways.  For example, we could have a given AsyncSubject that we want to use with our existing Dojo functionality.  Imagine we had an Observable sequence that waits for a second and then yields the value via an AsyncSubject.

function someAsyncSubjectFunction() {
    var subject = new Rx.AsyncSubject();
    
    setTimeout(
        function() {
            subject.OnNext(42);
            subject.OnCompleted();
        },
        1000);
        
    return subject;
}

Now, we might have some processing logic written in Dojo, so we want you to take full advantage of that.  Via the AsDeferred method provided to us through the “rx.dojo.js” script, we build that bridge between the two libraries.  After calling AsDeferred, we can then use it as a regular dojo.Deferred object.  So, now we can call then passing in the two continuations for the success and failure cases.

var subject = someAsyncSubjectFunction();
subject
    .AsDeferred()
    .then(
        function (value) {
            alert(value);
        },
        function (error) {
            alert(error);
        });

Now, how did we make this happen?  We’ll need to extend the AsyncSubject with our AsDeferred method.  Inside, we’ll create a new dojo.Deferred which will handle the values from our AsyncSubject.  Next, we’ll subscribe to our subject via the Subscribe method, calling the callback method with our result on success, and errback on the failure.  Finally, we return the deferred which completes our bridge.

Rx.AsyncSubject.prototype.AsDeferred = function () {
   var deferred = new dojo.Deferred();
   
    this.Subscribe(
        function (value) {
            deferred.callback(value);
        },
        function (error) {
            deferred.errback(error);
        });
        
    return deferred;
}

And there you have it, our bridge between the Dojo Toolkit and the Reactive Extensions for JavaScript, which allows you to have the best of both worlds.

Conclusion

Dealing with asynchronous programming has been in the forefront of many minds in the JavaScript community.  At JSConf, there were several examples of frameworks trying to get around the idea of callbacks and instead lean more towards composition.  By utilizing the Reactive Extensions for JavaScript, we’re able to compose together asynchronous and event-based operations together and transform the results in interesting ways.

The Reactive Extensions isn’t meant to be the only solution in your toolkit, but instead you can continue to use the JavaScript library of your choice, and when you run into callback hell and composition issues, that’s where RxJS shines.  To that end, we’ve provided a lot of bridges between various libraries such as the following, and more to come:

So with that, download it, and give the team feedback!

 
Idempotency vs Distibuted Transactions

This post comes from a discussion I had with Clemens Vasters at NDC but is a much bigger discussion than the context we had it in which was Azure specific. An exaple of where this also comes into play is with the REST crowd who also want all idempotent operations.

 

Sure distributed transactions are evil ... until well they aren't. There is a trade off between capital investment and *insert whatever reason you don't like distributed transactions*. First let's look at the alternative.

 

To briefly cover idempotency from Gregor

 

"In the world of distributed systems, idempotency translates to the fact that an operation can be invoked repeatedly without changing the result."

 

This is a huge benefit because in failure conditions etc, I can just send you the same message multiple times and it won't affect you in a bad way. As an example I peek a queue and process the message writing something. Then suddenly my power goes out before I have actually removed the item off the queue. When I start up, I can just re-process the message (it won't cause destructive change).

 

Many operations are idempotent to start with. Consider a read or the setting of a property. Other options are not idempotent and need code to be written to make them idempotent as an example charging someone's credit card what happens if you get that message twice, ouch. The trick here is the operations that are not naturally idempotent, they require us to write code to make them idempotent. That code has a cost associated with it in terms of creation, maintenance, testing, and conceptually.

 

Clemens gave an example of a system he worked on to illustrate his point, a credit card processing system.

 

A credit card processing system has a very limited number of possible transaction types. For sake of discussion let's say that there are 15 and they are called very often, each has a large amount of logic associated with each transaction type (a similar example might be a trading system, not a lot of command types but a lot of logic associated with each). In these cases the idempotency is not a big deal because the amount of code being added is very small for it as a percentage of the total code.

 

This is very different than a stereotypical business application where there will orders of magnitude more transaction types and they will be often called rarely with small amounts of logic behind them ... for sake of discussion let's say closer to 1500 types with the detection code being vastly larger as a percentage (I have seen business systems where there was more code to ensure the idempotency than to perform the actual operation).

 

We can make everything idempotent but we will have a very different cost associated with it in the stereotypical business system than with the first system. Out of our 1500 operations how many of them are not naturally idempotent? How much code will we have to write in order to make them idempotent (quite likely ALOT!). Looked at as a percentage of the overall code it will be much higher than in the first example as there is much less code per transaction type on average.

 

Now what are we gaining by this overhead in order to have all idempotent operations? Scalability and Availability are two things that come immediately to mind. It is very hard to scale distributed transactions to an extremely high level (think Azure scale), many other trade offs start to come in at this level of scaling.

 

Are you scaling to Azure level? What is the ROI of that code you wrote to make things idempotent for the business system? This is not to say to use distributed transactions haphazardly (you always want to limit their usage as much as possible) but to point out that there are certainly situations where they may be favorable, there is a tradeoff involved. Again this is not to pick on Azure alone, REST also does this.

 

It is important to consider this tradeoff when you decide to say put an application onto Azure. How well do their non-functional requirements as a platform meet yours as an application? What types of decisions like this have they forced upon you and what will be the long term cost of those decisions.

 
Eventual Consistency and Set Validation

I was just writing an email to the cqrs group http://groups.google.com/group/dddcqrs and figured it might be useful to put it up here as well as its a very common question I get.

The initial question was

Here's the example I'm using:
A system that handles user registration for 2 million+ active users. These users should be able to login with their email address and password. They should also be able to change their associated email.

I have the following design: 

Entities:
User(Guid Id, string email, string hashedPassword)

Events:
UserRegistered(Guid Id, string email, string hashedPassword)
UserEmailAddressUpdated(Guid Id, string email)

Commands:
RegisterUser(Guid Id, string email, string hashedPassword)
UpdateEmailAddressForUser(Guid Id, string email)

ViewCaches:
RegisteredEmailAddresses(emailAddress) - Used for client side validation on email prior to sending a RegisterUser command

When processing a RegisterUser command, I need to validate that no other user has registered with that email. How can I do that without loading every user in the system? I could use a view cache like the client side, but then I would have business logic outside of my domain. Any suggestions?

 

This is a very common question. There were many responses with various suggestions. Mine follows.

 

I am just replying to the last one on the list after reading through.

To me the most important concepts have been completely missed in this
thread and they are a big part of why eventual consistency is so cool
(it makes you think about things).

*What is the business impact of having a failure*

This is the key question we need to ask and it will drive our solution
in how to handle this issue as we have many choices of varying degrees
of difficulty.

Most of the time the business impact of such a failure is low and the
probability of it happening is low. If we query the eventually
consistent store at the time of submission (either from client or from
server as this is a big part of how one-way commands work) then our
probability of receiving a duplication is directly calculable based on
the amount of eventual consistency. We can drive this probability down
by lowering our SLA very often this is enough.

We can detect asynchronously if we broke our invariant. Imagine an
eventhandler that inserts into a table with a constraint. If it gets
an exception, we broke the constraint (note this is not really the
"read model" but the same db can be used if convenient, it is
important to note the distinction as if we scale to have 5 read models
we don't have 5 of these ...).

What do we do if we break the constraint? We need to come back to that
business impact statement above. For most circumstances, just raising
an alert to an admin etc is enough, these things are very low
probability of happening and are often not worth the time/cost of
implementing automatic recovery. Just imagine 1 username create out of
1,000,000 fails this way. How long would it take to automate the
process of handling the situation? Consider discussions with domain
experts etc. 5 minutes of admin time once a year is much better ROI in
most of these situations than a week of developer time to automate.

Continuing along it has now been decided that this has large enough
impact that it should be automated. The said process that finds the
duplicates could either raise an event DuplicateUsernameDetected or
directly call a command ResolveDuplicateUsername (which involves more
discussion). It is important to note that in either of these cases we
are discussing the "What" not the "How" it would never issue a command
"DeleteUser" etc, how to handle these situations is core domain logic
and should be modeled within the domain. In the username example
perhaps ResolveDuplicateUsername marks the user as not being able to
login (and as a duplicate) and it sends an email to the user saying
"Hey we screwed up but its your lucky day! you get to create a new
username ..."

But even after all of this if from a business perspective the impact
is too high we can still make things consistent. We could drop in a
service to the domain that deals with a consistent set. This would of
course be the last resort as its the most complicated of these
solutions and brings with it many limiting factors in terms of our
architecture.

Udi had a great example of this in his explanation of 1-way commands.
It was an ATM that would spit out money having only read your balance
from an eventually consistent read model. The reason this can work is
that from a business perspective the risk is low (and it is built into
the business model itself). You have a bank account with me, I know
your SS# and all of your information. For people who overdraw their
accounts I will recover atleast 90% of the money that has been
overdrawn. On top of that I charge a fee for each overdraw that
occurs. For these reasons the business impact of such a problem is
low.

To sum up I just want to reiterate that this is a *good* thing.
Eventual consistency is forcing us to learn more about our domain. It
is forcing us to ask questions that are otherwise often not asked.

Consistency is over-rated.

 

 
Exploring FubuCore: Convert a string value to any type of .Net class with the ObjectConverter

Several months ago the Dovetail team realized that our various OSS projects and our main codebase were starting to show a lot of duplication of utilities and extension methods. At one time ReSharper could find 6 different versions of a Cache<TKey, TValue> utility that we had originally written for what became Fluent NHibernate and at least 4-5 versions of the pesky IEnumerable<T>.Each() extension method that we all add to our new projects on day 1 (because some ivory tower guy at Microsoft had a dogmatic objection to that very extension and refused to put it into the BCL where it belongs).  Worse yet was the fact that I was finding myself needing to fix certain extension methods across several different projects.  The obvious answer was to consolidate many of our commonly used utilities and extension methods into a common library which we called “FubuCore” in the greater FubuMVC project.  I think there’s some potentially useful stuff in there that other teams could use or contribute to, so here’s to the start of a new series on the FubuCore library starting with the sleek, sexy ObjectConverter class that I use heavily in StoryTeller and our Dovetail code:

    public interface IObjectConverter
    {
        // Given a string and a .Net type, read this string
        // and give me back a corresponding instance of that
        // type
        object FromString(string stringValue, Type type);
        T FromString<T>(string stringValue);


        bool CanBeParsed(Type type);
    }

The real class behind this deceptively simple interface can take a string value and return to you an instance of the type you requested.  Pretty awesomely boring, right?  The TypeDescriptor and TypeConverter classes built into the BCL already does some of this (and by default ObjectConverter delegates to TypeDescriptor to handle most simple types), but they’re somewhat clunky to use.  No, the cool part about the underlying ObjectConverter object is that you can teach it how to convert a string to an entirely new type that you just created.  I always thought the old fashioned class oriented approach that FitNesse took was clumsy and high ceremony, so I decided to just register a Lambda of type Func<string, T> for a given T like this method on ObjectConverter:

        public void RegisterConverter<T>(Func<string, T> finder)
        {
            _froms[typeof(T)] = x => finder(x);
        }

The value of the Lambda approach is that you can add a small strategy to ObjectConverter inline without going to the trouble of defining another class that’ll live somewhere off to the side.

 

Putting ObjectConverter to Work

Let’s say that you have a value type in your system called Contact like this:

    public class Contact
    {
        public string FirstName { get; set; }
        public string LastName { get; set; }
    }

And you want to represent this in string form as “FirstName LastName” like “Jeremy Miller.”  You can quite happily teach ObjectConverter how to convert a string to a Contact object with the code from this unit test:

        [Test]
        public void register_and_retrieve_a_new_type_of_complex_object()
        {
            var finder = new ObjectConverter();
            finder.RegisterConverter<Contact>(text =>
            {
                var parts = text.Split(' ');
                return new Contact(){
                    FirstName = parts[0],
                    LastName = parts[1]
                };
            });

            var c = finder.FromString<Contact>("Jeremy Miller");

            c.FirstName.ShouldEqual("Jeremy");
            c.LastName.ShouldEqual("Miller");
        }

Simple enough, eh?  Now, how about if we want to get an array (or IEnumerable) of Contact objects from a string?  No problem, by default ObjectConverter can take a comma delimited string and convert each value into a Contact like this:

        [Test]
        public void how_about_getting_an_array_of_those_complex_objects()
        {
            // Same converter as before
            var finder = new ObjectConverter();
            finder.RegisterConverter<Contact>(text =>
            {
                var parts = text.Split(' ');
                return new Contact()
                {
                    FirstName = parts[0],
                    LastName = parts[1]
                };
            });

            // Now, let's pull an array of Contact's
            var contacts =
                finder.FromString<Contact[]>("Jeremy Miller, Rod Paddock, Chad Myers");
       
            contacts.Select(x => x.LastName)
                .ShouldHaveTheSameElementsAs("Miller", "Paddock", "Myers");
        }

Look at that, once I defined how to find a Contact from a string, ObjectConverter now knows how to handle an array (or IEnumerable / IList / List) of Contact’s as well.  That little feature has been very helpful to me in creating better StoryTeller DSL’s.

 

“Finding” Objects

I originally wrote ObjectConverter (née ObjectFinder in older versions) as a utility internal to the new StoryTeller testing engine.  I knew from my prior experiences with FitNesse that I’d need to frequently take test input data that would be stored as strings and convert that string to the real .Net types that the test code would need to do its job.  I also knew that I’d need to allow users to “teach” StoryTeller how to convert (or find) new types of objects from the domain of the system under test so that StoryTeller could take a set of key/value pairs of strings and turn that into the correct arguments to call a method like this one from Dovetail test harness code:

        // 'User' and 'Site' are both Entity types
        // in our system
        [FormatAs("User {user} is at Site {site}")]
        public void UserIsAtSite(User user, Site site)
        {
            user.Site = site;
            _system.Save(user);
        }

In the StoryTeller tests we can just specify the user name and the site name, but first we had to teach ObjectConverter how to find a user and site from a string with a function that:

  1. First tries to look up an existing User by the specified user name, and if it exists, return they existing user object
  2. If the user doesn’t already exist, create a new User with that user name and some default data, save it, and return the new User object

Using ObjectConverter within StoryTeller tests like this has done a world of good for us by making test input setup less laborious and making our tests much easier to understand and read because there’s less “noise” in the “Arrange” part of our StoryTeller tests.

 

 

Family of Types in One Swath

Instead of explicitly telling ObjectConverter how to deal with each specific type, you might be able to make your life easier by registering a single IObjectConverterFamily policy that can tell the ObjectConverter how to deal with several types at once.  Let’s just look at the interface:

    public interface IObjectConverterFamily
    {
        // ObjectConverter calls this method on unknown types
        // to ask an IObjectConverterFamily if it "knows" how
        // to convert a string into the given type
        bool Matches(Type type, IObjectConverter converter);

        // If Matches() returns true for a given type,
        // ObjectConverter asks this IObjectConverterFamily
        // for a converter Lambda and calls its
        // RegisterFinder() method behind the scenes to cache
        // the Lambda for later usage
        Func<string, object> CreateConverter(Type type, Cache<Type, Func<string, object>> converters);
    }

Deep in the guts of our proprietary (sounds special if you say it like that) rules engine we frequently need to reference an entity as part of a Json blob and find the correct entity later (think of rules like “send this case over to the urgent queue” where the Queue is an Entity in our Domain Model).  Like many projects we use a Layer Supertype class for our Entity objects for all the basic properties like Id.  Since all our Entities are identified the same way, we can tell an instance of ObjectConverter to convert a string to any type that inherits from Entity by looking up that entity in our Repository by Id in this code:

    public class DomainEntityConverterFamily : IObjectConverterFamily
    {
        private readonly IRepository _repository;

        public DomainEntityConverterFamily(IRepository repository)
        {
            _repository = repository;
        }

        // Matches any type deriving from DomainEntity
        // CanBeCastTo<> is an extension method in FubuCore as well
        public bool Matches(Type type, IObjectConverter converter)
        {
            return type.CanBeCastTo<DomainEntity>();
        }

        // In this case we find the correct object by looking it up by Id
        // from our repository
        public Func<string, object> CreateConverter(Type type, Cache<Type, Func<string, object>> converters)
        {
            return text =>
            {
                var guid = new Guid(text);
                return _repository.Find(type, guid);
            };
        }
    }

I can register the conversion family by just calling the RegisterConverterFamily<T>() method on ObjectConverter.

What else can ObjectConverter do?

Rather than make this blog post longer than its already absurd length, I’d invite you to simply look at the unit test class for ObjectConverter here.  If you’re interested in that sort of thing, I’d put ObjectConverter forward as a somewhat advanced example of using first class functions as the primary unit of composition.

 
FubuMVC at C4MVC Tomorrow

It's not up on the event page, but I'm penciled in to give an online brown bag talk on FubuMVC tomorrow at the C4MVC meeting at 12PM CDT.  I'm trying not to overlap too much with my earlier talk from the MVCConf last month.

This time out I'm going to talk about:

  • Technical Differences between FubuMVC and MSMVC, and why they matter
  • How FubuMVC's configuration model enables a lot of new scenarios and reduces common errors we see described in blog posts about MSMVC
  • Diagnostics -- You guys using ASP.Net MVC really, really wish you had this
  • "Global Filters" the Fubu way
  • Url resolution and Route determination
  • IoC and Composition are like Peanut Butter and Jelly
  • Swap out virtually everything
  • Modular Model Binding -- teach your model binding new tricks
  • "POCO" Controllers and why they're valuable.  No hokey Controller base class required.
  • How leveraging an IoC tool for composition and object lifecycle management simplifies the internals of Fubu compared to ASP.Net MVC.  I.e., why I think the ASP.Net team is missing the boat with the supposed "IoC friendliness" in MVC3.
  • Extensibility scenarios
  • Html Conventions the Fubu way (Time permitting)
 
IronRuby and the Reactive Extensions Together Again – Taming User Input

In the previous post, I talked about how IronRuby 1.1 now supports extension methods, and that it not only supports LINQ to objects, but with relative ease it also supports the Reactive Extensions for .NET.  We covered a little example of taking a Ruby Enumerable and turn it into an .NET Observable instance and then projecting and filtering the data.  In this post, we’ll go a little bit further by showing how you can also use events as well and how we can tame user input.

What Does the Future Hold?

Before we continue this series, I thought I’d address the elephant in the room in terms of what the future holds for IronRuby.  Jimmy Schementi, the former PM of the IronRuby project laid out what has been going on in his blog and the IronRuby core mailing list (which if you want to see it stick around, join the community).  Keep in mind that at the time of this writing, no decision has been made one way or the other as to its future.  My hope is that the community does indeed stand up and either continue on IronRuby, or a bridge effort such as RubyCLR as they are both valuable things to have.

Either way, this blog series, however many posts it may be, will continue as I think it might have some value to some folks out there.  So with that, onto today’s subject.

Taming User Input

In this post, I’m going to take an example we had from the Reactive Extensions for .NET Hands on Labs and apply it to a version using IronRuby and WPF.  The idea behind this example is that calls to such things as HTTP calls such as Web Services, REST and so forth can be expensive.  Imagine if you were a really fast typer at over 100 words per minute, and then having each new character cause a new request to be sent.  Chances are you’d overload the system, and not only that, but also have a chance of the results coming back in out of order.  So, how do we fix that?  By using the Reactive Extensions for .NET together with IronRuby, we can make this scenario work quite nicely.

Removing the Boilerplate

Getting started, we’ll need to get some boilerplate out of the way including referencing the proper assemblies for a .NET 4 solution.  In order for us to do so, let’s move the assembly includes to a separate file.  Since we’re going to be using WPF and the Reactive Extensions, we’ll create two separate files, rxnet.rb and wpf.rb.  The rxnet.rb contains all specific assemblies related to the Reactive Extensions such as the following:

load_assembly 'System.CoreEx'
load_assembly 'System.Interactive'
load_assembly 'System.Reactive'

And our wpf.rb contains the following assemblies:

load_assembly 'WindowsBase'
load_assembly 'PresentationCore'
load_assembly 'PresentationFramework'
load_assembly 'System.Xaml'
load_assembly 'System.CoreEx'
load_assembly 'System.Reactive'

Now we can make this available any number of ways, either through a quick gem, or just copying these files directly to the lib folder for IronRuby.  Once we’ve decided on a given path, then we could reference all we need by the following for this post:

require 'System.Core'
require 'wpf'
require 'rxnet'

Now that our boilerplate is out of the way, let’s get on to the solution.

Starting the Solution

Once we have finished that, we are free to include the namespaces and the extension method namespaces which include both System and System::Linq namespaces.  The IObservable extensions for the standard operators reside in System::Linq, and the overloads for the IObservable<T>.subscribe reside in the System namespace.

include System
include System::Linq
include System::Windows
include System::Windows::Controls

using_clr_extensions System
using_clr_extensions System::Linq

Now, let’s create a standard WPF Window class with a StackPanel which contains two elements, our TextBox for entering data and a ListBox for showing the results. 

class RubyWindow < Window

    def initialize
        self.Title = 'Hello Ruby!'
        
        stackpanel = StackPanel.new
        @textbox = TextBox.new
        @listbox = ListBox.new
        stackpanel.children.add(@textbox)
        stackpanel.children.add(@listbox)
        self.content = stackpanel
        
        initialize_handlers
    end

end

Once we’ve done this, now let’s get to the interesting part.  After setting the content of our Window to the StackPanel, let’s hook up the to the TextChanged event of our TextBox by calling Observable.from_event generic method (FromEvent) with the arguments of TextChangedEventArgs. 

def initialize_handlers

    @textbox_changed = Observable.
        method(:from_event).
        of(TextChangedEventArgs).
        call(@textbox, 'TextChanged')

end

This is all fine and interesting, but the TextChangedEventArgs doesn’t actually give us the text of our TextBox, so, let’s fix that to call select, passing in our IEvent<TextChangedEventArgs> argument and then retrieving the text property from the sender, which in this case was the TextBox.  In order for us to call generic methods using IronRuby, we need to make use of the following pattern in which we specify the method we want to call, of what type and then we can invoke it via the call method.

some_object.
    method(:some_method).
    of(SomeType).
    call(some_args)

To see this in action, we’ll apply the same pattern to call the from_event method of type TextChangedEventArgs, and then we invoke via call with our @textbox and the ‘TextChanged’ event.

def initialize_handlers

    @textbox_changed = Observable.
        method(:from_event).
        of(TextChangedEventArgs).
        call(@textbox, 'TextChanged').
        select(lambda { |event| event.sender.text })        

end

One distinct advantage we have here while using a dynamic language such as Ruby, we’re able to get the text from the sender without having to cast.  For example, using C#, we’d have to do the following:

var textChanged = 
    Observable.FromEvent<TextChangedEventArgs>(textbox, "TextChanged")
        .Select(ev => ((TextBox)sender).Text);

We can subscribe to the resulting observable and then have the items added to the ListBox.

def initialize_handlers

    @textbox_changed = Observable.
        method(:from_event).
        of(TextChangedEventArgs).
        call(@textbox, 'TextChanged').
        select(lambda { |event| event.sender.text })        

    @textbox_changed.
        subscribe(lambda {|text| @listbox.items.add(text))

end

This is great, but as you may notice, all of our input is immediately put into the below ListBox.  This may or may not be a problem, but imagine we’re calling an external service, then ultimately it’d be a huge issue to have hundreds of calls going out at once because we’re fast typers.  Let’s fix that by using the throttle method (Throttle) which allows us to specify a timeout time before the event is fired.  If there are no other events that happen in the specified timeframe, then we get the result, else we keep waiting until that timeout.  So, let’s now throttle our input.

def initialize_handlers

    @textbox_changed = Observable.
        method(:from_event).
        of(TextChangedEventArgs).
        call(@textbox, 'TextChanged').
        select(lambda { |event| event.sender.text })        

    @textbox_changed.
        throttle(TimeSpan.from_milliseconds(500))
        subscribe(lambda {|text| @listbox.items.add(text))

end

Now we can run it and watch the results, and it should just work, right?  Not so much…

image

The problem is that once we start throttling our input, we’re launching a new thread to make this happen.  This newly spawned thread cannot access the object to modify the items.  Instead, we have to think about Schedulers, which I’ll cover all of them in a later post, but sufficed to say, we have some options.  The easiest of which is the observe_on_dispatcher method (ObserveOnDispatcher) which allows us to observe this sequence on the control’s dispatcher.  Let’s change the code to fix this issue:

def initialize_handlers

    @textbox_changed = Observable.
        method(:from_event).
        of(TextChangedEventArgs).
        call(@textbox, 'TextChanged').
        select(lambda { |event| event.sender.text })        

    @textbox_changed.
        throttle(TimeSpan.from_milliseconds(500)).
        observe_on_dispatcher.
        subscribe(lambda {|text| @listbox.items.add(text))

end

And now we can start typing, and sure enough our text is throttled, including copying the sentence and pasting it.

image

But you’ll notice that we now have a duplicate because of the copy/paste operation.  What we really want is to have distinct values only in our ListBox.  To make that happen, we’ll need to use the distinct_until_changed method (DistinctUntilChanged) which then eliminates those duplicates.

def initialize_handlers

    @textbox_changed = Observable.
        method(:from_event).
        of(TextChangedEventArgs).
        call(@textbox, 'TextChanged').
        select(lambda { |event| event.sender.text })        

    @textbox_changed.
        throttle(TimeSpan.from_milliseconds(500)).
        observe_on_dispatcher.
        distinct_until_changed.
        subscribe(lambda {|text| @listbox.items.add(text))

end

And we can run the same experiment and sure enough our result is throttled and we receive no duplicates from item to item.

image

And there you have it, a simple case of taming user input, all with the use of IronRuby and the Reactive Extensions for .NET.

Conclusion

With the Reactive Extensions for .NET, we have a nice cross-language support for building a bridge to asynchronous and event-based programming.  Coming with the latest release of IronRuby, we now have the ability, and a rather nice one at that, to consume extension methods that are predefined in other languages, which opens a whole new set of opportunities for interoperability.  In the coming posts, I’ll take a look at what else I can do in this space to see that we can fact help solve some of the hardest problems around asynchronous and event-based programming.

As for the future of IronRuby, best to get involved and help where you can if you want to see it be a success!

So with that, download it, and give the team feedback!

 
Improving Your Audio: Hardware Edition

[ASIDE: Community Server (our current blogging engine here at CodeBetter.com) is swallowing the HTML5 <audio> tags in this post along with the Flash Player fallback. To hear HTML5 audio in your browser, jump to here with a browser that supports HTML5 audio such as FireFox 3.5+, Chrome, Safari, Opera, or IE9. I've successfully tested it in FireFox 4.0b2 and Chrome 6. I've also successfully tested it in IE8 where it falls back to the Flash Player. Sorry for the hassle.]

[ASIDE Part Deux: Brendan has tweaked the Community Server settings to allow CodeBetter to serve up <audio> tags. So the audio below should work now. Thanks, Brendan!]

Over the years, I’ve done a lot of audio work between podcasts, screencasts, and webcasts. So I know a thing or two about computer audio. I don’t claim to be an expert like my friends Carl Franklin or Richard Campbell, but I’ve done enough to be able to offer some helpful tips. We’re going to start with the hardware.

The quality of your computer audio can only be as good as the raw captured product. Use a bad microphone and no amount of software cleanup is going to magically produce good audio. You might be wondering how much difference the hardware can make? I’ve recorded the same audio track using four (4) different microphones on the same computer. (I didn’t record them simultaneously as multi-track recording is notoriously difficult, but I did say the same phrase into each microphone on after the other.) Let’s start with the LifeChat ZX-6000.

LifeChat ZX-6000

My voice sounds like I’m on a telephone. The sound is hollow and lacks depth. If we plot a frequency analysis using Audacity, we can easily see the problems.

LifeChat ZX-6000 Frequency Spectrum

OK, maybe not easily if you’re not familiar with audio. Let me explain some basic ideas and then you should be able to see the problems.

Normal human hearing discerns frequencies between 20 Hz and 20 kHz. The standard tuning note for musicians is the 440 Hz, which is an A above middle C on the piano. The lowest note on the piano (A0) is 27.5 Hz and the highest note (C8) is 4186 Hz. (I’m using the example of a piano since many people, even non-musicians, have at least played with a piano at one time or another.) Lower frequencies correspond to lower notes and higher frequencies to higher notes. The frequencies mentioned are the fundamental frequencies. When you play an A4 on the piano (or any other instrument including the human voice), the major frequency is 440 Hz, but there are many harmonics or overtones that occur. These harmonics give a colour and depth to the sound. This is one of the reasons why different instruments sound vastly different when playing the same note – the harmonics produced by each instrument are quite different. This is how we perceive different ranges of audio frequencies. (Taken from Wikipedia Audio Frequency.)

Frequency (Hz) Octave Description
16 to 32 1st The human threshold of feeling, and the lowest pedal notes of a pipe organ.
32 to 512 2nd to 5th Rhythm frequencies, where the lower and upper bass notes lie.
512 to 2048 6th to 7th Defines human speech intelligibility, gives a horn-like or tinny quality to sound.
2048 to 8192 8th to 9th Gives presence to speech, where labial and fricative sounds lie.
8192 to 16384 10th Brilliance, the sounds of bells and the ringing of cymbals. In speech, the sound of the letter "S" (8000-11000 Hz)

Note how 2048 to 8192 Hz gives a presence to speech, whereas 8192 to 16384 give a brilliance. Without these frequencies present, speech will sound hollow.

With this in mind, let’s take another look at the frequency spectrum from the LifeChat ZX-6000. We see virtually no frequencies above 4000 Hz, which is making my voice sound hollow. Old analog telephones transmitted 200 Hz to 3000 Hz, which is why it sounds like I’m talking on an old phone. You’ll also note that the lower frequencies (below 400 Hz) are attenuated (e.g. not as pronounced), which is why the sound is lacking some of the bass timbre of my voice.

Let’s try a different microphone and see how it performs… Next up the LifeChat LX-3000.

LifeChat LX-3000

The audio quality is vastly improved. Let’s take a look at the frequency spectrum.

LifeChat LX-3000 Frequency Spectrum

You can visibly see the difference. We have frequency response all the way up to 20 kHz with the majority of the response in the lower frequencies, which is expected due to the timbre of my voice. The lower frequencies are also not as attenuated. The quality of the sound is much warmer and vibrant with the LX-3000 than the ZX-6000.

As our last point of comparison, let’s listen to a semi-pro microphone – the one I use for my recording work – the audio-technica AT2020.

audio-technica AT2020

The differences are subtler this time, but still noticeable. The audio has more depth and presence than with the LX-3000. Let’s take a look at the frequency spectrum.

audio-technica AT2020 Frequency Spectrum

Notice the better bass response below 400 Hz giving a truer rendering of my low voice. We also have better harmonics in the 10 to 20 kHz range, providing a more life-like sound. We can also take a look at the frequency response of the microphone, which can be found on the manufacturer’s website here.

audio-technica AT2020 Frequency Response

Note the flat response curve across the entire range of frequencies. This means that the microphone records all frequencies with equal efficiency, which results in little distortion of the raw sound. For comparison, I would expect the response curve for the ZX-6000 to drop to virtually zero above 4 kHz and show attenuation below 400 Hz. You want a flat response curve for your microphone as it will not colour or distort the recorded audio.

I should note that both the LifeChat LX-3000 and ZX-6000 have hardware noise cancellation. (Noise cancellation will remove an annoying background hum originating from fans, pumps, and other sources of low background noise. It can’t do anything to clean up dogs barking, children screaming, or other sudden noises that disrupt your recording sessions.) Applying software noise cancellation on either of these microphones has little additional benefit. The audio-technica AT2020 does not have hardware noise cancellation and benefits from applying software noise cancellation. Assuming you are working in a quiet environment the audio quality of the AT2020 without noise cancellation is still better than the LX-3000 and far superior with noise cancellation. Software noise cancellation usually involves little more than selecting a checkbox in programs like TechSmith Camtasia Studio or similar recording packages. You can perform noise removal using Audacity too, though it’s a bit more work as you have to manually select a quiet region with just the background noise that you want to subtract.

The LX-3000 is a great microphone for conference calls and gaming. It is a good, though not great, microphone for recording podcasts/screencasts/webcasts. It is inexpensive ($30 to $50), easy to use, and can be bought at most computer stores. If you’re just getting started, this is a good microphone to buy.

If you’re looking to take your audio to the next level, the audio-technica AT2020 is a great semi-pro microphone that you can pick up at reasonable cost. You’ll have to go to an audio specialty store as you won’t find these in your regular computer stores. I purchased mine at Long & McQuade, which is a chain of well-respected Canadian musical instrument stores. Now what is a reasonable cost? You’ll need more than just a microphone. You’ll also need a pre-amp to power the microphone as semi-pro and pro microphones don’t have high enough output to jack directly into your computer microphone port. You’ll need a pop filter (which prevents “p” and “t” sounds from making “popping” sounds in your audio), a mic stand, an XLR cable for mic to preamp, and a 1/4” to 1/4” male cable for preamp to computer (or 1/4” to 1/8” male if you are using a normal mic-in on your computer).

Component Price*
audio-technica AT2020 $120
ART TubeMP Tube Mic Preamp $49
Pop Filter $20
Mic Stand $20
2 Cables (XLR & 1/4”-1/4”) $20
Total $229

* Prices are in Canadian dollars.

You can get the same microphone (AT2020) with a USB option, but at a higher cost of $170, which is basically the cost of the preamp and cables. The TubeMP preamp has an actual vacuum tube that gives a warmth to the sound that is hard to achieve otherwise. Given the similar costs, I would personally err on the side of using a tube preamp over USB.

You might want to invest in a decent sound card, such as a Creative Labs X-Fi Platinum or similar card, which has better audio recording qualities than the audio-in that comes on your motherboard. It’s hard to find the X-Fi cards anymore. So you’ll have to look around to find a good quality audio card, but expect to spend $100 to $200 on the audio card alone. Remember your audio is going to be no better than the weakest link in the chain.

Is $229 of the audio-technica AT2020 worth the improved audio over the $30 to $50 LifeChat LX-3000? That’s up to you to decide.

 
Upcoming online classes

As I mentioned previously I will only do another 2 online classes. I have to admit talking to the computer screen is too weird for me but I find it a good exercise to get the material better suited for the medium.

The next will be Monday (sorry for late notice guys) 9/8/2010. I think this will be a small group which is always better. dddcqrs2.eventbrite.com

The last one will be 4/9/2010 as many people asked me for a weekend class. This will be done in EU time as I will be in Oslo dddcqrs3.eventbrite.com. If there is a ton of interest I would consider doing the same weekend in US time but I would prefer to just have it be the last course.

 

The courses are by donation only as I hope them to be open to the community. I would much prefer if eventbrite supported donations after the fact 

 
IronRuby and the Reactive Extensions for .NET Together at Last

Recently, there was a release of IronRuby 1.1, which had a number of new features including targeting .NET 4 only, as well as the most interesting part, the support for Extension Methods.  Taking a cue from the examples, we can write a simple LINQ comprehension using extension methods defined in the System.Core library’s System.Linq namespace.  To make this happen, we need to load our assembly System.Core and then use the extension methods defined in the System.Linq namespace.

load_assembly 'System.Core'
using_clr_extensions System::Linq

We’ll also need a way to go from the Ruby Enumerable to .NET IEnumerable<T> instances.  To bridge that, we’ll extend the Object class and define a to_seq method, which takes a type, should we need to make our IEnumerable<T> instance anything other than a Object.  We then create the IEnumerable<T> instance by calling to_a or to array on the Enumerable itself.

class Object  
    def to_seq(type = Object)    
        System::Linq::Enumerable.method(:of_type).of(type).call(self.to_a)  
    end
end

Putting this all together, I take a Ruby enumerable and convert to a .NET IEnumerable<T> instance, square the numbers and get those which are divisible by 3 and then print the results.

load_assembly 'System.Core'
using_clr_extensions System::Linq

class Object  
    def to_seq(type = Object)    
        System::Linq::Enumerable.method(:of_type).of(type).call(self.to_a)  
    end
end

(1..10).to_seq.
    select(lambda { |x| x * x }).
    where(lambda { |x| x % 3 == 0 }).
    each { |x| puts x }

This works great as it prints out the numbers 9, 36 and 81.  This is a great interop story although I’m sure if I’m using Ruby, I might just stick with the Enumerable module.  But then I looked at the Observable Module in Ruby and I wasn’t really happy with what I saw there in terms of adding/deleting observers as well as the notification is pretty much a straight copy of the Gang of Four pattern, which I think we’ve moved past. 

This got me thinking, that if we can go back and forth with Ruby’s Enumerable module to the .NET IEnumerable<T> interface, then we should be able to do the same with the Reactive Extensions for .NET and the IObservable<T> interface.  We’ll recreate the example above, but instead of the push model of Enumerable, we’ll be pushing objects to it instead.

Let’s first load our assemblies that we’ll need and include the proper namespaces and extension method namespaces.  We’re including the System namespace as that contains the overloads for the Observable.Subscribe method to take lambdas instead of an IObserver<T> instance.

load_assembly 'System.Core'
load_assembly 'System.CoreEx'
load_assembly 'System.Reactive'

include System
include System::Linq

using_clr_extensions System
using_clr_extensions System::Linq

Once we’ve done that, let’s take the knowledge about taking an Enumerable and turning it into an IEnumerable<T> by taking that same Enumerable and turn it into the IObservable<T> via the to_observable (ToObservable) method.

class Object  
    def to_seq(type = Object)    
        System::Linq::Enumerable.method(:of_type).of(type).call(self.to_a)  
    end
    
    def to_observable(type = Object)
        System::Linq::Observable.method(:to_observable).of(type).call(self.to_a)
    end
end

Now we can focus on the problem at hand which is to create our IObservable<T> instance.  As above, we can take an Enumerable range of 1 to 10 and turn that into an observerable.  Once we do that, we can now call our select and where methods appropriately which creates our observable sequence.

observable = (1..10).to_observable.
    select(lambda { |x| x * x }).
    where(lambda { |x| x % 3 == 0 })

We now have our observable sequence, but now what?  Well, since we’ve imported the extension methods from the System namespace, we’re able to use the Subscribe overloads that take lambdas which cover the OnNext, OnError and OnCompleted cases.  In this case, I’ll handle all three, but in this instance I only really need to handle just the OnNext.  I’ll use the do syntax instead of the lambda just to show you that both work.

observable.subscribe(
    lambda do |next_value|
        puts next_value
    end,
    lambda do |error|
        puts '#{error.message}'
    end,
    lambda do 
        puts 'done!'
    end)

Having this code in place, I can run this and sure enough I have printed to my screen:

9
36
81
done!

Interesting possibilities here, including comparing and contrasting the Ruby Observable and the IObservable<T>.  More to come in the next post!

Conclusion

With the Reactive Extensions for .NET, we have a nice cross-language support for building a bridge to asynchronous and event-based programming.  Coming with the latest release of IronRuby, we now have the ability, and a rather nice one at that, to consume extension methods that are predefined in other languages, which opens a whole new set of opportunities for interoperability.  In the coming posts, I’ll take a look at what else I can do in this space to see that we can fact help solve some of the hardest problems around asynchronous and event-based programming.

So with that, download it, and give the team feedback!

 
Introduction to the Reactive Extensions for JavaScript – Error Handling Part II

We’ve covered a bit recently with conditional and looping operators on the Reactive Extensions for JavaScript, but I want to step back just a minute and cover exception handling.  This post will cover how we can compensate for errors as they happen in several ways and will largely follow Bart de Smet’s post on the same topic, but instead of covering the Interactive Extensions, we’ll stick primarily in JavaScript.  Last time, we covered the basics of error handling including Catch and Finally.

Before we get started, let’s get caught up to where we are today:

OnErrorResumeNext VB Style

The next error handling feature in the Reactive Extensions for JavaScript is one that should be familiar with those in the Visual Basic world, called OnErrorResumeNext (On Error Resume Next).   In Visual Basic parlance, this meant you specify that when a run-time error occurs, control goes to the statement immediately following the statement where the error occurred, and execution continues from that point.  The functionality is similar to the Concat operator which allows you append Observable sequences onto each other to create a single sequence, except that should an error occur, it is not bubbled up and instead continues to the next.  This operator comes in two flavors, one as a static method and one as an instance method:

// items : Observable[]
// scheduler : optional custom scheduler
// returns : Observable

Rx.Observable.prototype.OnErrorResumeNext(
    items,
    scheduler);

// items : Observable[]
// returns : Observable
Rx.Observable.OnErrorResumeNext(
    items);

Let’s create a simple example where we want to output the numbers 1 through 9, even if an exception occurs along the way.  In this case, we yield two arrays and the third then has an exception concatenated to the end before then yielding the final array.

Rx.Observable.OnErrorResumeNext([
    Rx.Observable.FromArray([1, 2]),
    Rx.Observable.FromArray([3, 4, 5]),
    Rx.Observable.FromArray([6, 7]).Concat(Rx.Observable.Throw("woops!")),
    Rx.Observable.FromArray([8, 9])])
    .Subscribe(
        function (next) {
            $("<p/>").html(next).appendTo("#results");
        });

This is great for handling data that may be potentially bad and is ok to drop the exception should it occur.

Cleaning Up With Using

Just as in .NET there is the using keyword to denote a try/finally with a call to Dispose, we need a similar capability in the JavaScript world for resource cleanup in the asynchronous world.  Let’s look at the signature of the function which takes a resource selector which returns the Disposable object, and then the usage which takes the Disposable object and you return an Observable sequence.

// resourceSelector : () -> Disposable,
// resourceUsage : Disposable -> Observable
// returns : Observable

Rx.Observable.Using = function(
    resourceSelector, 
    resourceUsage);

A Disposable object is nothing more than an object with a Dispose method which takes no arguments and returns nothing.  For example, we could create a Disposable object to encompass the jQuery data API which allows us to store arbitrary data associated with a given element.  This object would allow us to both get and set the value, and in our Dispose, we would remove the data.

DataDisposable = function (element, key) {

    this.data = function (value) {
        return element.data(key, value);
    };

    // Dispose method
    this.Dispose = function () {
        element.removeData(key);
    };
};

Using this, we can write two samples, the first of which sets the data and then retrieves the data without an exceptional condition whereas the second example does.  let’s take a look at the first example which yields to us the answer of 56088.

Rx.Observable.Using(

    // resourceSelector
    function () {
        return new DataDisposable($("#content"), "key1");
    },
    
    // resourceUsage
    function (disposable) {
        disposable.data(123);
        
        return Rx.Observable.Return(456 * disposable.data());
    })
    .Subscribe(
        function (next) {
            $("<p/>").html(next).appendTo("#content");
        });

The second example below is the exact same, except that we throw an exception and then resume the next sequence which returns 789 instead of the answer above.

Rx.Observable.Using(

    // resourceSelector
    function () {
        return new DataDisposable($("#content"), "key1");
    },
    
    // resourceUsage
    function (disposable) {
        disposable.data(123);

        return Rx.Observable.Return(456 * disposable.data())
            .Concat(Rx.Observable.Throw("woops"));
    })
    .Catch(Rx.Observable.Return(789))
    .Subscribe(
        function (next) {
            $("<p/>").html(next).appendTo("#content");
        });

In both cases here, we ensure that the underlying data is properly disposed when we leave the scope of our Using operator.

If At First You Fail, Retry

Let’s move onto the last operator for this section on error handling, Retry.  The idea behind this operator is that we can retry the observable sequence until it succeeds (meaning no exception) or hits the maximum try count.  Let’s look at the method signatures below:

// returns : Observable

Rx.Observable.prototype.Retry();

// count : int
// scheduler : optional scheduler
// returns : Observable

Rx.Observable.prototype.Retry(
    count,
    scheduler);

Now if we have a sequence that doesn’t throw an exception, then it should be a standard no-op, which means it does nothing for example, the following simply prints the numbers 1 through 3.

Rx.Observable.FromArray([1, 2, 3])
    .Retry()
    .Subscribe(
        function (next) {
            $("<p/>").html(next).appendTo("#results");
        });

However, if we indeed had a collection that did throw an exception each and every time, we could run into a big issue such as the following, where it would blow out the JavaScript stack as it would try to go on forever:

Rx.Observable.FromArray([1, 2, 3])
    .Concat(Rx.Observable.Throw("woah nellie!"))
    .Retry()
    .Subscribe(
        function (next) {
            $("<p/>").html(next).appendTo("#results");
        });

Luckily, we have ways around this where we can specify the maximum number of retries that are allowed with the provided overload.  Next, let’s show a case where Retry is really used.  For example, let’s say we have a loop which could produce three values, but will fail at different points for the first two iterations. 

var count = 0;

Rx.Observable.Return(4)
    .Concat(
        Rx.Observable.If(
            function() { return count == 0; },
            Rx.Observable.Throw("nope!"),
            Rx.Observable.Return(5)))
    .Concat(
        Rx.Observable.If(
            function () { return count == 1; },
            Rx.Observable.Throw("nope!"),
            Rx.Observable.Return(6)))
    .Finally(function() { count++; })
    .Retry()
    .Subscribe(
        function (next) {
            $("<p/>").html(next).appendTo("#results");
        });

In this situation, we will render the following result, since we hit two different exceptions:

4 // Hits exception
4
5 // Hits exception
4
5
6

And there you have it, we have composable ways around potential failures in our asynchronous push based operations that give us the flexibility that our synchronous imperative ones do.

Conclusion

Dealing with asynchronous programming has been in the forefront of many minds in the JavaScript community.  At JSConf, there were several examples of frameworks trying to get around the idea of callbacks and instead lean more towards composition.  By utilizing the Reactive Extensions for JavaScript, we’re able to compose together asynchronous and event-based operations together and transform the results in interesting ways.

When we start creating more advanced workflows through the Reactive Extensions, we also need ways of handling errors as well.  We have the ability do handle errors and compensate as they happen through a rich set of operators including Catch, Finally, Using, OnErrorResumeNext, Retry and more.

So with that, download it, and give the team feedback!

 
news feeds
CNET 
tech feeds
DevX 
tools
No items available...
Copyright © 2010 – Richard Gardiner