C# 5.0 and the sample that made me go “wow!”

In the Visual Studio Async CTP there is one sample project called 101 Asyncs. This is a browser displaying 101 code-snippets involving the new C# 5.0 async functionality. As I was flicking through I found the following sample, which I think speaks volumes for the elegance and simplicity of the feature:

public async void AsyncSwitchToCPU()
{
    Console.WriteLine("On the UI thread.");

    // Switch to a thread pool thread:
    await new SynchronizationContext().SwitchTo();  
                
    Console.WriteLine("Starting CPU-intensive work on background thread...");
    int result = DoCpuIntensiveWork();
    Console.WriteLine("Done with CPU-intensive work!");

    // Switch back to UI thread
    await Application.Current.Dispatcher.SwitchTo();                

    Console.WriteLine("Back on the UI thread.  Result is {0}.", result);
}

public int DoCpuIntensiveWork()
{
    // Simulate some CPU-bound work on the background thread:
    Thread.Sleep(5000);
    return 123;
}

Using Ready-made Virtual PCs to try out the Visual Studio Async CTP

The Visual Studio Async CTP containing the preview of C# 5.0 which was all the talk at Microsoft PDC 2010 installs on top of Visual Studio 2010. If, like me, you’re wary of trashing your main development machine, you might like to try out the ready made Visual Studio Virtual Machines that Microsoft provide. They are a pretty hefty download (over 9 GB), but I set my download manager to work on the task just after sitting down at my desk this morning, and it was done by mid-afternoon.

Having a Windows Server 2008 Virtual Machine ready built with Visual Studio 2010 (and 2008), Team Foundation Server 2010, SQL Server 2008 and Microsoft Office is a huge time-saver. Though they are all time-limited versions, you can apparently use your MSDN product keys to activate them into perpetual mode.

They come in three flavours, one for Hyper-V, one for Virtual PC 2007, and one for Windows 7 Virtual PC. If you hop over to Brian Keller’s blog he has a some lists of URLs that you can paste into your download manager to make the download job pretty painless, and then some instructions to get you started. One thing to note, once you’ve booted the machine, is that all the user accounts have password “P2ssw0rd”.

Having booted into the Virtual PC, you will need to install the Silverlight 4 SDK and then, of course, the Visual Studio Async CTP itself. Both are pretty small downloads and quite quick to install.

Have fun!

WPF to support hosting of Silverlight controls in vNext

To the great consternation of the blogosphere, Joe Stegman’s planned session on Silverlight quietly disappeared from the Microsoft PDC schedule, and no new announcements were made on that front. Instead we got a pre-recorded session from Rob Relyea entitled WPF Today and Tomorrow.

Microsoft adoption

Early on in his talk Rob reminded us that Microsoft is making the ultimate commitment to WPF in that many of their latest products are built on it, including the Expression Studio suite, Visual Studio, and most recently Web Matrix [see the demo in Rob’s talk]. He also hinted that there are other products on the way which cannot be announced yet.

As part of these efforts across Microsoft a number of controls have been developed which Rob feels would be widely applicable(one of which is WPF equivalent of the Silverlight ChildWindow) and he intends to work with the respective teams to release these components to the web.

Silverlight Integration

One welcome announcement Rob made was that the next version of WPF will support hosting of Silverlight controls [watch his demo]. Today, of course, this is somewhat possible by hosting a Web Browser inside WPF, then navigating it to some Silverlight content. What Rob announced was something deeper than that, in two ways. The first is that they want to enable good communication between the WPF host and the Silverlight components, including sharing data.

The second improvement was part of a bigger announcement: they are planning to solve the airspace issues with hosting embedded Hwnd content [watch]. This was something I didn’t expect – in fact, I thought the airspace issue was a problem here to stay.

To briefly recap what the airspace issue is: in current versions of WPF, whenever you need to embed Win32-based content inside WPF, the embedded content always ends up sitting on top of the WPF stuff. This is because Win32 Hwnds don’t layer well with WPF elements – they can’t share airspace. You also can’t rotate or scale the Hwnds or adjust their opacity like you can with WPF elements.

Rob announced that Microsoft will be making a large investment, and will remove the airspace restrictions. And I can see why it will be a large investment. From what Rob said [watch] it seems that they way they plan to tackle this is by rendering Hwnds off-screen so that their visuals can be composited into the appropriate place in the visual tree, then any keyboard our mouse input intended for the control will have to be redirected from the WPF surface back to the Hwnd. A tough challenge indeed – but with great rewards for anybody looking to port legacy applications.

Amongst other things that Microsoft are considering for WPF vNext, Rob mentioned the following:

• Proper support for Grouping in ItemsControls when doing UI Virtualization [watch]
• Improved support for data-binding to collections and properties that get updated in a background thread [watch]. For collections, this will involve adding APIs to allow the data-binding engine to interact with the locking scheme used in the collection.

C# 5.0 – Other News

Compiler as a Service

In the last 10 minutes of his talk on the future of C# 5.0, Anders Hejlsberg gave an update on a project that he first mentioned back at PDC 2008 – Compiler as a Service [watch]. Anders confirmed that what was a prototype back in 2008 is now being worked on in earnest. It has a codename – “Roslyn” (he didn’t actually say this, but you’ll see it appear as a namespace in the code samples, and in one of the Visual Studio tabs), but no ship dates as yet.

As Anders put it, during the compilation process, the compiler builds up a huge amount of knowledge about what your code actually means – and then throws it all away. The idea of Compiler as a Service is give you a Language Object Model encapsulating exactly the same knowledge about your code as the compiler has - right down to the finest semantic details like what an identifier means in a particular context, or precisely how a method call will bind at run time.

It seems that this Language Object Model will be available in a number of different ways, including to Visual Studio Extensions, which will be able to use it for things like performing Refactorings, or formatting code intelligently.

Anders showed one example of a Visual Studio Extension that could add outline regions to if/else blocks [watch]. Sadly, the extension failed to run in Anders demo, but it still conveyed the simplicity and power of the Language Object Model.In another example he showed how the Language Object Models for VB.Net and C# could be used together to create a Visual Studio extension allowing you to copy C# code and paste it as VB.Net [watch]. “I’m not saying this is super-trivial”, remarked Anders, “I mean, there is a couple of thousand lines of code here. But it’s not a complete compiler at all – it’s relatively modest.”

There is more to come

In answer to a question at the end of the session [watch], Anders confirmed that there will be other features added to C# 5.0 that they’re not ready to talk about yet. He did hint that the kind of areas they are exploring were Immutability, Isolated State and Pure Functions – all features associated with ensuring that programs continue to work correctly when parallelism is introduced. Check out the Programming Languages Panel session if you want to hear Anders talk in a little more detail about this.

C# 5.0: The Asynchronous Revolution

In his usual quiet and unpretentious way, Anders Hejlsberg stepped up on stage at Microsoft PDC yesterday – and proceeded to launch a revolution. Two new keywords added to the C# and VB.Net languages was all it took, but the impact of those changes will be felt by programmers for years to come. And not just programmers: unlike other changes made to the .Net languages across the versions, these new additions will produce tangible benefits for users.

The Status Quo

Anders started his presentation by noting a current trend in applications [watch]: they are becoming increasingly connected. We’ve had client-server applications for decades, but we’re now seeing more and more applications that have to call out to a number of different services and co-ordinate their responses. This brings a number of problems. First there’s the latency - the application having to wait as the requests and responses pass back and forth over the network. Then, if we’re not careful, the application appears to hang, unable to repaint the user-interface because it’s too busy waiting for the server to respond. Meanwhile, on the other side of the network, all the server’s threads are tied up, waiting for responses to come back from other servers, so it stops responding to incoming requests – and scalability plummets.

Now we all know what the answer is: asynchronous programming. Rather than tying up our main thread waiting for a response to come back, launch the request on another thread and set up some kind of event handler so that you can be notified when the response comes back, and then finish the work.

We have the answer, but it’s not a pretty one, as anybody who has programmed a serious client-server application in Silverlight will tell you. In Silverlight there is no way of calling the server synchronously: Silverlight forces you to call asynchronously so that the browser hosting your application can remain responsive. In fact, there’s no way of doing something as simple as showing a dialog synchronously: if the way your application continues is dependant on the users’ response, you have to move the continuation logic into an event handler attached to the dialog so that it can be executed once the user clicks their choice of button.

Inside-out Thinking

Programming like this requires you to turn your thinking inside out, because you have to prepare your response (in an event-handler or whatever) before you even make the request. And if in your response you’re going to make another asynchronous call, you have to prepare your response for that even before your first response. Anders gave a small example of that [watch]. Here’s a small method that queries the NetFlix web service. Nice straight-forward code:

Make that asynchronous and see what happens:

See how it’s turned inside out: in line 4 we prepare our what-happens-next by attaching a lambda to the DownloadStringCompleted event, and the call to download the string is the very last thing we do. Then notice that we can no longer return a value from the method: it isn’t available when the method returns. Instead we have to get the user to pass in an Action that we can call back when the result is ready. So we’re forced into changing all the places where we call the QueryMovies method too: the effects of this change ripple out through the code. As Anders said, “it becomes lambdas all the way down”.

No wonder we only do this stuff when absolutely necessary – and our customers pay a price, getting applications that become unresponsive when busy, and servers that are not as efficient as they could be. All because, for mere mortals at least, asynchronous programming is just too hard.

Not any longer.

Asynchronous Programming for the rest of us

Up on stage, Anders introduced his two new keywords, async and await [watch]. Here’s how the asynchronous version of QueryMovies looks in C# 5.0:

It’s identical to our nice straight-forward but sadly synchronous version, with the exception of four small changes (highlighed in orange). The first is the addition of the async keyword in front of the method definition. This tells the compiler that it’s going to have to work some magic on this method and make it return its result asynchronously. This produces the second change: the method now returns a Task<Movie[]> instead of just Movie[] – but notice how the return statement is unaltered – the compiler is taking care of turning the return value into a Task.

In case you’re wondering, Task is a class that was introduced in .Net 4.0 as part of the Task Parallel Library. It simply encapsulates a value that is going to become available at some point in the future (hence, why in some languages the equivalent concept is called a Future). Importantly, Task provides the ContinueWith that allows you to schedule an event handler to be called when that value is available.

The most significant change is the introduction of the await keyword in line 4, and the change to call DownloadStringTaskAsync. DownloadStringTaskAsync is just a version of DownloadString that, instead of returning string, returns Task<string>. Again, this Task<string> is something that will deliver a string sometime in the future. Adding the await keyword in front of the method call allows us to await that future moment, at which point the string will be popped out of its Task wrapper and the method will continue as normal.

But get this: it is not waiting by blocking the current thread. Under the covers the compiler takes everything in the method that comes after the await keyword, everything that depends on the value we’re waiting for, and packages it up into a lambda function – an event handler. This lambda function it attaches to the Task<string> returned by DownloadStringTaskAsync so that it can be run when the string becomes available.

There are very few restrictions on what the bodies of async methods can look like. All the usual control flow operations, if statements, foreach, while, etc. work. Best of all, exception handling works without needing any changes. The compiler takes care of all the twistedness of the asynchronicity, leaving you to think in straight lines.

All this means that, once C# 5.0 is here, there will be no excuses for applications that suffer white-outs while they do some work. And it means some cheap improvements to scalability on the server side. Anders showed an example of a server application that processes some RSS feeds over which he waved the async wand [watch]. The time needed to process the page dropped from about 1.25 seconds to 0.25 seconds – 5 times faster. Here’s a snippet of the code so you can see again how few changes are needed:

This is big news. There are other languages that can already do this kind of thing, F# being one of them. But I don’t know of any other mainstream language that does it, or does it as neatly as this. Anders and team have created an incredibly powerful feature, but exposed it a way that everybody will be able to use.

You can play with this now. Anders announced a Visual Studio Async CTP which contains new versions of the C# and VB.Net compilers, and the necessary supporting APIs in the framework (including Async versions of lots of the existing APIs). It installs on top of Visual Studio 2010, and I gather that it enables Asynchronous methods in both .Net 4.0 and Silverlight.

Further Reading

• Whitepaper: Asynchrony in .Net: an excellent overview of the new features being added in the languages and framework.
• Eric Lippert’s blog: Eric is a member of the C# compiler team, and he’s running a series of posts on the design and implementation of the async feature.
• Jon Skeet’s blog: Jon has been digging into various details of the async implementation, and (as if Anders hasn’t already given us enough!) also speculating about further additions to C# 5.0 to supplement async support.
• Ian Griffiths has his own summary of the async functionality which digs into the exception handling side of it.
• C# Language Specification for Asynchronous Functions
• The C# team blog has an interesting Silverlight 4 Facebook API example.

Microsoft PDC 2010 Speculations

We’re now less than a week away from Microsoft PDC 2010. The session list has just been published so I think it’s time to indulge in some speculation as to this years announcements.

C# 5

For me the highlight of recent PDCs has been discovering what Anders and his sous-chefs have been cooking in the C# kitchen. Of all the changes likely to be announced, it is changes to the C# language that are likely to make the most day-to-day difference to my work. In 2005 it was LINQ, in 2008 it was dynamic programming. What will it be in 2010? Well, Anders has been given a slot, and we already have some clues as to what he might say. At his presentation on the future of C# and VB.Net at PDC 2009, Luca Bolognese trailed some of the likely features, including:

• Compiler as a Service – the C# compiler completely rewritten in C#, and available to use programmatically, allowing all kinds of interesting meta-programming, including the ability to write refactorings.
• Better support for immutability built into the language
• Resumable methods.

It was this last one that interested me most. Luca showed an example involving asynchronous programming. Today we do this by calling a Begin* method, and passing in a callback that will be executed when the asynchronous invocation has finished (and we never, ever, forget to call the corresponding End* method!). This can get quite difficult to wrap your head around if you have a whole chain of methods that need to be invoked asynchronously – say in the callback of the second one you kick off another asynchronous method, passing in a callback that runs a third – and so on.

This style of programming actually has a name – Continuation Passing Style (or CPS) – and, from recent developments on Eric Lippert’s blog it seems they’re planning to add support for CPS to the C# language to make it easier to write this kind of code without blowing a fuse in your brain (in typical Eric Lippert style, he is neither confirming or denying what hypothetical features might or might not hypothetically be added to a hypothetical future version of C#, hypothetically speaking).

No doubt Anders will reveal all.

LINQ

Since it was released in 2007, LINQ has colonised .Net code bases at an incredible pace. Parallel LINQ was another great innovation in .Net 4. And Microsoft aren’t done yet. The Reactive Framework (aka Linq-to-Events) is coming on nicely. And this year Bart de Smet has a session tantalisingly entitled “Linq, Take Two: Realizing the LINQ to Everything Dream”. Linq-to-Everything sounds pretty ambitious, so I’m looking forward to hearing what he has to say.

Silverlight

It looks like we can expect some big announcements for Silverlight this year – check the session list for proof: there’s a note against Joe Stegman’s session on Silverlight saying that the abstract is embargoed until after the PDC keynote.

What would I like to see? Well, one of the biggest pain-points I’ve found in my Silverlight programming is that Continuation-Passing-Style programming is thrust upon you whenever you want to show a dialog, or make a call to the server: because Silverlight runs in a browser, you aren’t allowed to block the UI thread, so every potential blocking operation has to be done asynchronously. It would be really nice if C# 5 does simplify CPS, and if that also applied to Silverlight.

Azure

I’ve already made my PDC prediction for Windows Azure: I’m guessing that Microsoft will announce free, enthusiast-scale hosting for Web Applications in Azure. This is looking even more likely now that Amazon have announced a free entry-level version of their EC2 cloud hosting platform.

Thursday 28th October will reveal all. Stay tuned – once again my company has graciously given me time to follow along with the PDC sessions (and this year they’re being transmitted live, as well as being available for download within 24 hours). As on previous occasions, I’m hoping to find time to share with you all what I learn.

My PDC 2010 Prediction: Free Enthusiast App Hosting on Windows Azure

So Microsoft have announced that there will, after all, be a Platform Developers Conference this year. It will however, be an austerity PDC, chopped down to two days, held on Microsoft’s campus rather than in a big conference city, and with a much narrower focus. This is billed to be a Cloud Conference. And I think I know what Mr Steve “Developers, Developers, Developers” Balmer is going to be announcing.

Think about what Microsoft have revealed over the last couple of weeks:

• WebMatrix, a new streamlined IDE aimed at enthusiasts wanting to produce web applications. And skipping hand-in-hand with WebMatrix comes Razor, a new syntax for creating web views.
• SQL Server Compact Edition 4, a new version of the embeddable, in-process edition of SQL Server that has been enabled to run inside ASP.Net
• IIS Express, a version of IIS 7.5 with full support for the new IIS Integrated pipeline, SSL, etc. but capable of running in a locked down environment with no admin privileges.

Then there are the rumours about KittyHawk, a tool aimed a tech-savvy business users wanting to produce Silverlight applications without any coding.

Add all this up, and what do you get? Hundreds of enthusiasts with shiny new web applications, eager to share them with the world – but not wanting to go through the pain of ftp-ing their creation to a web host.

Hence my prediction: At PDC 2010 Microsoft will announce free hosting for enthusiast-scale web applications within Windows Azure. And they’ll throw in tools to publish web applications from WebMatrix and Visual Studio Express to Azure at the click of a button.

Take a look at the top two feature requests on mygreatwindowsazureidea.com (the official feature request list for Windows Azure): Make it less expensive to run my very small service on Windows Azure and Continue Azure offering free for Developers. Microsoft seem pretty serious about addressing issues on such lists just recently.

Google have a head-start in this arena: they’ve been offering this kind of free hosting in their AppEngine for years. But with their Visual Studio tooling, I think Microsoft could clean up.