#2782

HPC Pack SP3 Ships and an Update on LINQ to HPC

Sometimes things just don’t work out the way you’d expect. I mean really, really not the way you expect.

Today Microsoft announced Windows Azure HPC Scheduler and HPC Pack 2008 R2 Service Pack 3 that includes updates to the HPC Pack 2008 R2 and a new Windows Azure HPC Scheduler to allow you to run HPC workloads on Azure! HPC is now a major Azure workload and the HPC team is working hard to enable that for end users and ISV developers.

There’s also an update on the next steps for LINQ to HPC.

As part of this release we’ve also updated the preview version of LINQ to HPC, however, this will be the final preview and we do not plan to move forward with a production release.  In line with our announcement in October at the PASS conference we will focus our effort on bringing Apache Hadoop to both Windows Server and Windows Azure.  Hadoop has emerged as a great platform for analyzing unstructured data or large volumes of data at low cost, which aligns well with Microsoft’s vision for its Information Platform.  It also has a vibrant community of users and developers eager to innovate on this platform.  Microsoft is keen to not only contribute to this vibrant community, but also help its adoption in the Enterprise.  We expect a preview version on Windows Azure available by end of the calendar year.

You can read the full announcement and more about Microsoft’s Big Data strategy and their work with Hadoop at http://microsoft.com/bigdata.

Suffice to say that from a personal perspective I’m very, very disappointed. I was excited to join the Technical Computing team back in February of this year as the Program Manager for LINQ to HPC. I got to work with a great group of people from the HPC team and Microsoft Research’s Dryad team on a really interesting technology in an important, emerging space.

But really this post is about saying thanks…

I would like to take a moment and say thanks to all those who’ve spent time evaluating LINQ to HPC and provided us with feedback. Either after attending my talk at //BUILD, talking to me or other members of the team or just reading about it. Maybe you feel you could or should have been doing something else. If so, I’m sorry. Hopefully, like me, you learnt some valuable lessons about building and using distributed systems along the way. In particular, how to provide a generalized solution to big data analytics that went beyond the Map-Reduce model.

There’s lots of other exciting scenarios that are supported by SP3 and I’d encourage you to check them out. Especially if you are thinking of scaling HPC to the cloud.