Archive for the ‘PDC08’ Category

F# – Functional Programming for the Masses….

fsharp

 

Functional programming has been around for quite some time.  Many functional languages predate the languages we use today.  In fact Lisp was created in the late 50’s.

Functional programming is based on lambda calculus which is a branch of mathematics that describes functions and their evaluations (see wikipedia).  Pure functional languages are built from mathematical functions and do not allow for any mutable state or side effects.  This means that no state can be modified and it cannot have IO. Because of this, until recently functional programming languages and concepts were primarily the domain of mathematicians and scientists doing theoretical work. 

So, why the new interest in functional programming? 

Some of this is due to functional programming concepts creeping into main stream languages such as C#.  With .NET 3.5, came LINQ (Language Integrated Query) which is firmly rooted in the functional programming paradigm.  LINQ essentially builds an expressions tree used to describe a query operation.  This operation is then evaluated “Lazily” at the moment it is needed.  This idea of lazy evaluation is also a core concept in functional programming language.

So why is functional programming important? 

While Moore’s law (which is the doubling of the number of transistors that can fit on an IC every two years) continues to be true, we are not seeing this same trend in clock speeds which have traditionally given us the performance boosts that we have seen over the years.  With the advent of multi-core processors, the future shows us that to increase performance, instead of clock speeds increasing (which has a physical limitation that is due to the speed of light), we will see an exponential growth in the number of processor cores.  Currently, two and four cores are popular but soon we will see 8, 16, 32 and more cores.

To take advantage of all of these cores, multithreaded programming will become increasingly more and more important as a way to exploit the new multi-core processors.  We all know that multi-threaded programming is very difficult at best.  With mutable state and multi-threaded programming, the possibility of encountering race conditions, dead locks and corrupted data goes up exponentially with the number of cores causing a much greater possibility of producing “Heisen-bug” (which are bugs that are very difficult to debug because the behavior is changed by the act of debugging).

Because Functional programming does not use mutable state, it is very suitable for being able to easily and safely handle computationally expensive processing across many, many cores in parallel,

Enter, Microsoft F#

Microsoft has recently introduced a new, mostly, Functional programming language built on top of the .NET CLR.  This new language, called F#, came out of the Microsoft Research group and is in the process of being productized so that it can be included in a future release of Visual Studio. 

F# at the PDC 2008

At the PDC08, Luca Bolognese, the product manager for F#, gave an excellent presentation on F#.  In his talk he created a program in F# that performed various financial calculations on stock prices that were downloaded from the internet.  What was really interesting was how he went about writing the program.  Typically, when writing programs using imperative languages like C#, we tend to take a top down approach.  Sure, we break our programs into methods but we think more or less in a procedural fashion.  With functional languages, as was the case in this presentation, the solution is often composed from the inside out, much like you would compose a complex SQL statement or an XSL transform.  With SQL or XSL, you often start in the middle by starting with a basic function then running it to check the results.  Then you slowly filter and modify the results to slowly build out the desired solution.  Because of this, when you look at the final result, it can look complex and you might often scratch your head when you come back to it at a later date. 

This inside-out composition technique is exactly how Luca built his solution.  Slowly getting to the final desired result.  The amazing thing was how little code it took to reach his end result compared to what would have been necessary if it were written in C#.  Much of the “noise” was removed by the F# solution and in the end it really described the problem at hand.  What really helped this approach is that F# uses type inference so you don’t need to declare your many of your types, this made the composition process of creating the program much easier.  This is not to imply that F# is not statically typed, it is NOT dynamic language.  In fact, functions are types in F#. 

Another technique he showed was currying.  This is basically piping the results of one function into the input of another.  This made the code very readable and gave a clear understanding of what was being accomplished.

After creating the program, Luca showed how simple it was to parallelize his solution to run across multiple processors.  To do this, he needed to make only slight modifications to the code but he didn’t need to add any locks, monitors, wait handles or any of the other constructs we normally associate with multithreaded programming.  In fact, his solution would safely run against as many processors as could be contained on a chip.  Amazing…doing this in an imperative language like C# would scare the most seasoned developer.

Since F# is built upon the CLR, it is a functional language that can interact with both the rich class library of .NET framework as well as with other assemblies written in other .NET languages.  This opens up lots of interesting scenarios and allows us the possibility of writing computationally expense algorithms  in F# which provide a safe story for parallelization. 

Here are links to Luca Bolognese’s presentation of F# at the PDC 2008.  I highly recommend it.  Luca is presentation style is both very informative as it is entertaining.  The bulk of his presentation was done as a demo which should please the developer in all of us.

An Introduction to Microsoft F# presenter: Luca Bolognese

WMV-HQ  |  WMV  |  MP4  |  PPTX

 

Other resources for F#

www.fshare.net

http://research.microsoft.com/fsharp/fsharp.aspx

http://www.ffconsultancy.com/dotnet/fsharp/index.html

PDC08, from my perspective…

Welcom to the PDC08

Welcome to the PDC08

The PDC was a great conference showing much of Microsoft’s future vision for their products and platforms.

I would say that this conference could be broken up into several major areas of interest:

  • Oslo
  • Azure Cloud Operating System
  • Live Services
  • User Interface – Silverlight
  • Languages – C#, Dynamic Languages, F#
  • Windows 7

Overall, I was very impressed with Microsoft’s ability as a company to coordinate the efforts of many diverse groups and technologies throughout the company. It seems that many of the efforts and initiatives that Microsoft has undertaken are starting to come together into a common cohesive vision. That said, it was also evident to me that they still have a bit of work to do to bring all of this new technology together for prime time usage. Although the PDC is an event that usually occurs only every few years, I did hear a rumor that Microsoft has already announced a PDC09. This makes me believe that the timing of this PDC may have been a little aggressive. For many of the Azure and Live services sessions the presenter was in constant IM contact with the Microsoft datacenter. That tells me that there were lots of stability concerns regarding the products that had strong reliance on their cloud services.

My personal interests drove me to many of the sessions on their cloud-based services and Oslo. It was pretty well know that Microsoft would be announcing a cloud-based platform and Live Mesh client-side platform but many of the details were clouded in secrecy before the PDC. Oslo was also talked about a little before the PDC and was shown in a little more detail at the PDC.

Over the last several years Microsoft has been building datacenters at a record pace. During the 1st keynote address, Ray Ozzie talked about how Microsoft realized that in building up their own web-based internet properties that many of these activities were being undertaken by many large and smaller companies around the world. While many large companies could afford to build large datacenters that provide reliability, redundancy, fault tolerance, etc. Many not so big companies were struggling with this overwhelming task. Even very large companies were having trouble with scaling out their services to handle geo-location and fault tolerance. It was becoming clear to Microsoft that cloud based services to complement on premises software was needed. In his keynote, Ozzie recognized similar efforts by both Google and Amazon in this area.

With Windows Azure, Microsoft differentiates itself with cloud based services offered by both Google and Microsoft by offering a service that is:

  • Abstracted out to be completely elastic, by allowing computing power to by dynamically sized and scaled at runtime so that you can handle peak loads without paying for more than you need.
  • Geo-location, by being able to spread a cloud-based deployment across the globe.
  • Fully fault tolerant. All data and software in the cloud is located in several places in the cloud and always spread between servers at different locations.
  • An infrastructure that can easily be connected to on-premises software through an internet service bus.
  • And the list goes on…

With Windows Azure will come many interesting deployment tools that will make deploying and managing applications in the cloud easy.  Microsoft has created many new and interesting technologies to create what they call “Fabric”.  This “Fabric” is the abstraction that sits on top of the actual servers that are running inside their many datacenters.  You can think of this “Fabric” as an abstraction that is a few levels higher than virtual machines.

From all of the various sessions on Azure and cloud-based services it was very clear that the new world of cloud-based software was going to require us to think differently about how we architect, design and implement our software so that it is better suited to fit into a cloud-based paradigm which, in my opinion, is where things will be headed in the next ten years.  Many of these practices involve the proper decoupling of software as well as other practices that are just part of good software design.

As part of this new cloud-based initiative, it is clear that WCF and WF (Windows Workflow) are going to be two very fundamental enabling technologies. Up to the point, we haven’t seen too much usage of Windows Workflow but it is clear that this will be a large part of many cloud-based applications.  As part of this, Microsoft announced a new server product called “Dublin” which is an application server built using WCF services that front Windows Workflows.  This server product has many advances in Windows Workflow that make it an ideal choice for hosting workflow in the cloud.

There were also talks given on the new data technologies that expand Sql Server into the cloud.  These were dubbed “Sql Services”.  One of the new services is Sql Server Data Service.  This service provides a new way to retrieve data over the internet using REST-based protocols.  These new REST protocols provide ways to query and update data using standard HTTP verbs, such as Get, Post, Put and Delete.  These protocols are built on the idea that specific data resources are uniquely addressable using URIs.  REST based protocols are built to scale like the internet itself.  It was clear that in addition to SOAP based protocols, Microsoft was heavily investing in REST-based protocols as well.

I also attended a session on storing scalable data in the cloud. From what I could gather, one will need to rethink how data is organized in order to take full advantage over the scalability that the cloud offers  What was described was very close to what Amazon does with it’s data storage services.  Basically, there are three storage items in cloud based storage services: blobs, tables and queues.  From what I could understand, the table storage was pretty basic.  Much of the storage seemed to center around entities which play nicely into Microsoft’s Entity Framework (which was just released with .NET 3.5 service pack 1).  They alluded to providing more relational storage in then cloud in the future but none of the cloud storage options had any relational capabilities.  I would imagine that much of this is because they want to provide a massively scalable and reliable data platform and the relational aspects would make this task much more difficult (which is the same route that Amazon has appeared to take).  Not sure where relational cloud storage would fit it but I would think that they would need to have a story here.

In addition to recognizing how difficult it is for companies to write highly scalable software and the need for cloud-based computing, Microsoft has put a considerable investment into “Oslo” which is a software modeling technology.  In recent years, software has become more and more complex.  It is also recognized that there are many aspects of software development that needs to be coordinated. These aspects include business analysis, software architecture, software design and implementation, software deployment. etc.

Over the past 20 years, there have been many attempts to model the software development process.  Many of these attempts, like Rational Rose, have had very limited success.  With “Oslo” Microsoft has taken software modeling to the extreme.  This appears to be one of Microsoft’s most ambitious projects in recent years.  In “Oslo” all of the modeling is stored in a database repository.  On top of this repository there is a highly customizable user interface that is used to explore this repository.  The user interface is very interesting and provides very in depth views into a given software application and how it connects to other various applications and services.

In addition to providing a graphical view of software, “Oslo” also provides a new modeling language called “M” that can be used to model software.  “M” makes it easy to also create Domain Specific Languages (DSLs) to make it easy to create applications.

As I mentioned, all of the modeling is stored inside a Sql Server database repository.  Runtimes that are meant to drive WCF/WF etc. are then driven from the data stored in the Oslo repository.

While I was very impressed by what I saw in “Oslo” it appeared to be a long way off.  Done right, this could really change the way we write and think about software in the future although it will be many years, in my opinion, before “Oslo” will be able to make that type of impact.

Well…that’s the long drawn out overview of PDC.  I will add more posts that target the specific sessions I attended along with links to the online videos of those sessions.