Just my opinion...

Made ya’ look.  But had I been serious, my wife assures me that I’m magnificent.

Actually, I’m exposing how inept and bumbling I am with LINQ and F#.  Granted, I’ve only just begun trying to integrate the concepts into my toolbox but I haven’t felt this technologically-naked in a long time…  Like ever. 

It’s really not LINQ that’s causing issues.  It’s pretty straight forward and doesn’t apply to my functional programming (FP) ignorance.  Nor is it the F# language.  It’s trying to force myself to “think functional”.  I know I posted about this yesterday but since I mentioned that I’m doing essentially CS 101 exercises to learn the concepts, I might as well post the results for all to see.  You’re welcome to grade me.  I will point out that the solutions, while not really FP, do show off cool things about the technologies and introduce some FP building blocks.  Before we get started, I want to acknowledge the excellant tutorial written by Eric White that is referenced in this post.  Additionally, this blog post while editorial in nature, gives plenty to think about and a place to begin investigating the big, initial question “Does FP matter?”

The first problem on the Euler Project list is:  Add all the natural numbers below one thousand that are multiples of 3 or 5.  Simple enough, and for my purposes a perfectly trivial thing to go do in LINQ and F#.

Solution in LINQ

 I did implement the solution using the two different syntax notations (IEnumerable and query) so you could get a look at the coding options available. 

The first thing we notice about the solution is its really short.  One of the advantages of using FP techniques is that they reduce the amount of code that has to be written.  This clearly leads to fewer defects and higher developer productivity.  Next, look at how easy it is to process a list of anything.  In this case it’s just a list of integers but we can just as easily process a list of objects.  Consider doing exactly the same implementation using straight forward C# code to iterate over the list of integers and accumulate the answer.  Here we accomplish it all with one program statement.  Again, notice the two different syntax styles available.

Is VB back?  No…  You see the var keyword in the solution and some of you may be tempted to think of variants from Visual Basic 6.0.  Don’t.  var is a keyword introduced with C# 3.0 to implement a language feature called type inference (also called anonymous types).  The variables answerIEnumerable and answerQuery are both 100% strongly typed at compile time.

Examining the code a little closer we can dig out other FP influences.  Look at  Enumerable.Range(1, 999).Where(x => (x % 3 == 0) || (x % 5 == 0)).  See anything unusual?  The static call to Enumerable.Range(…) returns an IEnumerable interface.  It appears that IEnumerable implements a Where method.  It does but it does so because we included a reference to System.Linq.  What is going on here is the use of something called an extension method.  Extension methods allow existing classes to be extended without relying on inheritance or having to change the class’s source code.

Of course there’s “x => (x % 3 == 0) || (x % 5 == 0)” lambda expression.  Follow the link provided and read Eric White’s explanation, its quite good. 

Solution in F#

Nothing really exciting about the F# implementation other than an example of using the let key word.  We do essentially the same thing as in the LINQ solution but in the F# syntax.  Not much FP here; and totally breaking FP purity with the use of the printfn call.

Hmm…  Even shorter.

Specific to F# is that #light directive.  F# is derivative of OCaml the #light directive tells the F# compiler to relax some of it’s syntax rules to make your F# code portable back to OCaml.  Of course, our example above is not portable because we use the System.Console .NET namespace.

Notice the use of the |> notation.  This operator is the forward pipe operator and allows us to string the output of one operation into the input of another.

We also use the fun operator.  This is the syntax for anonymous functions (lambda expressions) in F#.  There is a second keyword for defining functions in F#, oddly enough, it’s called function.  One of the principle differences of the function keyword is that it allows you to to use pattern matching (not shown in our example).

We’ll drift for a moment to introduce a couple of F# concepts since we mentioned pattern matching.  Pattern matching in F# is a flow control construct that acts somewhat like a case statement in C#.  Here’s an example:

Essentially take a single value and match it to something else.  The right hand side of each -> could be an operation of some form instead of a literal value as shown.  Hence, you can use pattern matching to do conditional processing on the input.  Now, since the function keyword allows for pattern matching and pattern matching operates on a single value, the function operator can only take a single parameter.  So in order to define a function with 2 parameters, we’d have to resort to something like:

let x1 = (function x -> function y -> x + y ) 1 2

Nasty.

That’s where the concept of a tuple comes in.  A tuple is an immutable data type that can contain multiple values.  The syntax is simply to group the values inside of parentheses.  So our nasty looking function becomes:

let x2 = (function (x, y) -> x + y) (1, 2)

Much better.  Of course, this implies that pattern matching can handle tuples, which it can.

Since we’ve already seen it in action so much let’s talk about let.  Our example uses a the let keyword.  let operates much like the word let in mathematical disciplines.  It defines the preconditions of the problem to be solved or proof to be derived.  let binds a value to an identifier.  We don’t show it above but  let can be used to define an immutable value (think constant) as in:

Here x and y are immutable values of 10 and 5 respectively, and will never change.  While the 3rd statement binds the identifier diagonal to the function value System.Math.Sqrt(x*x + y*y).  Here we see type inference at work because the F# compiler will infer the types of x and y to be integers but will infer from the last statement that diagonal will be of type float.  We may have to come back to let in it’s own post as we’ve really just skimmed over how central it is to the heart of F#.

From an FP perspective, the concept of immutability is critical.  The reason is this, FP is one way to approach the synchronization issues of scaling across multiple computational units (whether those units are clustered computers, distributed computers, multiple processors, or multiple cores on a single processor).  No need for us to get into the issues of threading here but we all know the synchronization issues that arise with shared state.  Immutable data structures solve certain synchronization issues (contention, for example).  If we call the basic unit of computation a thread (or fiber, etc.) then each thread has it’s own immutable copy of the data.  Again, we could have another post strictly about this issue.

More to come

In this post we touched on some of the basic building blocks of FP even if we didn’t really do any FP coding.  We talked about brevity, type inferencing (anonymous types), extension methods, lambda expressions (anonymous methods), pattern matching, tuples, data immutability, and a scattering of syntax facts.  There’s lots more to learn including concepts like; lazy evaluation, deferred execution, aggregation, pure functions, and what functional style programming (known as, declarative) really is.

That just about exhausts what I’ve learned so far.  I appreciate you letting me ramble and dump it all on you.  It’s a great exercise for me.  The next Project Euler problem requires recursion.  Since the principle method of iteration in FP is recursion we’ll have more to talk about.

I’ve spent today looking at LINQ and F#.    I’m sure I’ll spend quite a bit of the future looking into them.  I’m certainly no clairvoyant or great prognosticator of technology but I do sense a technology shift coming.  It’s the same feeling I got in the late 80’s when object oriented programming began to gain momentum as the way of the future. 

I wish I could write an authoritative technical post on either LINQ or F# but so far the only thing I can say authoritatively is that I have a lot to learn.  I spent most of last week contrasting imperative programming with functional programming and doing searches on “Why should I care about functional programming?“  I decided that if I don’t want to become irrelevant in about 5 years, I’d better get on with learning functional programming techniques.  I only occasionally dabbled in Lisp years ago and trying to capture the functional mindset is proving really difficult for me.

I did use both LINQ and F# to solve the first problem on the Euler Project list.  I’ll post my solutions later once I believe they’re functional solutions instead of imperative; honestly, right now, I’m having trouble knowing the difference.  I’d like to go through the entire list solving all 237 problems in both LINQ and F# but I’m sure my work schedule won’t allow that.  Hopefully, I can work through enough of them that I can get over the hump of thinking like an imperative programmer.  Luckily (or unluckily) F# doesn’t force you into pure functional programming, it tries to ease you into it.

Toss the new features of Enterprise Library 4.1 into the mix and my time looks pretty limited.

No pressure though…  I’m only predicting that I have to change my entire approach to problem solving in order to stay relevant in my chosen profession.

C# 3.0 includes lambda expressions.  I won’t go into their explanation; there are plenty of people out there to tell you the syntax progression of delegates => anonymous methods => lambda expressions.  But I will point you to a cool example I found.  It’s cool enough to reproduce the code here.  Clearly, being a programmer, I have no problem stealing implementations from other programmers if they save me time.

This example simply loads a page via an HttpWebRequest and asynchronously downloads the page.  It makes liberal use of lambda expressions and I just think it looks cool.  A bit difficult to follow on a single-pass read but worth looking at.  I haven’t compiled and tested it so if you have questions or gripes, please address them to the original author.

While a buddy of mine feels it complicates the code, the one change I’d be tempted to make would be to in-line the readResult lambda directly into the call to responseStream.BeginRead(buffer, 0, buffer.Length, rc, null);.  That would spatially move it above the responseStream.EndRead(readResult) call.  That would preserve the flow of control a little better.  I do realize that creates a recursive situation, so in this example it doesn’t work but it does work in the typical BeginInvoke-EndInvoke scenario.  Food for thought.

For the last few years I’ve been following the progress of the Microsoft patterns & practices (p&p)group.  While the p&p group addresses many Windows and .NET development concerns, I’ve been most interested in what they genericly call “guidance”.  The p&p group offers guidance on a wide variety of topics as well as giving developers the tools to create their own integrated guidance packages.  There is a browser call the “Microsoft Guidance Explorer” to allow you to search, browse, and apply the various guidance packages.

One of the more important guidance topics available is the Application Architecture Guide (AAG).  The AAG is actually a book that describes the principles, patterns and practices recommended by Microsoft for developing solutions for the .NET platform.  AAG has excellent recommendations for selecting the correct technologies under different development scenarios (for instance, when to use WCF vs. ASMX).

The AAG sets the context for application architecture primarily using two “road maps” – the Architecture Meta-Frame (AMF) and the Reference Application Architecture (RAA).  The AMF helps the software architect focus on the the key questions for selecting architecture to employ for their application solution needs.  The RAA represents the canonical view of generic layered application.  The AMF and RAA diagrams from the guide are shown below.

Architecture Meta-Frame

Reference Application Architecture

As you can see, nothing earth shattering but it’s important to have a common frame of reference for discussions.  This is particularly true when we discuss the wide array of Microsoft development technologies.  When necessary, I will reference the AMF and the RAA when I present topics here.  You should read the section, “Fast Track: A Guide for Getting Started and Applying the Guidance” for a full explanation of these concepts.

Hopefully, with a stated frame of reference, I’ll be able to give better clarity on the topics I present here.

“Believing in luck is the act of taking probability personally.”  I heard Penn Jillette of the comedy team Penn and Teller say this on an episode of “Bullshit!“.  As a gambler, I tend to agree and really don’t believe in “luck”.  As a software professional, it’s a different story…

My entire career has been a series of tremendously lucky breaks.  My first professional job out of college was with a successful company that wrote various types of computer aided design (CAD) tools on various flavors of the Unix platform.  Pretty interesting stuff but the true break was the development team I fell backwards into.  The manager, Mike, of that team ended up asking me to be his best man at his wedding, and  the other guys, Tom and Mark, have been equally close friends since as well.  Professionally each of them happened to be some of the most technically proficient and driven people I know to this day.  I’ve said on numerous occasions that working with any of them makes me work “two gears” above my ability alone. 

Being a young, very opinionated (and embarrassingly arrogant) developer at the time I didn’t know I was about to get another break.  We were told to port all of our software to Windows 3.0.  Mark embraced the change and I fought it at every turn.  I loved Unix and would be damned to let my management take away my SPARCStation and plop a Windows box down.  After all, I’d invested a lot of time and effort into learning the inner workings of Unix, setting up a “first class” development environment made up of vi, cc, gdb, various csh/ksh scripts, xterms, and Perl (it was 1992 after all).  How could Windows development be better?  I argued endlessly with Mark while Tom laughed at both of us  (Tom avoided the fray because he was busy learning a little known, flash-in-the-pan language called HTML hence, he’d become platform agnostic… the coward).  Mark made the argument that the development tools were nicer.  Eventually, I had no choice because of pressing deadlines.  The biggest stroke of luck here was that I had to sheepishly eat my own words and admit to Mark that I’d been dead wrong.

At the same company, I was afforded the opportunity to learn a then little known technology called the component object model (COM).  Not only that, my group was given professional training in COM a couple years later by a guy named Chris Sells.  So I got COM instruction from Chris Sells for free (at least to me)…  How lucky is that?

Not long after that, I was moved to a position where my basic responsibility was “solve really hard problems”.  That group didn’t have much COM and Windows development experience so for about a year I got to learn anything I wanted and do so while applying it directly to pressing real development concerns.

That run of luck stuck into 2001-2003 years of the .NET era as well.  When .NET was released in early 2002, I had recently taken a job as architect doing a ground-up design on a new application to be done on the soon to be released .NET Framework in the new language C#.  That same job reunited me with Tom and Mark (and introduced me to Roy)  - both of whom had gone off, learned different areas of technology and frankly become experts in them.  One having focused on web-development and the other having focused on the Java technology.  They made me feel like I’d been wasting my time.  Given the opportunity, I soaked up any knowledge I could from them.  At that time, we investigated a young implementation on the early Microsoft patterns & practices website called the Enterprise Infrastructure Framework (EIF – later to become the Enterprise Library).  We didn’t use it but logged the information away for later.

Several years later around 2006, after another job in between, Tom and I teamed together to write a software development kit (SDK) and framework for unmanned aerial systems (UAS) for the Army’s Aviation Mission Planning System (AMPS).  We chose the now very mature Enterprise Library (EL) 3.1 as our underlying foundation.  Specifically, using ObjectBuilder for our underlying object creation and configuration pipeline.  Tom also used the Composite UI Application Block (CAB) to develop a user experience framework application.  We quickly set about using EL, ObjectBuilder, WCF, and the CAB to create a service oriented framework that let us add new vehicles with as little effort as possible.  By the time I left that position, our managing system engineer actually implented our last vehicle application (2 actually) in 4 days of effort.  Given the specifications for the aircraft he simply wrote new configuration files and we had a new deliverable application.  All that and at the same time in this job I get to work with what I would later come to know as the group that finally took over “best development team” for my career.  Thanks to Tom, Roy, Kory, Mike, Seth, and Bo for that new high in my career.

Which brings me to the job I just recently took.  I work for a team under the direction of the Army to build the military version of Flight Operational Quality Assurance (FOQA).  Not only is this an exciting and interesting problem domain, my managers here have asked me to learn everything I can about .NET 3.5 and related technolgies and their applicability to our new effort.  So, I’m getting paid again to learn all the latest technolgies like EL 4.1, Unity, WCF, WPF, WWF, Silverlight, and Prism.

I’ve restarted my blog because my managers here want me to dissiminate as much of that information as I can to my colleagues.  So, my lucky streak is alive.  In an effort to pay the universe back for it – I’ll try to share my findings here with others.  I promise to attempt to put things into the most logical context I can.  Where possible I will relate the various technologies to the p&p Application Architecture Guide and what it calls the “architecture meta-frame” and the “reference application architecture”.

There you have it.  That’s how I got here.  That’s how I came to be principly a Windows platform developer (I still love Unix though).   I didn’t even have to be all that good at this stuff.  I’m not kidding when I say…

I got here through shear luck.

OK, I’m not trying to be arrogant and compare myself to Einstein.  That’s just the default picture that comes with the theme I’ve chosen (at least, temporarily).   I’ve also determined that in order to want to maintain a blog, you have to be a pretty self-involved bastard.  It takes work to set this shit up.  My hat is off to those folks out there that blog daily.

I develop software for a living.  The last thing I want to do is more programming in order to use what should boil down to being a tool.  I’m using WordPress and by all accounts and recommendations from friends, it’s the best blogging tool out there.  I may have to write a blogging tool.

I’ll have actual content blogs up soon…  Stay tuned.

After about a week and a half of wrangling with my domain host, I have installed WordPress.   I won’t leave the blog looking so generic.  Hopefully, within the next few days I’ll get everything ironed out.