Just my opinion...

A long time friend and I have had a long-term discussion of sorts about mathematics.  He and I were Computer Science majors together in college and worked in the same lab during graduate school.  My first major as an undergraduate was physics so I had to have considerably more math than he did.  While an incredibly smart guy, he loathed math while I saw math as essential.

His issues with math were how it was taught, he recently sent me this article, “A Mathematician’s Lament“.  It’s a long article so don’t think you’ll breeze through it in 2 minutes but it’s worth reading.

I’d love to hear some feedback from others on this.

I will attempt to keep this post accessible to a general audience although it is about software development. My intent is to explain my long held philosophy of getting the domain experts as close to the software development process as possible. Since they are seldom experts in software development, delving deep into esoteric software topics is the quickest way to run them off. There will be some pretty geeky stuff buried in here but it should be easy to separate it from the general philosophy.

I’ve been interested in Language Oriented Programming (LOP) and Domain Specific Languages (DSL) since graduate school and my first professional software development job at Intergraph Corporation. I didn’t call the concepts LOP and DSL but that’s what I was doing. One of my last courses in graduate school was on compiler and language theory and I wrote parsers and translators early on at Intergraph. I’ll come back to this later.

First let me explain to the non-developer types who might be reading a little about how software gets developed.

The essence of software development is solving problems. To solve a problem, you have to understand the problem. To understand the problem, you have to learn about the discipline (or domain) where the problem resides. To learn the domain, you need to interact with an expert in that domain. This interaction is known as requirements gathering to programmers. Then the development team writes a requirements document and, if you’re lucky, the domain experts will verify that the development team interpreted the interactions with them correctly and captured the requirements accurately. In larger organizations and software development environments, programmers usually do not interact with the customer representative directly. Often the customer isn’t the domain expert, they are relaying information gathered from the domain experts within their organization. So, programmers get a “requirements specification” that was told to an intermediary (like a system engineer or project manager), from another intermediary who had spoken with the domain experts, then interpreted to the programmer.

Now, I want you to know that what I just described a pretty thin layer of abstraction between the domain expert and the programmer. In some instances, there are more layers between the two. There are plenty of codified software processes that try to address this issue. There are whole industries built around the concept of telling you how the process of software development is supposed to be done including the requirements gathering step (You don’t want to get me started on that, that’s a whole new post). The other developers reading this post will tell you that I just described a very small part of “the process”. There are a lot more steps. If you want to know how complicated it can get and how many more steps there are and you have millions of dollars to spend, the Software Engineering Institute at Carnegie Mellon will be happy to teach you how to become a CMMI Level 5 organization and spend 98% of your software development budget on “the process”, leaving 2% to actually write the software. Very profitable little hustle business the SEI has don’t ya’ think? But, I digress.

To be fair, there are software processes that are better. To learn about some of them, look up agile software development techniques.

Eventually, larger organizations started to understand that “process” could not address this problem of interpretation and they started hiring “subject matter experts” (SME – usually pronounced “smee”) to work directly with the programmers. So let’s proceed with the rest of this post assuming that the programmer and SMEs get to interact directly. As a point of interest here, I share an office with one of the SMEs for the project I’m currently working on. Even cooler is that his domain expertise is in flying Blackhawk helicopters. It gets better on my current job too. My direct contractor boss is also a SME (Apache helicopter pilot), another of our project managers is yet another Blackhawk pilot and I’m told we’re hiring another pilot. From a programmer’s perspective, this is a great situation. If I have a domain question I don’t even get up, I just say “Hey, Bernie.”

Right now Bernie has to define a lot of the concepts that I will be working on. These concepts include vocabulary and rules which describe the elemental properties of the problem domain. It sounds a lot like he’s putting the language of flying a Blackhawk down in requirements.

Finally, we’ve reached the core of this post – bridging the gap between subject matter experts and programmers.

What if Bernie and I were to define a language (or a collection of smaller languages) that captured the problem domain concepts and the software could be directly written from Bernie’s description in this language? What if that language was core domain logic for the software? That is a domain specific language (DSL).

Bernie does exactly what his job is supposed to be but writes down his expertise in our DSL. Now my job would become writing software that I understand very well, parsers, abstract expression trees, emitters, implementing visitor patterns, etc… Mmmm. Real geek stuff. I spend my time on the things I’m actually an expert. The defects I would be fixing have to do with how the requirements Bernie writes are translated into the actual instructions in the software. Think about that statement for a moment. I spend my time working on ensuring that the requirements are correct. Once the system had matured Bernie and I would reach a point where we knew the DSL was always being translated properly and we could now work on expanding the behavior of the software through our DSL and NOT by writing new code, C# and the .NET Framework in the case of my organization.

The many layers of interpretation described earlier have now been replaced by a single, automated step that can be refined and extended. This is such a powerful concept that someone should write a book on it!

What I’ve just described is what Martin Fowler calls an “External DSL”. My fellow developers that I’m trying to evangelize to here have all written what Fowler calls an “Internal DSL”. An internal DSL is what we write when we write an application programming interface (API), a software development kit (SDK), and even a basic object model (classes, methods, etc.) with limited scope. If you have to capture the concept of a “user” in your application you’re going to write some structure or class that represents a user. The C# language has no native construct for a user in your domain so you had to extend the C# language and once you did, you had created an internal DSL.

Plenty of DSLs exist that programmers are familiar with. Structured Query Language (SQL) used to query data from a relational database system is a prime example. Programmers and database administrators write queries in the SQL DSL, then Microsoft SQL Server, Oracle, or MySQL interprets that SQL and inside actual C, C++, or even assembly language instructions get executed. If you want a simple demonstration of the value of a DSL go write a complex JOIN in C++.

Let’s go back to my first job at Intergraph where I worked for the Electronic Design Automation (EDA) division (the division was later sold and is now Mentor Graphics). I was 24ish and just out of graduate school and honestly a little nervous. I’d finally be working beside professional software developers. We wrote software to help electrical engineers design application specific integrated circuits (ASIC), programmable logic devices (PLD), programmable logic controllers (PLC), etc. This was the 1989-1992 time frame, extensible markup language(XML) wasn’t around, hell, the WWW wasn’t around yet. So, there were lot’s of different file formats to describe all types of aspects of EDA. When I got to Intergraph I was asked to fix a bug in a “netlist parser”. I was told that “netlisters” were constantly breaking and that parsing netlist files was a huge issue and they usually put new programmers on fixing those bugs to get their feet wet. So, I got the source and much to my shock, it was a hard-coded conglomeration of nested conditionals… if this, then expect this or that, else if not that, then this if, switch/case… Horror. And this was the parser for just one of the many netlist formats our software had to support. I suddenly understood why netlist parsing was a big deal. Intergraph had 5 other additional and experienced developers working on maintaining netlist parsers.

Then I found out that many of these netlist formats either had relatively trivial grammars to capture or the Backus-Naur Form (BNF) grammar specification was around. So, I started up vi and started writing LEX and YACC specifications to describe the grammar of the parser I was supposed to just be fixing a bug in. I wired together new MAKE files, stabilized my builds and started tweaking the grammars until I could parse every netlist example we had for this particular format. It took me 4 weeks to fix one bug. There were about 12 different netlist formats. The netlist group accounted for about 30% of the data defects the EDA division had and accounted for about 2% of the total source code. I rewrote the rest of the parsers in LEX and YACC. The defect rate dropped to almost 0 about 4 months after I was asked to fix my first defect. The only defects we had after that typically dealt with non-standard netlist formats from other tools. And were always a pretty easy addition or exceptional case in the BNF specification held in LEX and YACC files. So, the netlist development “team” was spread out to other areas and netlist maintenance was no longer a huge issue. I then got assigned to write all parsers for anything we ever had to parse.

None of that was meant as bragging. Far from it, I had just come from a pure Unix environment in school into an environment that had only recently transitioned from VAX/VMS to Unix on workstations. They were still learning the tools and environment and I was already very familiar with it. Unix has a pretty rich environment based around DSLs like awk, troff, etc. based on grammars processed with LEX and YACC. I had also just recently had a graduate level compiler theory course and had been using LEX and YACC at school. The guys I was working with had never heard of LEX and YACC and many had long since forgotten school work. I just stumbled into a problem I was particularly well-suited to solve.

The real point of the story was that I used 2 DSLs to fix the problem. LEX lets you specify the lexical rules for a language (the alphabet and vocabulary) and YACC lets you define the language rules for combining the lexical elements (the sentences and paragraphs). This is the essential process programmers use to write a parser for a new language. We first specify a grammar that describes the language then construct a parser in terms of that grammar. Using LEX and YACC the construction step is automated. To be absolutely precise I used 3 DSLs because MAKE uses another DSL to automated building software.

Throughout the software development discipline DSLs are scattered into all corners. Clearly, developers know DSLs are useful. The problem is that many developers don’t call them domain specific languages. Hence, the problem gets extended because the power of LOP and DSLs aren’t recognized by many developers and pushed to the next level – the end user and domain expert – even though important publications have pointed out the power of DSLs for going on 15 years now. The concepts have obviously been around longer than that but the tools and support to effectively employ LOP/DSLs have really blossomed in the last 10-15 years.

Let’s speculate for a moment. Natural language processinghas advanced considerably in the last 10 years. It still has a way to go but if we worked within a restricted subset of the English language we can conceive of a DSL that was very intuitive for the domain expert. Not only would it be intuitive it could be pretty close to natural spoken English. I’m using English as my example because, 1) I can speak English and 2) syntactically English is pretty difficult to parse because it can be very context sensitive.

Plenty of graphical examples of DSLs exist as well. The Unified Modeling Language(UML) is an example of a graphical DSL. Schematics for electrical circuits are graphical DSLs.

In order to define the basic classes that constitute my object models I often approach the problem as if I were going to write a DSL.  It’s a great way to organize my thought process and it lays out a narative of use for the obejct model.  A very handy tool when you write APIs and SDKs like I do most of the time.  It tends to let me develop a natural way for other developers to approach using my SDKs.  I propose that if all developers of APIs and SDKs approached the problem from a LOP perspective that APIs would be more usable and intuitive.

Creating and maintaining a DSL does have overhead and additional development costs so one must weigh that cost against the long term benefit.  But, I contend that if your application will have any significant longevity then the benefit is clear, at least, to me.

If you’re not a developer I request that if you find yourself apporached by a developer to be a domain expert you remember this blog post and understand the developer’s efforts.  Perhaps you will have some additional shared understanding of the joint objectives you have with them whether a DSL is the goal or not.

If you’re a developer I hope that I’ve convinced you that a LOP approach and a DSL is an option you should consider.

So allow me to teach a little instead.

A friend of mine at work recently joined a pool league.  She knows I play and sent me this in an email:

I have a bridging question. When I played as a teenager, I hooked my index finger over the cue. When I started again, I changed my bridge and was getting pretty comfortable with it. Our team captain suggested I go back to hooking my finger over the cue, so I did. It was comfortable. However, I don’t know if it was my imagination or what, but I seemed to be missing shots that I had been able to make before. Is it possible that just a change in hand position could alter your aim? 

This is a pretty common question I get from novice pool players. I’m no champion but I can give a little advice here. Here’s my reply:

To “hook your finger over the cue” is known as a CLOSED BRIDGE. Bridging off of a “V” you form with your thumb and index finger is an OPEN BRIDGE.

Depending on the speed I have to hit the cue ball, the position of intervening balls, whether the cue ball is close to the rail, etc. I constantly switch between open and closed bridges. Which one you use really gets down to a matter of comfort and the situation at hand.

To answer your question: “Is it possible that just a change in hand position could alter your aim?”

Yes. It is possible but if your pre-shot routine (aim, stance, body positioning, falling into the shot position, practice strokes, stroke) is consistent, it should NOT alter your aim.

More than likely what has been altered is that you’ve made your bridge LESS stable because you’ve removed another finger from the surface of the table. You should have as much of your hand in contact with the table (including the ball of your hand) as is comfortably possible – no matter which bridge you use. Also, you’ve altered how your index finger interacts with the cue. A closed bridge naturally introduces more friction between your hand and the cue. That alone is enough to alter your stroke.

One of the biggest mistakes (and the mistake your APA coach has made) is to tell someone to shoot like they do. He probably uses a closed bridge so he thinks others should as well. Since we all have different physical stature, we all have different comfort positions. Again, comfort and… Fundamentals.

Like any other sport, pool has fundamentals.

Learn a fundamentally sound aiming technique, how to take a fundamentally sound stance, how to make a fundamentally sound bridge, how to execute a fundamentally sound stroke and I assure you eventually the balls will start to fall. Once you’re pocketing the balls, you can move on to position play. My guess is your APA coach is also trying to teach you how to get position… Why? Until you can consistently pocket the balls, getting position doesn’t matter at all.

All the questions I get from beginners I answer essentially the same way: Learn the fundamentals.

“How do I use English?” – Learn the fundamentals first.
“How do I stop the cue ball?” – Learn the fundamentals first.
“How do control the speed of the ball?” – Learn the fundementals first.

There are numerous resources to learning the fundamentals of pool.  Do a quick Google search on the topic.   Even in the response to my friend above, I think I have it out of order.  I should have put “learn a fundamentally sound aiming technique” after stance and stroke.

Trust me on this one because it took me years to get this simple rule through my head in pool. Learn the fundamentals.  Once I got better at the fundamentals, I started running-out.  Anytime I miss a ball or miss position now, I can almost always track it down to a fundamental error.