Stumbling Blocks For Domain Specific Languages

[moved from LanguageOrientedProgramming, edited slightly]

As for whether LanguageOrientedProgramming (and DomainSpecificLanguages and MetaProgramming in general) are a good idea, I see two big stumbling blocks:

The first is cultural. Programmers don't want to learn new languages or new paradigms. Lisp has provided the ability to create DSLs since the 1960s, and the arguments against it are never "because it only compiles to Lisp" and only rarely "because it doesn't let you define new syntax" (which is false, BTW. The semantics of Lisp are described in terms of nested lists, and it's not a huge deal to write a shift/reduce parser that outputs Lisp program fragments, which could then be macroexpanded/compiled/evaluated like any Lisp program. The reason it hasn't happened is that most Lisp programmers like EssExpressions and don't really want to switch to infix syntax. A more restricted form of adaptable syntax, ReaderMacro?s, is part of the CommonLisp standard and is fairly widely used.)

Instead, programmers usually argue that Lisp gives the programmer too much freedom. They worry that someone else will create a language that they'll then have to learn in order to maintain the code. Take a look at WhyWeHateLisp and SocialProblemsOfLisp. People argue that with Java and C++, then at least the control structures will be familiar, even if they have to learn huge class libraries.

It may be simpler to learn a DSL than an equivalent class library, but many people would prefer learning the class library. Class libraries give you something familiar (paradigm + basic language structures) to hold on to, while a DSL is likely completely new. That perception - that a class library is learnable while a new language is a big risk - is a big stumbling block to LanguageOrientedProgramming.

The second problem is technical. Defining a DSL is basically an exercise in compiler writing. There's a certain inherent difficulty to writing compilers which you can't paper over with editors and other DSLs. I really doubt that you can make it simpler than LispMacros. Lisp takes away all the syntax issues, gives you raw program structures in terms of forms and symbols, and then gives you a well-defined target language (Lisp) that already has a substantial number of features. Despite this, writing good Lisp macros is considered an exercise for the advanced programmer.

Now imagine that instead of compiling to high-level Lisp, which may already have been extended with macros, you want to compile to Java or machine code. Do you really think the average corporate programmer, who has enough trouble just writing Java, is going to be able to compile to it? According to the paper, "Language Oriented Programming: The Next Programming Paradigm the big timesink in ordinary programming is mapping your conceptual model of the problem into the implementation model of the target language. When you construct a DSL, you not only have to map your conceptual model of the problem onto the implementation language, but you have to formalize that map sufficiently to let the computer automatically transform between the two of them. The paper seemed to suggest that the transformation step is fairly simple with the help of editors and LOP DSLs, but as anyone who's taken a compiler-design course can tell you, there's a lot of conceptual trickiness that you need to deal with regardless of the tools.

I think LanguageOrientedProgramming is a good idea, but I really think one should begin with Lisp or ML and then try to extend that rather than reinventing the wheel from (ick) Java. Java was not designed for LOP; in fact, it took out most of the available LOP features in C++ (the preprocessor, OperatorOverloading, destructors, templates, etc.) It's probably necessary to provide Java bindings and extensions for marketing needs, but if you try designing based on what Java gives you, or even what IntellijIdea provides on top of Java, you'll be way behind where the Lisp folks were in the 1970s. -- JonathanTang

I think the paper being referred to is the one at the top of LanguageOrientedProgramming as this material was moved from that page some time ago. -- JohnFletcher

One might say that comprehension of a language is measured by knowing the syntactic constructs and how they translate to semantic components, the ability to predict evaluation orders, etc.

DSLs, syntax extensions, and macros violate this expertise, allowing for strange new syntax with strange new evaluation orders and thus introducing a learning burden on later maintainers. This is not an insignificant price to pay for the Meta features, so it must be weighed carefully... and avoided if a feasible solution (with acceptable characteristics) is readily available in the host language.

The problem of MetaProgramming proliferation can very likely be heavily curbed by use of community and convention. E.g. One can require by convention that the MetaProgramming proponent obtain from a few other programmers (that have stake in the code) endorsement for each new macro, syntax extension, DSL, use of reflection or MetaObjectProtocol, and even creation of a new framework (frameworks share many of the negative properties of DSLs). This should raise the bar just far enough to prevent the useless, unintuitive, and obnoxious uses of MetaProgramming, and would also help ensure discussion towards other possibilities the proponent might not have seen.

[Moved from MetaProgramming]

In addition, there are other reasons why writing a good DomainSpecificLanguage is difficult.

That said, if there is a significant amount of domain knowledge that can be expressed much more clearly in a meta-language than in a domain-independent language (including libraries written using the native facilities of that language), or if you need to let users customize stuff (without requiring them to learn a complete programming language) then MetaProgramming might be a good idea.

Regarding protecting the junior programmers from themselves:

In a large project you need both leaders and followers. Followers will carry out 50-80% of the project code. In such a case, you will appreciate order and punctuality over creativity. Please remember, it is not easy to manage too many creative people. Not to mention our salaries, and the time it takes to find us.

The followers do not care about changing the system architecture. They want to be productive. They will be more productive in a secure, well defined and measured environment. They need guidance and they will bless anyone who gives them more productivity for less programming power.

Personally, I think these 'followers' should be 'technical followers' and the leaders in domain expertise. -- ShaiBenYehuda.

Writing DSLs is not the task of junior programmers. Criticizing DSL-building tools in the hands of juniors is wrong-headed, IMO. Typically it's the technical lead(s) who define a new object language that the juniors use to express solutions to domain problems. This works well with pyramid team structures since the few experts at the apex provide the small, highly-factored object languages that the many juniors at the base use to describe the program.

When it comes to DSLs, the technical leads define the languages that the juniors use to implement the user stories. -- BryanEdds

In my opinion, one should try domain-specific (DS) API's first. That way you don't have to reinvent a language and compiler/interpreter from scratch. However, some existing languages don't seem up to this tasks. For example, named optional parameters are very handy for DS API's in my opinion. But many languages have awkward or missing support for this. Use of OOP attributes as a substitute can sometimes be too verbose (although having the option of both syntax techniques, or equivalent, based on specific instances is handy.) -t

See also an article by MartinFowler:


View edit of September 11, 2014 or FindPage with title or text search