This week's giveaway is in the EJB and other Java EE Technologies forum.
We're giving away four copies of EJB 3 in Action and have Debu Panda, Reza Rahman, Ryan Cuprak, and Michael Remijan on-line!
See this thread for details.
The moose likes IDEs, Version Control and other tools and the fly likes Background required for DSL Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of EJB 3 in Action this week in the EJB and other Java EE Technologies forum!
JavaRanch » Java Forums » Engineering » IDEs, Version Control and other tools
Bookmark "Background required for DSL " Watch "Background required for DSL " New topic
Author

Background required for DSL

nehaa arora
Greenhorn

Joined: Oct 27, 2009
Posts: 14
Hello Prof. Parr!

What computer language fundamentals are required to get on with building/designing a DSL beyond the ones required for writing 'theoretical' compilers?
Thanks!
Terence Parr
author
Ranch Hand

Joined: Jan 13, 2010
Posts: 35
nehaa arora wrote:What computer language fundamentals are required to get on with building/designing a DSL beyond the ones required for writing 'theoretical' compilers?


Hi Nehaa Well, first let's distinguish between designing and implementing a DSL. Designing a DSL means coming up with the syntax and the meaning (semantics) of each construct in the language. Implementing it means creating a language application that reads in statements in the DSL, decodes them, and executes them.

My book doesn't really concern itself with designing languages per se, though you do sort of get experience looking at different languages and so on. On to your question about fundamentals required for building non-compiler applications. Compilers are highly specialized and extremely complicated programs, at least for large general-purpose programming languages. If you can build such a beast, you have the background necessary to build interpreters and translators. The skill sets are very similar but you only need a subset of those skills to build interpreters and translators. Well, actually, you need a few different skills. For example, to build a translator you need to learn about template engines so you can generate source code without using print statements. The skills in common are things like parsing, tree construction, symbol table management, and type checking.
Burk Hufnagel
Ranch Hand

Joined: Oct 01, 2001
Posts: 814
    
    3
Terrence,

After reading many of your posts, Im getting the feeling that even something like a RESTful web service could be considered a small and rather limited DSL. Would you agree?

Burk


SCJP, SCJD, SCEA 5 "Any sufficiently analyzed magic is indistinguishable from science!" Agatha Heterodyne (Girl Genius)
Terence Parr
author
Ranch Hand

Joined: Jan 13, 2010
Posts: 35
Burk Hufnagel wrote:even something like a RESTful web service could be considered a small and rather limited DSL.


Sure. Anything that is not random text follows the format, hence, a language. URLs certainly follow a language. The output of the Web service itself must also follow a format. This is usually XML or JSON etc... Those are languages.

When your world is a parser generator, everything looks like a parsing problem ;)
Burk Hufnagel
Ranch Hand

Joined: Oct 01, 2001
Posts: 814
    
    3
Terence Parr wrote:When your world is a parser generator, everything looks like a parsing problem ;)


Cool. Reminds me of a friend who sees almost all programming in terms of a database.
nehaa arora
Greenhorn

Joined: Oct 27, 2009
Posts: 14
Terence Parr wrote:
Burk Hufnagel wrote:even something like a RESTful web service could be considered a small and rather limited DSL.


Sure. Anything that is not random text follows the format, hence, a language. URLs certainly follow a language. The output of the Web service itself must also follow a format. This is usually XML or JSON etc... Those are languages.

When your world is a parser generator, everything looks like a parsing problem ;)


Wow! that is indeed an interesting view to problem solving! but it would need a lot of experience to know you are not hammering down your screws?!

Neha
Terence Parr
author
Ranch Hand

Joined: Jan 13, 2010
Posts: 35
nehaa arora wrote:Wow! that is indeed an interesting view to problem solving! but it would need a lot of experience to know you are not hammering down your screws?!


Yep, experience helps in two ways: knowing when to go DSL and being able to get their quickly. It also helps in designing the language (syntax/semantics)...so three ways.

I cringe when see of the languages out there. I remember a friend, who was designing a new programming language, telling me about a bug he had in his parser. It allowed what he thought was a cool language feature. It was a weird special case. Every tool that parsed that language would have to have a bug to match it.
nehaa arora
Greenhorn

Joined: Oct 27, 2009
Posts: 14
Terence Parr wrote:
Yep, experience helps in two ways: knowing when to go DSL and being able to get their quickly. It also helps in designing the language (syntax/semantics)...so three ways.

I cringe when see of the languages out there. I remember a friend, who was designing a new programming language, telling me about a bug he had in his parser. It allowed what he thought was a cool language feature. It was a weird special case. Every tool that parsed that language would have to have a bug to match it.


Hmm...any hints as to how one can attain nirvana here? I know nothing beats making your mistakes and learning from them but a guru can sure set one on the right path to the needed grind. Any recommended references for the above three, besides your book?

Thanks,
Neha
Burk Hufnagel
Ranch Hand

Joined: Oct 01, 2001
Posts: 814
    
    3
nehaa arora wrote:Hmm...any hints as to how one can attain nirvana here? I know nothing beats making your mistakes and learning from them but a guru can sure set one on the right path to the needed grind. Any recommended references for the above three, besides your book?


Great question Neha! Please Terrence, inquiring minds want to know. Is there a mailing list or Google group for DSL designer/implemeters that you're aware of? Or, perhaps some web site where they congregate to share mistakes and successes? If not maybe you could start some centered around the book's web site.

Thanks,
Burk

Terence Parr
author
Ranch Hand

Joined: Jan 13, 2010
Posts: 35
Burk Hufnagel wrote:Great question Neha! Please Terence, inquiring minds want to know. Is there a mailing list or Google group for DSL designer/implemeters that you're aware of? Or, perhaps some web site where they congregate to share mistakes and successes? If not maybe you could start some centered around the book's web site.


Let's take design first. I think exposing yourself to lots of languages is the key to avoiding "sins". Try to pick out what you like and dislike from some languages. Think about how languages evolve. For example, Java has gotten WAY to complicated for my taste. Some of the weird edge cases are truly frightening in complexity. It also lacks gotos, which sucks for a guy like me that has to *generate* Java a lot. Try to look at everything through the lens of a language designer. during the day tomorrow while doing your job, try to pick out the languages you see (and imagine the hidden ones): command-line shell, URL, HTTP, SMTP (mail), network protocols, programming languages, HTML, XML, config files, ...

As for implementation, you can hang out on the antlr-interest list or comp.compilers newsgroup (though I admit I don't read that anymore). Stuff pops up on various lists/forums, but too much content focuses on compilers. That's why I wrote this book and teach a non-compiler language course at Univ. of San Francisco. Almost no one is writing a compiler but all of us are writing little languages for everything from dat files to graphics DSLs etc...
Burk Hufnagel
Ranch Hand

Joined: Oct 01, 2001
Posts: 814
    
    3
Terence Parr wrote:
Burk Hufnagel wrote:Great question Neha! Please Terence, inquiring minds want to know. Is there a mailing list or Google group for DSL designer/implemeters that you're aware of? Or, perhaps some web site where they congregate to share mistakes and successes? If not maybe you could start some centered around the book's web site.


Let's take design first. I think exposing yourself to lots of languages is the key to avoiding "sins". Try to pick out what you like and dislike from some languages. Think about how languages evolve. For example, Java has gotten WAY to complicated for my taste. Some of the weird edge cases are truly frightening in complexity. It also lacks gotos, which sucks for a guy like me that has to *generate* Java a lot. Try to look at everything through the lens of a language designer. during the day tomorrow while doing your job, try to pick out the languages you see (and imagine the hidden ones): command-line shell, URL, HTTP, SMTP (mail), network protocols, programming languages, HTML, XML, config files, ...

As for implementation, you can hang out on the antlr-interest list or comp.compilers newsgroup (though I admit I don't read that anymore). Stuff pops up on various lists/forums, but too much content focuses on compilers. That's why I wrote this book and teach a non-compiler language course at Univ. of San Francisco. Almost no one is writing a compiler but all of us are writing little languages for everything from dat files to graphics DSLs etc...


Terrence,
I agree that Java's gotten more complicated over the years. Annotations and generics are very powerful, but I think it's easy to get lost in there and I don't even want to think about tossing AOP into the mix.

Lately I've been learning more about unix, and it seems chock full of little languages - and the ones I've seen so far all play nicely with each other because they all accept text as an input and create it as an output. Seems like a good plan...
nehaa arora
Greenhorn

Joined: Oct 27, 2009
Posts: 14
Burk,

IMHO Java as is a very bloated language with umpteen features, which one could pick and choose to use. While designing a DSL, we would need to have a more focused set of language features.As Terrence advised we'd need to pick pretty features that fit the context from across languages (procedural/functional/object oriented/etc), moving beyond the language of implementation (though the language itself could show limitations as discussed here). And I surely agree with you in that the unix model would be a neat way to model a DSL on.


Thanks,
Neha
Burk Hufnagel
Ranch Hand

Joined: Oct 01, 2001
Posts: 814
    
    3
Neha,
I agree with most of your comment. And Java's definitely been putting on weight. But just because a feature is there, that doesn't mean you have to use it - so I don't see that as big deal. Though you may feel that you need to explore each feature so you can better decide which ones make sense to use - which could take some time. But in the end that means you have more flexibility in deciding what to include and how to implement it.

Does that make sense?

Burk
nehaa arora
Greenhorn

Joined: Oct 27, 2009
Posts: 14
Burk,

Quite agreeable. In the same line of thought, I assume language features more than a rich api would support DSL development better. And even to have an intermediate knowledge of Java one would need to be well versed with the language features ( though they keep on adding more as they did with enums for java 5 - quite nice!). I have a very vague feeling as to constructs like closures in a language are way more handy tools than probable roadblocking ones like Generics (in Java) as they are not so elegant in their behavior (is this my inexperience speaking?! ) This takes me back to Terrence's comment

I remember a friend, who was designing a new programming language, telling me about a bug he had in his parser. It allowed what he thought was a cool language feature. It was a weird special case. Every tool that parsed that language would have to have a bug to match it.


So here I am bothered about corner cases in my development language. I guess, then, the choice of language used for coming up with a DSL does have a bearing on its working and success. Thoughts?

Thanks,
Neha
Burk Hufnagel
Ranch Hand

Joined: Oct 01, 2001
Posts: 814
    
    3
nehaa arora wrote:I guess, then, the choice of language used for coming up with a DSL does have a bearing on its working and success. Thoughts?


I'm going to have to think on this a bit. I started answering this several times and have deleted each one because I have a feeling that the choice is important, but not because of any particular language feature or lack thereof but because in many cases it's a complicated task so you need to pick a tool (language) you can use well and can make yourself understood clearly.

So it's not necessarily the language itself that's important, it's how good a craftsman are you with it. Can people read your code and understand it, or are you doing lots of cool, cleaver things that only experts understand? While you can write clever code in most languages, some make it easier than others and I suggest staying away from them if there are other developers involved.

I hope that makes some kind of sense,
Burk
nehaa arora
Greenhorn

Joined: Oct 27, 2009
Posts: 14
Hi Burk,

Yes i agree..its more like programming for any other solution DSLs are not a different problem in that regard. A programming language helps in ease of development but no more than a good knowledge of the language tool.

Thanks,
Neha
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: Background required for DSL
 
Similar Threads
What is the difference between a DSL and Metaprogramming
How Fast is Your Broadband Connection?
for when you have outgrown dominos
Confugure Tomcat To Host a Server
Need help in using apache camel routing