Domain specific languages (DSL) are great tools to communicate with non-programmers. Normally this group includes business users that would like to configure a system / rule using a fluent language (as in – a natural language). It also includes those like my 8 year old neighbor that knows absolutely nothing about programming. He would love to tell the computer how to perform a small series of operations, without delving into the specifics. Coincidentally, I have been reading up on methodologies to approach DSLs and was introduced to ANTLR.

Enter ANTLR:

What my neighbor needs is an English like grammar. This grammar needs to be parsed into something meaningful at runtime. Every time the grammar changes, the parser would need to change too. ANTLR, is a ‘parser generator’. Once a grammar is defined, ANTLR can code-gen a lexer and a parser for this grammar. The lexer identifies tokens in any input that adheres to the grammar and the parser makes sense of these tokens.

A GUI by the name ANTLRWorks helps you generate the grammar and its corresponding lexer+parser code. Lets take a look at an example that asks the computer if a number A is ‘bigger than’ or ‘smaller than’ number B.

Before you delve into the code, it might be a good idea to download the ANTLR library and the related ANTLRWorks language workbench.

The grammar:

Here is the grammar presented in full. Lets analyze it one piece at a time. (Some parts of the grammar are based on examples from ANTLR. You will find it easier to delve into documentation that way. )

grammar SimpleCalc;
 
@header {
  package com.example.antlr;
}
 
 
 
@members {
    private String answer = "";
 
    private void setAns(String num1, String num2)
    {
        Integer a = Integer.valueOf(num1);
        Integer b = Integer.valueOf(num2);
        if(a>b)
        {
            answer = "yes";
        }
        else
        {
            answer = "no";
        }
    }
}
 
 @lexer::header {
  package com.example.antlr;
}
 
/*------------------------------------------------------------------
 * PARSER RULES
 *------------------------------------------------------------------*/
 
is    returns [String expr]:    'IS' 
                (
                a=NUMBER 'BIGGERTHAN' b=NUMBER {setAns($a.text,$b.text); } 
                | 
                a=NUMBER 'SMALLERTHAN' b=NUMBER {setAns($b.text,$a.text); }
                )
                {$expr=answer;};
 
/*------------------------------------------------------------------
 * LEXER RULES
 *------------------------------------------------------------------*/
 
NUMBER    : (DIGIT)+ ;
 
WHITESPACE : ( '\t' | ' ' | '\r' | '\n'| '\u000C' )+     { $channel = HIDDEN; } ;
 
fragment DIGIT    : '0'..'9' ;

Grammar definition:

grammar SimpleCalc;
 
@header {
  package com.example.antlr;
}

This defines the name of the grammar and the header under which the code needs to be generated. Lexer and parser code can be generated under different packages.

Custom java code:

@members {
    private String answer = "";
 
    private void setAns(String num1, String num2)
    {
        Integer a = Integer.valueOf(num1);
        Integer b = Integer.valueOf(num2);
        if(a>b)
        {
            answer = "yes";
        }
        else
        {
            answer = "no";
        }
    }
}

This is a piece of java code. It is embedded into the parser and can be called when a rule is executed. We maintain a private String by the name ‘answer’. When the rule that compares two numbers is invoked, the setAns() method can be called with the numbers in order to compare them.

Lexer rules:

/*------------------------------------------------------------------
 * LEXER RULES
 *------------------------------------------------------------------*/
 
NUMBER    : (DIGIT)+ ;
 
WHITESPACE : ( '\t' | ' ' | '\r' | '\n'| '\u000C' )+     { $channel = HIDDEN; } ;
 
fragment DIGIT    : '0'..'9' ;

In the lexer section, we identify the bits of the input grammar that the parser can make sense of. A DIGIT ranges from 0-9. When the parser encounters one or more DIGITs, it represents a NUMBER. We also tell ANTLR that any whitespace needs to ignored.

Parser rules:

/*------------------------------------------------------------------
 * PARSER RULES
 *------------------------------------------------------------------*/
 
is    returns [String expr]:    'IS' 
                (
                a=NUMBER 'BIGGERTHAN' b=NUMBER {setAns($a.text,$b.text); } 
                | 
                a=NUMBER 'SMALLERTHAN' b=NUMBER {setAns($b.text,$a.text); }
                )
                {$expr=answer;};

The parser section of the grammar, defines a ‘is’ operation that returns a String. The pattern to match is ‘IS NUMBER ‘BIGGERTHAN’ | ‘SMALLERTHAN’ NUMBER’. The numbers are stored in variables a and b. The code inside the {} brackets is embedded into the parser, just like the code inside @members. As such this code can call the methods in @members and is executed when an appropriate match is made.

The match ‘IS NUMBER BIGGERTHAN NUMBER’ results in the following call setAns($a.text,$b.text);. When the match is for the ‘SMALLERTHAN’ input, simply reverse the numbers in the method call.

Running the pieces:

Now that the grammar is in place, it can be debugged in ANTLRWorks or code-gened and ported to a java project. Debugging in ANTLRWorks reveals interesting internal details. Once can visualize how ANTLR goes about generating a parse tree for our grammar or what the syntax tree for the ‘is’ rule looks like.

ANTLRWorks:

Syntax tree:

The code-gen route allows one to execute the parser for a user defined input.

import org.antlr.runtime.ANTLRStringStream;
import org.antlr.runtime.CommonTokenStream;
 
public class CalcTest
{
    public static void main(String... args)
    {
        new CalcTest().go();
    }
 
    public void go()
    {
        try
        {
            SimpleCalcLexer lex = new SimpleCalcLexer(new ANTLRStringStream("IS 19 SMALLERTHAN 20"));
            CommonTokenStream tokens = new CommonTokenStream(lex);
            SimpleCalcParser parser = new SimpleCalcParser(tokens);
            String eval = parser.is();
            System.out.println(eval);
        }
        catch (Exception e)
        {
            e.printStackTrace();
        }
    }
}

Modifying bits of the program above would allow the input to be obtained from the console. So my neighbor types in ‘IS 20 BIGGERTHAN 19′ and is happy with the result. Now he wants me to make a language that will print multiplication tables. His homework is due this week

As most of you are aware by now, developers can write java robots that can aid a conversation that happens in google wave. A conversation is a wavelet and each reply in this wavelet is called a blip. There are some ‘getting started’ tutorials available out there that are of great help. These links should help you

Official google wave guide
Google wave getting started – Sort of an abridged version of the official guide written by Vogella.

Grasping the overall picture of a java robot is a little difficult. This is because there are no flow or architecture diagrams (at least none that I know of) that show you the sequence of events. Given below is a diagram that does that. Assume that you wrote a java robot that is meant to edit blips in a wavelet. The robot should provide a profanity filter service which will delete objectionable words from the wave. This is how the series of events happen.

Google wave robot sequence of events:

1. User logs in

2. Replies to a blip.

3. Our java robot is already deployed under a .appspot.com address. Its capability allows it to trigger an event that signals that a blip was created.

4. The triggered event is actually a HTTP request that goes out to the java robot servlet from google wave. The servlet then contacts wave using the wave client library API.

5. The blips in the wave are accessed using the client API. An event object return all the blips that were created.

6. The blip in question is identified and edited if needed.

7. Optionally you can include a “gadget” into your wave.  The robot can append a blip and add a gadget to it. A gadget is an XML file that houses, at times, javascript and HTML related content. The gadget can be hosted on any site. This is unlike the java robot, which at the time of this writing is only allowed to be hosted in google app engine (GAE). There are plans to allow developers to host it elsewhere on their own servers.

More on gadgets:

To get a feel for what a gadget is and how it can be used, check out the ISBN gadget and the gadget help files. Gadgets that are integrated with wave have a sense of state. This state is just a saved variable that is accessible by all users on that wave. This is like having a variable exposed to several threads at the same time. If you modify the state variables in the gadget, do so carefully since it is equivalent to accessing a non thread safe variable.

So there you go. The wave is still a developer’s playground and less of a lay man user’s tool. It will soon mature. If you happen to have a wave account, give the robots a try. Let others know what robot you made.

If you are a newbie to working with java threads, this post will help you. Certifications like the SCJP require you to understand java threads to a fair degree. Threads behavior can be difficult to understand even for experienced programmers, so I will try to present some examples which will help candidates identify how threads wait / lock and synchronize.

Lets cut the chit chat and jump into a problem. A program increments a counter in a for loop as shown below.

Synchronizing:

public class StaticSync
{
    public static final Object s_lock = new Object();
    public static int s_counter=0;
    public static void main(String... args)
    {
        new StaticSync().go();
    }
 
    public void go()
    {
        for(int iCounter=0; iCounter<50; iCounter++)
        {
            Thread thread = new Thread(new Incrementer(new StaticSync()));
            thread.start();
        }
    }
}
 
class Incrementer implements Runnable
{
    StaticSync m_staticSync=null;
    public Incrementer(StaticSync staticSync_INP)
    {
        m_staticSync = staticSync_INP;
    }
 
    public void run()
    {
        synchronized (m_staticSync.s_lock)
        {
            m_staticSync.s_counter++;
            System.out.println(m_staticSync.s_counter);
        }
    }
}

This program is pretty simple. This is the sort of code that you can expect on the SCJP exam. Advanced versions of the code above can also appear by adding notify() wait() sleep() etc into the picture. Let us concentrate on the program for now. What can be guaranteed about the output of this program ?

Can we say that Numbers 1 through 50 will be printed for sure ? What can be said about the order in which they are printed. Is there a guarantee about that ?

This program prints numbers 1 through 50 and does it consistently. What that means is that we can guarantee that this program will print 1->50 in ascending order. Why ?

Instances of a thread object lock on the same static object:

One thread manages to obtain a lock. The others wait:

We are creating a new Incrementer instance while starting each thread. We are also passing it a new StaticSync instance for every thread. However when the code tries to update the counter, it synchronizes over a static object. The static object has only one instance, so when one of the 50 thread instances holds this lock, the other thread will have to wait until the lock is released.

Once the lock is released the next thread will update the counter variable. Since only one thread can obtain the lock at any given time, the operation of incrementing the counter and printing its value is atomic. What that means is that the ++ operation and the System.out.println() operation will happen without any other thread intefering in the middle. It happens as one single unit. Note that m_staticSync.s_lock is actually referring to the public static variable through a non static reference. This is not a recommended practice but it was used anyway to help readers understand the concept. The code also throws encapsulation out the window. Our focus is on the thread behavior of the code alone.

Now let us replace synchronized (m_staticSync.s_lock) with synchronized (this). What do you think will change (if there is any) and why ? What does “this” refer to here and how is the locking different from the locking we did in the first example ? If the program is modified so that we synchronize over a static method, which Object’s lock do we obtain ?

I will leave these questions to you as an exercise. The images below should provide a hint.

Threads locking on distinct object instances:

Everyone is a winner:

Moving on to thread interruptions

Interrupting:

public class ThreadInterrupt
{
 
    public static void main(String... args) throws Exception
    {
        new ThreadInterrupt().go();
    }
 
    public void go() throws Exception
    {
        InterruptTest interruptTest = new InterruptTest();
        Thread thread = new Thread(interruptTest);
        thread.start();
        Thread.sleep(1000);
        thread.interrupt();
        Thread.sleep(1000);
    }
 
}
 
class InterruptTest implements Runnable
{
    public void run()
    {
        try
        {
            synchronized (this)
            {
                System.out.println("Waiting");
                wait(30000);
            }
        }
        catch (InterruptedException e)
        {
            System.out.println("Inerrupted");
        }
    }
}

This class shows you how interruption of threads work. When a thread waits or sleeps, it goes into a blocking state. Such threads can be interrupted to come out of the blocking state. When such a thread gets “interrupted” from its blocking state, it throws an InterrutpedException. The output of the program above is

Waiting
Inerrupted

I have never used the interrupt() method at work. But it is useful to know that this can be done. Just remember that only threads that are blocking can be interrupted. Trying to interrupt threads in the running state will not work.

Notification:

public class NotificationExample
{
    public static Object lock = new Object();
    public static void main(String... args)
    {
        new NotificationExample().go();
    }
 
    public void go()
    {
        for(int iCounter=0; iCounter<100 ; iCounter++)
        {
            Thread thread = new Thread(new ThreadWorker());
            thread.start();
        }
        try
        {
            Thread.sleep(2000);
        }
        catch (InterruptedException e)
        {
            e.printStackTrace();
        }
        synchronized (lock)
        {
            lock.notifyAll();
        }
    }
}
 
class ThreadWorker implements Runnable
{
    public void run()
    {
        synchronized (NotificationExample.lock)
        {
            try
            {
                NotificationExample.lock.wait();
                System.out.println(Thread.currentThread().getName() + " finished waiting");
            }
            catch (InterruptedException e)
            {
                e.printStackTrace();
            }
        }
    }
}

The SCJP exam expects you to understand how waiting and notifications happen in Threads. In this example, several threads synchronize over the same lock and then wait. A few seconds later a notifyAll() call is made to notify all the threads that were waiting on this Object’s lock. As an exercise, try to change the notifyAll() call to notify(). Observe what change that does to the execution of this program.

Some of the code displayed above uses the sleep() method to better explain and understand Thread behaviour when you run the examples. However making a Thread sleep does not guarantee any given result. You need to keep that in mind. The scenarios may also play out differently when you remove the calls to sleep(). What would happen for example if the notify() method is called before the threads actually begin waiting ?

You can toy around with those three examples to get a better grasp on threads.