Recognizing The Letters as Being Drawn With The Mouse

While playing-out with a Kinect device, I think I will be writing more about this later, I had an idea. I could detect letters being drawn with the mouse and convert them to key presses enabling the user, use the mouse  like a keyboard. It's basically same problem like they do with these pens.

So while drawing the above with the mouse my app. prints "BABA" on where the cursor is. There are two parts to this problem, hard part is obviously recognizing what actually is written. Easy part is producing the key presses. Here are the libraries I used :
java.awt.Robot
A Robot is actually created with automating tests in mind. It has the methods for moving, clicking the mouse, producing key presses and capturing the screen.
java.awt.MouseInfo
MouseInfo class gives you the location of the mouse. Unfortunately with just Java I couldn't find a way to tell if any of the mouse key's are being pressed out side the java application window. Here are some discussions on that. Apparently one would have to use a native library. Anyway I didn't really need to handle clicks at the beginning anyway.
How to Recognize The Characters
My approach is capturing the changes in the movement of 'x' and 'y' axis.
Here is how 'x' and 'y' axis moves while a the letter 'A' is being drawn :

One might also choose to compare the values of increases and decreases in order to eliminate some false matches.
I use 90ms mouse position samples to compute the differences from the previous location. If the differences are less than 10 pixel I ignore them.
Here is how the 'B' recognition function looks like :

  @Override  
     public void execute(Double x, Double y) {  
       if (state == 0 && (y < 0 || (y == 0 && prevY < 0))) {  
         state++;  
       } else if (state == 1 && x > 0 && y > 0) {  
         state++;  
       } else if (state == 2 && x < 0 && y > 0) {  
         state++;  
       } else if (state == 3 && x > 0 && y > 0) {  
         state++;  
       } else if (state == 4 && x < 0 && y > 0) {  
         state++;  
       } else {  
         state = 0;  
         setIndex(saveI);  
         saveI++;  
       }  
       if (state == 5) {  
         setResult(getIndex());  
         breaked();  
       }  
       prevY = y;  
     }  

This function is continuously fed with ordered X and Y axis values. State variable is increased if the difference is expected, reseted otherwise. When the state 5 is reached than we know that it's a 'B'.
I found this approach working quite well even with not taking account if the mouse button is pressed, though that would be required to match some simpler characters like 'I' or '.'.