Handy Groovy for Batch Processing, Systems Integration and Data Migration

Groovy comes in very handy when doing batch processing, database access, file and directory modifications and data migrations. You can easily read data from one host and store it without intermediate storage directly at the destination hosts database system. Groovy accesses flawlessly JEE servers (EJB), web services and other remote services. With it’s polyglot behaviour, most data integration scenarios are addressable.

I frequently use it’s capabilities instead of shell scripting. Many handy file and directory access and manipulation methods and much more are made available from within the scripting language through AntBuilder, a DSL like interface to well known Apache Ant. It is also possible to start external processes, manage these and pipe ones output as the input to another process’s input (Groovy Process Management).

Maintenance scripts written in Groovy, need not be compiled. They are simply stored as text files and may be executed directly from the shell, cron or other scheduling tools like UC4. Technical support personnel and system administrators then may look at the source of the scripts in case of errors or other difficulties (ok – that’s a matter of taste – one could argue that if the manual is detailed enough, there is no need to look at a batch job’s code – unless you are on late shift 🙂 ).

If you have a full Groovy installation located at your production machine, then starting a Groovy script is as easy as this (supposing that the directory {groovy-home}/bin is included in your PATH variable):

1
> groovy my-groovy-script.groovy

If your systems administrator is not fond of installing Groovy, then that’s fine also – just need to deploy somewhere the groovy-all-x.x.x.jar (found in {groovy-home}/embeddable) and reference it in your class path.

I have shell scripting files that set up a decent execution environment that serves our needs in respects to logging and the like, prior to executing the actual Groovy code. A simple line like this then starts the Groovy script:

1
> /opt/jdk/bin/java -classpath $CLASSPATH groovy.ui.GroovyMain $@

We usually include helper classes in the classpath that give the Groovy scripting code access to all resources needed, such as databases and JEE servers. Those helpers are aware of the execution environment. If the script is executed on a integration test server, then only connections and service locators to integration testing databases and application servers are accessible. Which is fully transparent to the programmer.

The following interface gives an outline of an utility, that processes local configuration files from the application directory and then offers JDBC connection creation and JEE service lookups.

1
2
3
4
5
6
7
interface ConfigUtil {
 
    public Connection createTradesDbConnection() throws SQLException
    public Connection createMasterDbConnection() throws SQLException
    public ServiceLocator createServiceLocator() throws Exception
 
}

Now it’s very easy to create for instance an index maintenance script for SAP Sybase ASE database servers, that’s periodically called and checks if the changed rows in a table make 30 % or more. If so, it executes an UPDATE INDEX STATISTICS command against that table.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
import java.sql.Connection;
 
import de.tmw.batch.DefaultConfigUtil;
import groovy.sql.Sql;
 
configUtil = new DefaultConfigUtil()
sql = new Sql(configUtil.createMasterDbConnection())
 
highWaterMark = 30.0 //we do an update index statistics when 30% of the rows have been changed
 
tables = ["Trade", "Inventory", "Customer", "Instrument"]
 
tables.each() { table ->
    if (sql.firstRow("select object_id(?) as 'id'", table).id != null) {
        datachange = sql.firstRow("select datachange(${table} ,null ,null) as 'change'").change
        if (datachange != null ) {
            println "${table} has datachange = ${datachange} %; threshold is ${highWaterMark}."
            if (datachange >= highWaterMark) {
                println "   Updating indices ..."
                sql.execute("update index statistics ${table}".toString())
            }
        }
    }
    else {
        println "Skipping $table (unknown) ..."
    }
 
}

In reality, this script is much more complex and searches through dozens of partitions via an to our needs tailored algorithm. Thanks to the excellent SQL integration in Groovy, this is really easy and helpful, transparent and fun to work with.

Quickstart NetBeans Editor Shortcuts and Code Templates

NetBeans is an optimal Java development environment. Just recently, while visiting the JAX Conference in Mainz/Germany, I attended a live coding session by Adam Bien. I was telling later a fellow developer and Eclipse user how convenient NetBeans is and how Adam, even though he broke his arm in a bicycle accident, used NetBeans during his live coding session. Later I met my colleague again during lunch time: he just had installed NetBeans on his notebook and was already fond of the NetBeans usage experience.

And yes, numerous keyboard shortcuts and code templates make editing and writing java classes with NetBeans a breeze. Here are my favourites to give you a quick start within the NetBeans Code Editor (in brackets the Eclipse pendants) :

Keyboard Shortcuts

  • Alt+ENTER (Ctrl+1)
    Fixes Code – whenever there is a yellow light bulb on the left hand side of the editor, try Alt+ENTER: a list of suggestions how NetBeans can fix the code for you pops up. It also may add missing JavaDoc and assigns a variable to the returned value of a method call.
  • Alt+Insert
    The code generation menu opens up and let’s you insert getters and setter, equals and hashCode, toString and many more.
  • Shift+ENTER (same)
    Insert a new line and move the cursor to the new lines beginning.  Shift+ENTER may be pressed at any position on the current line without splitting it.
  • Ctrl+e (Ctrl+d)
    Removes the current line.
  • Shift+Alt+period/comma
    Selects next/previous element (such as name, expression, line, method body, method etc).
  • Ctrl-r (Alt+Shift+r)
    Rename
  • Ctrl-o (Ctrl+Shift+t)
    Goto Type
  • Shift+Alt+o (Ctrl+Shift+r)
    Goto File
  • Shift+Ctrl+Arrow Up/Down (Ctrl+Alt+Arrow Up/Down)
    Copy line up/down
  • Shift+Alt+Arrow Up/Down (Alt+Arrow Up/Down)
    Move line up/down
  • Ctrl+Arrow Up/Down (same)
    Scroll without loosing cursor position
  • Shift+Ctrl+c (same)
    Comment/Uncomment line
  • Shift+Alt+f (Shift+Ctrl+f)
    Format file
  • Shift+Ctrl+i (Shift+Ctrl+o)
    Organize imports
  • Shift+Ctrl+v
    Insert clipboard content formatted
  • Ctrl+F12 (Ctrl+o)
    Select Navigator Window. Inside start typing method names and the matching method will be selected. Press ENTER to navigate to it in your source file. Very handy!
  • F7/F8 (F5/F6)
    Debugger: Step into/Step over
  • Ctrl+Shift+ENTER (Ctrl+m)
    Show only Editor (hit key combination a second time to return to normal)
  • Ctrl+s (same)
    Save File
  • Ctrl+g (Ctrl+l)
    Goto Line

Code Templates

Code Templates may be entered at any line in the current file. After hiting the TAB-Key, the template is expanded.

  • pu
    Expands to “public”
  • pr
    Expands to “private”
  • re
    Expands to “return”
  • fore
    Creates a for-each loop wih smart detection of collection classes to use.
  • Pm
    Creates a skeleton public method declaration.
  • pm
    Creates a skeleton private method declaration.
  • sout
    Expands to “System.out.println(“|”)”

If you never tried the code templates before, you should at least try the method declaration templates (pm and Pm). They are clever implemented and really convenient to use.

 

Integrate JBoss AS Maven Plugin with NetBeans Actions

Deploying JEE Artefacts such as EAR Files to a JBoss EAP Server instance is unfortunately not working with NetBeans out-of-the-box. The NetBeans Application Server Plugin as well as the additionally available Wildfly Plugin do not help. If you are a corporate developer whose company licensed a fairly new JBoss EAP 6.2.x (at least at the time of writing this entry) – you’re out of luck.

Fortunately NetBeans excellent maven integration comes to the rescue in form of the JBoss Application Server 7 Maven Plugin. There is also Wildfly Maven Plugin available. Just add the following lines to your pom.xml’s “plugins” section (in my case a maven EAR module):

1
2
3
4
5
6
7
8
9
10
11
  <plugin>
    <groupId>org.jboss.as.plugins</groupId>
    <artifactId>jboss-as-maven-plugin</artifactId>
    <version>7.6.Final</version>
    <configuration>
      <hostname>localhost</hostname>
      <port>9999</port>
      <username>jbossadmin</username>
      <password>*******</password>
    </configuration>
  </plugin>

Hostname, port, username and password have to be changed according to your needs.
Now you have quite a few additional maven goals at hand – deploying, undeploying and redeploying as well as starting and stopping the JBoss EAP Server instance. I usually have an already running JBoss instance on my notebook, hence the deployment related goals matter to me:

  • jboss-as:undeploy — undeploys your already deployed artefact from the running JBoss instance
  • jboss-as:deploy — deploys the artefact that is generated by your maven project to the running JBoss instance
  • jboss-as:redeploy — redeploys your artefact

You could bind the redeploy goal to the packaging phase, so that every time the artefact is built, it is also redeployed to your local JBoss EAP Server instance. I prefer explicit execution of the redeployment goal. NetBeans offers two ways doing that:

Deploy through the NetBeans Navigator Window

Select the maven artefact project – in my case this is the pla-ear-2.0.0. The Navigator Window displays all maven goals that are not bound to maven life cycles. Double click the jboss-as redeploy goal and 8 seconds later, the artefact is deployed and ready to be debugged. At least when you have your debugger attached to the running JBoss EAP Server instance.

navigator-window-redeploy

Deploy with custom NetBeans Actions

I have three custom actions, that let me deploy, undeploy and redeploy the built artefact to my locally running JBoss EAP Server instance. The actions can be reached through a right click on the maven JEE artefact project node. It opens up the projects context menu. Here select the Custom entry, that again opens a sub-menu with all your custom NetBeans actions refering to maven jboss-as plugin goals.

context-menu

Set-up the goals as actions in the projects properties screen. Select the Actions category.

actions

With the Add Custom… button I created three additional actions: Deploy, Undeploy and Redeploy. For each action  fill in the relevant maven goal: jboss-as:deploy, jboss-as:undeploy and jboss-as:redeploy. These show up later under the projects context menus Custom entry.

Make sure to uncheck the Compile On Save checkbox! Otherwise NetBeans own server deployment mechanism interferes with the maven plugin:

compile-on-save

Happy deploying!

NetBeans join line macro

I was searching for a way to join lines in NetBeans in the same way you can join lines in vi (shift+j) or Eclipse (ctrl+alt+j). As far as I know, there is no predefined way to achieve this in my favourite IDE.

The powerful macro dialog lets you design your own editor macros and assign a keyboard shortcut to it. Hey – sure enough – here it is, the “join line macro”:

macro

 

 

 

 

 

 

 

 

 

 

 

 

My Five Favorite NetBeans IDE Features!

I was asked by Geertjan Wielenga, one of the Oracle product managers on the NetBeans Team, to participate in a series of articles focusing on NetBeans users and their five favorite NetBeans features.

Here is what I wrote: My Favorite NetBeans IDE Features