When Premature Optimization Isn't

Earlier this month, I decided I wanted to write a post on not all optimization being premature optimization after hearing more than one developer use this mantra as an excuse for not making a better decision in the same week. Bozhidar Bozhanov beat me to it with his post Not All Optimization Is Premature, which makes some excellent but different points than I had planned to make in postulating that there is nothing wrong in early optimizing "if neither readability nor maintainability are damaged and the time taken is negligible."

Bozhanov's post reminded me that I wanted to write this post. I use this post to provide additional examples and support to backup my claims that too many in our community have allowed "avoid premature optimization" to become a "bumper sticker practice." In my opinion, some developers have taken the appropriate advice to "avoid premature optimization" out of context or do not want to spend the time and effort to really think about the reasons behind this statement. It may seem easier to blindly apply it to all situations, but that is just as dangerous as prematurely optimizing.

Good Design is Not Premature Optimization

I like Rod Johnson's differentiation between "design" and "code optimization" in his book Expert One-on-One J2EE Design and Development (2002). Perhaps the most common situations in which I have seen developers make bad decisions under the pretense of "avoiding premature optimization" is making bad architecture or design choices. The incident earlier this month that prompted me to want to write this post was a developer's assertion that we should not design our services to be coarser grained than he wanted because that was premature optimization and his idea of making numerous successive calls on the same service was "easiest" to implement. In my mind, this developer was mistaking the valid warning about not prematurely optimizing code as an excuse to not consider an appropriate design that might require a barely noticeable amount of extra effort.

In his differentiation between design and code optimization, Johnson highlighted, "Minimize the need for optimization by heading off problems with good design." In that same section he warns against "code optimization" for four main reasons: "optimization is hard" ("few things in programming are harder than optimizing existing code"), "most optimization is pointless," "optimization causes many bugs," and "inappropriate optimization may reduce maintainability forever." Johnson states (and I agree), "There is no conflict between designing for performance and maintainability, but optimization may be more problematic."

Applying Appropriate Language Practices is Not Premature Optimization

Most programming languages I'm familiar with often offer multiple ways to accomplish the same thing. In many cases, one of the alternatives has well-known advantages. In such cases, is it really premature optimization to use the better performing alternative? For example, if I'm writing Java code to append characters onto a String within a loop, why would I ever do this with a Java String instead of StringBuilder? Use of StringBuilder is not much different in terms of maintainability or readability for even a relatively new Java developer and there is a known performance benefit that requires no profiling to recognize. It seems silly to write it with String in the name of "avoiding premature optimization" and only change it to StringBuilder when the profiler shows it's a performance issue. That being stated, it would be just as silly to use a StringBuilder for simple concatenations outside of a loop "just in case" that code was ever placed within a loop.

Similarly, it's not "premature optimization" to write a conditional such that the most likely condition is encountered first as long as doing so does not make the code confusing. Another example is the common use of short circuit evaluation in conditionals that can be effective without being premature optimization. Finally, there are cases where certain data structures or collections are more likely to be appropriate than others for a given operation or set of expected operations.

Writing Cleaner Code is Not Premature Optimization

Some developers might confuse more "efficient" (cleaner) source code with premature optimization. Optimizing source code for readability and maintainability (such as in refactoring or carefully crafting original code) has nothing to do with Knuth's original quote. Writing cleaner code often leads to better performance, but this does not mean writing cleaner code is a form of premature optimization.

Others' Thoughts on Misunderstanding of Premature Optimization

Besides the Not All Optimization Is Premature post, other posts on the misapplication of the "avoid premature optimization" mantra include Joe Duffy's The 'premature optimization is evil' myth and 'Avoid Premature Optimization' Does Not Mean 'Write Dumb Code'.

Duffy puts it so well that I'll quote him directly here:

I have heard the "premature optimization is the root of all evil" statement used by programmers of varying experience at every stage of the software lifecycle, to defend all sorts of choices, ranging from poor architectures, to gratuitous memory allocations, to inappropriate choices of data structures and algorithms, to complete disregard for variable latency in latency-sensitive situations, among others. Mostly this quip is used defend sloppy decision-making, or to justify the indefinite deferral of decision-making. In other words, laziness.

The James Hague post points out one of the signs of potentially having crossed into premature optimization: "The warning sign is when you start sacrificing clarity and reliability while chasing some vague notion of performance." Hague also writes, "What's often missed in these discussions is that the advice to 'avoid premature optimization' is not the same thing as 'write dumb code.'" I like this last sentence because I believe that just as some developers have adulterated the agile concept to mean (to them) "no documentation," some developers have adulterated the sound advice to "avoid premature optimization" to mean (to them) "blindly write code."

Premature Optimization is a Real Problem

Premature optimization is a problem we developers must guard against. As Johnson states in the previously cited book, "Few things in programming are harder than optimizing existing code. Unfortunately, this is why optimization is uniquely satisfying to any programmer's ego. The problem is that the resources devoted to such optimization may well be wasted." There makings examples of where premature optimization wastes significant resources and in some cases even makes things perform worse. There is indeed a reason that the well-regarded Knuth wrote that "premature optimization is the root of all evil." I'm not saying that premature optimization doesn't exist or that it's not harmful. I'm just saying that avoiding this admitted dysfunctional behavior is often used an an excuse to avoid thinking or to avoid implementing sound decisions.

Conclusion

Like pithy bumper stickers on cars that naively boil down complex issues to a few clever and catchy words, use of "avoid premature optimization" is often used much more broadly than it was intended. Even the best of recommended practices can cause more harm than benefit when applied improperly and the misuse of "avoid premature optimization" is one of the best examples of this. I have seen the high cost paid in maintainability, readability, and even in performance when supposed "optimization" was implemented too early and at the expense of readability and maintainability for no measurable benefit. However, just as high of a cost can be incurred by blindly using "avoid premature optimization" as an excuse to avoid designing and writing better performing software. "Avoiding premature optimization" is not an excuse to stop thinking.

Type-safe Empty Collections in Java

I have blogged before on the utility of the Java Collections class and have specifically blogged on Using Collections Methods emptyList(), emptyMap(), and emptySet(). In this post, I look at the sometimes subtle but significant differences between using the relevant fields of the Collections class for accessing an empty collection versus using the relevant methods of the Collections class for accessing an empty collection.

The following code demonstrates accessing Collections's fields directly to specify empty collections.

Using Collections's Fields for Empty Collections

   /**
    * Instantiate my collections with empty versions using Collections fields.
    * This will result in javac compiler warnings stating "warning: [unchecked]
    * unchecked conversion".
    */
   public void instantiateWithEmptyCollectionsFieldsAssigment()
   {
      this.stringsList = Collections.EMPTY_LIST;
      this.stringsSet = Collections.EMPTY_SET;
      this.stringsMap = Collections.EMPTY_MAP;      
   }

The code above compiles with javac, but leads to the warning message (generated by NetBeans and Ant in this case):

-do-compile:
    [javac] Compiling 1 source file to C:\java\examples\typesafeEmptyCollections\build\classes
    [javac] Note: C:\java\examples\typesafeEmptyCollections\src\dustin\examples\Main.java uses unchecked or unsafe operations.
    [javac] Note: Recompile with -Xlint:unchecked for details.

Specifying -Xlint:unchecked as an argument to javac (in this case via the javac.compilerargs=-Xlint:unchecked in the NetBeans project.properties file) helps get more specific warning messages for the earlier listed code:

    [javac] Compiling 1 source file to C:\java\examples\typesafeEmptyCollections\build\classes
    [javac] C:\java\examples\typesafeEmptyCollections\src\dustin\examples\Main.java:27: warning: [unchecked] unchecked conversion
    [javac]       this.stringsList = Collections.EMPTY_LIST;
    [javac]                                     ^
    [javac]   required: List<String>
    [javac]   found:    List
    [javac] C:\java\examples\typesafeEmptyCollections\src\dustin\examples\Main.java:28: warning: [unchecked] unchecked conversion
    [javac]       this.stringsSet = Collections.EMPTY_SET;
    [javac]                                    ^
    [javac]   required: Set<String>
    [javac]   found:    Set
    [javac] C:\java\examples\typesafeEmptyCollections\src\dustin\examples\Main.java:29: warning: [unchecked] unchecked conversion
    [javac]       this.stringsMap = Collections.EMPTY_MAP;      
    [javac]                                    ^
    [javac]   required: Map<String,String>
    [javac]   found:    Map

NetBeans will also show these warnings if the appropriate hint box is checked in its options. The next three images demonstrate ensuring that the appropriate hint is set to see these warnings in NetBeans and provides an example of how NetBeans presents the code shown above with warnings.

Fortunately, it is easy to take advantage of the utility of the Collections class and access empty collections in a typesafe manner that won't lead to these javac warnings and corresponding NetBeans hints. That approach is to use Collections's methods rather than its fields. This is demonstrated in the next simple code listing.

Using Collections's Methods for Empty Collections

   /**
    * Instantiate my collections with empty versions using Collections methods.
    * This will avoid the javac compiler warnings alluding to "unchecked conversion".
    */
   public void instantiateWithEmptyCollectionsMethodsTypeInferred()
   {
      this.stringsList = Collections.emptyList();
      this.stringsSet = Collections.emptySet();
      this.stringsMap = Collections.emptyMap();
   }

The above code will compile without warning and no NetBeans hints will be shown either. The Javadoc documentation for each field of the Collections class does not address why these warnings occur for the fields, but the documentation for each of the like-named methods does discuss this. Specifically, the documentation for Collections.emptyList(), Collections.emptySet(), and Collections.emptyMap() each state, "(Unlike this method, the field does not provide type safety.)"

Use of the Collections methods for empty collections shown in the last code listing provided type safety without the need to explicitly specify the types stored within that collection because type was inferred by use of the Collections methods in assignments to known and already declared instance attributes with explicitly specified element types. When type cannot be inferred, compiler errors will result when using the Collections methods without an explicitly specified type. This is shown in the next screen snapshot of attempting to do this in NetBeans.

The specific compiler error message is:

    [javac] C:\java\examples\typesafeEmptyCollections\src\dustin\examples\Main.java:62: error: method populateList in class Main cannot be applied to given types;
    [javac]       populateList(Collections.emptyList());
    [javac]       ^
    [javac]   required: List<String>
    [javac]   found: List<Object>
    [javac]   reason: actual argument List<Object> cannot be converted to List<String> by method invocation conversion
    [javac] C:\java\examples\typesafeEmptyCollections\src\dustin\examples\Main.java:63: error: method populateSet in class Main cannot be applied to given types;
    [javac]       populateSet(Collections.emptySet());
    [javac]       ^
    [javac]   required: Set<String>
    [javac]   found: Set<Object>
    [javac]   reason: actual argument Set<Object> cannot be converted to Set<String> by method invocation conversion
    [javac] C:\java\examples\typesafeEmptyCollections\src\dustin\examples\Main.java:64: error: method populateMap in class Main cannot be applied to given types;
    [javac]       populateMap(Collections.emptyMap());
    [javac]       ^
    [javac]   required: Map<String,String>
    [javac]   found: Map<Object,Object>
    [javac]   reason: actual argument Map<Object,Object> cannot be converted to Map<String,String> by method invocation conversion
    [javac] 3 errors

These compiler errors are avoided and type safety is achieved by explicitly specifying the types of the collections' elements in the code. This is shown in the next code listing.

Explicitly Specifying Element Types with Collections's Empty Methods

   /**
    * Pass empty collections to another method for processing and specify those
    * empty methods using Collections methods. This will result in javac compiler
    * ERRORS unless the type is explicitly specified.
    */
   public void instantiateWithEmptyCollectionsMethodsTypeSpecified()
   {
      populateList(Collections.<String>emptyList());
      populateSet(Collections.<String>emptySet());
      populateMap(Collections.<String, String>emptyMap());
   }

The Collections class's methods for obtaining empty collections are preferable to use of Collections's similarly named fields for that same purpose because of the type safety the methods provide. This allows greater leveraging of Java's static type system, a key theme of books such as Effective Java. A nice side effect is the removal of cluttering warnings and marked NetBeans hints, but the more important result is better, safer code.

Scripted Reports with Groovy

Groovy has become my favorite scripting language and in this blog I look at some of Groovy's features that make it particularly attractive for presenting text-based reports. The post will show how custom text-based reports of data stored in the database can be easily presented with Groovy. I will highlight several attractive features of Groovy along the way.

I use the Oracle Database 11g Express Edition (XE) for the data source in my example in this post, but any data source could be used. This example does make use of Groovy's excellent SQL/JDBC support and uses the Oracle sample schema (HR). A visual depiction of that sample schema is available in the sample schema documentation.

My example of using Groovy to write a reporting script involves retrieving data from the Oracle HR sample schema and presenting that data via a text-based report. One portion of the script needs to acquire this data from the database and Groovy adds only minimal ceremony to the SQL statement needed to do this. The following code snippet from the script shows use of Groovy's multi-line GString to specify the SQL query string in a user-friendly format and to process the results of that query.

def employeeQueryStr =
"""SELECT e.employee_id, e.first_name, e.last_name,
          e.email, e.phone_number,
          e.hire_date, e.job_id, j.job_title,
          e.salary, e.commission_pct, e.manager_id,
          e.department_id, d.department_name,
          m.first_name AS mgr_first_name, m.last_name AS mgr_last_name
     FROM employees e, departments d, jobs j, employees m
    WHERE e.department_id = d.department_id
      AND e.job_id = j.job_id
      AND e.manager_id = m.employee_id(+)"""

def employees = new TreeMap<Long, Employee>()
import groovy.sql.Sql
def sql = Sql.newInstance("jdbc:oracle:thin:@localhost:1521:xe", "hr", "hr",
                          "oracle.jdbc.pool.OracleDataSource")
sql.eachRow(employeeQueryStr)
{
   def employeeId = it.employee_id as Long
   def employee = new Employee(employeeId, it.first_name, it.last_name,
                               it.email, it.phone_number,
                               it.hire_date, it.job_id, it.job_title,
                               it.salary, it.commission_pct, it.manager_id as Long,
                               it.department_id as Long, it.department_name,
                               it.mgr_first_name, it.mgr_last_name)
   employees.put(employeeId, employee)
}

The Groovy code above only adds a small amount of code on top of the Oracle SQL statement. The specified SELECT statement joins multiple tables and includes an outer join as well (outer join needed to include the President in the query results despite that position not having a manager). The vast majority of the first part of the code is the SQL statement that could be run as-is in SQL*Plus or SQL Developer. No need for verbose exception catching and result set handling with Groovy's SQL support!

There are more Groovy-specific advantages to point out in the code snippet above. Note that the import statement to import groovy.sql.Sql was allowed when needed and did not need to be at the top of the script file. The example also used Sql.newInstance and Sql.eachRow(GString,Closure). The latter method allows for easy application of a closure to the results of the query. The it special word is the default name for items being processed in the closure. In this case,it can be thought of a a row in the result set. Values in each row are accessed by the underlying database columns' names (or aliases in the case of mgr_first_name and mgr_last_name).

One of the advantages of Groovy is its seamless integration with Java. The above code snippet also demonstrated this via Groovy's use of TreeMap, which is advantageous because it means that the new Employee instances placed in the map based on data retrieved from the database will always be available in order of employee ID.

In the code above, the information retrieved from the database and processed via the closure is stored for each row in a newly instantiated Employee object. This Employee object provides another place to show off Groovy's brevity and is shown next.

Employee.groovy

@groovy.transform.Canonical
class Employee
{
   Long employeeId
   String firstName
   String lastName
   String emailAddress
   String phone_number
   Date hireDate
   String jobId
   String jobTitle
   BigDecimal salary
   BigDecimal commissionPercentage
   Long managerId
   Long departmentId
   String departmentName
   String managerFirstName
   String managerLastName
}

The code listing just shown is the entire class! Groovy's property supports makes getter/setter methods automatically available for all the defined class attributes. As I discussed in a previous blog post, the @Canonical annotation is a Groovy AST (transformation) that automatically creates several useful common methods for this class [equals(Object), hashCode(), and toString()]. There is no explicit constructor because @Canonical also handles this, providing a constructor that accepts that class's arguments in the order they are specified in their declarations. It is difficult to image a scenario in which it would be easier to easily and quickly create an object to store retrieved data values in a script.

A JDBC driver is needed for this script to retrieve this data from the Oracle Database XE and the JAR for that driver could be specified on the classpath when running the Groovy script. However, I like my scripts to be as self-contained as possible and this makes Groovy's classpath root loading mechanism attractive. This can be used within this script (rather than specifying it externally when invoking the script) as shown next:

this.class.classLoader.rootLoader.addURL(
   new URL("file:///C:/oraclexe/app/oracle/product/11.2.0/server/jdbc/lib/ojdbc6.jar"))

Side Note: Another nifty approach for accessing the appropriate dependent JAR or library is use of Groovy's Grape-provided @Grab annotation. I didn't use that here because Oracle's JDBC JAR is not available in any legitimate Maven central repositories that I am aware of. An example of using this approach when a dependency is available in the Maven public repository is shown in my blog post Easy Groovy Logger Injection and Log Guarding.

With the data retrieved from the database and placed in a collection of simple Groovy objects built for holding this data and providing easy access to it, it is almost time to start presenting this data in a text report. Some constants defined in the script are shown in the next excerpt from the script code.

int TOTAL_WIDTH = 120
String HEADER_ROW_SEPARATOR = "=".multiply(TOTAL_WIDTH)
String ROW_SEPARATOR = "-".multiply(TOTAL_WIDTH)
String COLUMN_SEPARATOR = "|"
int COLUMN_SEPARATOR_SIZE = COLUMN_SEPARATOR.size()
int COLUMN_WIDTH = 22
int TOTAL_NUM_COLUMNS = 5
int BALANCE_COLUMN_WIDTH = TOTAL_WIDTH-(TOTAL_NUM_COLUMNS-1)*COLUMN_WIDTH-COLUMN_SEPARATOR_SIZE*(TOTAL_NUM_COLUMNS-1)-2

The declaration of constants just shown exemplify more advantages of Groovy. For one, the constants are statically typed, demonstrating Groovy's flexibility to specifying types statically as well as dynamically. Another feature of Groovy worth special note in the last code snippet is the use of the String.multiply(Number) method on the literal Strings. Everything, even Strings and numerics, are objects in Groovy. The multiply method makes it easy to create a String of that number of the same repeating character.

The first part of the text output is the header. The following lines of the Groovy script write this header information to standard output.

println "\n\n${HEADER_ROW_SEPARATOR}"
println "${COLUMN_SEPARATOR}${'HR SCHEMA EMPLOYEES'.center(TOTAL_WIDTH-2*COLUMN_SEPARATOR_SIZE)}${COLUMN_SEPARATOR}"
println HEADER_ROW_SEPARATOR
print "${COLUMN_SEPARATOR}${'EMPLOYEE ID/HIRE DATE'.center(COLUMN_WIDTH)}${COLUMN_SEPARATOR}"
print "${'EMPLOYEE NAME'.center(COLUMN_WIDTH)}${COLUMN_SEPARATOR}"
print "${'TITLE/DEPARTMENT'.center(COLUMN_WIDTH)}${COLUMN_SEPARATOR}"
print "${'SALARY INFO'.center(COLUMN_WIDTH)}${COLUMN_SEPARATOR}"
println "${'CONTACT INFO'.center(BALANCE_COLUMN_WIDTH)}${COLUMN_SEPARATOR}"
println HEADER_ROW_SEPARATOR

The code above shows some more addictive features of Groovy. One of my favorite aspects of Groovy's GString support is the ability to use Ant-like ${} expressions to provide executable code inline with the String. The code above also shows off Groovy's GDK String's support for the center(Number) method that automatically centers the given String withing the specified number of characters. This is a powerful feature for easily writing attractive text output.

With the data retrieved and available in our data structure and with the constants defined, the output portion can begin. The next code snippet shows use of Groovy's standard collections each method to allow iteration over the previously populated TreeMap with a closure applied to each iteration.

employees.each
{ id, employee ->
   // first line in each output row
   def idStr = id as String
   print "${COLUMN_SEPARATOR}${idStr.center(COLUMN_WIDTH)}${COLUMN_SEPARATOR}"
   def employeeName = employee.firstName + " " + employee.lastName
   print "${employeeName.center(COLUMN_WIDTH)}${COLUMN_SEPARATOR}"
   def jobTitle = employee.jobTitle.replace("Vice President", "VP").replace("Assistant", "Asst").replace("Representative", "Rep")
   print "${jobTitle.center(COLUMN_WIDTH)}${COLUMN_SEPARATOR}"
   def salary = '$' + (employee.salary as String)
   print "${salary.center(COLUMN_WIDTH)}${COLUMN_SEPARATOR}"
   println "${employee.phone_number.center(BALANCE_COLUMN_WIDTH)}${COLUMN_SEPARATOR}"

   // second line in each output row
   print "${COLUMN_SEPARATOR}${employee.hireDate.getDateString().center(COLUMN_WIDTH)}"
   def managerName = employee.managerFirstName ? "Mgr: ${employee.managerFirstName[0]}. ${employee.managerLastName}" : "Answers to No One"
   print "${COLUMN_SEPARATOR}${managerName.center(COLUMN_WIDTH)}${COLUMN_SEPARATOR}"
   print "${employee.departmentName.center(COLUMN_WIDTH)}${COLUMN_SEPARATOR}"
   String commissionPercentage = employee.commissionPercentage ?: "No Commission"
   print "${commissionPercentage.center(COLUMN_WIDTH)}${COLUMN_SEPARATOR}"
   println "${employee.emailAddress.center(BALANCE_COLUMN_WIDTH)}${COLUMN_SEPARATOR}"
   println ROW_SEPARATOR
}

The last code snippet is where the data retrieved from the database is output in a relatively attractive text format. The example shows how handles in a closure can be named to be more meaningful. In this case, they are named id and employee and represent the key (Long) and value (Employee) of each entry in the TreeMap.

There are other Groovy features in the last code snippet worth special mention. The presentation of commission uses Groovy's Elvis operator (?:), which makes even Java's conditional ternary look verbose. In this example, if the employee's commission percentage meets Groovy truth standards, that percentage is used; otherwise, "No Commission" is printed.

The handling of the hire date provides another opportunity to tout Groovy's GDK benefits. In this case, Groovy GDK Date.getDateString() is used to easily access the date-only portion of the Date class (time not desired for hire date) without explicit use of a String formatter. Nice!

The last code example also demonstrates use of the as keyword to coerce (cast) variables in a more readable way and also demonstrates more leverage of Java features, in this case taking advantage of Java String's replace(CharSequence, CharSequence) method. Groovy adds some more goodness to String again in this example, however. The example demonstrates Groovy's supporting extracting the first letter only of the manager's first name using subscript (array) notation ([0]) to get only the first character out of the string.

So far in this post, I've shown snippets of the overall script as I explained the various features of Groovy that are demonstrated in each snippet. The entire script is shown next and that code listing is followed by a screen snapshot of how the output appears when the script is executed. The complete code for the Groovy Employee class was shown previously.

generateReport.groovy: The Complete Script

#!/usr/bin/env groovy

// Add JDBC driver to classpath as part of this script's bootstrapping.
// See http://marxsoftware.blogspot.com/2011/02/groovy-scripts-master-their-own.html.
// WARNING: This location needs to be adjusted for specific user environment.
this.class.classLoader.rootLoader.addURL(
   new URL("file:///C:/oraclexe/app/oracle/product/11.2.0/server/jdbc/lib/ojdbc6.jar"))


int TOTAL_WIDTH = 120
String HEADER_ROW_SEPARATOR = "=".multiply(TOTAL_WIDTH)
String ROW_SEPARATOR = "-".multiply(TOTAL_WIDTH)
String COLUMN_SEPARATOR = "|"
int COLUMN_SEPARATOR_SIZE = COLUMN_SEPARATOR.size()
int COLUMN_WIDTH = 22
int TOTAL_NUM_COLUMNS = 5
int BALANCE_COLUMN_WIDTH = TOTAL_WIDTH-(TOTAL_NUM_COLUMNS-1)*COLUMN_WIDTH-COLUMN_SEPARATOR_SIZE*(TOTAL_NUM_COLUMNS-1)-2



// Get instance of Groovy's Sql class
// See http://marxsoftware.blogspot.com/2009/05/groovysql-groovy-jdbc.html
import groovy.sql.Sql
def sql = Sql.newInstance("jdbc:oracle:thin:@localhost:1521:xe", "hr", "hr",
                          "oracle.jdbc.pool.OracleDataSource")

def employeeQueryStr =
"""SELECT e.employee_id, e.first_name, e.last_name,
          e.email, e.phone_number,
          e.hire_date, e.job_id, j.job_title,
          e.salary, e.commission_pct, e.manager_id,
          e.department_id, d.department_name,
          m.first_name AS mgr_first_name, m.last_name AS mgr_last_name
     FROM employees e, departments d, jobs j, employees m
    WHERE e.department_id = d.department_id
      AND e.job_id = j.job_id
      AND e.manager_id = m.employee_id(+)"""

def employees = new TreeMap<Long, Employee>()
sql.eachRow(employeeQueryStr)
{
   def employeeId = it.employee_id as Long
   def employee = new Employee(employeeId, it.first_name, it.last_name,
                               it.email, it.phone_number,
                               it.hire_date, it.job_id, it.job_title,
                               it.salary, it.commission_pct, it.manager_id as Long,
                               it.department_id as Long, it.department_name,
                               it.mgr_first_name, it.mgr_last_name)
   employees.put(employeeId, employee)
}

println "\n\n${HEADER_ROW_SEPARATOR}"
println "${COLUMN_SEPARATOR}${'HR SCHEMA EMPLOYEES'.center(TOTAL_WIDTH-2*COLUMN_SEPARATOR_SIZE)}${COLUMN_SEPARATOR}"
println HEADER_ROW_SEPARATOR
print "${COLUMN_SEPARATOR}${'EMPLOYEE ID/HIRE DATE'.center(COLUMN_WIDTH)}${COLUMN_SEPARATOR}"
print "${'EMPLOYEE NAME'.center(COLUMN_WIDTH)}${COLUMN_SEPARATOR}"
print "${'TITLE/DEPARTMENT'.center(COLUMN_WIDTH)}${COLUMN_SEPARATOR}"
print "${'SALARY INFO'.center(COLUMN_WIDTH)}${COLUMN_SEPARATOR}"
println "${'CONTACT INFO'.center(BALANCE_COLUMN_WIDTH)}${COLUMN_SEPARATOR}"
println HEADER_ROW_SEPARATOR

employees.each
{ id, employee ->
   // first line in each row
   def idStr = id as String
   print "${COLUMN_SEPARATOR}${idStr.center(COLUMN_WIDTH)}${COLUMN_SEPARATOR}"
   def employeeName = employee.firstName + " " + employee.lastName
   print "${employeeName.center(COLUMN_WIDTH)}${COLUMN_SEPARATOR}"
   def jobTitle = employee.jobTitle.replace("Vice President", "VP").replace("Assistant", "Asst").replace("Representative", "Rep")
   print "${jobTitle.center(COLUMN_WIDTH)}${COLUMN_SEPARATOR}"
   def salary = '$' + (employee.salary as String)
   print "${salary.center(COLUMN_WIDTH)}${COLUMN_SEPARATOR}"
   println "${employee.phone_number.center(BALANCE_COLUMN_WIDTH)}${COLUMN_SEPARATOR}"

   // second line in each row
   print "${COLUMN_SEPARATOR}${employee.hireDate.getDateString().center(COLUMN_WIDTH)}"
   def managerName = employee.managerFirstName ? "Mgr: ${employee.managerFirstName[0]}. ${employee.managerLastName}" : "Answers to No One"
   print "${COLUMN_SEPARATOR}${managerName.center(COLUMN_WIDTH)}${COLUMN_SEPARATOR}"
   print "${employee.departmentName.center(COLUMN_WIDTH)}${COLUMN_SEPARATOR}"
   String commissionPercentage = employee.commissionPercentage ?: "No Commission"
   print "${commissionPercentage.center(COLUMN_WIDTH)}${COLUMN_SEPARATOR}"
   println "${employee.emailAddress.center(BALANCE_COLUMN_WIDTH)}${COLUMN_SEPARATOR}"
   println ROW_SEPARATOR
}

In this blog post, I've attempted to show how Groovy provides numerous features and other syntax support that make it easier to write scripts for generating readable and relatively attractive output. For more general Groovy scripts that provide text output support, see Formatting simple tabular text data. Although these are nice general solutions, an objective of my post has been to show that it is easy and does not take much time to write customized scripts for generating custom text output with Groovy. Small Groovy-isms such as easily centering a String, easily converting a Date to a String, extracting any desired character from a string based on array position notation, and easily accessing database data make Groovy a powerful tool in generating text-based reports.

RMOUG Training Days 2013 Presentations Announced: A Mobile Emphasis

The Rocky Mountain Oracle Users Group (RMOUG) has announced the presentations to be given at Training Days 2013. As shown on that page, I will be presenting "Charting Oracle Database Data with JavaFX and Groovy." The abstract for my presentation as shown on the conference materials is shown next:

JavaFX 2.x makes generation and display of data-powered charts powerful and straightforward. JavaFX’s simple APIs allow data sets to be used to create numerous types of charts including pie charts, bar charts, line charts, area charts, bubble charts, and more. JavaFX also supports dynamic charts that are automatically updated when the underlying data changes. With JavaFX poised to become part of Java SE, it is destined to be a standardized technology that is readily available in many environments.

Groovy is a dynamic scripting language that also runs on the JVM and can be used to write concise but powerful scripts. Groovy makes retrieving data from a database relatively painless and is much easier to use than even JDBC. Groovy’s XML parsing is also similarly easy to use and makes Groovy an ideal language for parsing XML to generate charts. JavaFX and Groovy together make it easy to write simple scripts that easily create visually impressive charting results. JavaFX and Groovy can be used with other Java/JVM-based libraries to generate these charts in various medium from websites to desktop clients to PDFs.

I also intend to discuss in this presentation an obvious topic when discussing use of JavaFX and Groovy together: GroovyFX.

One of the major themes of this year's RMOUG Training Days 2013 conference appears to be developing mobile applications, either with Oracle Application Express (APEX) or with Oracle ADF Mobile. The main Oracle web page on Oracle ADF Mobile describes it as "an HTML5 and Java mobile development framework that enables developers to build and extend enterprise applications for iOS and Android from a single code base" and is "based on a hybrid mobile architecture" and "supports access to native device services." The following are some of the sessions at RMOUG Training Days 2013 that are obviously covering mobile application development with Oracle ADF Mobile or with APEX.

• Mobile Integration through Oracle Technologies
  • Objectives and outline indicate this session will cover general mobile development with Oracle technologies.
  • Jordan Braunstein, Visual Integrator Consulting
• Oracle ADF – The No Slides Overview
  • "See how far you can get with Oracle ADF in one hour of building web and mobile applications."
  • Shay Shmeltzer, Oracle Corporation
• Develop Compelling On-Device Mobile Apps – The Simpler Way
  • "This session will discuss and demonstrate developing an on-device mobile application using the technologies and tools you are familiar with: JDeveloper, Java, and ADF Mobile, without writing a single line of device-native code. You can deploy this single code base to both iOS and Android-based devices."
  • Joe Huang, Oracle Corporation
• ADF Mobile eCourse Pilot
  • "Through interactive presentation and hands-on exercises, you’ll examine the principles to consider when designing mobile applications, as well as understand the road map for developing them."
  • Lynn Munsinger, Oracle Corporation
• Bring Your iPads! (Because You’re Gonna Build a Mobile APEX App in One Hour!)
  • "Oracle Application Express integrates very easily with JQuery Mobile and this combination can be used to create mobile applications that are supported on a wide range of mobile devices relatively simple."
  • Chris Ostrowski, Avout
• Building Mobile Applications with Oracle Application Express
  • "This session provides an overview of the basics of jQuery mobile, explains how it is integrated with Oracle Application Express, and discusses how customers can quickly build mobile web applications and extend their existing Oracle Application Express applications with mobile capabilities."
  • David Peake, Oracle Corporation

RMOUG Training Days 2013 looks to have a plethora of database-related presentations as normal, but several of them are focused on MySQL as well as the Oracle database. I will likely try to attend as many of the mobile application development sessions as I can and look forward to presenting on generating charts for data using JavaFX and Groovy.

RMOUG Training Days 2013 will be held 11-13 February 2012 at the Colorado Convention Center in Denver, Colorado.

Design Patterns: Mogwai or Gremlins?

The 1994 book Design Patterns: Elements of Reusable Object-Oriented Software introduced many software developers to the concept of "a catalog of simple and succinct solutions to commonly occurring design problems" that nearly every object-oriented software developer knows of today as "design patterns." Like most technical concepts (whether real or hype or somewhere in between), "design patterns" seemed to go through the normal stages of acceptance, rising rapidly from new idea to the prevalent way of thinking. As is always the case, this rapid rise in popularity led to backlash as design patterns were overused, abused, and otherwise used inappropriately. Today, design patterns seem to have become accepted as a useful tool when used correctly, but are generally recognized as dangerous in the wrong hands.

I have generally avoided devoting an entire blog post to a discussion of the good, the bad, and the ugly of use of design patterns, but a fellow software developer recently made up an analogy related to his observations of the use and misuse of design patterns that motivated me to write this post. Andy pointed out that there seems to be a tendency among some software developers to take an innocent and well-intentioned design pattern and turn it to evil, like turning a Mogwai like Gizmo into a Gremlin. In this post, I look in more detail at why this is a particularly fitting movie-themed analogy for turning effective use of design patterns into misuse and abuse of design patterns.

The 1984 movie Gremlins begins with an inventor and father in Chinatown purchasing a Mogwai. Mr. Wing, the owner of the store, does not want to sell the Mogwai to the inventor/father because "with Mogwai comes much responsibility." However, Mr. Wing's grandson sneakily sells the Mogwai to the inventor/father while warning him of three important things to be aware of related to care of the Mogwai. These three things are:

1. "Keep him out of the light. He hates bright light, especially sunlight. It will kill him."
2. "Keep him away from water. Don't get him wet."
3. "But the most important rule, the one you can never forget, no matter how much he cries or how much he begs never, never feed him after midnight."

The appropriate use of design patterns is not affected by bright light, water, or eating after midnight, but the effects of not taking care when applying design patterns can have effects similar to not taking care of Mogwai properly.

Rapidly Spawning Design Patterns

When Mogwai or Gremlins get wet, spontaneous reproduction of more Mogwai or Gremlins occurs. The effect can be very similar for developers with design patterns. It is easy for a developer new to design patterns (or a developer who is excited about a new design pattern that he or she has recently learned) to apply too many design patterns to the same problem. If design patterns are good, more of them must be better. It is similarly easily for developers to fall into the trap of applying the same favorite design pattern to too many different, diverse, and unrelated problems (Maslow's Hammer).

Design Patterns Turned Evil

The Mogwai turned into mischievous Gremlins if they ate after midnight. Similarly, design patterns can be more evil than good if used inappropriately. A misapplied design pattern can obfuscate the intention of the code. Several design patterns used together can obscure the intent as well. Design patterns that are meant to facilitate better design can and often do lead to worse design when not used carefully. What is a design pattern in one situation might be an anti-pattern in a different situation.

One of the benefits of cataloging the common design principles as patterns is the ability to aid communication among developers and designers. However, design patterns can have the exact opposite effect (hindering understanding and communication) when misapplied or overused. I have also seen the case when a developer insists he or she is using a particular design pattern when he or she is really using a totally different design pattern or even an anti-pattern. In such cases, use of "design pattern" terminology also confuses rather than clarifying.

Well-Intentioned But Ill-Conceived

These problems most commonly arise when developers apply the design patterns because they believe they should rather than because they truly understand their value in a particular situation. Rather than applying the same design pattern to every problem or shoe-horning a design pattern into a situation in which it does not fit well, the developer needs to understand the advantages and objectives of different design patterns, along with trade-offs associated with the design patterns, to make an informed decision about application of design patterns.

Effective Use of Design Patterns

It seems to me that the best use of design patterns occurs when a developer applies them naturally based on experience when need is observed rather than forcing their use. After all, when the Gang of Four compiled their book on design patterns, they were cataloging existing design patterns that developers had been using already. Indeed, some of the patterns covered in their book became so popular that they were incorporated into Java language syntax and into other newer languages. For example, Java provided the interface (which aids many of the design patterns covered in the original design patterns book) and newer languages such as Scala and Groovy have added their own pattern implementations.

Use of Design Patterns: Gizmo or Gremlin?

When used properly, design patterns are attractive and desirable just as Gizmo the Mogwai is a desirable pet. However, when used inappropriately or applied without appropriate care and consideration, design patterns can turn into Gremlins, wreaking havoc on one's code base and hindering the ability to understand and maintain one's design. Note that the design patterns themselves are not necessarily the problem, but rather the people entrusted with proper use of design patterns determine whether they maintain desirable like Gizmo or undesirable like the Gremlins.

Java/NetBeans: Overridable Method Call in Constructor

I wrote about the NetBeans hint "Overridable Method Call in Constructor" in the blog post Seven Indispensable NetBeans Java Hints. In this post, I look at why having an overridable method called from a parent class's constructor is not a good idea.

The next class, Employee, is a contrived example of a class in which the extensible class's constructor calls an overridable method (setSalaryRange()).

Employee.java

package dustin.examples.overridable;

/**
 * Simple employee class that is intended to be a parent of a specific type of
 * employee class. The main purpose of this class is to demonstrate the
 * insidious dangers associated with a constructor calling an overridable method.
 * 
 * @author Dustin
 */
public class Employee
{
   private String lastName;
   private String firstName;
   private JobTitle jobTitle;
   protected int minWeeklySalary;
   protected int maxWeeklySalary;

   public enum JobTitle
   {
      CHIEF_EXECUTIVE_OFFICER("CEO"),
      COMPUTER_SCIENTIST("Computer Scientist");

      private String displayableTitle;

      JobTitle(final String newDisplayableTitle)
      {
         this.displayableTitle = newDisplayableTitle;
      }

      public String getDisplayableTitle()
      {
         return this.displayableTitle;
      }
   }

   public Employee(
      final String newLastName, final String newFirstName, final JobTitle newJobTitle)
   {
      this.lastName = newLastName;
      this.firstName = newFirstName;
      this.jobTitle = newJobTitle;
      setSalaryRange();
   }

   public void setSalaryRange()
   {
      this.minWeeklySalary = 5;
      this.maxWeeklySalary = 10;
   }

   @Override
   public String toString()
   {
      return  this.firstName + " " + this.lastName + " with title '"
            + this.jobTitle.getDisplayableTitle()
            + "' and with a salary range of $" + this.minWeeklySalary + " to $"
            + this.maxWeeklySalary + ".";
   }
}

NetBeans flags the existence of an overridable method called from a constructor as shown in the next screen snapshot (NetBeans 7.3 in this case):

To demonstrate a common problem associated with overridable methods called in a constructor, a child class is needed. That is shown next with the code listing for ComputerScientist, which extends Employee.

ComputerScientist.java

package dustin.examples.overridable;

/**
 * Class representing a specific type of employee (computer scientist), but its
 * real purpose is to demonstrate how overriding a method called in the parent
 * class's constructor leads to undesired behavior.
 * 
 * @author Dustin
 */
public class ComputerScientist extends Employee
{
   private final int MIN_CS_WEEKLY_SALARY_IN_DOLLARS = 1000;
   private static final int MAX_CS_WEEKLY_SALARY_IN_DOLLARS = 60000;

   private int marketFactor = 1;

   public ComputerScientist(
      final String newLastName, final String newFirstName, final int newMarketFactor)
   {
      super(newLastName, newFirstName, JobTitle.COMPUTER_SCIENTIST);
      this.marketFactor = newMarketFactor;
   }

   @Override
   public void setSalaryRange()
   {
      this.minWeeklySalary = MIN_CS_WEEKLY_SALARY_IN_DOLLARS * this.marketFactor;
      this.maxWeeklySalary = MAX_CS_WEEKLY_SALARY_IN_DOLLARS * this.marketFactor;
   }
}

Finally, a simple test driving application is required to run this example. That is shown in the next simple executable class (Main.java).

Main.java

package dustin.examples.overridable;

import dustin.examples.overridable.Employee.JobTitle;
import static java.lang.System.out;

/**
 * Simple driver of the demonstration of why calling an overridable method in
 * the constructor of an extendible class is a bad idea.
 * 
 * @author Dustin
 */
public class Main
{
   public static void main(final String[] arguments)
   {
      final ComputerScientist cs = new ComputerScientist("Flintstone", "Fred", 5);
      final Employee emp = new Employee("Rubble", "Barney", JobTitle.CHIEF_EXECUTIVE_OFFICER);

      out.println(cs);
      out.println(emp);
   }
}

One might expect the Computer Scientist, Fred Flintstone, to earn a weekly salary in the range of $1,000 to $60,000. However, that is not what is shown when the simple main application is executed (command line output and NetBeans output shown).

The reason the Computer Scientist's salary range is not correct is that the parent class's (Employee's) constructor must first be run before the extending class (ComputerScientist) is completely instantiated. The child class does override the setSalaryRange() method, but this overridden implementation depends on an instance variable (marketFactor) that is not yet initialized in the child instance when the parent's constructor calls this child class's overridden method.

There are multiple ways to avoid this problem. Perhaps the best and easiest approaches are those recommended by the NetBeans hint that flagged this issue.

As the screen snapshot above shows, NetBeans provides four easy and effective ways to deal with the issue of an overridable method called from the constructor of a class. Because the issue involves an "overridable" method and a child class that is not fully instantiated when that overridable method is invoked during parent's constructor, an obvious tactic is to make the parent class final so that it cannot be extended. This obviously will only work for new classes that don't have classes extending them. I have found that Java developers often don't put a lot of consideration into making a class final or planning for it to be extensible, but this is an example of where such consideration is worthwhile.

When it is not practical to make the parent class final, NetBeans offers three other approaches for addressing the problem of an overridable method called from the parent class constructor. Even if the class cannot be final, that method can have the final modifier applied to it so that the constructor is no longer calling an "overridable" method. This allows a child class to still extend the parent class, but it cannot override that method and so won't have an implementation that depends on instance level variables that have not yet been initialized.

The other two ways shown that NetBeans uses to handle the issue of an overridable method being called from a constructor likewise focus on making that invoked method not overridable. In these other two cases, this is accomplished by making the invoked method static (class level rather than instance level) or by making the method private (but then not accessible at all to the child class).

In my particular example above, another way to fix the specific issue would have been to convert the instance-level marketFactor variable into a class-level (static) variable because that would move its initialization forward enough to be used in the invocation of the overridden method. The problem with this method is that there is still an overridable method invoked from the parent constructor and that method can have more than one state issue to worry about.

In general, I try to be very careful about a constructor calling methods as part of an instance's instantiation. I prefer to use a static factory that constructs an object and calls static methods to set that instance appropriately. It is always wise to be cautious with implementation inheritance and this issue of overridable methods called from a constructor is just another in the list of reasons why caution is warranted.

My example shown in this post is relatively simple and it is easy to figure out why the results were not as expected. In a much larger and more complicated example, it might be more difficult to find this and it can lead to pernicious bugs. NetBeans helps tremendously by warning about overridable methods called from constructors so that appropriate action can be taken. If a class is not extended and there are no plans to extend it, then making the class final is easy and appropriate. Otherwise, NetBeans presents other options for ensuring that all methods called from a constructor are not overridable.

The Checker Framework

One of the interesting tools I learned about at JavaOne 2012 is The Checker Framework. One of the Checker Framework's web pages states that the Checker Framework "enhances Java’s type system to make it more powerful and useful," allowing software developers "to detect and prevent errors in their Java programs." One way to look at the Checker Framework is as an implementation of what JSR 305 ("Annotations for Software Defect Detection") might have been had it not fallen into the Dormant stage.

The intention of JSR 308 ("Annotations on Java Types") is to "extend the Java annotation syntax to permit annotations on any occurrence of a type." Once JSR 308 is approved and becomes part of the Java programming language, annotations will be allowed in places they are not currently allowed. Although JSR 308 is still in Early Draft Review 2 stage, Checker Framework allows a developer to include commented-out annotation code in places not currently allowed until made available by JSR 308. It is important to note here that JSR 308 only makes annotations more generally available (specifies more types of source code against which they can be applied) and does not specify any new annotations.

The Checker Framework requires Java SE 6 or later. The Checker Framework can be downloaded as a single ZIP file at http://types.cs.washington.edu/checker-framework/current/checkers.zip. The downloaded file can be unzipped to the directory checker-framework and then an environmental variable called CHECKERS can be set to point to that expanded directory's subdirectory "checkers." For example, if the checkers.zip is unzipped to C:\checker-framework, then the environmental variable CHECKERS should be set to C:\checker-framework\checkers.

One the Checker Framework checkers.zip has been downloaded, expanded, and pointed to by the CHECKERS environmental variable, it is time to try the Checker Framework out. The "long way" of running the Checker Framework is shown next and is used with the -version tag to verify that Checker Framework is applied:

Windows

java -Xbootclasspath/p:%CHECKERS%/binary/jsr308-all.jar -jar %CHECKERS%/binary/jsr308-all.jar -version 

Linux

java -Xbootclasspath/p:$CHECKERS/binary/jsr308-all.jar -jar $CHECKERS/binary/jsr308-all.jar -version 

The above should lead to output that looks something like that shown in the next screen snapshot.

The installed Checker Framework can now be applied to compiling code. The next code listing shows a simple class that specifies that a method argument should not be null via the checkers.nullness.quals.NonNull (@NonNull) annotation.

Example of Using Checker Framework's @NonNull

package dustin.examples;

import checkers.nullness.quals.NonNull;
import static java.lang.System.out;

public class CheckersDemo
{
   public void printNonNullToString(@NonNull final Object object)
   {
      out.println(object.toString());
   }

   public static void main(final String[] arguments)
   {
      final CheckersDemo me = new CheckersDemo();
      final String nullStr = null;
      me.printNonNullToString(nullStr);
   }
}

The above code listing shows a null being passed to a method with the argument annotated with @NonNull. NetBeans 7.3 flags this with the yellow squiggles and warning if hovered over. This is shown in the next screen snapshots.

Although NetBeans flags the null setting of a parameter marked with the @NonNull annotation, the compiler builds that code without complaint. This is where the Checker Framework comes in. Because it's a pain to type in the long command I showed previously, I either run the command shown above with a script or set up an alias as described in the Checker Framework Installation Instructions. In this case, I'll use an alias like this:

Setting Windows Command Line Alias for Java Checker

doskey javachecker=java -Xbootclasspath/p:%CHECKERS%\binary\jsr308-all.jar -jar %CHECKERS%\binary\jsr308-all.jar $*

The setting of this alias and running it with the -version flag is demonstrated in the next screen snapshot.

It is far easier to apply this approach with the alias set. This can be used to compile the class in question as shown next (command using my 'javachecker' alias and image showing result).

javachecker -d classes src\dustin\examples\*.java

The above command demonstrates that I am able to use normal javac options such as -d to specify the destination directory for compiled .class files and pass along the Java source files to be compiled as normal. The example also demonstrates that without specifying the checker processor to run as part of the compilation, the @NotNull additional typing is not enforced during compilation.

Before showing how to specify a processor to force the @NonNull to be be enforced during compilation, I want to quickly demonstrate that this compilation approach will still report standard compiler errors. Just for this example, I have renamed the "nullStr" variable passed to the method of interest on line 17 to "nullStry" so that it is a compiler error. The next two screen snapshots show this change (and NetBeans's reported compilation error) and how the Checker Framework compilation approach also reports the javac error.

Having shown that this approach to compilation compiles compilable code normally, reports compiler errors normally, and shows version appropriately, it is time to apply it to stronger type enforcement. I fix the compiler error in my code by removing the extra "y" that I had added. Then, I need to pass -processor checkers.nullness.NullnessChecker as an additional flag and argument to the compilation process. Note that there are other processors besides NullnessChecker, but I am using NullnessChecker here to enforce the @NonNull at compile time.

The following shows the command along with the output window demonstrating that command in action. Note that the compilation process is not allowed to complete and an error based on violation of the @NonNull typing is reported.

javachecker -processor checkers.nullness.NullnessChecker -d classes src\dustin\examples\*.java 

This blog post has introduced the Checker Framework and shown how to quickly apply it to stronger type enforcement in Java source code. I only focused on one type of stronger typing here, but the Checker Framework supplies other built-in type checks and supports the option of writing custom type enforcement checks.