And what can I do next?

Exploring interesting features in Android

I've been playing around with Android lately (which brings me right back to Java - I may have to change this blog's tag line) and it has some very interesting, innovative concepts behind it. Like the iPhone OS its application model has been developed from scratch, rather than relying on decade-old technologies like desktop apps. I'm sure that some of these concepts will migrate back to the desktop at some time, in fact, I believe that's exactly what Apple is trying to push along with the iPad (including the dubious benefits of a completely closed and company-controlled App Store).

Two innovations are pretty cool in Android. First applications are broken up into components, rather than being a monolithic unit. What the user perceives as an application, is a so-called task, which can be comprised of components belonging to multiple applications. For example an app can borrow a browser window from the Browser app. That is different from the iPhone: here browser integration happens via a OS-level widget. And of course, Apple alone determines which services should be available, while on Android any application can offer its services to others.

The other interesting functionality is Android's management of background applications. I've kind of agreed with Apple: burdening the user with managing which applications are active and take memory and battery power is not a good idea. But Android got an elegant way around it: It can shut down applications as needed. If they manage their state correctly, the user wouldn't even see the difference between an active or an re-launched application. Also, Android can shut down different parts of the applications separately: for example terminate UI components, while leave the background process running.

Giving the OS the ability to shut down apps would be useful on the desktop too. How many users end up with a dock or a task bar full of active applications until - if they don't routinely reboot - the computer starts getting sluggish.

My software design principles

I haven't done much programming lately and was more busy with the design of a large application. In the realm of enterprise software development the process is often much less formal than everybody pretends it to be, so it is very important to stick to some general principles in order to come up with a good, long living application. Here are my principles:

Keep it simple. Simpler programs mean less code to be written, tested and maintain. I believe the elegance of an application lies in its simplicity. That's not as obvious as it sounds as many designs pride themselves with being as flexible and have as many functions as possible, which usually leads to complex code.

Good design is if an application can be used for unknown requirements it was never designed for. I read this somewhere, but is sums it pretty much up. In two ways. First, a design should be flexible enough to be easily adapted to future requirements. Second, those requirements are unknown. Don't try to analyze them upfront, because at some point you have to deliver something of value. Making a design hold for future unknown requirements, requires a lot of domain experience, educated guesses and luck.

Optimize for the most common, not the most complex use case. Complexity usually balloons during requirement gathering phase. One main reason is that the users of an application often try to avoid making hard decisions that may prove wrong in the future. So you get often something like "I've never seen this use case, but it doesn't mean that it can never happen". Coding for all these rare use cases often lead to complexities like many-to-many relationships, name/value pairs or unlimited hierarchies and thousands of lines of code to handle them. If this code is rarely or never needed, it's difficult to be tested and maintained and it becomes stale over time, means that after a few iterations nobody is really sure if it is still working. This doesn't necessarily mean that rare use cases cannot be supported: often there's a work around that still fulfills requirements, maybe at the cost of usability - which doesn't really matter if it's happening only rarely.

Don't go overboard with abstraction. Abstraction is an important tool to achieve flexibility, reuse and better testability and maintainability. But there is a too much, which is often observed in Java application. Compulsively abstract away every concrete implementation has a negative effect on code readability. There are common sense abstractions, like UI vs. business logic, logical vs. physical data model and data manipulation vs. business methods. But often it's better to choose an implementation and stick with it, rather than squeezing in another layer of abstraction.

Configuration does not always trump code. There's a general aversion against "hard-coding" configuration parameters in code. But it doesn't necessarily make it better to put them in a, for example, XML file. As long as the configuration is encapsulated from code, I don't mind if it's Java or XML or a properties file. Java code is especially useful if the configuration doesn't only contains strings but also logic, like validation rules. Pure text configurations make a lot of sense only if they are maintained by a non-programmer or they can be changed at runtime.

If it looks like some of those principles conflict - yes, they do. That's what makes good design a challenge. There's a middle ground to be found. Perfection is impossible and only leads to missed deadlines.

Don't prefix Java method calls with this or super

Here's one thing I stumbled over in Java code several times. Some developers believe that prefixing method calls with "this" or "super" produces more readable code, as in

this.method();
super.method();

Supposedly this would make it clearer where to find the actual method definition. I don't think so. When prefixing with "this", the method can be defined anywhere in the class hierarchy. Also, since Java doesn't support functions outside a class context, "this" would be redundant.

Prefixing with "super" is plain wrong as it forces the execution of a method in one of the super classes, even if the method is overridden in the actual class. As the code evolves, this may lead to tricky bugs. Also, it basically breaks polymorphism, which means that Java decides at runtime which method implementation is called. In the case of an implementation in the super class this is not obvious as Java doesn't allow to change the class hierarchy at runtime, but functionally it's very much the same thing as casting an interface down to its implementation class.

Bottom line. Never prefix method calls with "super", except when explicitly calling the implementation in the super class from an overridden method. And I don't see a reason to prefix it with "this".

Exceptions in Java are your friend

Exception handling is one of my pet peeves as I believe it's essential for building robust applications. In an enterprise environment with its short development cycles, it's not only about deploying high-quality software, but also, in the unfortunate reality, about finding bugs in production code as quickly as possible. And that's where correct exception handling helps a lot.

In Java there are two types of exceptions: checked and unchecked ones. The former need to be declared or caught explicitly in the code, the latter remain invisible until they are caught somewhere in the call stack. If uncaught, they just terminate the program.

There's a lot of discussion about if checked exception make sense at all or if they just pollute the code. I do believe they make a lot of sense, if used correctly. Unfortunately in third party libraries and even in the Java runtime that's often not the case. But here's how they should be used:

• Checked exceptions signal a condition that can and will happen during normal program operation and that needs to be handled somehow. The application is able to recover.
• Unchecked exceptions should be thrown id an unexpected error occurs. The application cannot recover and needs to shut down or - more often - cancel the request and show a generic error message. Those errors stem from bugs and system issues.

This categorization applies per component or API. An application usually is composed of many components and hence the meaning of exceptions can change as they are thrown up the call stack. For example, every JDBC method throws a checked SQLException. From the (narrow) view point of JDBC this is justified with the fact that databases are external systems that can be unavailable, not configured correctly. Also much of the data that is passed to the database is unverified and may lead to an error once it hits the database engine.

For an database-driven enterprise application a SQLException most likely is unrecoverable and should be wrapped into a unchecked exception (I use a UnexpectedException for this purpose). For a database query tool, where the user enters SQL, the same exception is a business exception and should be reported to the user in a nice error message. In most cases you would still wrap it, but into another checked exception.

But why dealing with checked exceptions at all? Because they force the developer to think about the fact that things can go wrong right at the place where they can go wrong. Because it not only matters which exception is thrown, but also where it is thrown. For example, a FileNotFoundException can just mean that the user entered an incorrect filename, or that an essential configuration file could not be found because the hard disk is corrupted.

Unchecked exceptions tempt to have a central exception handler, for example a error page configured for the web container. But in many cases the message attached to the exception doesn't really tell what actually went wrong and dumping the stack trace is not very user friendly (and has some negative security implications as well).

But by handling a checked exception when it actually happens, makes it possible to react in a meaningful way (just never do this), even it's only by wrapping it into a another exception with a explanatory, custom error message. Whoever has to analyze a logfile in order to fix an urgent production issue will thank you big time.

What's really the next big thing after Java?

Recently I stumbled over a discussion at TheServerside about "10 good reasons to look for something better than Java". It starts promising, stating

"Don't get me wrong. During my professional life I have written tons of Java code and of course I think it is a great language still. For sure it has been a great improvement from C++ and Smalltalk. But now even Java is starting to feel the weight of its 15 years."

But then what follows is the usual list of low level language features, like closures or getting rid of checked exceptions. Most of them are highly debatable, with a lot of devil in the details (e.g. closures) and would show productivity gains only after a long time, if at all, when they are widely adopted.

However, the initial statement holds, but the real issue with Java lays far deeper. The thing is that Java was conceived as a language for embedded systems (where it never really took off) but today today most frequently used for multi-user, database-oriented server applications. But the language and its runtime library has almost no support for this class of applications. So you need to pile layer upon layer of frameworks, libraries, application servers etc. in order to build an environment that can implement even the most simple service. Before developers can start with actual coding, they have to make a lot of decisions about architecture, design and implementation of all the components involved.

In a way the situation reminds me on the RISC vs. CISC discussion in processor design, back in the 90s. The idea was that simpler and thus faster processors would be beneficial to overall performance as compilers could not really take advantage of the complex instructions of a more powerful processor. Same with development: The Java ecosystem has everything available for setting up an environment that is optimized for a certain task, but real corporate development teams most of the time cannot take advantage of this as time to market is paramount and time for architecture and design very limited. So usually they stick to what they know best, no matter how much sense it makes for the problem at hand.

That's why I believe that a successor to Java is an language and environment tailored to build server-side applications, with first class support of concepts like transactions, concurrency, distributed computing or clustering. It does not need to be an all-purpose language like Java as we can have a another tailored language, let's say for GUI-based apps. It would require dev teams to code more themselves, instead of looking around to find a library that provides sort of the needed functionality, but in the end lead to a far better understanding of the code.

The most productive language I've ever programmed in was Clipper, a programming language optimized for file-based databases (dBase). It wouldn't hold for today's requirements, but at that time it provided dead-simple ways of putting together an UI (not graphical) for maintaing data in a database, all without an IDE or other supporting tools. And it was efficient - I saw an application with hundreds of clients running against database files stored on a single Novell Netware server.

For a successor for Java, I would say, that is the way to go.

Groovy Notes: When dynamically typed variables bite you

Most developers love dynamically typed variables as they save some key strokes and - in most cases - don't really reduce the reliability of the code. On the other hand, there are cases where data types do matter and Groovy's way of silently assuming data types and convert one into each other, can lead to subtle bugs.

For example, I had to read a file produced by the Sybase bcp tool. Since the database itself contained new line characters (my advise: don't do this), I had to use a null byte as the line terminator. This means I cannot use the standard readLine() methods, but I have to read the char by char and process the field and line delimiters manually.

Suppose there's a tab delimited text file (this is a simplified example - not the file I was working with):

10	ABC	cde
20	XZY	rst
30	MNO	qwr

Using the following piece of code to read the file and print it again, will not produce the wanted results:

def file = new File("tab.txt")

file.withReader() {
	reader ->

	def c = 0
	while (c >= 0) {
		c = reader.read()
		if (c>0) print c
	}
}
println ""

Instead of printing the characters, it's printing the ASCII codes. This is because Reader.read() returns (for some not obvious reason) an int instead of a char. Solution is to cast the int into a char before printing:

if (c>0) print "${(char)c}"

Another issue arises with fields loaded from a database. JDBC will take care that the internal data type of the variable matches with whatever data type the field has in the database. If you then use, for example, a numeric field from the database and compare it with a piece of String data loaded from a text file or user input it won't match, even though the data looks by all means the same when you print it to the console.

Bottom line: Groovy is nice and simple, but without a knowledge of the underlying Java implementation, you can pretty quickly end up with mind-boggling bugs.

Groovy Notes: Closures, break and continue

Turns out I had a pretty narrow view on the issue with break and return in Groovy closures. The actual issue is bigger, as a return in a closure sometimes acts as a break, sometimes as a continue, depending on the method that executes the closure. And apparently there's not really a solution to this.

Groovy Notes: reading Excel files directly with POI

Here's another episode of little pieces of work that can be handily automated with Groovy. This one selects three columns by their names from an Excel sheet and writes their content into a text file. It uses Apache POI for reading the excel file, which took me only a couple of minutes to figure out, even though I've never used it before. Note that I didn't use the new SS interface as well as had to fall back to some deprecated methods as I had only an older version available:

import java.io.*
import org.apache.poi.hssf.usermodel.*
 
def dir = "/tmp"
def inFile = "sheet.xls"
def fileName = "outfile.txt"

def COLUMN_NAMES = ["Name", "Full Name", "City"]
InputStream inStr = new FileInputStream(inFile)
HSSFWorkbook workbook = new HSSFWorkbook(inStr)
HSSFSheet sheet = workbook.getSheetAt(0)
HSSFRow headerRow = sheet.getRow(0)
 
def columnIndexes = []
for (HSSFCell c : headerRow) {
 if (COLUMN_NAMES.contains(c.getStringCellValue())) {
 columnIndexes << c.getCellNum()
 }
}
 
new File(dir, fileName).withWriter() {
 out ->
 for (HSSFRow r : sheet) {
 if (r.getRowNum() > 0) { // Skip header
  columnIndexes.each() {
  def c = r.getCell(it)
  if (c) {
   def text = c.getStringCellValue()
   text = text.replaceAll(/[^A-Za-z0-9 ]/, "")
   out.writeLine(text)
  }
  }
 }
 }
}
 
inStr.close()

Sweet!

Groovy Notes: generating SQL from Excel

There's an Excel sheet with data and it needs to be inserted into a database. There are certainly tools for this, but it can be easily done with a Groovy script as well as shown in the following example:

System.in.eachLine() {
	def fields = parseExcelCsvLine(it)
	print "insert into some_table values ("
	print "'${fields[0]}', cvnull(${fields[1]}), '${fields[3]}'"
	print ")" 
	println ";"
}
 
def parseExcelCsvLine(line) {
	def fields = []

	// Flags
	def insideQuote = false
	def word = new StringBuilder()

	line.each() {
		switch (it) {
			case '"':
				insideQuote = !insideQuote 
				break 
			case ',':
				if (!insideQuote) { 
					fields << word.toString() 
					word = new StringBuilder() 
				} else { 
					word.append(it) 
				} 
				break
			default: word.append(it) 
		} 
	}

	fields << word.toString()

	return fields
}

// Creates SQL for adding notification filters
def cvnull(string) {
	if (string == null || string == '')
		return 'null'
	else
		return string
}

This scripts assumes a comma-separated file exported from Excel. The most complicated piece is the logic splitting the lines: Excel will put quotes around field values which contain commas and the parseExcelCsvLine function makes sure that the quotes are removed. cvnull is a little utility that converts empty fields into the string "null" values, which comes handy for nullable columns.

From there this script can be called with:

groovy generateSQL.groovy < data.csv > inserts.sql

It took me less time to write this script (except the parseExcelCsvLine function, which I had implemented earlier an put into our common library) than to figure out how to use our database tool to do this for me.

What is going on in Apple's Interface Builder?

Besides HTML stuff, I've done relatively little user interface stuff. That's why I'm pretty excited to look into iPhone development, which is centered mostly around UI work.

The architecture on the iPhone is pretty much standard: MVC, with a hierarchy of views that display the various user interface elements. In order to quickly layout the views on the screen, there's the Interface Builder tool, which allows to position views as well as set properties that determine their look and feel.

The interesting part is that Interface Builder does not generate code. The UI design tools I've seen in Java would in the end generate code with some blanks to be filled by the developer. Interface Builder, on the other hand, creates a serialized object graph that is loaded and instantiated at application launch time. That means that Interface Builder cannot get out of sync with the code - something that can happen more easily with code-generating UI designers.

Besides the visual layout, the developer needs also define the dependencies of the view as well as the controllers and model components. Coming from Java, this reminds me a lot of IoC frameworks like Spring. In fact, Spring configuration files are nothing more than a serialized descriptions of the objects that are instantiated at runtime. This means that Interface Builder could be used for more than just the UI - in theory you could design the entire application with it. Unfortunately Interface Builder doesn't support setting user defined properties in objects other than the standard user interface elements without doing some serious work. Maybe it would be a good idea to add this. Just a thought.