Poju4s obviously stands for “Plain old JUnit for Scala”

It is not much yet and it has some distance to go before being really useful but I think a few sparks of elegance have found their way into the code (or that is just the fever talking). Anyway, before I understand better than to make this stuff public. Without further due… here it is for what its worth:

A quick tour

At the moment I’m focusing on how to interact with JUnit from the REPL. Lets get a quick feel for what poju4s can do in this department. First of all fire up a REPL (I use SBT for this purpose):

Next import poju4s:

Now we’re ready to run tests from the REPL. Lets start by running an invdividual test. This consists of two steps:

1. Mix in an interaction point for poju4s
2. Run and render the output

Here we use InteractionSpec that is a spec in poju4s itself:

What happened here? By mixing in Basics into our plain old JUnit test it is runnable from the REPL. Basics just contains default choices for poju4s such as outputting information to standard out, adding methods for finding and running tests. The next question is what brief is? Brief runs the tests one by one and outputs a green dot for each passing test and ends the run with a summary of the tests run. To make things a bit more interesting. Lets use an example spec with some tests that don’t succeed:

How pretty! :) Ok, as you see, since this is an experiment I can do whatever I want so adding silly colors came higher up on my list than things like being able to find and run all tests on the class path for instance :). For those poor bastards on some crippled terminal incapable of displaying such state of the art stuff like tty colors there is of course a way to avoid the tty control-codes to be sent:

As you can see brief is just a shortcut for new BriefReport with Color

I could show how pending and fixed works but that requires a bit of a scenario, perhaps a video so I’ll save it for later. Feel free to browse the source.

The obvious missing feature now is to be able to glob or grep for tests to run. It’s being worked on.

Please tell me what you think so far. Should I even bother with making the binaries available as maven dependencies?

I’ve been looking forward to write this post for some time. I really don’t like Maven but I realized that I don’t exactly know why. It will be useful to explore that. I don’t want this to be just another of the many maven rants out there. I’ll try to be fair but I also reserve the right not to think that some things are good  even if I can’t present a better alternative :) This is a subjective post.

I have yet to try Maven 3 though so things may have improved in that version.

When it comes to my experience with maven: My first experience with Maven must have been in 2004 or so. We investigated if we could use maven for a project we were starting. A few of us looked into it and found Maven to be a bit opaque “what does this thing actually do?”. Since then I have that part figured out of course but back then we stuck with our ant-script. That was probably the wrong choice. The feeling of an untransparent build-system still sticks somehow. I find that Maven is sometimes a bit hard to penetrate. I find that my maven usage is a bit google intensive and that maven is a little bit of a black box at times. I don’t know if it’s just me or if the Maven philososphy is a bit like “don’t you worry about your build, we take care of it. Just write mvn clean install and you’re good to go”. I’m outsourcing the details of my build to strangers.

I have used maven on and of through the years. Maven does a decent job most of the time. Can’t complain. Above all Maven gets you started quickly; just arrange a project so that Maven finds it’s way and your’re all set. Maven has also made wonders for the Java community when it comes to how to distribute your binaries. Before Maven there weren’t really any good options. In fact Maven is pretty decent over all so what’s the problem really? Maven stands in the way of innovation, Maven is not excellent, Maven assumes there is one right way to build for everyone, Maven does too much and sometimes bites off more than it can chew, Maven is hard to extend, Maven is a bit of a black box, Maven is slow, Maven is unreliable.

Ok, that was a lot of unfounded opionons. They all deserve to be motivated. I will explain what I mean with each of them one by one in reverse order:

Maven is unreliable

Ever deployed something you just built and found that for some reason your last change did not make it to the build? No? Good for you! An experience I know I share with many is that I build clean quite often with Maven. It happened a few times too many that I deployed something after a dreadfully long build-cycle just to find that everything was not in there in mysterious ways. Build clean, wait again.

I especially find Maven unreliable in combination with eclipse. The Maven plugin for eclipse can really mess up eclipse’s build-cycle. Infinite loops, persistent errormarkers that are never removed etc. You often get different results from the prompt and from within eclipse. It happened a few times that we had infinite loops of rebuilds in Eclipse (ok, I can’t prove that the loops were infinite since we did not wait for an infinity but we did see the same projects being rebuilt over and over again). The incremental build is about the only truly excellent feature of Eclipse and with Maven that pretty much goes down the drain. If there is one area where Maven really does suck it’s the eclipse integration. As usual though Maven gives you enough to make you think it’s worth the hassle. I’m not sure if it is.

Maven is slow

I heard that Maven 3 is more snappy. Good thing. Maven 2 is prettty awful performancewise. Why is that a problem? Slower builds, slower feedback. Slower feedback, worse quiality. Simple as that. Hardly anyone that use Maven does not know how to disable tests in a build just to make the build times bearable. While it’s fair to point out that most developers could learn a thing or two about how to make their tests run faster maven still has a pretty long ramp up time when it does not output any useful feedback to the developer.

Maven is a bit of a black box

This is one of my key issues with Maven. Partly the black boxyness is due to java as the language of choice for creating plug ins. In Buildr for example, if I want to know how something works I can just find the source and look at it (Buildr is made in ruby). If I get a stacktrace I can do the same. If there is a bug in the plugin I can fix it by editing the plugin source. If I feel like it, I can generate a patch and submit it so I won’t have to fix it again when upgrading.

While I can download the source for a maven plugin and do the same I also have to think about how to distribute my plugin to others after I figured out how to get the source and compile it. If I want to know how things work I will at the very least have to find the documentation for the plugin (if there is any good documentation to be found). Due to the maven strategy of plug-ins the documentation is fragmented and scattered.

Maven could compensate for this better. For instance when I run mvn –help it would be nice if I could somehow see what the build-cycle will look like. What will happen when. It would also be nice to get a listing of all pugins I have in my Repo and their options, heck, it would be cool if I could see a listing of all plugins available in any listed online repository. I bet there are plugins that can provide this information but then I would have to know about them. I could google but I think I should also be able to find out via mvn –help.

If I supply an option to maven like -Dmaven.test.skipp it would be pretty nice if Maven told me that maven.test.skipp is not a valid option. Why not tell me which options are available whenever I supply an unrecognized option? Sure, Maven can’t know which options plugins accept… or could it? At least maven could have been designed to be able to do this if it was considered important. But obviously it wasn’t… consiered important. Maven doesn’t even tell me about my alternatives when I run jetty:start. It would indeed be helpful to know my alternatives then and that one of them is jetty:run. In all projects that used maven I have heard a lot of questions among teammembers: “Do you remember how you do X in maven…”. We’re not talking about hard stuff. Just basics like how to run the integration tests. Symptomatic of an opaque system.

Maven is hard to extend

When it comes to plugins maven could do what SBT does. Just supply the source in the correct directory and they will be compiled when needed. It would simplify distribution, versioning of your whole build in one place etc. Of course with an interpreted language it all becomes even simpler and perhaps build tools should be done in interpreted languages (as is the case with Buildr and Gradle for instance). Given that the source is always available it’s much easier to learn those inner details that so easily get lost otherwise.

Maven does too much and sometimes bites off more than it can chew

I often feel that Maven is not satisfied with being my buildtool. It wants to be my whole development process. Maven dictates how I should do my versioning, packaging, deploying, directory structures you name it. There are very few areas of development where maven does not meddle. In this Maven sort of becomes a jack of many trades, master at none.  I think the maven plugin is what will kill Eclipse (ok, Eclipse wont die easily but the maven plugin totally takes over eclipse’s build and turns it into an average IDE from the early 2000s buildwise…). I often feel that Maven’s approach sort of works but is a bit dated and far too rigorous for the simple case… but hey… it sort of works so why complain?

Maven assumes there is one right way to build for everyone

I have to conform to Maven to a large degree and the compromise is always on my side. This is not by necessity a bad thing. The maven folks are right that all builds do essentially the same thing so why shouldn’t we standardize the process and solve it once and for all? To be honest, if Maven hadn’t beaten me to it I would probably have started (and if I know myself not finished) a project to create something like Maven. Maven’s idea makes total sense. The bad news is that there is no one true model for every problem. Even if it sounds totally crazy, the way you do your builds can be a competitive advantage. How you build is a key component to how you get your feedback and how often you can deploy. Your build tool is a strategic tool!

There are also ways in which Maven punishes totally valid choices. For instance in multi site it’s hard to get an aggregated view. If I chose to rely mostly on acceptance tests and not unit tests I can’t see how well every module in my system is covered by those tests. The test coverage reports with cobertura for instance are generated per module from the tests in each module. I cannot see how well module B is covered from module A. You may argue that this way of testing a system is not pure, good, cocher or whatever you call it but that would just be you imposing your opinion on everyone else. Maven should not make that choice for developers.

You may make the point now that it’s not Maven’s fault that the cobertura plugin works how it works. Fair, and it would be ok too if it would be a simple thing for me to adapt to my needs but as far as I’m aware there is no way I can express my needs in XML so that the cobertura plugin understands me. If it would be a simple thing to fix given that I have downloaded the source for the cobertura plugin I don’t know. Perhaps it’s in Mavens nature?

Another point is this thing with versions. What if I release every 10 minutes? I can really get a versioning nightmare in Maven. If I use snapshots for everything it can become a performance nightmare (so that I in efect can’t release every 10 minutes).

Couldn’t agree more that each build is not as unique as a snowflake. They all look pretty much the same from a long enough distance but so does snowflakes. Mavens build process works ok for everyone. Mavens process is optimized to suit average needs (and succeded quite well). On average Mavens model works well but it will just be average :)

All builds do essentially the same thing but the focus can make a world of difference.

Maven is not excellent

If Maven was the end of the road when it comes to builds; the ultimate one and only answer to how builds should be done then that would indeed be excellent. Unfortunately I doubt that there will ever be a one and only ultimate build solution out there. I have no doubt that Maven is a perfect match for some but I know it’s not a perfect match for everyone. It is all fine if Maven would recognize this but it doesn’t. Maven really tries to be the ultimate build solution and is bound to fail like every other tool that attempts this.

There are so many dimensions that you could optimize for and it’s highly unlikely that one tool would excel in all of them.

Maven stands in the way of innovation

Maven does a good enough job for most problems to have become popular. When a tool becomes as popular as Maven most people don’t even look at another tool. There are other initiatives out there that deserves more attention but teams play safe and choose the tool that everyone knows (though most people I have met know little more than how to type mvn clean install).

Maven is now an established tool. As such there is not much innovation happening at Maven’s initiative. Most new cool things like shells etc are created in other tools first.

I really can’t say why but I also have this strong feeling that the Maven team does not share my values when it comes to how I want to build my software. I have specific needs when it comes to testing, deploying etc. I don’t really believe in the one tool to rule them all idea.

If I’m going to outsource something as important as my whole build process to a team of people I don’t know I better feel that I trust that team to look after my needs. I don’t. Especially since the Maven team has already failed twice to create an excellent build tool. Why would they succeed a 3rd time? What indications are there that the Maven team learned any important lessons from Maven 1 and Maven 2 that will make Maven 3 the final one tool to rule them all?

So pardon me for not jumping up and sown with glee when I heard that Maven 3 was released. I’m sure I will find out soon enough just how excellent Maven 3 really is (wether I want to or not). As for my curiousity and learning that comes from it it is currently aimed at gradle and SBT.

I think Maven works fine but in the past some logical choices have been made that do no longer make sense (XML, a compiled language for plugins etc). Other tools have fixed those and it will be hard for Maven to change in fundamental ways (and Maven shouln’t, it does what it does). When I will have to use Maven I wont bitch about it but I will sure keep my options open. Will you?

Cheers!

I hate best practice. If you follow best practice you will end up with Maven, SVN and programming in java (the language) #rest_my_case

I was promptly corrected by @jchyip (Jason Yip):

@johlrogge That’s equating “common practice” to “best practice” which is a very common failure mode. Popularity != best option.

He couldn’t be more right. In fact, I have often felt that what is marketed as best practice is often common practice with a more attractive label. The two are confused a lot. It sort of makes sense too, I mean – if everyone is doing it, it must work. Right? If it wasn’t the best then why would everyone do it? Granted it will probably work just fine, but will probably not be the best, so don’t call it that. Assuming that excellent software development is rare, then best practices should also be pretty uncommon. Given this, you probably shouldn’t be doing what everyone else is doing, if you are going for “best”.

So what is a best practice, really?

First some definitions from googling define:”best practice”

A process, technique or innovative use of resources that has a proven record of success in providing significant improvement in cost, schedule, quality, performance, safety, environment or other measurable factors that impact the health of an organization – www.reliableplant.com/Glossary

The optimal solution to a business problem. – campuspol.chance.berkeley.edu/GlossaryofTerms.doc

A way or method of accomplishing a business function or process that is considered to be superior to all other known methods. -www.qaproject.org/methods/resglossary.html

A best practice is a , method, process, activity, incentive, or reward that is believed to be more effective at delivering a particular outcome than any other technique, method, process, etc. when applied to a particular condition or circumstance. – en.wikipedia.org/wiki/Best_practice

Those quotes do not all say exactly the same thing. Some of them appear to be narrowed down to a context such as business processes or organization issues. Others are just plain naive like “the optimal solution to a business problem”. It is important to realize that a best practice is context bound. I got a reply to one of my tweets about that from Viktor Klang

@johlrogge “Best” implies that all use-cases are equal in both resources and restrictions, which never is the case. What is being measured?

As we have all experienced over and over, what may work brilliantly for someone else may not work so well for you. As Viktor points out, we must be aware of the context in which a practice is supposedly “best”. All things considered, the number of contexts for software development is a very large number. That makes it hard to provide any useful general advice at all. The context must always be considered and measuring the correct things is a science of it’s own (I’d even say that for software it’s an unsolved problem so some of those definitions cannot apply to software, in my humble opinion).

But that is not the only problem with best practices; “best” implies there is nothing better. Isn’t there? I’d say that if we look at what we are doing today and find that it’s exactly what we were doing 2 years ago, it probably means that we don’t spend enough time looking for better practices.

Looking for better practices

“Best” implies “ultimate”. Way too often the term ‘best practice’ is associated with something someone is trying to sell you: “my method is founded solely on best practices” (like ITIL). I think that is why we don’t call our best practices “currently accepted theories” which would be more science-like in the sense that they invite better theories to replace them. To illustrate, try this sentence: “we use best practice until a better best practice replaces it”. Sounds kind of silly, doesn’t it? (That is right: good, better, best; only a hobbit would add “bestestest” to that)

I found this page that describes the scientific method. It goes on and on about how good theories “invite replacement” like in this quote from the conclusion:

Any system of thought that proclaims itself to be “ultimate” or beyond correction is dogmatic and wrong: The best theories are the ones that happily give way to better theories.

The worst are the ones that do not budge and refuse to admit new evidence that disputes them. They become stagnant and outdated.

Science is revolutionary because it accepts new facts, new evidence and new thought. New knowledge acquired from the scientific method frequently changes society with technology and ideas. It is refreshing, challenging and above all the scientific method continues to dynamically improve its description of the world.

I currently subscribe to many theories of software development. Like – the theory of continuos refactoring, the theory of continuos integration, the theory of release early and often, the theory of self-verifying specs  and so on. I use, support and promote them because they, among other theories, are the best I know of when it comes to developing software. But even such fine theories cannot be prescribed without a context. Releasing early and often may not be a good idea, if what you get payed for is upgrades to new versions of your software. It may not be a good idea if you are unable to verify the quality of your system as fast as you can deploy new versions of it. Also, no matter how good a theory seems to be – compared to what you were doing yesterday, always assume there is a better way (that may even be the complete opposite of what you’re currently doing).

I’ll sum up my point here in a tweetable way: Don’t settle for best, always look for better.

My pet-project is becoming more and more Scalafied and I’m starting to wonder why I ever thought it was important that I should be able to use it from java-applications. Given my detachment from Java some mental barriers are falling and I am adding new strategies to my active toolbox. Today I created some simple traits that I thought would be worth exploring in a blog.

Here we go, logging with slf4j is something I’ve done a lot. I found myself sprinkling a lot of this around in my code:

import org.slf4j.{Logger, LoggerFactory}
...
class SomeClass {
   private val log = LoggerFactory.getLogger(getClass)
   def aMethodThatLogs  {
      log.debug("This is a value: {}", 4711)
   }
}

Nothing new (unless you never saw slf4j before in which case you should check it out now wether you program in Scala or Java. The code so far is not dependent on scala other than syntax-wise.)

What I keep forgetting about

Scala has traits, it’s easy to forget that traits are more than just interfaces. Over the years I’ve gotten pretty used to not being able to add any generic functionality other than via composition or inheritance.

Thinking out of the java-box I created this little trait:

import org.slf4j.{Logger, LoggerFactory}

trait Log {
 private val log = LoggerFactory.getLogger(getClass)

 def trace(message:String, values:Any*) = 
     log.trace(message, values.map(_.asInstanceOf[Object]).toArray)
 def trace(message:String, error:Throwable) = log.trace(message, error)
 def trace(message:String, error:Throwable, values:Any*) = 
     log.trace(message, error, values.map(_.asInstanceOf[Object]).toArray)

 def debug(message:String, values:Any*) = 
     log.debug(message, values.map(_.asInstanceOf[Object]).toArray)
 def debug(message:String, error:Throwable) = log.debug(message, error)
 def debug(message:String, error:Throwable, values:Any*) = 
     log.debug(message, error, values.map(_.asInstanceOf[Object]).toArray)

 def info(message:String, values:Any*) = 
     log.info(message, values.map(_.asInstanceOf[Object]).toArray)
 def info(message:String, error:Throwable) = log.info(message, error)
 def info(message:String, error:Throwable, values:Any*) = 
     log.info(message, error, values.map(_.asInstanceOf[Object]).toArray)

 def warn(message:String, values:Any*) = 
     log.warn(message, values.map(_.asInstanceOf[Object]).toArray)
 def warn(message:String, error:Throwable) = log.warn(message, error)
 def warn(message:String, error:Throwable, values:Any*) = 
     log.warn(message, error, values.map(_.asInstanceOf[Object]).toArray)

 def error(message:String, values:Any*) = 
     log.error(message, values.map(_.asInstanceOf[Object]).toArray)
 def error(message:String, error:Throwable) = log.error(message, error)
 def error(message:String, error:Throwable, values:Any*) = 
     log.error(message, error, values.map(_.asInstanceOf[Object]).toArray)
}

Using it is now a matter of:

import johlrogge.Logging
...
class SomeClass extends Object with Logging {
   def aMethodThatLogs  {
      debug("This is a value: {}", 4711)
   }
}

It’s not a huge difference but I kind of like the with Logging and the lack of boilerplate code in the class-body.
I also like that I can drop the log. part of the logging.

Traits are on the menu

Of course this my wetted my appetite for traits. And it didn’t take long before an opportunity arose. How often have you written code like this in java:

long startTime = System.currentTimeMillis();
AValue value = doSomethingTimeConsuming(parameter);
log.debug("SomethingTimeConsuming took {}ms to complete", System.currentTimeMillis()-startTime);
return value;

I was about to repeat the process in Scala when I thought: function… pass by name… and then… TRAIT!

To make a long story short: Enter Timing:

trait Timing {
  def time[T, R](f: => T, res:(T, Long) => R):R = {
    val startTime = System.currentTimeMillis
    res(f, System.currentTimeMillis - startTime)
  }
}

How this beast came to be:

1. I realized that the timing-code in the java-example could measure anything given that anything given that anything was executable. And thats what pass by name is for.
2. I realized that sometimes you want to have the timed value, sometimes you want to log it as a side effect. So, a second pass by name function made sense. The res-function that takes the timing-value and does whatever it likes with it.
3. You may want to return the timervalue, the result of the timed function, both or something completely different. So I made the result type generic and different from the result-type of the timed function.

How to use it:

import johlrogge.Logging
...
class SomeClass extends Object with Logging
                               with Timing {
   def aMethodThatTimesAndReturns =  {
      time(timeConsumingMethod(4711),
              debugMs("The timeConsumingMethod generated {} in {}ms", _:SomeValue, _))
   }

   def debugMs[T](message:String, result:T, spent:Long):T = {
    debug(message, result, spent)
    result
  }

   def timeConsumingMethod(interval:Int):SomeValue =  {
       Thread.sleep(interval);
       new SomeValue
   }
}

Pretty neat. Of course you can compose all kinds of pre-packaged res-methods. The point is that time becomes reusable and you don’t have to log. You could for instance update and return a data-set for a graph that you’re plotting somewhere else in your code. We have abstracted away timing.

Just so you know

If I would mind that anyone cut and pasted the code I wrote in this post and even make money from it (like thats going to happen) I wouldn’t post it on the net. I hope that the code is useful and/or inspires someone in some way. So use it however you want and have fun!

I have this idea that all practices we apply are somewhere on a scale from dampening to amplifying. Did I hear a WTF? I’ll try to explain…

Dampening and amplifying – some background

I use the terms dampening and amplifying metaphorically. If you studied electronics you know that you often use a combination of dampening[attenuation] and amplifying to separate the desired signal from undesired noise. Think of a radio, you tune in to your favorite radio-show at 104.8 MHz FM. Part of what happens in your radio is that the receiver suppresses, dampens all frequencies except 104.8 (+- tolerance) MHz and amplifies the 104.8 signal so that it can be decoded into music or whatever you listen to.

You could get away with doing dampening exclusively, like when your signal is strong enough but you want to filter out undesired noise. The opposite is also true, if you have  a signal that has no noise, you could just amplify it. This is of course extremely simplified but works as an illustration.

Now imagine software development as a signal, the desired signal is continuous customer satisfaction being deployed regularly. Or to simplify things, delivering excellent software to our target group efficiently. You want to tune your team into that channel. How do you do it?

 Dampening bad behavior

It seems to me that a lot of processes start close to the extreme assumption that people have a desire to do bad things and unless you prevent them from doing that… they will. Using our radio-metaphor this would be an environment with extreme static.

If your starting assumptions are close to this extreme it makes sense to look for dampening practices and introduction of control. Such dampening practices can be implemented as process, technically or both. An example would be a tollgate system where at each tollgate a select few, typically non team members,decide wether or not whatever is being built can pass to the next tollgate or be rejected.

Technical enforcements could be the use of various tools. I am a Java-programmer and I find that Java often has a dampening strategy. Remove things that people find difficult in order to keep the language easy to use; at least for the vast majority of programmers. The downside of course is that programmers that would be comfortable and more efficient with those advanced features are also deprived of them.

Keywords: prevent, reject, suppress.

Amplifying good behavior

At the other extreme we assume that people have a desire to do good things and with freedom and opportunity that is what they will do. Again, using our radio-metaphor this would be an environment with no static.

If your starting assumptions are close to this extreme it makes sense to look for amplifying practices. The signal is clean, turn the volume up to ten. Amplifying practices tend to align views and remove obstacles. An example would be communicating a clear goal in terms of purpose and business value so that everyone knows what is developed for what purpose. Another example is removal of obstacles: the team says that the off the shelf product is not adequate for their needs, lose it! The team needs some infrastructure, get it! Some company internal process prevents the team from being effective, fix it! (either the process or the teams views)

As some of you may know I have been playing with Scala recently. The philosophy of Scala is very different from Java. Most programmers will find a lot of the things in Scala difficult, but hey, they can learn them and if they do they will be more effective so why should they be removed? What about dynamic languages? What if you choose a more powerful language than Java and instead of removing all “difficult things”, even for the ones that know them, and instead educate as necessary? That would be amplifying.

Keywords: enable, accept, encourage

Remove interference

A third option is to remove interference. Removing interference is deceptively similar to dampening but with the exception that you don’t suppress the unwanted noise, you find the source and remove it (or move away from it). In some hospitals there are still signs that you must keep your cell-phone switched off since it can interfere with sensitive medical equipment (at least that is the reason that is given on the signs). Interestingly enough the hospitals trust complete strangers with this great responsibility. Flight crews do too. You have to put your toothpaste in a plastic bag but something so lethal as a cell-phone they kindly ask you to switch off :)

Going back to our team, perhaps we don’t have to be so careful making everything harmful impossible to do. Perhaps sometimes we can assign some great power and freedom to each other and simply agree on how to and how not to use that power… then turn the volume up to ten…

Keywords: eliminate.

I will probably have reason to revisit this subject with more concrete examples later but this post is already hideously long. Until I get around to that (with my regular posting frequency it will be sometime next year), please read about the party planning exercise here and think about how it relates to the dampening/amplifying metaphor.

Also, to my great surprise someone uploaded my source to github here. There is even a Maven pom now to build and run the thing.

I thought I’d share the links since the showed up in my statistics. Enjoy :)

Update: Randin pointed out that the link to xscalawt was broken so I changed the destination. Also Randin thought this link seemed interestong so I provide it here aswell.

I decided to make the source for my DSL available under the beerware license. I choose beerware for several reasons:

1. I don’t have the energy to turn his into a full blown traditional open source project.
2. I believe that libraries without restrictions would make making GUI’s in Scala more appealing if you only need to convince whoever you need to convince that using Scala is a good idea and not having to deal with additional licensing issues imposed by some guy on a power-trip (me)
3. I would need a hand if this should turn into something really useful and I invite any of you to without restrictions take this initiative further and perhaps even tweak it into your own library with or without my help.
4. This project has been about learning for me, I hope it can help others learn too from it.

That I give it away like this does not mean that I’m not interested in what you do with it or use it for. I’m very curious so if you have the energy to do so, keep me posted! But you don’t have to.

I attach a zip with the code in it’s current state. It’s perhaps not pretty but it’s what I have so far. Enjoy the code!

To start with I had to be torn kicking and screaming from my beloved object orientation. I was told I suffer from the Stockholm syndrome which was quite a hilarious way to put it. Especially since I live in Stockholm and was immediately comforted by the same person with the words “I suffer from it too, I still love C++”. So I thought maybe leaving my comfort zone is not so bad?

Yesterday it payed off and I experienced a kind of flow that was new to me in Scala. I thought I’d share that experience here step by step. Don’t expect to be blown away by my Scala-skills, If you want to be impressed in that way there are other blogs :) What follows is a real world example of  things coming together into a moment of productiveness. I guess another real world example of applying Scala to a problem could not harm?

In a previous post I wrote about how to make a DSL for creating SWT-guis in Scala. My source of inspiration was Glimmer. One thing that is obviously lacking in that example is some interaction with other code (in any direction) so I set out to solve that. I have come up with a few different ways and one of the several ways, more on them in a different post, I would like to support is to observe one or several other objects from the GUI. That requires an update mechanism. Some way to tell the created GUI to update itself. I have that solved too but I want to zoom in on one interesting step in the solution. A refactoring. I have ended up with a lot of small snippets like this:

trait Labels {
    def label(setups:(Label => Unit)*)(parent:Composite) = {
        val label = new Label(parent, SWT.NONE)
        setups.foreach(setup => setup(label))
    }
 

    def progressBar(setups:(ProgressBar => Unit)*)(parent:Composite) = {
        val progress = new ProgressBar(parent, SWT.NONE)
        setups.foreach(setup => setup(progress))
    }
}

Basically, I can create a progress bar, a label and both of them can be manipulated with 0 or more setups. Before we continue, let’s rewrite the first of the methods in a more verbose way so that even I would have understood that method 3 months ago:

def label(setups:(Label => Unit)*):(Composite) => Unit = {
   def createWidget(parent:Composite):Unit = {
      val label = new Label(parent, SWT.NONE)
      setups.foreach(setup => setup(label))
   }
   createWidget
}

This is exactly the same as the first example but more verbose and explicit. The double parameter-lists is syntactic sugar in Scala to return a function. Once you get used to it’s easier to read IMO so I have adopted it. It should be pretty clear now that the evaluation will occur in two steps:

1. provide a list of setups (functions that takes a label as their only argument). This step will return a function that takes a composite (the createWidget function).
2. Interestingly enough the returned function is “charged” with the setups and they will be invoked in the setups.foreach part when the composite, parent is provided. Not before, not after.

Now to the problem I wanted to solve. There is obvious duplication between the two functions in the first code-listing. The most obvious duplication (as I saw it) was the setup part. Lets extract a function for setting up the label:

    def label(setups:(Label => Unit)*)(parent:Composite) = {
        val label = new Label(parent, SWT.NONE)
        performSetup(label, setups)
    }
    def performSetup(target:Label, setups:Seq[Label =>; Unit]) = { 
        setups.foreach(setup => setup(target)) 
    }

This step really did not gain us that much. One thing to notice is that we pass the variable argumet setups into a parameter of type Seq. So variable arguments are just sequences. There is not an awful lot less code and we can’t use the performSetup method for progress-bars. Lets fix that. If you look at the performSetup-function there is not anything specific to labels in that function. So what type should it be? It’s really not important as long as it’s the same type that the functions in setups take. You won’t get a more obvious case for generics, and a pretty basic flavour also. Lets change the label to a generic type:

    def performSetup[T](target:T, setups:Seq[T => Unit]) = { 
      setups.foreach(setup => setup(target)) 
    }

That was easy, now we can use the performSetup method in both the label and progressBar functions:

    def label(setups:(Label => Unit)*)(parent:Composite) = {
        val label = new Label(parent, SWT.NONE)
        performSetup(label, setups)
    }
    def progressBar(setups:(ProgressBar => Unit)*)(parent:Composite) = {
        val progress = new ProgressBar(parent, SWT.NONE)
        performSetup(progress, setups)
    }
    def performSetup[T](target:T, setups:Seq[T => Unit]) = { 
        setups.foreach(setup => setup(target)) 
    }
 

Ok, slightly better than what we started with but we still have obvious duplication. The only thing that really differ between the two functions (besides the names)is the creation of different widgets. If we could parameterize the creation of widgets we would have a reusable template. Lets try by copying and changing the label function. I will rename it to createWidget and then try to remove the label-specifics:

    def createWidget(setups:(Label =>; Unit)*)(parent:Composite) = {
        val label = new Label(parent, SWT.NONE)
        performSetup(label, setups)
    }
    def performSetup[T](target:T, setups:Seq[T => Unit]) = { setups.foreach(setup => setup(target)) }

So, we obviously have to remove the creation of the label. As a first step lets make a factory-function and insert it in the parameterList:

    def createWidget(factory:Composite =>Label)(setups:(Label => Unit)*)(parent:Composite) = {
        val label = factory(parent)
        performSetup(label, setups)
    }
    def performSetup[T](target:T, setups:Seq[T => Unit]) = { setups.foreach(setup => setup(target)) }

By inserting an extra set of parameters we have created:

1. A function that takes a factory that creates lables that…
2. returns a function that takes setups that… (note that at this point we have an equivalent function to our old label-function)
3. returns a function that takes a composite…

We can now update our label function to step use 2 if we just have a factory. Time to bring out a new trick from the bag of tricks:

    def label = createWidget(new Label(_, SWT.NONE))_
    def createWidget(factory:Composite=>Label)(setups:(Label => Unit)*)(parent:Composite) = {
 val label = factory(parent)
 performSetup(label, setups)
 }
    def performSetup[T](target:T, setups:Seq[T => Unit]) = { setups.foreach(setup => setup(target)) }
 

Ehhhh, what happened? Well, at first the underscores in Scala are a bit confusing and above we use them in two places.
First there is an underscore in the Label’s constructor where we used to pass in parent. SWT.NONE is still where it used to be. What we have done is to leave the parent-parameter to be filled in later. If you think of it, a constructor is in essence a function that returns a new instance of some class. In the Label-case: (Composite, Int) => Label.

Pre fill the params part: new Label(_, SWT.NONE) and we have a (Composite) => Label which is exactly what we need for the first argument of our createWidget function.

The second _ is a placeholder for “the rest”, which is (setups:(Label => Unit)*)(parent:Composite).
That parameter-list should look familiar from our first version of the label-function.
Now we want to use our createWidget function for our progressBar function aswell. Lets make it generic, we don’t want to create only labels:

def createWidget[T](factory:Composite => T)(setups:T => Unit*)(parent:Composite) = {performSetup(factory(parent), setups)}

I took the liberty to also inline the val label – statement. Sweet, lets use it for progressbars as well:

    def label = createWidget(new Label(_, SWT.NONE))_
    def progressBar = createWidget(new ProgressBar(_, SWT.NONE))_
    def createWidget(factory:Composite=>Label)(setups:(Label => Unit)*)(parent:Composite) = {
 val label = factory(parent)
 performSetup(label, setups)
 }
    def performSetup[T](target:T, setups:Seq[T => Unit]) = { setups.foreach(setup => setup(target)) }
 

Not too bad. Since we have other traits with widgets than just Labels we want to be able to reuse this pattern there as well. For that purpose we move the functions to a new object, Framework

object Framework {
	def createWidget[T](factory:Composite => T)(setups:T => Unit*)(parent:Composite) = {
		performSetup(factory(parent), setups)
	}
	def performSetup[T](target:T, setups:Seq[T => Unit]) = {
		setups.foreach(setup => setup(target))
	}
}

That leaves us with the final result for our Labels trait looking like this:

import Framework._  
trait Labels {     
   def label = createWidget(new Label(_, SWT.NONE))_     
   def progressBar = createWidget(new ProgressBar(_, SWT.NONE))_ 
}

Not too bad, definatly better than what I started with anyway :)

It must be possible to do in Scala!

I had already started playing with Scala and I thought it would be a cool experiment to see if I could do something similar in Scala. After all, scala is a pretty expressive language.

I made the first attempt a few months ago and I think that qualifies as a failure. I got something to work but all the mutable state and redundant classes I had to create bothered me big time.

I showed what I did to colleague and I got some pointers on how to make my code more function-oriented and hence get rid of some redundancy and mutable state. But my lacking knowledge of Scala and FP prevented me from finishing the task back then.

Success!

Yesterday I gave it another shot and it took me only an evening to make this code:

        shell(
            text("User Profile"),
            composite (
                gridLayout(2, false),
                group (
                    text("Name"),
                    gridLayout(2, false),
                    gridData(FILL, FILL, true, true),
                    label(text("First")), edit(text("Bullet")),
                    label(text("Last")), edit(text("Tooth"))
                ),
                group (
                    gridData(FILL, FILL, true, true),
                    text("Gender"),
                    radioButton (text ("Male"), selected),
                    radioButton (text ("Female"))
                ),
                group (
                    gridData(FILL, FILL, true, true),
                    text("Role"),
                    checkBox (text("Student"), selected),
                    checkBox (text("Employee"), selected)
                ),
                group (
                    text("Experience"),
                    rowLayout,
                    gridData(FILL, FILL, true, true),
                    spinner(selection(5)), label {text ("years")}
                ),
                button (
                    text("save"),
                    gridData(RIGHT, CENTER, true, true)
                ),
                button (
                    text("close"),
                    gridData(LEFT, CENTER, true, true)
                )
           )
        )

Display this:

Pretty cool! I will admit that I have gracefully stolen most of the ideas from Glimmer. Even the example above is a slightly modified Glimmer-example. I will probably go over it a couple of times to see what I can improve from a Scala perspective. I particulary dislike the gridData functions. I think there is room for improvement there.

How does it work?

The general pattern is simple. Take a look a the group element in the code above. Group is implemented like this:

def group(setups:(Group => Unit)*)(parent:Composite) = {
    val group = new Group(parent, SWT.NONE)
    rowLayout(group);
    setups.foreach(setup => setup(group))
}

If you, like me when I saw my first Scala examples, have some dificulty with the double parameterlist notation it is a shorthand in Scala for returning a function. So the above is a function that takes a variable number of functions that take a Group as the only parameter. Unit in Scala is equivalent to void in java. The group-function will return a function that takes a Composite, parent, as it’s only parameter.

The cool thing about this is that you can pass the first set of parameters (the variable number of functions that takes a Group and return Unit) and save the returned function for later evaluation. When the composite parent is later passed what will happen is the following:

• A new Group widget will be created with parent as parent
• as a default the layout of the group will be set to row layout, this may be overridden in the next step.
• Each function in setup will be applied to the newly created group.

Remember, the setup-functions take a group as a parameter so they can perform whatever action on the Group. You have full freedom to do whatever can be done with a group in any of those functions.

The variable length argument is one of the missing pieces I needed to solve the puzzle, I failed to find information about variable arguments via google. Ok, I did not spend an awful lot of time trying…

What does one of those setup-methods look like then?

Let’s start with something simple, the label:

 def label(setups:(Label => Unit)*)(parent:Composite) = {
    val label = new Label(parent, SWT.NONE)
    setups.foreach(setup => setup(label))
}

The pattern repeats itself, lazy creation of the label onto the parent composite (for instance a Group) and the posibility to furter customize the label. Now comes the next secret I had to learn in order to solve the puzzle. Take a look at the text-function:

def text[T <: {def setText(txt:String)}](txt:String)(subject:T) = {
    subject.setText(txt)
}

Pretty straight forward right? Look again at the generic type T. I don’t know what that construct is called, I call it declarative duck-typing, you know if it walks like a duck and quacks like a duck it’s a duck!

Here I declare that T can be any type that has a function named setText that takes a String as an argument and returns nothing. If I did not declare this then the subject.setText(txt) line would not compile. But I’m getting ahead of myself. Why did I need to find this construct?

Several different widgets in SWT has a setText-method, for instance, shell has a setText, label has a setText etc but some controls do not. There is no common interface between all different widgets that has a setText-method. That makes things a bit tricky, my first attempt was to overload different setText-functions, one for label, one for shell etc… This was not only tedious, it didn’t work since the difference between all of those overloaded methods is in the parameterlist of the returned function. Luckily, this little inconvenience forced me to discover this NEAT feature of the Scala-type system. I’m happy I did.

In conclusion

I was able to make a DSL that is very comparable to glimmer which was my original goal. I was able to make the whole thing initialize lazily. Basically I just build a tree of constructing functions and then I sort of light the fuse by passing a shell to the root of that tree and the GUI is created. I did not have to introduce any mutable state other than what is already mutable in SWT. I have a feeling that this approach is very extendable. Maybe the GUI is a good place to introduce Scala in Java-projects? My feeling is that Scala would slowly eat it’s way in to other parts of the system from there :)

Perhaps this is a good candidate for an open source project? Would you want to use it?

  def importResource(name:String, resource:Resource):Unit = {
    log.debug("importing " + name + " into " + root)
    val path = pathOf(name)
    createResource(path.toList)(resource)
  }

  private def createResource(nodes:List[String])(resource:Resource) =  {
    val directory = nodes.dropRight(1).foldLeft(root)((directory, name) => {
      if(name.equals("")) {
        directory
      }
      else {
        val directoryOption:Option[AbstractDirectory] = directory.getDirectory(name)
        directoryOption.getOrElse({
          val subDir = directory.createDirectory(name) match {
                         case result:AbstractDirectory => result
                       }
          directory.add(subDir)
          subDir
        })
      }
    })
    directory.createIfNewer(nodes.last, _:Resource)
  }

If it’s not crystal clear to you what the code does. Here is an overview:

• The first method takes a name and a resource
• The name is split up into it’s path-elements
• The path is passed to the createResource method that will create all directories that does not yet exist on the way to the resource and finally return a function that takes a resource as input and creates it in the already given directory.

A big issue I have with the code is the createResource-method. It simply is very hard to name it. createAllDirectoriesAndReturnResourceCreatingMethod would illustrate better how smelly that method really is. I have an idea on how to refactor this with scalas pattern matching. This is the new version (after some red/green bar cycles):

  def importResource(name:String, resource:Resource):Unit = {
    log.debug("importing " + name + " into " + root)
    val path = pathOf(name)
    createResource(path.toList, root, resource)
  }

  private def createResource(nodes:List[String],
                             directory:AbstractDirectory,
                             resource:Resource) {
    nodes match {
      case head :: Nil => directory.createIfNewer(head, resource)
      case "" :: tail => createResource(tail, directory, resource)
      case head :: tail => {
        val subDir = directory.getDirectory(head).getOrElse {
          val result = directory.createDirectory(head) match {
                                   case dir:AbstractDirectory => dir
                                 }
          directory.add(result)
          result}
        createResource(tail, subDir, resource)
      }
      case nil =>
    }
  }

The first thing that strikes me about the second version is the unbalance between the different cases. The head :: tail case is not exactly a one-liner… but it should be. This smells like feature envy. We could ask the directory to getOrCreateDirectory and it would read much better. But that is another refactoring. First let’s go through this one:
One thing I did not first get in scala is how you can iterate through lists using pattern-matching. In order to get how that works it is important to realise one thing about scalas lists:

List(“a”, “b”, “c”, “d”)

is equivalent to

“a” ::(“b” ::(“c” ::(“d” ::(Nil))))

which due to the right associativityness of :: is equivalent to:

“a” :: “b” :: “c” :: “d” :: Nil

Or plain english: Lists are not flat structures, lists are recursive structures. What that means in practical terms is that given any element in the list it is very easy to split the list at that element into head (the current element) and tail (the rest of the elements.

For instance, given the element “b” above, head will be “b” and tail will be “c” :: “d” :: Nil.

Ok, not that hard. Now to the cool part: The :: operator is a case class which in short means that it can be used in pattern matching:

“a” :: “b” :: “c” :: Nil match { case h :: t => println(head +” -> ” +tail); …}

will assign “a” to the variable h and “b” :: “c” :: nil to the variable t

We can now understand the cases above. In pseudocode:

  private def createResource(nodes:List[String],
                             directory:AbstractDirectory,
                             resource:Resource) {
    nodes match {
      case head :: Nil =>  //the last element of the list, create the resource using head as a name
      case ""   :: tail => // special case, an empty directory name. Skip to the next name:
                           // createResource(tail, directory, resource)
      case head :: tail => //head will be a directory-name that we use to get or create the 
                           //nextDirectory that is used with tail to call ourselves recursively:
                           // createResource(tail, nextDirectory, resource)
      case nil => //no more elements to traverse
    }
  }

I’m far from done with the refactoring. I want to get rid of the special case of “” to begin with and as mentioned above remove some feature-envy but I think that the cases are more readable than the original code. At least in terms of where the problems lie in terms of special cases and bloated cases. Of course, it’s just an opinion and I reserve the right to change my mind tomorrow :)