The point is missed

Museum of public mistakes and unfinished projects

Phasing over from java to scala, retrospect

with 10 comments

I have written quite a few posts, 4 to be exact, about my experiences with phasing over from java to scala in my pet-project. I thought it is about time to take a look in the rear-view mirror now when some time has passed and I am more comfortable with Scala.

Setting the stage

I hope that my experiences can help someone else to actually migrate from java to scala. I hope to be able to point at a few things that were difficult to me, what made me understand it and what I still don’t get. I also want to review whether making the switch was actually worth it, which value does scala add compared to java?

Gathering data

In order to have some foundation to draw conclusions from I brainstormed with myself:

Hard stuff

Some things seemed hard at first and I had to study them or they dawned on me later

  • Reading Scala doc
  • Academic examples
  • Shortcut syntax for returning functions and _
  • Case classes
  • Seing how FP can be memory efficient
Things that don’t work well

There are some things that are bothering me to some degree

  • Specs
  • Tool support is lagging
  • I still write “java in scala”. I need to push myself a bit further
Good stuff

Some things I learned to appreciate with Scala

  • Compact
  • Clearer than java
  • Clever constructs
  • Expressive (DSL etc)
  • Anonymous functions
  • Implicits
  • Everything returns something
  • Lazy fields
  • Constructor properties(!!!)
  • Supports evolution well

dynamicness

One thing that I was really interested in when I started looking at Scala was how it compared to dynamic languages. I had the idea that a more competent type-system than javas dito would be better at staying out of my way than Java’s typessytem.

  • The type system is more in the way than helpful
  • Most things can be accomplished if you learn the syntax
  • performance over simplicity

I could probably do this for longer and come up with more stuff. A team would have been good for this exercise :)

Generate insights

I found the Scala doc very hard to understand at first. To use the Agile Manifesto format the scala-doc seems to value correct definitions over descriptive examples. Compactness seems to be valued so it is not uncommon to find examples that use one letter variable-names, it all seems very academical which seems a bit unnecessary to me. But I guess some people prefer that so I think I can get used to the more scientific approach in describing the language than I’m used to. In retrospect there are a few things that I found extra hard and once I got them it started to become easier

Reading funtion syntax

x: => Int

x: (Int, String) => Long

At first I read these as “A function that returns an Int” and “A function that takes an Int and a String and returns a Long”. Somehow it started to read easier to me when I started to read it as Something that transforms to an int and Something that turns an Int and a String into a Long. In short, instead of thinking of the => as a marker for a function I now think of <left> => <right> as turns left into right or yeilds right from left or converrts left to right.

Functions that returns functions

It is not that weird that functions can return functions:

def functionReturningAFunction(a:Int):Int => String = {

def sumToString(b:Int):String = {
“the sum is: ” +(a+b)
}

sumToString
}

invoking it with

println(functionReturningAFunction(12)(13))

prints:

the sum is 25

It is weird at first with the double parameter-lists (12)(13) but understandable. What confused me A LOT in the beginning is that there is also a shorter way to do the same thing:

def functionReturningAFunction(a:Int)(b:Int):String = {
“the sum is: ” +(a+b)
}

The two forms mean the exact same thing!!!

Hangman notation

Now throw in the _ into the picture:

def addFiveTo = funtionReturningAFunction(5)(_)

That can be called like this:

addFiveTo(12)

and returns

the sum is 17

It really starts to get confusing.

I have invented the term hangman notation inspired by the game hangman where you use _ to indicate a missing letters that need to be filled in. _ in scala is in this context a placeholder for something to be filled in later. This is the way to return a partly evaluated function in scala (in this case functionReturningAFunction with 5 already given).

Insight: Understanding the above three things made many more scala code examples understandable to me

Specs

Specs has been a bit of a disapointment to me. I find that it misses the mark as a BDD framework. I feel I have to be very explicit about what I want to do and quite often the syntax is redundant, unintuitive or inconsistent. I am considering dropping specs in favor of ScalaTest.

Tool support is lagging

I had some problems with Buildr and the eclipse-plugin is a bit buggy. The former seems to be worked on and I expect it to be fixed soon. The latter probably makes scala an option mainly for the innovators and early adopters while the early majority may be put off by having to close and reopen edit-windows and doing clean builds from time to time.

Scala goodness

Most of the time scala is a joy to code in. The typesystem is much less in your way than in java and the code becomes much clearer due to removal of type system noise and very clever solutions such as constructor properties. Anonymous functions really do add another dimension of possibilities that I don’t want to be without once you learned to appreciate them. Over all I find the switch worthwhile even in spite of lacking tool support. Partly scala makes up for that tools by simply being more managable and less verbose.

Insight: The only thing I regret about switching to scala in my pet-project is that I am even more painfully aware how much java sucks ;)… and java happens to be what I work with :S (most of the time)

Dynamicness

I can’t help but compare scala and ruby while the latter is dynamic. I found this post about how you can do most of the cool things you can do in ruby in scala: Scala, the statically typed dynamic language and it answered a lot of questions I had on the topic. In short, I think scala allows itself to be persuaded to do a lot of things. The downside is that you have to figure out how to make scala obey you. No matter how you look at it there is an extra dimension of static typing that needs to be learned and added to the code. In a dynamic language like ruby, you don’t have to learn that. For example with duck-typing, all you have to do is to understand that you can use it and… start using it. No magic rituals involving special syntax, possibly chanting and some goat-blood. Just do what makes sense and Ruby assumes you know what you are doing. If not you will find out runtime (and that seems to scare some people to death).

Speaking of runtime, and this is not in relation to Scala, the statically typed dynamic language but with posts I read here and there. There is a lot of runtime language bashing going on, that dynamically typed languages are lesser languages and that high quality systems can’t be written in the due to lack of safetynets in the form of static typing. To me that is just ignorant and I immediately, perhaps wrongly, assume that the
person making that statement do not routinely test-drive his code. To me building enterprise systems in any language statically or dynamically typed without automatic verification is just stupid. So, my advise to you static typers out there, you may have found something I haven’t and if that is the case, make a reference, explain what you mean but general statements like static-typing = good, dynamic-typing = bad is not convincing. Be open-minded, if you did not have static typing, what would you do to make sure things worked and how would that be different?

Insight: To me scalas static typing adds performance, not safety. My automatic specifications handles safety just as well in a dynamic language as in a statically typed one. The dynamic language is easier to manage and satisfy and I don’t have to haggle with the compiler, I can just do. But the dynamic language is slower and I can change some elegance for performance to a certain degree, Scala does come out ahead here in my opinion.

Decide what to do

I need to let go of my safety and explore FP more thoroughly to be able to compare dynamic languages and scala more properly. I still have a lot to learn and I still allow the possibility that I will learn how to put more trust in the compiler (after all it is not the same as java that I now find annoying and in the way without adding much value) and gain some momentum from that.

I need to do more greenfield Scala and not just convert stuff. I have an idea to use scalas parser-library to rewrite a javaCC-generated parser for a simple rule-matching syntax in pure scala.

I will kick out specs and try scala-test instead.

I’ll continue to think about the best way to introduce scala in an assignment without scaring people :)

Close

I hope to write shorter posts in the future and more to the point. My phasing over posts have been very verbose in order to convey the feeling what migrating is really like. In the future I will try to write more in the form of nuggets which is a style that I personally like better both to read and write. Until then, convert your pet-project to scala, you might learn something ;o)

About these ads

Written by johlrogge

October 1, 2008 at 10:32 pm

Posted in scala

Tagged with , , ,

10 Responses

Subscribe to comments with RSS.

  1. If you have specific examples for Specs where “the syntax is redundant, unintuitive or inconsistent”, I am sure Eric would like to hear about them. He is usually very responsive.

    For example, in the past “in” clauses had to be nested in “should” clauses, but someone complained that this was burdensome for simple tests and Eric removed the limitation.

    P.S. Sorry if this is a duplicate comment, but there was a problem when I submitted it for the first time and it was unclear if it had been submitted for moderation or not.

    Ismael Juma

    October 2, 2008 at 2:52 am

  2. Ismael:
    Thanks for your comment (and it arrived only once. besides I rather have duplicate comments than no comments, it’s no biggy to remve one)

    Of course I will give an example since I made a statement it’s only fair. My main problem with specs is that it doesn’t resonate with me. I find it unintuitive and I have not really been blown away by it. My gut feeling is that I’m not too happy with it.

    But you and Eric (I’m sure he is a great guy) deserve some more concrete examples and I will try to elaborate. I realize that “syntax” was sloppily put of me, some of my remarks are related to other areas.

    Here are some of the things that I don’t care much for:

    the ‘in’ keyword, I don’t see the logic. From the specs 5 minute tutorial:

    “‘hello world’ has 11 characters” in {
    “hello world”.size must_== 11
    }

    To me ‘in’ does not make sense there. You could of course replace ‘in’ with ‘>>’ which is what I have been doing but… i don’t get in. I rspec it is easier since do is dictated by ruby, so no choice there but in scala specs ‘in’ is chosen for some reason I don’t get.

    I think the specs don’t read well in the equal case which is a really common one:

    “hi” must beEqual(“hupp”)

    a ‘to’ would help readabiliy:

    “hi” must beEqualTo(“hupp”)

    or

    “hi” must equal(“hupp”)

    To me readability is a big deal and overlooking such an obvious thing tells me that the framework maker does not share my opinion here which is fine but makes it less appealing to me.

    This leads me to inconsistency:

    Instead of:
    “hi” must beEqual(“hupp”)

    you can write:
    “hi” must_== (“hupp”)

    all of a sudden you have a different version of must while the form in general is value must matcher. These small things makes it slightly harder to grasp since there are exceptions to the general rules.

    Another example of inconsistency:

    “hi” must equalIgnoreCase(“hupp”)
    “hi” must beEqual(“hupp”)

    why “be” in the second case but not in the first?

    About redundancy:

    For every matcher there is a “not” version

    “hi” must notEqualIgnoreCase(“hupp”)

    (inconsistently there is not a notBeEqual)

    But I much prefer the

    “hi” must not(beEqual(“hupp”))

    form since it renders all the notXxxx methods redundant. Why specs supports both is beyond me. Specs is not very opinionated but seems to provide as many options as possible in the hope that there is something for everyone. To me it’s just confusing. My remove duplication bells ring all over the place :S)

    Often naming is something I have a problem with.

    What does:

    “myCar must be also null”

    mean?

    myCar must beAlsoNull

    Well: “a must beAlsoNull(b) is ok if a is null when b is null or a is not null when b is null”

    Que? That one is so weird that it must make perfect sense in some context that I’m not aware of. An example would help :S

    One feature I really liked at first glance was contexts but what’s up with the ->- operator? And I can never remember if it goes before or after “should”

    Also contexts can be a bit scary since in general you want to set up some fixture with the context like:

    var list = List[String]()

    “lists with one item ” ->-(beforeContext({list = List(“oneItem”)})) should {
    “have a size of one” >> {
    list.size must beEqual(1)
    }
    }

    Now when I find this part in the documentation:

    “By default, the execution mode of examples of a Specification is “on demand”. The examples are only executed when a Runner requests the number of failures or errors. This enables a runner to do a concurrent execution of examples.

    However, if you need to execute samples sequentially (as in a literate specification), you can use the method setSequential:

    object mySpec extends Specification {
    “The first functionality” should { setSequential

    “have example 1 execute first” in {…}
    // example 2 uses the results of example 1
    “have example 2 execute in second” in {…}
    }
    }”

    You start wondering what happens when you run a few tests like that concurrently. The list is a variable so it mutates and a spec is an object. Unlike JUnit you don’t get a fresh instance for each test so you have to make sure that you don’t have unpleasant side-effects due to concurrency but contexts seem to work against that, since you don’t store things on the context instace but on the singleton spec. I don’t want to have to think about those issues when I write specs, I want to focus on the actual problem I’m trying to solve.

    In a BDD framework I would expect the concept of contexts more similar to:

    var list =
    given(“a list with one item”, List(“one item”}) {
    when “an item is added” {list = list ::: List(“another item”) }
    then list.size must be(2)
    }

    pardon my poor scala but I hope my point comes through…

    Those are some reasons why I’m not happy with specs, of course YMMV, these are my highly subjective opinions.

    Did I manage to make any sense?
    /Joakim

    johlrogge

    October 2, 2008 at 7:34 pm

  3. Hi Joakim,

    I think there are some valid points there, even if the importance given to them will vary from person to person. I posted a message to specs-users:

    groups.google.com/group/specs-users/browse_thread/thread/cb409fdffc95911c

    Maybe some of the issues can be mitigated or fixed.

    Ismael

    Ismael Juma

    October 2, 2008 at 11:24 pm

  4. Hi Joakim,

    Great feedback, thanks!

    Following Ismael’s post on the list I have made a long answer on the list itself as other users may have their word on the subject too.

    Can you please follow-up the discussion over there?

    Thanks,

    Eric.

    Eric

    October 3, 2008 at 3:36 am

  5. I’d be happy to, I’ve joined the group, read your comments there:

    http://groups.google.com/group/specs-users/browse_thread/thread/cb409fdffc95911c

    I will get around to answer them as soon as I have time.
    /J

    johlrogge

    October 3, 2008 at 5:21 am

  6. Hi,
    This line:
    def addFiveTo = funtionReturningAFunction(5)(_)
    gives me an error, at least with 2.6.1. The syntax for the partially applied function is:
    def addFiveTo = funtionReturningAFunction(5)_
    I don’t see the _ exactly as a parameter placeholder. Note that you can also have:
    def gg(a:Int)(b:Int)(c:Int) = a+b+c
    and:
    def add2to = gg(2)_
    Whatever additional parameters are missing in gg here will be required when actually invoking add2to.

    I understand the whole idea of currying is being able to compose functions, in such a way that you can make your solutions more generic and reusable. Say your actual need is to add 2 to something:
    def add2to(x:Int) = 2 + x
    You can keep that signature but build a more generic solution:
    def add(a:Int)(b:Int) = a + b
    def add2to = add(2)_
    def add5to = add(5)_
    (Sorry for the silly example) Anyway here’s a blog post on the subject:

    http://joelneely.wordpress.com/2008/03/22/currying-without-lab-rats/

    Germán

    October 10, 2008 at 2:01 pm

  7. Germán

    That’s odd, I did run and compile the example with scala 1.72.RC1 and it works fine. The reason I wrote (_) was because I was not able to get this example:

    def sumInts = sum(x => x)_
    def sumSquares = sum(x => x * x)_

    def sum(f: Int => Double)(a: Int, b: Int): Double = {
    def iter(a:Int, result:Double):Double = {
    if (a > b) result
    else iter(a+1, f(a)+result)
    }
    iter(a, 0)
    }

    to compile (from scala by example). But now when I try it again it compiles just fine. I was probably tricked by the eclipse plugin. But I can also get it to compile if I change the curried functions to this:

    def sumInts = sum(x => x) (_,_)
    def sumSquares = sum(x => x * x)(_,_)

    And if I give the type I can even do this:

    def sumIntsFrom2 = sum(x => x) (2, _:Int)
    def sumIntsTo2 = sum(x => x) (_:Int, 2)

    Perhaps that is new in scala 2.7x ? It makes sense given the _+_, _*_ etc operators in List(1,2,3,4).foldLeft(0)(_+_) // 10
    List(1,2,3,4).foldLeft(1)(_*_) // 24

    With that logic I can also see how _ without the () can be a placeholder for the whole parameter list.

    I wish I found your blog about it earlier. Being one of those people who does not miss the lab-rats from the explanations :)

    Thanks for pointing out the a + b + c case. I fount that I can also make that one compile this way:

    def funtionReturningAFunction(a:Int)(b:Int)(c:Int):String = {
    “the sum is: ” +(a+b) * c
    }

    def addFiveTo = funtionReturningAFunction(5)(_:Int)(_:Int)

    Which allows me to do this:

    def addFiveTo = funtionReturningAFunction(5)(_:Int)(8)

    Interesting! Thank you for your comment it made me look again and pick up some things I missed and understand other things a bit better.

    johlrogge

    October 10, 2008 at 7:53 pm

  8. Heeeyyyy that was not *my* blog (I don’t have one), that post was one of the reference links I’ve collected. I’m still pretty much a scala newbie!
    Cheers

    Germán

    October 11, 2008 at 12:49 am

  9. Love the name “hangman notation”! Hope it sticks :D

    Gabriel C

    October 13, 2008 at 7:47 pm

  10. […] for a retrospect Possibly related posts: (automatically generated)Scala Nugget – Pattern matching and […]


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: