Saturday, April 17, 2010

Starting with Akka and Scala

Update to this post: I have upgraded the examples to Akka 1.0-RC1. this newer post might be more helpful.

I recently started playing around with Scala. I had read a lot about Scala before but never got around really doing anything with it. I was looking for some (probably Java) libraries that would make a couple of things really easy:
  • Easy concurrency
  • Interop with Java or in Java
  • Distributed computing nodes
  • Event driven
  • Highly scalable
  • Low bandwidth usage for communicating between nodes (so no XML)
  • Pluggable data format
  • Easy development of message exchanges, asynchronous, one way, two way, synchronous, etc
  • Possible (but hopefully not necessary) distributed state/database
  • Reliable when failures happen
  • Easy integration with a range of communication protocols / endpoints
At first I found BSON as a data format, as part of the mongodb project. Quick after that I found Google Protobuf which was a better candidate for me. That lead me to JBoss Netty, which supports protobuf out of the box. Netty was a bit too low level for my needs this time and by chance I saw a tweet come by from Dean Wampler, 'Akka 0.7 released'. Didn't know what it was (Akka is apparently a mountain in Sweden) checked it out, and to my surprise Akka uses JBoss Netty and protobuf for Remote Actors! If you look at the many (pluggable) features of Akka it's easy to get enthusiastic about it very quickly. I read through the source code, which (as far as I can tell at this moment of course) looks good. Plenty of tests, well structured, easy to read. There are some smart people behind this framework with the right know-how, which is very important as well.

Akka does have a Java API, but once you see the Scala version, I don't think you will want to use it. That's got nothing to do with the quality of the Java API by the way, more the succinct way of expressing code and the amount of typing that you will save yourself from :). Well, for some years now I follow the rule to learn at least one new programming language every year, so this year I'm adding Scala to the list.

So I started with an Akka project, did some playing around with Eclipse, Intellij, maven, sbt, Scala of course, Apache Camel integration. I used the akka-template-rest template first to get started quickly. I quickly found out that the Scala Eclipse plugin is not really ready yet, so now I'm using Intellij IDEA. in the next post I'll talk about the use of an IDE.

Disclaimer: Since I'm not a Scala expert (yet :) and I don't always work on linux I just got things to work to get the job done, if you have any tips for improvement let me know. I'm using Ubuntu 9.10 Karmic Koala.

Install sbt (0.7.3):
Follow the setup here.

sbt will install scala inside the project that we will create. Create a new project by running sbt in a newly created project:



$ mkdir akka_start
$ cd akka_start
$ sbt
Project does not exist, create new project? (y/N/s) y
Name: akka_start
Organization: agilesquad
Version [1.0]: 
Scala version [2.7.7]: 2.8.0.Beta1 
sbt version [0.7.3]: 


It's very important to use 2.8.0.Beta1, the RC1 still has some issues. sbt itself is running on Scala 2.7.7.

I find sbt a very nice tool so far, it is very similar to maven, can use maven dependencies, but you write your build file in Scala in an internal DSL, which is a lot nicer than XML.

Lets write a simple project file:



$ cd ~/akka_start
$ mkdir project/build
$ gedit project/build/Project.scala


Project.scala:


import sbt._

class Project(info: ProjectInfo) extends DefaultWebProject(info){
 override def repositories = Set(
  "Java.Net" at "http://download.java.net/maven/2",
  "jBoss" at "http://repository.jboss.org/maven2",
  "service mix" at "http://svn.apache.org/repos/asf/servicemix/m2-repo/",
  "Apache Camel" at "https://repository.apache.org/content/repositories/releases/",
  "Akka Maven Repository" at "http://scalablesolutions.se/akka/repository",
  "Multiverse Releases" at "http://multiverse.googlecode.com/svn/maven-repository/releases/",
  "GuiceyFruit" at "http://guiceyfruit.googlecode.com/svn/repo/releases/",
  "DataBinder" at "http://databinder.net/repo",
  "Configgy" at "http://www.lag.net/repo",
  ScalaToolsSnapshots)

 override def libraryDependencies = Set(

  /* servlet implementation */
  "org.eclipse.jetty"  % "jetty-server"   % "7.0.1.v20091125" % "test",
  "org.eclipse.jetty"  % "jetty-webapp"   % "7.0.1.v20091125" % "test",
    "org.scalatest" % "scalatest" % "1.0.1-for-scala-2.8.0.Beta1-with-test-interfaces-0.3-SNAPSHOT" % "test->default",
  /* camel */
  "org.apache.camel" % "camel-ftp" % "2.2.0" % "compile",
  /* akka dependencies */
  "se.scalablesolutions.akka" % "akka-kernel_2.8.0.Beta1"  % "0.8.1" % "compile",
  "se.scalablesolutions.akka" % "akka-core_2.8.0.Beta1"    % "0.8.1" % "compile",
  "se.scalablesolutions.akka" % "akka-servlet_2.8.0.Beta1" % "0.8.1" % "compile",
  "se.scalablesolutions.akka" % "akka-rest_2.8.0.Beta1"    % "0.8.1" % "compile")
 override def jettyPort = 9012
  val junit = "junit" % "junit" % "4.5" % "test"
}


I am not using any distributed database yet so I removed those dependencies (redis etc). I have already added some dependencies that I will use in the next post (camel-ftp2, junit, scalatest). As you can see I am using akka 0.8.1 which is dependent on Scala 2.8.0.Beta1 at the moment.

update the project dependencies:



$ cd ~/akka_start
$ sbt update


You will now find a lib_managed directory in your project directory.
Add a webapp:



$ cd ~/akka_start/src/main
$ mkdir webapp
$ mkdir webapp/WEB-INF
$ cd webapp/WEB-INF
$ gedit web.xml


web.xml:



<?xml version="1.0"?>

<!DOCTYPE web-app PUBLIC "-//Sun Microsystems, Inc.//DTD Web Application 2.3//EN"
 "http://java.sun.com/dtd/web-app_2_4.dtd">

<web-app>

 <listener>
  <listener-class>se.scalablesolutions.akka.servlet.Initializer</listener-class>
 </listener>

 <servlet>
  <servlet-name>AkkaServlet</servlet-name>
  <servlet-class>se.scalablesolutions.akka.rest.AkkaServlet</servlet-class>
 </servlet>

 <servlet-mapping>
  <servlet-name>AkkaServlet</servlet-name>
  <url-pattern>/*</url-pattern>
 </servlet-mapping>

</web-app>

Check if a WAR is being created:



$ cd ~/akka_start
$ sbt package


Add a akka.conf to the resources:


$ cd ~/akka_start/src/main/resources
$ gedit akka.conf


akka.conf:


####################
# Akka Config File #
####################

# This file has all the default settings, so all these could be removed with no visible effect.
# Modify as needed.

<log>
  filename = "./logs/akka.log"
  roll = "daily"  # Options: never, hourly, daily, sunday/monday/...
  level = "info" # Options: fatal, critical, error, warning, info, debug, trace
  console = on
  # syslog_host = ""
  # syslog_server_name = ""
</log>

<akka>
  version = "0.8.1"

  # FQN to the class doing initial active object/actor
  # supervisor bootstrap, should be defined in default constructor
  boot = ["com.agilesquad.Boot"]

  <actor>
    timeout = 5000              # default timeout for future based invocations
    serialize-messages = off    # does a deep clone of (non-primitive) messages to ensure immutability
  </actor>

  <stm>
    service = on
    fair = on                     # should transactions be fair or non-fair (non fair yield better performance)
    max-nr-of-retries = 1000      # max nr of retries of a failing transaction before giving up
    timeout = 10000               # transaction timeout; if transaction has not committed within the timeout then it is aborted
    distributed = off             # not implemented yet
  </stm>

  <remote>
    compression-scheme = "zlib" # Options: "zlib" (lzf to come), leave out for no compression
    zlib-compression-level = 6  # Options: 0-9 (1 being fastest and 9 being the most compressed), default is 6

    <cluster>
      service = on
      name = "default"                                                        # The name of the cluster
      actor = "se.scalablesolutions.akka.cluster.jgroups.JGroupsClusterActor" # FQN of an implementation of ClusterActor
      serializer = "se.scalablesolutions.akka.serialization.Serializer$Java$" # FQN of the serializer class
    </cluster>

    <server>
      service = on
      hostname = "localhost"
      port = 9999
      connection-timeout = 1000 # in millis (1 sec default)
    <server>

    <client>
      reconnect-delay = 5000    # in millis (5 sec default)
      read-timeout = 10000      # in millis (10 sec default)
    <client>
  </remote>
</akka>

In the akka.conf you need to specify your boot class, in this case com.agilesquad.Boot
(boot = ["com.agilesquad.Boot"])

Now write a very simple Actor to test if it runs (inside jetty). I have chosen to write an Actor that integrates with an Apache Camel file endpoint.



$ cd src/main/scala/
$ gedit Boot.scala 


Boot.scala:


package com.agilesquad

import se.scalablesolutions.akka.actor.{Actor, SupervisorFactory}
import se.scalablesolutions.akka.camel.{Message, Consumer}
import se.scalablesolutions.akka.config.ScalaConfig._

class Boot {
   val factory = SupervisorFactory(
  SupervisorConfig(
   RestartStrategy(OneForOne, 3, 100, List(classOf[Exception])),
   Supervise(new CamelConsumer(), LifeCycle(Permanent)) :: Nil))

  factory.newInstance.start
}

class CamelConsumer extends Actor with Consumer {
  
  def endpointUri = "file:///home/rroestenburg/input"
  def receive = {
    case msg:Message => {
      log.info("received camel message...")
    }
  }
}

For simplicity I have just added all the classes in the Boot.scala file.

Now run sbt without arguments, execute compile, execute jetty-run



$ sbt 
$ compile
$ jetty-run


You should now have a running akka server! as shown above the endpointUri is pointing to an input directory of your choice (in  my case /home/rroestenburg/input), copy a file in there and check if the message ends up in the logging ("received camel message).

That's it for now, in the next post I'll get into the use of Intellij IDEA Community Edition with the latest Scala Plugin for 2.8.0.Beta1, as well as some Test Driven Development with Scala and Akka, and how to use some extra Camel components.