Nearly a year ago to the day, my freind and coleague Brian O’Neill blogged about building storm on OSX. I had been through that pain two years ago, and largely forgot about it (once you get 0mq and JZMQ installed you’re largely in the clear). That is until today, when I had to set up a storm development environment on a new laptop…

Things have changed since then. Apple has released OSX 10.9 (Mavericks) and turned over development of the OSX JDK to Oracle, adding a little more salt to the wound.

Hopefully this will spare a few others from that pain.

JDK 6/7

On a fresh install of OSX 10.9 (no JDK installed), when you java -version from the command line you will get a prompt to download Java 7 from Oracle. But what if you need JDK 6?

Some Java applications can trigger the install of JDK 6 via software update. In my case it was Intellij IDEA. But one utility that Apple added to OSX was the /usr/libexec/java_home executable, which will output a path suitable for use as JAVA_HOME value. Without arguments, it will output the path for the default JDK:

1
2
3
4
5
6
7

$ export JAVA_HOME=$(/usr/libexec/java_home)
$ echo $JAVA_HOME
/Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home
$ java -version
java version "1.7.0_45"
Java(TM) SE Runtime Environment (build 1.7.0_45-b18)
Java HotSpot(TM) 64-Bit Server VM (build 24.45-b08, mixed mode)

You can also specify a certain version (running this should trigger an install of JDK 6 via software update – I’m not sure because Intellij had already triggered it in my situation):

1
2
3
4
5
6
7

$ export JAVA_HOME=$(/usr/libexec/java_home -v1.6)
$ echo $JAVA_HOME
/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home
$ java -version
java version "1.6.0_65"
Java(TM) SE Runtime Environment (build 1.6.0_65-b14-462-11M4609)
Java HotSpot(TM) 64-Bit Server VM (build 20.65-b04-462, mixed mode)

So now we have a way to easily switch between JDK 6 and 7.

If you need to download Java 6 manually, there is a .dmg file available from Apple.

Install Xcode and Command Line Tools

Xcode is easy to install via the App Store. You will also need the command line tools for compiling various things. You can trigger the install of the command line tools by invoking one of the stubs of the tools included, e.g.:

1

/usr/bin/strings

Install Homebrew

Homebrew is like a package manager for OSX that handles download, compilation, and installation of various tools and libraries. We will use it later to install dependencies.

1
2

ruby -e "$(curl -fsSL https://raw.github.com/mxcl/homebrew/go/install)"
brew doctor

If the brew doctor command completes without error, you’re ready to move on.

Install 0mq

The Storm documentation recommends using 0mq version 2.1.7, however I’ve found that 2.1.7 will not build on OS X 10.9 (Mavericks). The closest version I’ve found that will build under Mavericks is 2.1.9.

The new/preferred method is to use brew tap homebrew/versions, however that repository does not have 0mq 2.1.9. The fallback method is to use the brew versions command to find and checkout a specific version:

1
2
3
4

brew versions zeromq
cd /usr/local
git checkout 381c97f Library/Formula/zeromq.rb
brew install zeromq

Install JZMQ (Java bindings for 0mq)

Compiling and instlling JZMQ is where most of pain comes in, and most of the errors that come out of the build process are cryptic.

Follow the steps below exactly and you should be spared that pain:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

git clone https://github.com/nathanmarz/jzmq.git

brew install pkg-config
brew install automake
brew install libtool

export JAVA_HOME=$(/usr/libexec/java_home -v1.6)
sudo ln -s /System/Library/Frameworks/JavaVM.framework/Versions/Current/Headers/ /Library/Java/Home/include

cd jzmq
./autogen.sh
./configure
touch src/classdist_noinst.stamp
cd src
javac -d . org/zeromq/*.java
cd ../
make
sudo make install

The pkg-config, automake, and libtool are dependencies of the JZMQ build process. After installing those, we switch to JDK 6. Next we symlink the JDK 6 header directory where the JZMQ build process will be able to find it. Without it you will get an error like:

1

cannot find jni_md.h in /Library/Java/Home/include

The touch src/classdist_noinst.stamp command prevents the following error:

1

No rule to make target `classdist_noinst.stamp'

Next, we manually compile the JZMQ java sources. Without that step, you will get the following error:

1
2
3
4
5
6

error: cannot access org.zeromq.ZMQ
class file for org.zeromq.ZMQ not found
javadoc: error - Class org.zeromq.ZMQ not found.
Error: No classes were specified on the command line.  Try -help.
make[1]: *** [org_zeromq_ZMQ.h] Error 15
make: *** [all-recursive] Error 1

Finally, we compile and install JZMQ.

Install Leiningen

Leiningen is the build tool used by storm. It is similar to Maven and Gradle, but much more suited to clojure development.

Installing Leiningen is just a matter of downloading the script and making it executable:

1
2

curl https://raw.github.com/technomancy/leiningen/stable/bin/lein -o ~/bin/lein
chmod +x ~/bin/lein

The above commands assume you have a ~/bin directory and it is added to your PATH. The easiest way to do that is in a .bash_profile file:

1
2

mkdir ~/bin
echo PATH=\$PATH:~/bin >> ~/.bash_profile

Build Storm

Now we’re finally ready to build storm:

1
2
3

git clone https://github.com/nathanmarz/storm.git
cd storm
lein sub install

This will build and install the storm jars so they can be used as dependencies in other Leiningen/Maven/Gradle/etc. projects. If you want to create a distribution ZIP archive, run the following script:

1

sh ./bin/build_release.sh

Hopefully this will save some future storm contributors some time Googling cryptic errors.