dd-bd-storm-2014

A Clojure storm example cooked up from the following projects.

storm-starter

rolling-topology

For the Feb/14 Big-Data meetup in Dresden:

DD-BD-Meetup

Also thx to Michael G. Noll.

Abstract:

This example should compute rolling trending topics via storm. For presentation purpose i added a Redis Spout and the resulting ranks will be indexed to elasticsearch thx to this lib storm-elastic-search.

In combination with this storm-vagrant setup, it's very easy to get some play time with storm

Usage

Build steps:

lein deps

lein compile

Run steps:

You can run the example local with:

lein run

to run the example against a storm cluster you want to set up your storm client and stuff, find help in the storm starter project mentioned above.

If you pass a name to the main function call the topology will run not in dev mode.

lein run -m storm.rolling-topology my_test_run

Some configs you might want to consider:

src/clj/storm/rolling_topology.clj

is the place to make some elasticsearch settings, which is

      "elastic.search.cluster" "your cluster name"
      "elastic.search.host" "your host"
      "elastic.search.port" 9300

and the redis list to pop items from is set on top of the file,

(def REDIS_LIST "mylist")

also in,

src/clj/utils/redis.clj

you can set the

 :host "127.0.0.1" :port 6379 

to your redis instance. On the $redis-cli i manually added token sequeces like this.

127.0.0.1:6379> rpush mylist "An awesome sentence with token."

Words of warning

All stuff is intended for testing purposes. Don't use this .-) .

Sojoner Version:

• v0.0.1

License

Copyright © 2014 sojoner

Distributed under the Eclipse Public License either version 1.0 or (at your option) any later version.