Ecstortive

I recently saw the announcement of Google Dart being part of Webkit as patch. Lots of people really started yelling. This post is just an opinion and a wish-list.

I would love to see dart as plug-in to browsers rather than a patch to Webkit!

If Google is really looking forward to have Dart as next language for webpages; I think its time to open the Pandora box and again start using the long forgotten language=”Dart” in script tags (other options may include language=”Lua”, language=”Coffescript”, language=”Brainfuck!” and who knows what). Remember when two giants were fighting we used to have “VBScript” in the language property? It’s long gone, but here is a chance for Microsoft again! Here is my wish list:

• Scripting languages that ship as extension to browsers, just like Flash today! This is will open up true competition between developers (Google vs. Microsoft vs. Mozilla vs. Opera?) truly promoting the nature of open platform, where you get choice of choosing your language. Just think about it; if Javascript also becomes an extension to browsers, you would be able to plug v8 to Firefox (Ya I sound crazy)!
• Script tags remain there with no changes, execution however depending upon the availability of language (plugin) mentioned in attribute:
  

  <script type=”text/dart” language=”Dart”>/* My Dart Code */</script>

• BAL, yes a Browser Abstraction Layer. Since all of the extensions would need a interface to interact with DOM, and various components in browser we would definitely need one!
• Since we can switch between languages, we can also get options for compiled scripts! Which will take out the overhead of compression, minification and at times obfuscation crap. We would be able to ship byte-codes over the wire which means lesser size (compiled .pyc files ennnh) and more efficient VMs!

I have not yet calculated the Pros and Cons completely but as a web developer this is my wish list. What do you think?

Yes I know you are really happy with your “persistent” Key Value store. But did anybody notice hstore that comes along Postgresql. I find Postgresql to be a really great RDBMS that has been ignored all the time. It even has some great publisher/subscriber system as well (or LISTEN/NOTIFY in terms of Postgresql) that a lot of people may have implement using Redis, RabbitMQ etc. For people who have not lived anything other than MySQL. I would simply ask them to try out Postgres.

Instead of looking at benchmarks, I will be focusing on a key value store that is ACID compliant for real! Postgres takes advantage of its storage engine and has an extension on top for key value storage. So plan is to have a table can have a column that has a datatype of hstore; which in turn has a structure free storage. Thinking of this model multiple analogies throw themselves in. It can be a Column Family Store just like Cassandra where row key can be PK of the table, and each column of hstore type in table can be imagined like a super column, and each key in the hstore entry can be a column name. Similarly you can imagine it some what like Hash structures in Redis (HSET, HDEL), or 2 or 3 level MongoDB store (few modifications required). Despite being similar (when little tricks are applied) to your NoSQL store structures, this gives me an opportunity to demonstrate you some really trivial examples.

Lets setup our system first. For my experiment I will be using Postgres 9.1 and I will compile it from source. Once in source directory you can: ./configure && make install to install your Postgres. Don’t forget to install the extensions in the contrib directory: cd ./contrib && make install. Once you have setup the database you can create your own database and start the server (Hints: use initdb and pg_ctl). Then launch your psql and make sure you install your hstore extension:

CREATE EXTENSION hstore;
SELECT 'foo=>bar'::hstore;

If everything goes well you should be able to see table output. Now we are ready to do some DDL. I created a table my_store as schema definition below:

CREATE TABLE my_store
(
  id character varying(1024) NOT NULL,
  doc hstore,
  CONSTRAINT my_store_pkey PRIMARY KEY (id)
)
WITH (
  OIDS=FALSE
);

CREATE INDEX my_store_doc_idx_gist
  ON my_store
  USING gist
  (doc);

As you can see I’ve created a table with hstore column type and one GiST index (for operators ? ?& ?| etc.). You can checkout of documentation to have a look on different type of operators you have.

Now that we have database and tables setup I wrote a simple script to populate it with about 115K rows from twitter stream. Now keep in mind that its a real life data and I was interested in querying few basic things from collected data. For example, how many people are putting hash tags, or doing mentions, or were posting links in the tweets? For doing this I wrote a simple python script using tweepy and psycopg2 and ran it for about few hours. For each tweet in my store I added a key value pair of ‘has_hashtags=>:t’ if there were any hash tags in the tweet, similarly I introduced has_urls and has_mentions if they were present in tweet, I will be using these keys along with my GiST index to query my table later on.

So after populating my data with 115,142 tweets the database grew to a size of 239691780 bytes (Just 228MB). Now comes the fun part. I was totally blown away by what I can achieve by combining the power of relational and key value style under 1 store. So for example I want to  query all the tweets tweeted at unix timestamp of 1323446095 (since I stored the timestamps as a string here is what my query looks like):

SELECT doc -> ‘text’ as tweet, doc -> ‘created_at’ as created_at
FROM my_store
WHERE doc @> ‘created_at=>00001323446095’;

I can add simple count or any other SQL famous aggregate function without going into any complications of my data store specific map reduce or new language to learn hustle. Do note that I padded my timestamp value with zeros since I am only storing strings as values. Also I am utilizing @> operator, thats gonna use the GiST to really do a quick bitmap index scan instead of sequential scan. That was pretty good for starter. Lets try to fetch out all the tweets that had hash tags in them:

SELECT doc -> ‘text’ as tweet, doc -> ‘created_at’ as created_at
FROM my_store
WHERE doc @> ‘has_hashtags=>:t’;

Yes querying complete database pulling out complete data (That you won’t probably do because you page the data :) ) gives me 14689 rows just under 360ms on average. Since we have SQL at hand lets make a condition little more complicated, and use a different operator for same stuff and also sort the data by created_at:

SELECT doc -> ‘text’ as tweet, doc -> ‘created_at’ as created_at
FROM my_store
WHERE doc @> ‘has_hashtags=>:t’ AND doc ? ‘has_urls’
ORDER BY doc -> ‘created_at’ DESC;

It already sounds tasty! This is not it Postgres has more operators, so pulling out hash tagged tweets with urls or mentions is also possible,

SELECT doc -> ‘text’ as tweet, doc -> ‘created_at’ as created_at
FROM my_store
WHERE doc @> ‘has_hashtags=>:t’ AND doc ?| ARRAY[‘has_urls’, ‘has_mentions’]

This is not it! hstore comes with all sort of operators and index systems that you can ask for hash store. Check them out here. Now, despite the NoSQL boom I think we have some great examples and reasons of why RDBMS still remains core part of many market giants (Facebook being something everyone knows). Postgres just gives me one more reason to not ignore RDBMS systems, So If you have been moving around on some document stores just because the reason that RDBMS don’t provide them; think again! You can get the same rock solid durability with structure free systems.

I will be pretty soon revisiting the FriendFeed use case with MySQL to store structure free data with Postgresql approach. Stay tuned, leave your comments and thoughts.

Today I will be doing the long awaited initial benchmarks of Oracle’s NoSQL. Before I jump in I would like to mention that I had a previous post on Oracle’s NoSQL that examines the philosophy and design of Oracle NoSQL store. It’s been few days I’ve been playing around with system. I found it pretty simple due to its major and minor key system; some great multi-get, iterator constructs for read, basic delete, and put system with versions. I did find some short comings, and we will see them in details at the end of this post. So lets get started.

For sake of benchmarking I am going to make things really simple. I will be benchmarking the insert and read speeds of a single node (1 machine no replication, no distribution) and look how a single node performs, based on that we can make rough estimates of performance gains by adding more machines.

Machine used for benchmarks is a pretty normal laptop machine with Dual Core 2.1GHz processor, 3 GB RAM, and commodity HDD. As an example I implemented a twitter like user stream of 100,000 tweets for 100 users randomly (that make approx 1000 tweets per user) each tweet with size of a little more than 100 bytes. So what we will be benchmarking is a speed test for inserting 100,000 tweets (approx 100+bytes) for inserting and reading them all. They should be pretty good experiments (close to a real world scenario) to give us an idea of what Oracle NoSQL can do.

Here is the piece of code that inserts and reads the 100,000 million entries and benchmarks the total time consumed. Compiling and running this I get output as following:

Write Time consumed 143156
Iteration Total time taken 223

Please note the time consumed in above benchmark is number of milliseconds. It turns out that inserting 100,000 entries in random order (since user ids generated are random) take 143156 ms and iterating over each entry of each user take 223 ms. I am pretty satisfied with read speed, and for write speed I found and average time of 1.4 ms (which is slower than reads but almost 700 inserts per second). Its important to note that each insert here is disk synchronized (with durability of Durability.SyncPolicy.SYNC on master, and since we have only 1 node means we are doing the most disk write with no buffering). If I lower the durability value (Durability.COMMIT_WRITE_NO_SYNC write with buffering), the average time drops to 0.5 ms; which is almost double the performance of previous version (or Durability.COMMIT_SYNC). It’s worth noting that you can also customize durability per transaction, which is great for letting a programmer choose what he wants.

I didn’t stop here and continued to implement a example twitter class, which proves the simplicity of what such this simple structure can do. I think the core power lies in not being a monolithic (JSON like) tree like object, and allowing us to pull data on partial paths as well as from full major key path. The class is used in this example code to create a user, authenticate, tweet, and query time-line of user.

Somebody may wonder how it compares with other data stores (document free, column based, key value). I can definitely achieve same effect by other key value stores (some what close as well), but I must say this style was much breeze and easy to imagine (just look at the code for reading tweets and you will know what I am saying), Oracle must rename store type from Key Value store to a new title. Because it’s pretty clear that its not a key value store; it overlaps between the document and key value style (Again I achieve this same effect in key sorted KV stores like LevelDB or TokyoCabinet).

In closing I must also give out some negatives I found, plus the some extras. Currently the number of partitions on Oracle NoSQL are fixed! What does this mean? It implies that at storage can’t be scaled horizontally yet since they can’t rebalance; once partitions have been made they remain fixed. Although Oracle in it’s documentation mentions about the next version having the re-balancing feature. Second (may be most for people) hurting part is dependence on Java platform. Now this will hurt lot of Ruby, and Python etc. people but again, as I said in my previous post, with help of a Java programmer you can write a REST API for yourself (I am planning a Protobuff server to eat this plague). Third, I found administration of system to be a little tricky (could not be understood without reading complete administration docs), but this is no hurdle for a programmers since it provides launch and go script to run the server. Right now I am planning to write a Scala wrapper (syntactic awesomeness) for Oracle NoSQL, so once I am done I will put it on GitHub. You folks in the mean while have few more calories of this NoSQL deliciousness.

Today I finally downloaded a community version copy of Oracle NoSQL, I am pretty happy to see a market giant like Oracle in the NoSQL era (Now waiting for DB2 guys to come in too).  I’ve used Oracle 9i way back in my history and have been  using MySQL for like decades (almost daily basis) and I know how feature rich these platforms are. Now since Oracle has landed with their NoSQL machinery, I just went through documentation from developers perspective to have a glimpse of feature sets. Lets go through highlights really quickly:

• It’s a Key Value store that’s scalable, HA, predictable, and really simple.
• Keys can behave like a composed structure of major, and minor keys. Major keys can be imagined just like path in your storage files (Hassan/Zohaib where there are two major keys Hassan and Zohaib) and minor keys can just be imagined like files in the (given major key) path holding your data in bytes (Hassan/Zohaib.birthdate, Hassan/Zohaib.phonenumber, Hassan/Zohaib.image, Hassan/Zohaib.userID holding different type of data). There can be multiple major and minor keys. I find this depth-limitation free structure to be extremely powerful, and really close to JSON structures (Yes it can act as document oriented database when crafted carefully).
• Supports typical CRUD operations with high definition over consistency and durability, which lets you choose the amount of chocolate sauce you can afford on your taste buds.
• A powerful Multi-Get, Get-Iterators, Set-Iterators, and depth controls that can set your retrieval on steroids.
• One word that says it all. Transactions!
• Last but not least. Well documented! Some of NoSQL projects have been suffering with this problem, so right from version 1.1 you are getting a really good documentation which is a big plus.

Despite the good features, I feel it has some set backs or I may say a wish list that can actually be fulfilled:

• You need to talk Java, this will benefit the JVM dialects like Coljure, JRuby, Scala etc. But we can always write some REST API like HBase did in shape of Stargate. I won’t recommend that and I think Oracle should quickly figure out a good protocol (something like Memcache protocol will work) to do so.
• Intelligent client library makes client end thick by maintaining state tables, calculating bounded network hops efficiency, and using major-minor keys to exploit the shards for efficient storage. I find it something like libmemcached or mongos (workings of mongos as library). I can’t call it a set back because some people prefer this way of sharding and distribution. But my favorite is Riak or Cassandra styled model of peers.

I can pretty much see how basic and really powerful this tool can turn out to be. I’ve only considered the Pros and Cons from programming point of view (API) completely ignoring administration and other parts (Will be doing a detailed benchmarks next). I must say I am pretty satisfied until now, and my first impressions are pretty good. Stay tuned for a detailed benchmarks, and stress test.

OK, so I am writing this post after a whole day spent in effort to get Thrift working on Ruby 1.9.2, so that I can run the Cassandra gem. Now what I always expected out of Gem system is breezy a experience. But today it was really rough. Now some people and Google results suggest that its due to my Ubuntu 11.10 distribution, but if its due to Ubuntu guys why ain’t anybody reported on Ubuntu bug tracking! Some people say its due to Ruby, and there are lots of endless discussions with no conclusions.

Anyways I don’t want any of you to waste your time on the same exercise that I did today and waste a Sunday! Hopefully this post will help you fix issue. So here we go if you right now do gem install thrift on your system and you get error like

…(Some compile commands)
ruby/missing.h:157:20: note: previous declaration of ‘strlcpy’ was here
make: *** [struct.o] Error 1”

You are victim of RRT (Really Really Trash)! Here is how to fix it. First download the original Thrift 0.7.0 source code distribution from apache’s site. Extract, configure, and install it with your Ruby 1.9.2 activated (in case you use RVM). Here are commands to do so:

rvm 1.9.2
tar vxzf [your thrift archive].tar.gz
cd thrift-0.7.0
./configure
make && make install

Be sure you have ruby development library installed, and things will go fine. Next just to make sure you get the native thrift client for your activated Ruby, inside thrift source directory go to lib/rb (cd lib/rb) and run rvm 1.9.2 && ruby setup.rb. In case of Ruby 1.9.3, (Running ruby setup.rb) I encountered same as before error saying “struct.c:28:1: error: static declaration of ‘strlcpy’ follows non-static declaration”, to fix this error just open up the ext/struct.c (inside the same lib/rb directory) and remove the static keyword from line 26. Once you have done that you can now install native client of thrift. Lets install the gem now!

First download the thrift gem from here. Make sure its version is 0.7.0 (matches thrift protocol you compiled). Once done do gem unpack thrift-0.7.0 to unpack the gem contents into thrift-0.7.0 directory. Go to thrift-0.7.0 director and you will find similar structure as lib/rb in thrift source code (not exactly similar but I am mentioning ext directory specifically). Again open up the same file (struct.c) and remove the static keyword from line 26. Running ruby setup.rb inside gem directory should be error free now. Since I am using bundle to do package management a one liner in Gemfile should do the job for me.

gem “thrift”, :path => ‘thrift-0.7.0’ #path to modified gem directory

And thats about it. Launch your IRB an things should load up. Hope it saves you a day!

I’ve been using Redis lately and I really love the work they have done. In one of my recent projects I had a typical scenario of having users on different channels, and since the old front-end (Javascript) was doing a JSONP through jQuery, I had to take in account for people who were not able to get back to channel due to some disconnection. Yes, its a typical scenario of “timeout” operation. Within a particular time limit (30 seconds in my case) if a user is not coming back or responding back, you have to mark him as offline user (or may be different in your cases).

Timeout is a very typical operation that you usually come across all the time, In my case there were few options I had to get around this issue (I was using tornado for handling requests, and Redis for storage):

• On every request before returning use add_timeout, keeping a Dictionary of ‘user_id’ -> add_timeout_returned_id; each time user comes back we remove the timeout event by looking up the user id in dictionary. If user doesn’t come back within time limit the callback fired will clean the things up.
• Keep time stamps against user id in Redis, run a background process cleaning those who didn’t updated their time stamp and have their entry left in redis. The background worker will be a separate configurable instance running after intervals and cleaning up the dead users.
• Use the lovely EXPIRE feature of Redis to remove the entries from DB and the worker process which looks for missing (expired) entries in channel and notifies other users with PUBLISH.

Now first entry, won’t be scalable since we are making the single python instance responsible for maintaining timers for same user, plus it complicates the code (yes I hate complicated code). You can although use cookies with NGINX to load-balance and assign same server again to same user, but this is what I am looking forward to avoid. Second approach is not real-time since we are using worker with update intervals to clean the mess I simply don’t like it. Third approach sounds fine from DB point of view, but again its not real-time. Exploring and digging deeper I found there was not way to have PUBLISH foo bar DELAY 30 (Publish after delay of 30 seconds). *GOSH* that would have killed the ugly duckling! Unfortunately for meeting deadline I took the 3rd approach since it was memory effective on DB side and keeping my code cleaner when compared to other approaches.

Last night I sat with this idea in my mind to have a PUBLISH event on a configured channel when a key expires (Cure for the itch). Walking around Redis community I found some really complicated discussions going on; with ideas of meta publish channels and some really brainy ideas stuff. I wanted to keep it simple, clean, configurable and that’s where opensource really helps; you have a different opinion go create your own flavor. I created my own fork and implemented what was really simple (and yummy chocolate flavored :P). So with new fork, Redis configuration can now contain an additional parameter named keys-expire-notify (so keys-expire-notify !KEYS will make Redis do a “PUBLISH !KEYS <expired_key>” every-time a key expires).

This feature will open up a whole new world of possibilities for the Redis users, including DB updates on cache invalidation, cascaded updates in Redis, time out events from Redis (Yes you can use keys to make it happen), Imagination is your limit! I’ve been playing around with the thing for a bit longer now and I am pretty much satisfied. It should be pretty much stable, since its built on top of existing infrastructure. You can grab your copy here and enjoy the new feature! I am really looking forward to see this idea as official part of Redis.

In my previous post I did some benchmarks for NodeJS vs Java’s NIO. For benchmarking I wrote my custom event driven loop handling, and parsing HTTP connections; and what else you can expect! Hell broke loose when the NodeJS fans read that. They had sort of weird comments and objections. From those all the weirdness I learned a few things. Things that were pretty simple and straight forward. This post will be my last post on all I have to say about NodeJS, and Java (Yes, I won’t be comparing them again!), and other than just theory (as usual) I will present the experiments. While reading this post please bear in mind, I am not a technology maker or an Omega of any big platform; I am one of the developers among the race of curious ones, who when encounters a new technology readily accepts it, experiments with it and then places it in the right shelve (just like a weapon) for the right use case. The reasons behind my last blog post include clarifications (to my self and people around) about pros and cons, a solid idea of what’s going on inside, and what if I never had NodeJS on hand (more precisely how Facebook and Twitter guys handle so much load). I totally believe in greedy approach to problems, where I say “To optimize the bigger problem, you have to optimize the subpart of the problem”.

I know I annoyed alot of people just by comparing Apples and Bananas (Java and NodeJS). Both languages have a ridiculously different paradigm (OO vs Prototypal) and totally different maturity level (15 years of engineering on Java). So this post is to put both tools in “same situation” and experiment what are the results. Some people thought JIT, and JVM has nothing to do with IO based system, well I won’t say anything in reply to it yet. Let’s proceed with first benchmark.

This time I am taking the same NodeJS with Cluster (Thanks to suggestion in previous post), to add multi-core CPU utilization support (Like always you can suggest if there is a better option). Also I changed the code to do some really basic and simple processing, using a counter with timestamp system that just prints a simple message with number of hit counts, and timestamp. For Java I have a special arrangement this time; every body last time objected that my code was pretty unfair to the amount of processing NodeJS (http module) has to do, so I am making it fair this time. Netty NIO is the new contender in the ring for Java this time; with its fully implemented HTTP support, NodeJS people now cannot object on anything regarding the event loop and parsing system. The code gets divided into four classes now main launcher, pipeline factory, handler (NodeJS guys would be interested in this one), and a singleton. Pretty fair pieces of codes for both sides, both utilizing 1 worker per CPU core (I have a dual core), and having full fledged HTTP protocol support. Lets have look at benchmarks again! I used concurrency level 1000 and NodeJS managed to get only 5.2K req/sec [Raw Dump], a increase of 200 over last time (That’s pretty sweet you wasted my time again!). Java Netty NIO on other hand manages to handle 7.3K req/sec [Raw Dump], a decrease of 700 over the last time (now you know why I have decided not to write on both of these again); thats still over 1.7 times faster.

A basic application

Somebody suggested me to put them in a real scenario (do something with some storage and serve JSON and all typical stuff) and then do the benchmarks. Well taken, I picked Redis (Good enough for speed) for both of them and made an application where hit ‘counter’ now lies in Redis along with the hash ‘channel’ that contains a mapping of ‘timestamp’ => ‘counter value’. On each request I will check if the ‘counter’ exists; if no then I initialize it to 1 by set operation and set the hash value by hmset, if yes I increment the existing value and save that incremented value to map with hmset. Pretty basic and simple to understand (Like assigning a unique id to a user and making him join a chat channel). It can’t get any simpler and practical. So I am putting NodeJS redis (used npm install hiredis redis; its all steroids!) client’s “asynchronous coolness” with Java’s uncool asynchronous Jedis and Jackson for JSON serialization. Later on I discovered lettuce for Netty NIO as well, but I was too lazy to re-do benchmarks. Using lettuce will definitely improve the performance (or will it?). Let NodeJS have that advantage this time and see what results are we looking at. Here is the updated NodeJS code, and updated handler class and singleton class of Java.

Now back to benchmarks; running benchmarks with concurrency 1000 on NodeJS had a knockout, I started receiving connection reset by peer, super long delays (even with total 10000 requests it seemed like taking hours) and Redis running out of file numbers (Yes I’ve opened my limits and done everything possible but it was not working). So we have to lower the concurrency a little bit (since all this is running on same machine); with concurrency level 100 I was able to run benchmarks so I took it as standard point for Java too. With concurrency 100 NodeJS was able to make 1.1K req/sec [Raw Dump], and Java Netty NIO still managed 1.7K req/sec [Raw Dump]; after concurrency level 100 I tried shots on concurrency level 500 and results were pretty much same [NodeJS ~0.9K max][Java ~1.4 max]!

Update

After some fiddling around I was able to make Node work! Pretty much same results (Infact they went down ~0.7K [Raw Dump]), Netty keeps itself solid state [~1.3K Raw Dump].

So in conclusion:

• NodeJS is good for long open connections (I’ve experienced that), but after this experiment I am pretty confident to roll something out on Netty NIO as well, the same design choice was made by plurk people (using NodeJS) but they reverted back to Java Netty NIO.
• Speed of execution has alot to do with what you can push out, Ruby is suffering today due to its bad garbage collector and speed (That’s why Twitter abandoned it, that’s why Linked in is using JRuby with Jetty stuff to make things happen). Reason I always praise Java is when it provides you with so many options to tune your application according to your requirement (Yes its not easy but its fruitful).
• Buzz words are dangerous! Stop looking at top trends and then adapting the technology.
• If you are good at X you can achieve almost the same scale in X as that of Y (Not always true) but only if you are really good at X (Facebook took PHP to next level, MySQL still lives in their core, Heroku guys have been scaling alot of apps on notorious Ruby, there are many examples out there, just dig!).
• If NodeJS was the best tool for scaling your requests it should have killed us all (even cockroaches).
• Node will mature over the time; I have a strong believe over that, and I will be a happy coder using Javascript as server side too.
• I am not asking you to leave NodeJS (who will help it grow then), things evolve out of experiments and we have to do it, but before holding the sword in your hand you should know the pros and cons.
• Stop yelling and bragging about NodeJS being fast!
• Thanks to everyone, I was pushed to do this experiment and make picture even clear! Lets move web forward.

Before I start this post let me be very clear and precise “I am not advising you to leave NodeJS or shift to Java [PERIOD!!!]”.

I’ve been in this debate and there is a misunderstanding among my geek friends that NodeJS is the future and blah blah blah… I totally love Javascript (No bragging I have a history of being Javascript expert and I’ve been writing some love for JS too); there is no doubt about the beauty of closures, and the prototype styled programming. But when it comes to have Javascript as back-end, its totally a different story.

I find it pretty amusing when people are doing benchmarks of some solid technology stack to NodeJS, and declaring NodeJS to be the fastest thing on earth (just Google NodeJS vs *anything that comes to your mind* and you’ll find results like this, this, and what not). Beyond any doubt NodeJS model is worth attention, but will I use it in my production environment? I doubt that. I’ve worked on NodeJS on real serious basis; and experience was pretty bad. I had to write a complete HTTP client library with Multipart support (Today known as Reston) so that I can send files to Amazon S3 and few other REST services (at that time there was no existing library with HTTP Multipart support, HTTPS had issues and it was all really frustrating for me), to summarize it I have to say following things to my audience:

• Not every web application is many connections application, you don’t code a chat system or a comet system everyday. NodeJS is good at few things and should be used for them. If you ask me to code a websocket based chat system using a IRC server, I would vote for NodeJS; but if you ask me to pull emails from all your email accounts and store them in some RDBMS or NOSQL store I would rather think twice.
• Technology stack is important! Accept it! Live with it! I’ve seen people making wrong choices on technology stack (then bragging about it just because they used NodeJS) and then abandoning it, because later they discovered a better tool for doing the job.
• If its about event based code and its readability I am pretty sure everybody agrees on the point that callback code is more spaghetti code than normal flowing code.
• Static typed languages have a clear advantage over dynamic typed languages. You won’t have any trouble absorbing this fact if you know compiler internals.

Enough of my experience lets proceed to benchmarks. I have a typical commodity machine (3GB RAM, Dual Core processor) and as ground rules I will be doing straight forward benchmarks of Java NIO receiving a HTTP request, and responding hello world to it; same applies for NodeJS.

NodeJS code is pretty straight forward. I’ve used Node version 0.4.9. Notice that it relies on the typical ‘http’ module, which in turn relies on ‘net’, ‘stream’ etc. all of them are NodeJS libraries (nothing special from me) and they rely on JIT of V8 (accept the C parts) for speedy execution.

Now for Java I took NIO of Java and selector channels to achieve the same effect as NodeJS (single thread event dispatch). Code as expected is a little longer since I had to write the looping part too, but results are worth the experience. I’ve tried to put everything into a single file as much as I can, so code is not modular/production-ready at all! Here are the two files Runner.java and core.SocketSelectorCore.java. I’ve used HashMaps, String split, indexOf etc. and done basic HTTP header parsing of the request to simulate a normal request flow (read, parse, respond cycle). I’ve used pretty inefficient methods, but they are only there to make things worst for Java.

So launching NodeJS with “node test.js” and running Apache Benchmark (ab -c 1000 -n 100000) 3 times with concurrency level of 1000 yields [detailed dump] an average of almost 4 to 5 K requests per second.

Now before putting the custom written Java NIO event based system on trial; I would like to mention few things. Java is a beast, you get so many options on how to tune the JVM ranging from garbage collection to heap sizes (Checkout Java documentation in-case you are interested). So for my scenario, I used following flags to run my JVM “java -server -XX:+PrintCompilation -XX:+UseConcMarkSweepGC Runner”. Notice I’ve used verbose mode for JIT compilation, so that I can know when JVM is warmed up and I can take my benchmark samples; plus I changed the GC method (I tried different methods, but I liked this one the most). After the JVM warmed up (Google it n00bs) and running same Apache Benchmarks [detailed dump], Java was able to manage 8 to 8.5 K requests per second.

I tried different Heap Sizes and played around with other parameters; results were interesting. I was always above 6K requests per second on my machine. Lowering the concurrency (-c 100) takes figures as high as 11K. If you notice closely I was shipping more bytes per request (as compared to NodeJS), but that didn’t hurt Java and even with my really stupid approach for storing headers I was able to make it this far. Having these results at hand I also played around with JRuby for syntactic awesomeness and wrote a throw away code (Complete replica of Java code). And with a pretty rough code and a minor different flags passed to JRuby, I was again able to manage 4 - 4.5 K requests per second.

So question now remains is why did I do this, and whats the take away? I think answers are pretty clear. I personally love Javascript and NodeJS but I won’t take the argument NodeJS can do X that language Y can’t do. I believe comparing Java or PHP or some X to NodeJS is a stupid buzz that appeared recently. JIT of Java is pretty much state of art, and Google guys are putting in a great effort to bring V8 up-to mark. Frameworks like Netty NIO and Mina have been sitting out there for long time, but have been ignored due to Java’s weird syntax, memory hunger or steep learning curve reasons (I am just saying). Having said all that I’ve just busted the myth, “NodeJS can handle more connections due to its async style, and it enforces you to write code in callback(async) style; thus you write better code”. My answer is pretty simple “Write the core part in Java, wrap it in the JRuby or Scala beauty; and you have a much better way of expressing event driven system”.

EDIT: September 2nd 2011

Well I’ve been busy with work lately and seems like I had a nuclear melt down; and before I knew it the whole reactor exploded! In my previous post Node on Nails (I just realized I am good in making titles like these), just to purse the curiosity I tried to create a basic Java based NIO HTTP system and see the numbers with Node. Here is what I think reactions concluded for me (Will be helpful to you and me when writing a blog post):

• Never ever use the word n00bs (although I tried to point the people who hear about node and switch to it instantly) it makes me sound like attacking you personally.
• Avoid scandalish titles which I think is right; but I can’t avoid it (I will try to I promise).
• Never criticize the technology or any tool I have a crush on (NodeJS in this case); initial roll was pretty encouraging towards Java but later comments really show the people about to bang their head in the walls and not being able to convince me with anything concrete.
• Starting an article with such disclaimer never matters; you have to show the other site no matter if it is openly available.
• Languages and tools evolve over time; one of the good factors for evolution is criticizing mistakes! Fanboys don’t take criticism constructively so there must be another way around.
• People in today’s world believe that static typed can be beaten with dynamic typed languages.
• I always used this blog as personal space for letting out such benchmarks, ideas, discussions. But since people take it too seriously I have to be more careful making it more formal (People commented on English spell mistakes at times as if it was an English exam).

Although I was busy at work, but the exploding hatred called me to address the issue; don’t worry I will still be doing more work on it as I get more time.

Edit: 6-Sep 2011

I’ve done some more benchmarks with a full fledged framework. If you have objections on this benchmark try new ones out.

I’ve been pretty excited about Google’s LevelDB, not to mention there are some really old tanks already in the battle field like BDB, Tokyo Cabinet (Kyoto Cabinet as new one), HamsterDB etc. Fortunately I’ve already worked with Kyoto Cabinet and when I looked at the benchmarks I was totally blown away. I have a strong prejudice to name Google, but the benchmark results were way too good to believe. I also had an objection; LevelDB benchmarks compared itself only to TreeDB of Kyoto and not the HashDB (If ignoring key ordering and treating LevelDB to be true key value only). Some of you may start yelling right away that comparing Hash structure to an SSTable is not fair or something. Well by end of this post I will be comparing TreeDB with LevelDB as well and my results were different from officially posted benchmarks.

So to start off I decided to do a simple insertion, and iteration comparison (ignoring order) of simple HashDB and LevelDB without any compression or any sugar both stores come with, it will be a simple (no forced writes) benchmark of both comparing how much time each takes. I have a pretty normal machine (aka a commodity machine) with a normal HDD, and Intel Dual Core 3GB RAM. So without wasting anymore time; lets dive into code. Here is code for HashDB Kyoto and LevelDB. Pretty straight forward; I’ve carefully placed benchmarks point to only measure the “real time” consumed by both excluding additional key preparation and value preparation parts. The idea is to insert and iterate 100,000 key/value pairs. Running the benchmarks yield (Please note the first line indicates the time to insert 100,000 KV pairs, and second line indicates the time it took to iterate all those KV pairs, every time db was created from scratch i.e. previous copy was deleted. I took total 3 runs of each):

Kyoto Cabinet 100,000 entries:

• Time consumed was 0.328945
  Time consumed was 0.207315
• Time consumed was 0.286653
  Time consumed was 0.210296
• Time consumed was 0.284954
  Time consumed was 0.208595

LevelDB 100,000 entries:

• Time consumed was 0.44742
  Time consumed was 0.171842
• Time consumed was 0.449287
  Time consumed was 0.169108
• Time consumed was 0.444432
  Time consumed was 0.169381

Iterated reads are pretty fast in LevelDB, but writes are almost twice as slow as Kyoto Cabinet. I didn’t stop here because I’ve not hit the weak spot of LevelDB yet (yes larger values). So in my next test I fixed the length of value buffer to 512 bytes instead of the small string length, results took the expected form:

Kyoto Cabinet HashDB 512 bytes 100,000 entries:

• Time consumed was 0.549354
  Time consumed was 0.249182
• Time consumed was 0.451567
  Time consumed was 0.236295
• Time consumed was 0.568072
  Time consumed was 0.240299

LevelDB 512 bytes 100,000 entries:

• Time consumed was 5.09164
  Time consumed was 0.302542
• Time consumed was 5.16017
  Time consumed was 0.301258
• Time consumed was 5.32948
  Time consumed was 0.300175

At this point I became suspicious about the TreeDB benchmarks by LevelDB being true on my machine; and guess what, I was right! Here are benchmarks of TreeDB of Kyoto Cabinet:

TreeDB with Fixed 512 byte value length:

• Time consumed was 0.393798
  Time consumed was 0.445125
• Time consumed was 0.389259
  Time consumed was 0.439923
• Time consumed was 0.392562
  Time consumed was 0.438211

TreeDB with small length values (same as 1st HashDB benchmarks above):

• Time consumed was 0.39401
  Time consumed was 0.322735
• Time consumed was 0.445422
  Time consumed was 0.349465
• Time consumed was 0.287259
  Time consumed was 0.258459

Finally I took the total entries to 1,000,000 and ran a benchmark again with TreeDB, and LevelDB; results are as following:

LevelDB 1,000,000 entries (small values to give it an advantage):

• Time consumed was 5.9836
  Time consumed was 1.91519
• Time consumed was 6.03402
  Time consumed was 1.90402
• Time consumed was 6.01079
  Time consumed was 1.88251

TreeDB 1,000,000 entries (small values):

• Time consumed was 3.9333
  Time consumed was 2.56793
• Time consumed was 3.87633
  Time consumed was 2.56836
• Time consumed was 3.93033
  Time consumed was 2.57917

Surprised? Same here! Now the question is am I missing something? I am planning to put up some more benchmarks with some real life scenarios and values (may be a tweet sized stuff). Until then I would love to have feedback, and know if I’ve done any mistake while benchmarking.