There's been a lot great work happening in the VM performance space over the last few years. The problems of performance are beginning to be well understood as even dynamic languages begin to challenge the incumbents. This article reviews a project which aims to bring more empirical testing to the language/runtime performance debate. Rather than argue about the theorectical, we let the code speak for itself. You may be suprised by what it says.
The author of this article is well aware of its provocative nature. This is just one test and readers should take care when extrapolating these results to dissimilar use cases. The C code provided by the project being reviewed follows an Object Oriented style. It is certainly possible to optimize the C code further, for instance by using a more procedural style with less memory allocation. But then a similar approach should be used by the other versions (languages) presented as well. I whole heartedly encourge you to submit pull requests with these new versions in separate directories based on the style/technique used.
git clone git://github.com/gflarity/peg-performance.git
This was all developed and tested under OS X however there shouldn't be any platform dependencies. You'll need a number of things for the test itself. I've put the version I'm using in brackets: gcc (4.2.1), Mono (2.10.1), Java (1.6.0_24), node.js (0.4.2), PHP (5.3.3), Python (2.7.1), Ruby (1.8.7).
Run the tests!
Note this will take a few minutes as some platforms are definitely faster than others.
For each of the languages the test is run 10 times. I throw out the best and worst result then compute an average. A chart is created to show the results.
Results can be viewed by opening .reports/report.html.
Here is the graph produced by this test on my MacBook Air:
Here's what I find interesting about these results:
2) I'd also like to see the results of this test if this problem was solved in a non object oriented style. That said, there are distinct advantages to using OOP.
3) Much respect to the Mono project. While I'm sure Microsoft's .NET runtime approaches the HotSpot VM in terms of performance, Mono is hanging in there.