MG3 and CF Framework Performance Benchmarks
Whenever I work on the core of a Model-Glue version, I use a suite of sample applications and JMeter tests to make sure I'm not wrecking anything in a performance or resource context. I measure against both prior versions of MG and other frameworks, making sure nothing anomalous is sneaking into MG by watching for spikes in the numbers. I'm not trying to "compete" as much as I'm just looking for comparable benchmarks for Model-Glue, and performance profiles of the other frameworks are my best bet.
For running these tests, I maintain a set of applications (Model-Glue, Fusebox, Mach-ii, and now ColdBox) that are the simplest possible request - they don't even show any output. For Mach-II and Model-Glue, no listener functions are run. For ColdBox, it's just the minimum required handlers and a simple "main" handler that adds one view.
Basically, it's a test of how well each can spin up its equivalent of a request context, execute it, render one empty view, and shut down.
This morning, I slaved my PC to running these tests, which ramp up from 25 to 100 simultaneous users, each making 100 back-to-back requests. The following chart shows the results:
Note: don't use these numbers for choosing a framework. Application code is much more likely to be a bottleneck than the framework!
If anyone else would like to try these out (or if any framework authors would like to make sure I've made the sample apps as simple as possible!), I've attached the Eclipse project I use for all of this to this entry. Maybe it'd be a good RIAForge project?
9 comments - Posted by Joe Rinehart at 10:27 AM - Categories: Model-Glue | Frameworks