« Unable to load source file - GeoRSS for Virtual Earth API | Main | Mosquito destruction via acoustics »

Scorchin' in-memory db benchmark with 1 TB RAM

I've been investigating machines with large amounts of RAM for quite a few months now. I'm convinced that the distributed-centralized computing paradigm oscillates, and that the next few years will bring about monster computers with, in particular, tons of RAM, but maybe fewer processors. My sleuthing this week led to the Altix 4700. Emerging out of the ashes of bankruptcy, it is SGI's new dream machine. They boast a mind-numbing 128TB of shared memory capacity decoupled from the number of processors. This is important. For a number of years, SGI has had their NUMA* technology, which essentially promises shared access to all RAM by all processors. In most multi-processor machines, each processor really has direct access to about 16GB RAM or so (multiplied by number of cores...). If you've ever done any sort of parallel computing, it makes a difference. Essentially you have N machines each with M GB of RAM, instead of N processors sharing N*M GB of RAM. Think about inverting matrices... or hosting databases in memory.

The scoop today is a database benchmark on the University of Louisiana's 160-core Altix 4700. In particular, the database is an in-memory database system (IMDS), which means that the database is entirely hosted in memory. This is important, because there are a lot of optimizations that SQL Server, MySQL, Oracle, do to compensate for the fact that databases sit on slow hard drives and memory is hard to come by. Fancy things like memcaches, etc. These all go away if you have enough memory to store the whole thing simply in RAM. Here are the goods:

For a simple SELECT against the 1.17 Terabyte, 15.54 billion row database, eXtremeDB-64 processed 87.78 million query transactions per second using its native application programming interface (API) and 28.14 million transactions per second using a SQL ODBC API. To put these results in perspective, consider that the lingua franca for discussing query performance is transactions per minute; In more complex JOIN operations, the benchmark report documents performance of 11.13 million operations per second with the native API, and 4.36 million operations per second using SQL ODBC; [emphasis mine]

In case you were wondering, IMDS speed >> database hosted on RAM drive. (I also asked this question.) You would expect it to be a little bit faster, but it turns out that all the extraneous operations when you don't plan on being in RAM really take up a lot of time. Of course, the benchmark comes directly from McObject's marketing department, so YMMV."

Post a comment

Please enter the security code you see here