[Kolab-devel] Test: MongoDB for caching

[Thomas Brüderli]

Hi there

I just made some tests by using MongoDB as storage backend for the
Roundcube/libkolab cache. I thought one should give it a try as an
alternative to MySQL. And the results are quite promising:

While inserting was slightly faster, querying the cache shows nearly equal
results in speed. There isn't a big difference for contact autocompletion
either were MySQL is queried with LIKE and MongoDB with regex.

Action    | MongoDB   | MySQL     *
-----------------------------------
Insert    |  100 sec  |  120 sec  (loading 10k events from IMAP into cache)
Query 1   |  1.1 sec  |  1.5 sec  (sync + select events from-to out of 10k)
Query 2   | 0.36 sec  | 0.37 sec  (sync + select 30 contacts of a folder)
Search    | 0.36 sec  | 0.37 sec  (sync + search contacts by name)

* The listed times are the runtime of the entire PHP request including
login to IMAP, connecting to DB, cache syncing, etc. Run on a 2 GHz CPU
accessing a Kolab server running in a VM on the same machine.

The big drawback with MongoDB, however, is the disk space it uses to store
the data. 12k events and some contacts eat up ~465 MB while MySQL shows a
disk usage of ~80 MB.

On the other hand, Kolab objects are saved as structured data in MongoDB
which has the big advantage, that one can query them by almost every
attribute directly. Searching the cache becomes very efficient with MongoDB
whereas with MySQL, the payload data is stored as a serialized string and
in order to search objects, one has to iterate over all of them and compare
the attributes "manually" in PHP. There's an exception for autocompletion
where the relevant contact data is stored in a text column which is search
using LIKE queries.

Last but not least, some sharing and/or redundancy considerations could
give extra points to MongoDB but I guess that highly depends on the
individual Kolab setups.

Now that we have a working implementation to use MongoDB for caching we can
actually offer both options for the storage backend.

Any suggestions, comments or serious concerns about MongoDB?

Cheers,
Thomas