[Kolab-devel] redis vs. MongoDB for zpush kolab backend

Fri Jan 7 17:55:55 CET 2011

On Friday, January 07, 2011 06:32:41 pm Alain Abbas wrote:
> Gunnar Wrobel a écrit :
> > Hi Bogo,
> > 
> > Zitat von Bogomil Shopov <shopov at kolabsys.com>:
> >> Hi all,
> >> I am actively testing MongoDB + PHP driver on my machine right now and I
> >> am getting better results than Redis.
> > 
> > Sounds good.
> > 
> >> In fact MongoDB is what we need:
> > I wonder if this is really so much a question of the backend
> > technology. I admit I don't have much experience with something like
> > MongoDB but my assumption is that there are different technologies
> > around that provide the same feature set. Do we really need to choose
> > a single one?
> > 
> > I looked at the code you added in the "backend-mongodb" branch and so
> > far this is a key-value store. That is something you get from many
> > backend technologies and in fact the interface for such services is
> > near to trivial. It boils down to what you established in
> > mongoDBcache.php.
> > 
> > "backend-redis" wasn't very different concerning the interface. So I
> > would suggest to complete the abstraction at that point and really
> > rely on a simple interface. In the long term you'll be able to switch
> > to any newer technology that suites your needs better than what might
> > perform best right now.
> > 
> > Of course there are already interfaces like that around. Horde_Cache
> > is one of them (it abstracts about 10 key-value storage backend
> > types). And in fact I'd be very happy if we could keep such code
> > segments together.
> > 
> > I much rather convert your code into modules to Horde_Cache and add
> > support for both Redis and MongoDB there (both are missing from
> > Horde_Cache at the moment).
> > 
> > What is the advantage of keeping such code consolidated: We can use
> > the same caching backend for other applications as well. Did you know
> > that we use the db functions Alain used initially in the Kolab
> > free/busy code as well?
> > 
> > You obviously found them to be too slow for our needs. If Redis and/or
> > MongoDB is a better solution I'd like to use it for other systems as
> > well.
> > 
> > Does any of this make sense to you and would that be a direction you'd
> > be willing to go into?
> 
> Yes we must have an abstraction to switch the technology . An Abstract
> class at the high level who provide
> method that we need.
> 
> >> - We can make queries (it's usefull for Calendar)
> >> - We can create indexes on whatever we want
> >> - We can use authentication (salt+md5) (with Redis we don't (we should
> >> use "trusted environment"));
> >> - There are more than 10 language drivers
> >> (http://www.mongodb.org/display/DOCS/Drivers) - we can access the
> >> data in more
> >> than 10 languages
> > 
> > I'm aware that many of these are features that are not part of the
> > simple key-value interface I mentioned above. But that doesn't change
> > the fact that the key-value aspect is part of what is required. All
> > other parts can be handled separately from that.
> 
> We have all the same problem with Imap storage, The Performance when
> Calendars (especialy calendar who move often) and
> we must have index not only key values to search the dates . Iterations
> are not good. Try in horde actualy to display 10 shared calendar who have
> each 1000 messages in the folder. This is impossible or take minutes and
> minutes. The deal i think is to have only one process or method to
> populate the cache and each can retrieve what he want quickly.
> a good scheme would be :
> 
> Webmail ---------------->
> third party (eg : Caldav) ->    CACHE <-----> Imap
> zpush/backend  --------->
> 
> if the webmail ask for an event for exemple why to read again the event
> (with zpush) for example ) if there are already in the cache ?
> for me the best solution is to have a GLOBAL namespace for all the datas
> (contacts, calendar ...)
> and a private space for the application ( for example i need to store
> state of the sync for zpush but thos datas are not instesting for
> anothers application.

Can you try to draw it in a sheme? 

> 
> Regards
> Alain
> 
> > Cheers,
> > 
> > Gunnar
> > 
> >> I have tried xCache (lighttpd project) under PHP, but it does almost the
> >> same like current DB and it's not so good for real chaching, but for
> >> opcode optimization.
> >> 
> >> 
> >> 
> >> --
> >> Bogomil "Bogo" Shopov
> >> Senior Web Engineer
> >> 
> >> 
> >> Kolab Systems AG
> >> Zürich, Switzerland
> >> 
> >> e: shopov at kolabsys.com
> >> t: +41 43 5016691 (ext. 2115 649)
> >> w: http://kolabsys.com
> >> b:http://talkweb.eu
> >> 
> >> pgp: E69A226A Bogomil "Bogo" Shopov
> >> 
> >> _______________________________________________
> >> Kolab-devel mailing list
> >> Kolab-devel at kolab.org
> >> https://kolab.org/mailman/listinfo/kolab-devel
> > 
> > --
> > Gunnar Wrobel
> > Developer, Kolab Systems AG
> > 
> > e: wrobel at kolabsys.com
> > t: +49 700 6245 0000
> > w: http://www.kolabsys.com
> > 
> > pgp: 9703 43BE
> > 
> > ----------------------------------------------------------------
> > This message was sent using IMP, the Internet Messaging Program.
> > 
> > _______________________________________________
> > Kolab-devel mailing list
> > Kolab-devel at kolab.org
> > https://kolab.org/mailman/listinfo/kolab-devel
> 
> _______________________________________________
> Kolab-devel mailing list
> Kolab-devel at kolab.org
> https://kolab.org/mailman/listinfo/kolab-devel

-- 

Bogomil "Bogo" Shopov
Senior Web Engineer

Kolab Systems AG
Zürich, Switzerland

e: shopov at kolabsys.com
t: +41 43 5016691 (ext. 2115 649)
w: http://kolabsys.com
b:http://talkweb.eu

pgp: E69A226A Bogomil "Bogo" Shopov