[Kolab-devel] redis vs. MongoDB for zpush kolab backend

[Alain Abbas]

Gunnar Wrobel a écrit :
> Hi Bogo,
>
> Zitat von Bogomil Shopov <shopov at kolabsys.com>:
>
>   
>> Hi all,
>> I am actively testing MongoDB + PHP driver on my machine right now and I am
>> getting better results than Redis.
>>     
>
> Sounds good.
>
>   
>> In fact MongoDB is what we need:
>>     
>
> I wonder if this is really so much a question of the backend  
> technology. I admit I don't have much experience with something like  
> MongoDB but my assumption is that there are different technologies  
> around that provide the same feature set. Do we really need to choose  
> a single one?
>
> I looked at the code you added in the "backend-mongodb" branch and so  
> far this is a key-value store. That is something you get from many  
> backend technologies and in fact the interface for such services is  
> near to trivial. It boils down to what you established in  
> mongoDBcache.php.
>
> "backend-redis" wasn't very different concerning the interface. So I  
> would suggest to complete the abstraction at that point and really  
> rely on a simple interface. In the long term you'll be able to switch  
> to any newer technology that suites your needs better than what might  
> perform best right now.
>
> Of course there are already interfaces like that around. Horde_Cache  
> is one of them (it abstracts about 10 key-value storage backend  
> types). And in fact I'd be very happy if we could keep such code  
> segments together.
>
> I much rather convert your code into modules to Horde_Cache and add  
> support for both Redis and MongoDB there (both are missing from  
> Horde_Cache at the moment).
>
> What is the advantage of keeping such code consolidated: We can use  
> the same caching backend for other applications as well. Did you know  
> that we use the db functions Alain used initially in the Kolab  
> free/busy code as well?
>
> You obviously found them to be too slow for our needs. If Redis and/or  
> MongoDB is a better solution I'd like to use it for other systems as  
> well.
>
> Does any of this make sense to you and would that be a direction you'd  
> be willing to go into?
>
>   
Yes we must have an abstraction to switch the technology . An Abstract 
class at the high level who provide
method that we need.
>> - We can make queries (it's usefull for Calendar)
>> - We can create indexes on whatever we want
>> - We can use authentication (salt+md5) (with Redis we don't (we should use
>> "trusted environment"));
>> - There are more than 10 language drivers
>> (http://www.mongodb.org/display/DOCS/Drivers) - we can access the  
>> data in more
>> than 10 languages
>>     
>
> I'm aware that many of these are features that are not part of the  
> simple key-value interface I mentioned above. But that doesn't change  
> the fact that the key-value aspect is part of what is required. All  
> other parts can be handled separately from that.
>   
We have all the same problem with Imap storage, The Performance when 
Calendars (especialy calendar who move often) and
we must have index not only key values to search the dates . Iterations 
are not good. Try in horde actualy to display 10 shared calendar who have
each 1000 messages in the folder. This is impossible or take minutes and 
minutes. The deal i think is to have only one process or method to
populate the cache and each can retrieve what he want quickly.
a good scheme would be :

Webmail ---------------->
third party (eg : Caldav) ->    CACHE <-----> Imap
zpush/backend  --------->

if the webmail ask for an event for exemple why to read again the event 
(with zpush) for example ) if there are already in the cache ?
for me the best solution is to have a GLOBAL namespace for all the datas 
(contacts, calendar ...)
and a private space for the application ( for example i need to store 
state of the sync for zpush but thos datas are not instesting for
anothers application.

Regards
Alain
> Cheers,
>
> Gunnar
>
>   
>> I have tried xCache (lighttpd project) under PHP, but it does almost the same
>> like current DB and it's not so good for real chaching, but for opcode
>> optimization.
>>
>>
>>
>> --
>> Bogomil "Bogo" Shopov
>> Senior Web Engineer
>>
>>
>> Kolab Systems AG
>> Zürich, Switzerland
>>
>> e: shopov at kolabsys.com
>> t: +41 43 5016691 (ext. 2115 649)
>> w: http://kolabsys.com
>> b:http://talkweb.eu
>>
>> pgp: E69A226A Bogomil "Bogo" Shopov
>>
>> _______________________________________________
>> Kolab-devel mailing list
>> Kolab-devel at kolab.org
>> https://kolab.org/mailman/listinfo/kolab-devel
>>
>>     
>
>
>
> --
> Gunnar Wrobel
> Developer, Kolab Systems AG
>
> e: wrobel at kolabsys.com
> t: +49 700 6245 0000
> w: http://www.kolabsys.com
>
> pgp: 9703 43BE
>
> ----------------------------------------------------------------
> This message was sent using IMP, the Internet Messaging Program.
>
> _______________________________________________
> Kolab-devel mailing list
> Kolab-devel at kolab.org
> https://kolab.org/mailman/listinfo/kolab-devel
>