[Kolab-devel] Z-push optimization
wrobel at kolabsys.com
Mon Oct 4 15:41:48 CEST 2010
Zitat von "Jeroen van Meeuwen (Kolab Systems)" <vanmeeuwen at kolabsys.com>:
> Alain Abbas wrote:
>> i ve got a big problem :
>> Each time that a mobile synchronize i must read all the annotations of
>> all the folders to know if a folder is for sync or not
>> i ve got some customers with big trees ( arround 300 folders in Inbox)
>> become a bottleneck.
>> 1) how to do not read at each time all the annotation folders ?
>> even if i cache some informations that become the hell
>> and i must know if the annotations are changed or not
>> to put or remove the folder to the sync
>> I thought to make that ,to make a special flag in the INBOX annotation a
>> counter :
>> If the user modify the annotation of one folder this number could be
>> STEPS :
>> 1 when the user modify an annotation the "ANNOVERSION" is
>> incremented in the INBOX folder
>> Zpush would check this ANNOVERSION and compare with the old one
>> if ANNOVERSION is not equal Zpush read again the annotations
>> Stay now the problem of the user space where to put this ANNOVERSION ?
>> I think at this step we see the limitation for this method (annotations
>> per folder) and a simple
>> list in LDAP should be
>> more efficient and simple .
>> I waiting your feedback and ideas to resolve this big performance issue
> Hi Alain,
> I understand your problem is the iteration of (many) IMAP folders and the
> inherent cost of having to use the IMAP protocol to get to the data that you
> actually require.
Alain, is this indeed your problem? Meaning you are calling
imap_get_annotation 300 times if there are 300 folders? As far as I
know this problem is inherent to using the imap_get_annotation call
from the patched Kolab PHP package.
On the IMAP protocol side there is no limitation to using a wildcard
to retrieve all annotations of a specific type for 300 folders *at the
same time within one IMAP call*
C: a GETANNOTATION "INBOX.%" "/comment" "value.priv"
S: * ANNOTATION "INBOX.1" "/comment" ("value.priv" "My
comment for 1")
S: * ANNOTATION "INBOX.2" "/comment" ("value.priv" "My
comment for 2")
S: * ANNOTATION "INBOX.3" "/comment" ("value.priv" "My
comment for 3")
S: a OK GETANNOTATION complete
I did test this once and if I remember correctly then the Kolab IMAP
server supports this just fine but imap_get_annotation does not.
This is a significant problem for the Kolab webclient, too. I still
need to resolve that in Horde4 but this is no big deal as Horde4 does
not use the c-client based IMAP features of PHP anymore.
For your specific problem it might be an option to look into the
imap_get_annotation patch, too. Or you'd switch to using another IMAP
library but that might be a lot of work, too.
> I understand this to result in great performance penalties, and hence I was
> thinking the following scenario might make sense to pursue for the future;
> If Cyrus were enabled to keep it's mailboxes.db, annotations.db, seen.db, and
> possibly even it's .index, .cache and .header in a true SQL database backend,
> Kolab_Z-Push would be able to use Cyrus' SQL backend as if it were cache.
> I've got several threads going on this approach already, so please
> let me know
> what you think of using Cyrus' "live" as your "cache".
But that would mean that Kolab-Zpush does open a standard IMAP
connection to Cyrus and for reading annotations in an efficient way it
takes a short "peek behind the scenes" misusing the fact that is
installed on the same server as Cyrus imapd.
This sounds completely broken to me. You'd give a webapp access to the
Cyrus IMAP internal databases. You'd enforce installation of
Kolab_Zpush on the same server on which Cyrus IMAP is installed
(unless you want to use networked SQL databases and this is a no-go,
too). And you'd use two type of communication channels. No way :)
> Kind regards,
> Jeroen van Meeuwen
> Senior Engineer, Kolab Systems AG
> e: vanmeeuwen at kolabsys.com
> t: +316 42 801 403
> w: http://www.kolabsys.com
> pgp: 9342 BF08
> Kolab-devel mailing list
> Kolab-devel at kolab.org
Developer, Kolab Systems AG
e: wrobel at kolabsys.com
t: +49 700 6245 0000
pgp: 9703 43BE
This message was sent using IMP, the Internet Messaging Program.
More information about the devel