imap id cache and sync strategies (was: Kolab XML specification)

[Bernhard Reiter]

Steffen:
my memory about the discussion was, that it actually is better
to use Martin's tactic here.

And we want all Kolab clients (including the webclient)
to ideally implement this one day, so the scalability approach pays off.

As for the implementation details we should not go too deeply
in them on this list, as the format should allow a good implementation
(and as a compromise allows a few worse ones).

Martin: I think that a really good describtion of the sync
strategy would help many client developers. I also gave some hints
to the synckolab people, but this takes a while to explain,
so I am not sure they fully understood this.

Best,
Bernhard


On Thursday 12 May 2005 03:03, Steffen Hansen wrote:
> On Thursday 12 May 2005 02:31, Martin Konold wrote:
> > Am Mittwoch, 11. Mai 2005 11:57 schrieb Steffen Hansen:
> >
> > Hi Steffen,
> >
> > > There has never been talk about making the php script a
> > > disconnected imap client (which it essentially is if it has a local
> > > cache of imap ID<->SID relations). I'd rather not implement this
> > > because it is going to be a) error-prone, and b) quite an effort.
> > > And it doesn't solve the problem for the webclient, which is still
> > > online imap and faces exactly the same trouble as the invitation
> > > handling script.
> >
> > Actually there was a _lenghty_ discussion with the webclient
> > developers about exactly this problem. It is a misunderstanding that
> > maintaining a Imap ID <-> SID map means implementing a dimap client.
> > It only means to have a simple _cache_ semantic on the client side.
> > Of course online imap clients use in-memory or on-disk storage of
> > temporary data in order to speed up operations all the time.
>
> This cache stores message content (partial message content, the SID to
> be exact) and needs to be kept in sync with the actual messages on the
> server -- this is exactly the disconnected imap sync algorithm. Maybe
> some details like subfolder sync etc. can be left out, but all in all
> it's the same.
>
> > Due to the simple fact that php based scripts are restarted over and
> > over again using on-disk storage for caching the Imap ID <-> SID map
> > is much more efficient than rebuling this structure in memory over
> > and over again.
>
> It doesn't build any structure over and over again. It uses the cached
> freebusy list to get an overview of the calendar folder. Freebusy
> creation could be faster, but that's another topic...
>
> > > > Thus causes significant load on the server for large
> > > > installations.
> > >
> > > Can we agree that it can at most cause a linear traversal of the
> > > metadata for a folder? This is no harder for the server than a
> > > sync.
> >
> > No, we can't aggree on that as the load for doing searches on the
> > custom mail headers is much more expensive than a sync.
>
> Custom mail headers? I am only searching on the SUBJECT -- and that one
> is in cyrus' header cache, so it is definitely faster to let the server
> traverse it's cache and return a single item compared to a sync that
> requests a list of items from the same cache. For the search to be
> slower than a sync, either scanning the cache or the string-compare
> function has to have worse than linear complexity. I doubt that is the
> case :-)
>
> regards
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2145 bytes
Desc: signature
URL: <http://lists.kolab.org/pipermail/format/attachments/20050513/dbe40c6d/attachment.p7s>