[Kolab-devel] Kolab2 architecture/format flaws

Zachariah Mully zmully-kolab at smartbrief.com
Sat May 14 01:23:01 CEST 2005

On Fri, 2005-05-13 at 17:41 +0200, Bernhard Reiter wrote:
> Hello Zach,
> no problem. Our goal is a good design and it is never to late to improve it,
> even when this might take a while until a new version is build.
> [ As  side note: We should agree upon which list we want to discuss this,
> but about continuing on kolab-format only?]


> No, I said some caching, it does not need to be a full cache,
> even a slim cache can do the job. A list of appointments times and their
> imap ids will be say 10 bytes *  number of appointsments.
> Another possibility for thin clients is decide on the level of what to beam
> there (thin client usually get applications from somewhere else).
> You could beam Kontact on the clients and have the cache on the application server.

Assuming that Cyrus would be able to handle the queries in a reasonable
time if the XML was exposed to its native caching and searching, adding
yet another set of caches to maintain is excess complexity. I've pinged
the Cyrus list to see if there is an easy to expose the Kolab data, as
it stands, to the squatter. I'll let you know what results I get from
that... I'm not hopeful though.

> Each "thin" client will then pose more load on the server,
> and while you are technically right in that there is no inherit problem
> with making the contents available, we saw the psychological danger
> that we only get "poor" clients, if we optimise for this case.
> As there are strategies to implement web or other clients,
> no one felt the need, which explain why we did not do this.

I'm not sure that you're losing any functionality, nor are you
optimizing for "poor" clients. You're simply leveraging the built-in
functionality of your storage backend. The "optimization" is simply
switching to a mime-type that is understandable as regular text. The
true optimizations take place in the webclient code, not via any special
jiggering of the IMAP server.

> Yes, each unnecessary  IMAP search is suboptimal.

In what way? With properly implemented indexes ala Dovecot and Cyrus,
it's fast and completely transparent to client.

> > > As written above, we cannot just use any mime type.

Why not? Since the IMAP store should be completely transparent to the
user through whatever client, why does it matter as long as the
mime-type is distinguishable from the text/plain part of the message?

> The MIME type in general containts information about the contents of
> the message, like application/pgp-signature says, we now get a signature.
> So I think we wanted to give the information that this attachment is in the 
> Kolab 2 XML format. It is arguable if we could have used a different one
> to carry this information. rfc2046.txt has "text" as textual information
> and "application" does fit the description much better of our Kolab objects.

What if the spec was changed to "text/x-vnd.kolab.XYZ" with the
requirement that the object is stored as plain text, would that
negatively affect the fat clients? 

> Sure, but it breaks a format standard of the mail message,
> we would redefine MIME and we would need a very strong reason to do this.

I don't see how specifying a different mime-type (such as suggested
above) would break MIME as nothing but Kolab clients should be reading
that data, and even then it's arbitrary as to the MIME chosen. As you've
pointed out, you're not using any IMAP caching or searching on that
data, so what the heck does it matter? Presently nothing but the client
actually touches the mime data or even needs to know about it...

> Kolab-format and kolab-devel if my memory is working allright.

I can't find anything in the list archives, if you've got a link, I'd
love it.

> You don't need to, one simple example:
> You cache the imap id for each appointment and start and end date
> of its activity (this means first date and last date with recurring events).
> When you want to show a specific day, you know which imap objects to fetch,
> so there is no need to parse the entire calendar.

It looks like this might be the only way... And if I'm going to mix the
Kronolith sql driver with the kolab driver, I might as well cache the
entire message in the db... No need to do multiple fetch ops.

> It is possible to have a good webclient, but it has not happened
> yet, because no one in the community did it.
> I fail to see why this is shortsighted in any way. :-)

I think I was unclear... I think it's odd to pick a storage backend that
you don't use any of it's built-in optimizations, and then pick a
storage format that prevents the backend from even reading the contents

> We could design and implement a web client,
> but with offline usage and some security thinging,
> most people do not need it. So we are not limited to it
> in the long run.

But a webclient insures that regardless of the support for a particular
platform or client, the users will always have access to their data, and
will be able to collaborate with their collegues regardless of what OS,
webbrowser, or where they are in the world. For instance, we need it
here as we can't afford the license costs of Outlook+Toltec for every
desktop. Nor can we move these people to Linux to use the KDE. And those
already on Linux don't want to use the KDE client because no one likes
it. And I also have 20 freelancers to support... Which I would rather
support their use of a webapp because I don't have physical control of
their machines and cannot guarantee a quality of service to them.

> It is unsecure, but if you like, you can of course attach any imap webclient
> for emails to Kolab and we can build a scalable webclient. 
> Any contributions to it are welcome.

Access to email isn't a problem, there are any number of webmail clients
I could install... This is about web access to their scheduling data,
address data, etc. Besides, any webclient is going to have the same
problems with the Kolab XML as the current Horde one does.

> I believe that our current design decision does not pose a limitation.
> They make client development not as easy as a simple fat server design,
> but we gain advantages out of this.

I'm not asking that you cripple your current implementations, but that
you open up the currently implement formats, at no penalty to your
clients,  so that other clients can be developed in an efficient manner.

> Actually there is. A GPRS connection can be fully sufficient
> if the client does the disconnected imap mode nicely.

I don't know of any mobile IMAP clients that have the ability to store
10s of MB of data on the device... Because they don't have that much
storage usually and because I'd rather not try and sync 25MB of data
over a 56kpbs connection, simply to display today's appointments. If you
look at the RIM Blackberry model, they only sync X days worth of data to
the device... NOT the entire store.

> We can have optimisations for tiny clients on _a_ server, but this server
> would be like a client to the Kolab Server.

I admit that my whole argument hinges on the Cyrus' built-in searching
and caching. I'm still testing this and I'm still trying to figure out
if it's able to index the Kolab objects without any changes. Should this
*not* be the case, then you are totally right, some sort of caching will
be absolutely necessary for the webclient.


More information about the format mailing list