[Kolab-devel] Kolab2 architecture/format flaws

[Bernhard Reiter]

On Saturday 14 May 2005 01:23, Zachariah Mully wrote:
> On Fri, 2005-05-13 at 17:41 +0200, Bernhard Reiter wrote:

> Assuming that Cyrus would be able to handle the queries in a reasonable
> time if the XML was exposed to its native caching and searching, adding
> yet another set of caches to maintain is excess complexity. 

This depends on the viewpoint, from the server doing more is
adding more complexity. This is another requirement on the server.
We need to enchance cyrus imap in a couple of places,
but in principle an IMAP server with ACLs should still be close
for the Kolab Concept to use.

> I've pinged
> the Cyrus list to see if there is an easy to expose the Kolab data, as
> it stands, to the squatter. I'll let you know what results I get from
> that... I'm not hopeful though.

It should be fairly easy then even for us to switch on indexing on 
another mime-type, if they can already do text/xml.

> I'm not sure that you're losing any functionality, nor are you
> optimizing for "poor" clients. You're simply leveraging the built-in
> functionality of your storage backend. The "optimization" is simply
> switching to a mime-type that is understandable as regular text. The
> true optimizations take place in the webclient code, not via any special
> jiggering of the IMAP server.
>
> > Yes, each unnecessary  IMAP search is suboptimal.
>
> In what way? With properly implemented indexes ala Dovecot and Cyrus,
> it's fast and completely transparent to client.

It adds load and requirements to the server 
that could be handled on the clients.

> > > > As written above, we cannot just use any mime type.
>
> Why not? Since the IMAP store should be completely transparent to the
> user through whatever client, why does it matter as long as the
> mime-type is distinguishable from the text/plain part of the message.

Keeping the stored emails being real emails (RFC2822 and other MIME
standards applying) seems reasonable to me.

> > The MIME type in general containts information about the contents of
> > the message, like application/pgp-signature says, we now get a signature.
> > So I think we wanted to give the information that this attachment is in
> > the Kolab 2 XML format. It is arguable if we could have used a different
> > one to carry this information. rfc2046.txt has "text" as textual
> > information and "application" does fit the description much better of our
> > Kolab objects.
>
> What if the spec was changed to "text/x-vnd.kolab.XYZ" with the
> requirement that the object is stored as plain text, would that
> negatively affect the fat clients?

In my reading of rfc2046 that would not be correct MIME.
And it should be quite easy to convince cyrus imap to do its
caching on some more content types.
Beside text/xml certainly can also be encoded with quoted-printable
or base64. If you think international character sets, this seems reaonable to me.

> > Sure, but it breaks a format standard of the mail message,
> > we would redefine MIME and we would need a very strong reason to do this.
>
> I don't see how specifying a different mime-type (such as suggested
> above) would break MIME as nothing but Kolab clients should be reading
> that data, and even then it's arbitrary as to the MIME chosen. As you've
> pointed out, you're not using any IMAP caching or searching on that
> data, so what the heck does it matter? Presently nothing but the client
> actually touches the mime data or even needs to know about it...

Any email client can display the stored emails just fine,
any library that can handle email, can handle it fine.
Why should be give this up?

> > Kolab-format and kolab-devel if my memory is working allright.
>
> I can't find anything in the list archives, if you've got a link, I'd
> love it.

It is probably burried under a different topic.
Martin Konold would also know.

> > It is possible to have a good webclient, but it has not happened
> > yet, because no one in the community did it.
> > I fail to see why this is shortsighted in any way. :-)
>
> I think I was unclear... I think it's odd to pick a storage backend that
> you don't use any of it's built-in optimizations, 

You have got to note that the current Kolab-Server based on Cyrus imap
is just one implementation of the Kolab Concept, we do not want to depend
on cyrus imap as much as we can.

> and then pick a
> storage format that prevents the backend from even reading the contents
> of...

Again: This must be easy to enable, though because of the load question,
we want to encourage clients to cache _some_ things.

> > We could design and implement a web client,
> > but with offline usage and some security thinging,
> > most people do not need it. So we are not limited to it
> > in the long run.
>
> But a webclient insures that regardless of the support for a particular
> platform or client, the users will always have access to their data, and
> will be able to collaborate with their collegues regardless of what OS,
> webbrowser, or where they are in the world. For instance, we need it
> here as we can't afford the license costs of Outlook+Toltec for every
> desktop. Nor can we move these people to Linux to use the KDE. 

What about using a "beamed" windows of Kontact.

> And those
> already on Linux don't want to use the KDE client because no one likes it.

Hopefully more Kolab clients will appear
and this is a matter of taste, they might not like your webclient either. :,) 

> And I also have 20 freelancers to support... Which I would rather
> support their use of a webapp because I don't have physical control of
> their machines and cannot guarantee a quality of service to them.

You could give them the KDE or Outlook/Toltec client then,
it might server them better in a few ways, e.g. they do not need to be online.

> This is about web access to their scheduling data,
> address data, etc. Besides, any webclient is going to have the same
> problems with the Kolab XML as the current Horde one does.

I am no expert about Horde's design, but what I have gathered it is geared
towards having a database in the background. There is a good chance
that the design of other web client is different.

> > I believe that our current design decision does not pose a limitation.
> > They make client development not as easy as a simple fat server design,
> > but we gain advantages out of this.
>
> I'm not asking that you cripple your current implementations, but that
> you open up the currently implement formats, at no penalty to your
> clients,  so that other clients can be developed in an efficient manner.

I am merely explaining the motivation, having many badly implemented
clients is not as nice as getting better ones, of course.

> > Actually there is. A GPRS connection can be fully sufficient
> > if the client does the disconnected imap mode nicely.
>
> I don't know of any mobile IMAP clients that have the ability to store
> 10s of MB of data on the device... Because they don't have that much
> storage usually and because I'd rather not try and sync 25MB of data
> over a 56kpbs connection, simply to display today's appointments. If you
> look at the RIM Blackberry model, they only sync X days worth of data to
> the device... NOT the entire store.

I do not think it is necessary to sync huge amounts of data.

> > We can have optimisations for tiny clients on _a_ server, but this server
> > would be like a client to the Kolab Server.
>
> I admit that my whole argument hinges on the Cyrus' built-in searching
> and caching. I'm still testing this and I'm still trying to figure out
> if it's able to index the Kolab objects without any changes. Should this
> *not* be the case, then you are totally right, some sort of caching will
> be absolutely necessary for the webclient.

If the only change is the mime-type, then this should be easily resolvable 
within Cyrus.

Best,

Bernhard
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2145 bytes
Desc: signature
URL: <http://lists.kolab.org/pipermail/format/attachments/20050518/f483ec42/attachment.p7s>