[Kolab-devel] Kolab2 architecture/format flaws

[Bernhard Reiter]

Hello Zach,

On Friday 13 May 2005 16:41, Zachariah Mully wrote:
> On Fri, 2005-05-13 at 15:22 +0200, Bernhard Reiter wrote:

> Thanks for the response, I apologize for being late to the game with my
> comments...

no problem. Our goal is a good design and it is never to late to improve it,
even when this might take a while until a new version is build.
[ As  side note: We should agree upon which list we want to discuss this,
but about continuing on kolab-format only?]

> > So any client that does not cache the connection of imap id and data it
> > needs from them, is a suboptimal implementation.
>
> Unless, of course, you need to implement a thin client. Then this design
> decision has the exact opposite affect, it severely cripples an
> implementations where heavy use of disk based caching is not available.

No, I said some caching, it does not need to be a full cache,
even a slim cache can do the job. A list of appointments times and their
imap ids will be say 10 bytes *  number of appointsments.
Another possibility for thin clients is decide on the level of what to beam
there (thin client usually get applications from somewhere else).
You could beam Kontact on the clients and have the cache on the application server.

> And to argue your point, you lose no functionality or scalability by
> exposing the Kolab object contents to the IMAP server. Fat clients can
> continue to maintain disk caches, and thin clients can exploit the
> native caching and searching that Cyrus offers. 

Each "thin" client will then pose more load on the server,
and while you are technically right in that there is no inherit problem
with making the contents available, we saw the psychological danger
that we only get "poor" clients, if we optimise for this case.
As there are strategies to implement web or other clients,
no one felt the need, which explain why we did not do this.

> Even fat IMAP clients
> utilize these features, are they too suboptimal implementations?

Yes, each unnecessary  IMAP search is suboptimal.

> > > > Any chance to lobby for a change from the kolab
> > > > specific mime-type to text/xml for instance?
> >
> > As written above, we cannot just use any mime type.
>
> This I don't understand... The mime-type has zero to do with the
> contents of the message,
>  and is replicated in the header of the message.
> Unless I'm totally missing something, it was chosen arbitrarily. If it
> was to allow mixed use mailboxes, and allow the client to recognize
> different types of Kolab objects, the header is there and far more
> easily parsable than the entire mime-body.

The MIME type in general containts information about the contents of
the message, like application/pgp-signature says, we now get a signature.
So I think we wanted to give the information that this attachment is in the 
Kolab 2 XML format. It is arguable if we could have used a different one
to carry this information. rfc2046.txt has "text" as textual information
and "application" does fit the description much better of our Kolab objects.

> > > > 3) I noticed that the Toltec connector base64 encodes the Kolab
> > > > objects making any type of searching via the IMAP server impossible,
> > > > is there a specific reason for this?
> >
> > That is a question of implementation you
> > have to ask Joon about., the spec specifically does not limit the
> > encodings. And as explained above, the design does not want to rely on
> > deep server searches.
>
> Again, I have to argue that you're restricting functionality for no good
> reason, other than semantics. 

You have not heard Joon's reasons yet, there might be technical reasons
I just did not list. 
I only said that we did not see a reason to deviate
from using standard RFC2822 format in the emails.

> I realize that it's your opinion that
> client should never have to rely on the server backend to filter the
> returned messages, but enabling that functionality is not exclusive of
> maintaining your primary design theory.

Sure, but it breaks a format standard of the mail message,
we would redefine MIME and we would need a very strong reason to do this.

> > > > 4) Without format changes, I consider the possibility of a workable
> > > > *scalable* webclient, in any form, dead in Kolab2.
> >
> > Why?  There are several ways to implement a scalable web clients
> > as was discussed in depth a while ago on the lists.
>
> ?? Which list? I want to go back and read the discussions. 

Kolab-format and kolab-devel if my memory is working allright.

> I am not sure
> how you can design a scalable thin client when it has to fetch and parse
> the entire calendar store for *any* operation.

You don't need to, one simple example:
You cache the imap id for each appointment and start and end date
of its activity (this means first date and last date with recurring events).
When you want to show a specific day, you know which imap objects to fetch,
so there is no need to parse the entire calendar.

> The only way I can
> envision doing it is to replicate all that data into an sql database,
> which would allow fast searches for specific date ranges. But that would
> relying on a deep server search, no?

No, this looks like another way to do the local cache.
Many indices will be much smaller, so you can keep them in memory.

> > > > Was a robust webclient ever part of the plan for Kolab2?
> >
> > No, as Proko2 did not require and and nobody put of the effort
> > or the money to have one. This makes some sense as web clients
> > are only rarely needed with good native clients as in any reasonable
> > secure environment you need to touch the client installation anyway and
> > then you could as well install a native client which better usability.
>
> I understand that this is part of your contract, 
> and I am very
> appreciative of the work and code that you've contributed to the
> community... but this is extremely shortsighted. 

Proko2 is a contract, but we all are the Kolab community.
What creates the community is our common goals,
but as always everybody participates for a reason.
It is possible to have a good webclient, but it has not happened
yet, because no one in the community did it.
I fail to see why this is shortsighted in any way. :-)

> You've limited yourself to desktop only deployments, 

We could design and implement a web client,
but with offline usage and some security thinging,
most people do not need it. So we are not limited to it
in the long run.

> and offered nobody a means to access their
> data from an web terminal somewhere on the road. 

Any sensible it-security person should forbit this,
as you need to trust the machine which handles the connection.
PDAs or a laptop are much better.

> Most the salespeople I
> work with, make heavy use of public terminals to check mail through our
> webinterface as it's far more convenient and faster to sit down, login,
> check and leave, than it is to fish their laptop out, plug it in, find a
> network to connect to ....

It is unsecure, but if you like, you can of course attach any imap webclient
for emails to Kolab and we can build a scalable webclient. 
Any contributions to it are welcome.

> So I suppose what I'm worried about is that this design decision
> severely hinders the development of any viable webclient, or any other
> thin clients (mobile devices, for instance, 

I believe that our current design decision does not pose a limitation.
They make client development not as easy as a simple fat server design,
but we gain advantages out of this.

> no eff'in way I'm syncing
> the entire calendar store over a GPRS connection),

Actually there is. A GPRS connection can be fully sufficient
if the client does the disconnected imap mode nicely.

> and it has no bearing
> on the development or design decisions of fat clients. You're still free
> to use disk caches, but you open up a whole host of server-side
> optimizations for thin clients.

We can have optimisations for tiny clients on _a_ server, but this server
would be like a client to the Kolab Server.

Best,
Bernhard
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2145 bytes
Desc: signature
URL: <http://lists.kolab.org/pipermail/users/attachments/20050513/0796e466/attachment.p7s>