[Kolab-devel] "questions and critical comments"

Fri Jun 4 18:15:07 CEST 2010

Quoting "Mailingliste TBits.net GmbH" <mailinglists at tbits.net>:

> [quote=Gunnar Wrobel]
>      Can you do you me a favor and pipe as many questions and  
> critical comments
>      as possible concerning the Kolab_* libraries to this channel?
> [/quote]
>
> Perhaps the greatest criticism I have is also the most obvious;
> however I'm aware you will not heed this advice unless you actually
> *do* throw *everything* away and start over from scratch (which you
> should): The whole IMAP storage thing. Seriously? Storing
> contacts/tasks/etc. as XMLs attached to mail? Please! Just imagine
> having a module that requires lookups by anything other than UID
> and/or joins (in SQL parlance). (And at least now, you do; anything
> more complicated than Mnemo will eventually do this...) This will
> quickly give you O(n^2) runtime complexity or worse.

Of course I won't heed that advice :) It is a core Kolab concept and  
one I do like. I accept that people consider it to be a strange idea,  
though.

Most of the times the IMAP storage concept is compared to a database  
solution. You do the same here and I know that it is an obvious  
comparison. Most other groupware server vendors also use a database.

This comparison does not work however. A SQL database is about storing  
and accessing data in an efficient way. The Kolab storage concept *is  
solely concerned with storing data*. Nothing at all will prevent your  
client from reading the data from IMAP and caching it in a database  
for efficient access. This is really considered to be the job of the  
Kolab clients. They are required to handle the data on their side in a  
way that Kolab data access is convenient for the end user.

Whether Horde uses an efficient approach in that area (serialized data  
in files) can certainly be debated. But I'm certain the Kolab server  
offers applications that are far more complex than mnemo and remain  
usable nonetheless. The free/busy code is one example. It efficiently  
aggregates event data over several users and calendars and does this  
very quickly.

I'm not 100% certain what the original idea behind storing the  
groupware data in IMAP was. But I do know what I like about it: The  
user is able to control his own data. IMAP is a very general protocol  
and people are used to handle their data in IMAP. There are many  
different clients available and it is not uncommon to change your IMAP  
server provider. You just don't do that with an SQL database. Look at  
all the web 2.0 providers: you never have direct access to their  
storage system / SQL server even though it holds your personal data.

Whether that is really relevant in the future remains to be seen. But  
concerns about how data is stored by the different providers seem to  
be rising.

>
> I say, when (and note my use of 'when' rather than 'if') you redo
> everything from scratch, I advise you ditch the level of indirection
> that is OpenLDAP and dump everything (including the 'groupware
> objects') right into an embedded database (BerkeleyDB or somesuch,
> SQLite if you feel nostalgic) and replicate it right back to the
> clients, which can then work with their local copy of the database for
> offline availability. No servers needed, nothing.

Then I could also program for Open-Xchange or Zarafa or the like.  
Which I won't as I know why I chose Kolab when I started entering the  
groupware server world a few years ago.

>
> Next, I feel compelled to mention that the parts I've seen are quite
> broadly distributed across like two dozen files and classes -- not a
> bad thing per se (if done right, quite on the contrary), but since
> *none* of it is properly documented (or at all) someone looking for a
> problem will be lost in the depths forever. If you want people
> developing stuff for it, you should *seriously* consider giving out
> API docs for your stuff.

Agreed. Documentation is sparse in Horde land and also for the Kolab  
specific PHP code. I started documenting the libraries in the wiki  
once but I admit that this is really not enough. Even after four years  
of working on these packages they are still not where I want them to  
be and most of them really only work in Horde context. Which is pretty  
bad and makes using the libraries hard. Still it is no reason for not  
documenting them better, I know.

> Also, do some runtime checks so bad data
> (notably malformed XML) doesn't cause PHP to die and/or spit out
> incomprehensible messages; I've run into this more than once during
> development and ended up wiping the IMAP account because testing had
> driven the storage logic into a corner.

Kolab_Format is the package that has been the most stable one in the  
past years :) I agree it could have some more safeties and could  
probably use some refactoring but I think that will come later. At the  
moment Kolab_Storage is still a more important target for me.

>
> It also seems that the caching dealie is somewhat buggy, on more than
> one occasion I created an object, then had to log out and back in
> before it actually showed up in the list -- other times it worked
> without doing anything. Curious.

Kolab_Storage is currently in a major refactoring and I'm pretty  
certain the Horde 3 library still has a few significant bugs.

>
> Of course, you already know that requiring people to modify both the
> Horde *and* Kolab code when adding new modules (or rather, storage
> formats) is not the best of ideas. One better way would be doing what
> Horde itsself does quite frequently: Dumping definitions into separate
> files and include()ing *.php there.
>
> As for questions, what I always wondered: How can you assign a storage
> format (i.e. one from $HORDE/lib/Horde/Kolab/Format/XML) to a folder
> (i.e. "share")? My modules often have more than one such format and I
> routinely used the union of all of them as "the" format; surely there
> is a better way?

This has to be done by the application itself. You can specify the  
data type you want to retrieve when using the getData() method on a  
folder object (e.g. "$data = &$this->_folder->getData('jobtype');".  
None of the base applications do this however. I only use this in an  
experimental patch to the horde time tracking application.

>
> Also, I've seen that even the latest (release) version of Kolab uses a
> Kolab::_storage of Kolab_Storage_Deprecated -- does that mean there is
> some brand-new system being developed that will kick in shortly? Just
> curious... Funnily enough, the other modules (and, since "copying"
> code from them is the only form of docs available, mine as well) seem
> to happily access aforementioned field in their storage driver, even
> though the underscore suggests that this should not be done. Is there
> any more canonical way?

The "Kolab_Storage_Deprecated" thing just represents a change of the  
API. The background code is the same being used now but I had to keep  
the outside API stable until Horde 4. The access to the folders and  
data won't really change that much in Horde 4.

>
> Those are the ones I can think of for now.

Cool, I could have imagined worse comments :) If I compare your  
critical remarks to what you actually did in code I'm outright happy.  
You did create four completely new Kolab client applications in a very  
short amount of time. To my knowledge nobody did so before for any  
other Kolab client. If the structure Horde and the Kolab_* libraries  
have allowed you to do that then this means I did really reach one of  
my goals.

Of course this could be much easier. Of course a lot of the code I  
produced is crap and if you would have been in the same room with me  
while you were coding your apps you would certainly have hit me a few  
time for the one or the other horrible design choice in the libraries :D

But the simple fact that you were able to code these four apps really  
makes me happy :) Now I hope I find the time to get them into four  
packages we can offer as extensions for the Kolab server. And write  
some API docs ;)

Cheers,

Gunnar

>
> Sincerely,
> Simon Bausch
>
>
> ----------------------------------------------------------------
> Diese Nachricht wurde versandt mit Webmail von www.tbits.net.
> This message was sent using webmail of www.tbits.net.
>
> _______________________________________________
> Kolab-devel mailing list
> Kolab-devel at kolab.org
> https://kolab.org/mailman/listinfo/kolab-devel
>

-- 
______ http://kdab.com _______________ http://kolab-konsortium.com _

p at rdus Kolab work is funded in part by KDAB and the Kolab Konsortium

____ http://www.pardus.de _________________ http://gunnarwrobel.de _
E-mail : p at rdus.de                                 Dr. Gunnar Wrobel
Tel.   : +49 700 6245 0000                          Bundesstrasse 29
Fax    : +49 721 1513 52322                          D-20146 Hamburg
--------------------------------------------------------------------
    >> Mail at ease - Rent a kolab groupware server at p at rdus <<
--------------------------------------------------------------------

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: Digitale PGP-Unterschrift
URL: <http://lists.kolab.org/pipermail/devel/attachments/20100604/6c53bb28/attachment.sig>