[Kolab-devel] Kolab 3.0: Mime Message Storage

[Christian Mollekopf]

On Thursday 17 May 2012 10.55:25 Jeroen van Meeuwen wrote:
> On 2012-05-17 2:27, Christian Mollekopf wrote:
> > On Wednesday 16 May 2012 21.58:07 Jeroen van Meeuwen wrote:
> >> On Wednesday, May 16, 2012 09:58:57 PM Christian Mollekopf wrote:
> >> > On Wednesday 16 May 2012 16.55:24 Jeroen van Meeuwen wrote:
> >> > > As to the X-Kolab-* headers;
> >> > > 
> >> > > X-Kolab-Type is a duplicate of the Content-Type header to the
> >> > > attachment. I don't really see what purpose this serves, as in
> >> 
> >> addition to the
> >> 
> >> > > duplication, the object is also only supposed to be stored in a
> >> 
> >> folder
> >> 
> >> > > marked as containing rfc822 encapsulated payload of a particular
> >> 
> >> type.
> >> 
> >> > The Content-Type header is the same for all xcal/xcard headers and
> >> 
> >> doesn't
> >> 
> >> > give us this differentiation. And doing that by the xml is painful
> >> 
> >> to say
> >> 
> >> > the least.
> >> > 
> >> > (...snip...)
> >> > 
> >> > Generally I would rather keep it, as it IMO is a useful hint for
> >> 
> >> client
> >> 
> >> > implementors which don't have the foldertype available right away
> >> 
> >> and
> >> 
> >> > allows to double check the type. Also I'm not that fond of relying
> >> 
> >> on the
> >> 
> >> > folder type to "guess" the kolab object type.
> >> 
> >> I understand where it comes from, but I doubt whether it's needed.
> >> 
> >> When a client without any (current) caches opens up a folder that
> >> contains
> >> the same MIME content-type attachments, it does need to read all
> >> attachments in order to get to the display name, summary, duration
> >> and
> >> whatnot, right? What use does a X-Kolab-Type header have if all
> >> attachments
> >> need to be read regardless?
> > 
> > It contains the same Kolab-Objects stored as MIME message. Of each
> > message it only needs to read the xml attachment. If you don't have
> > the X-Kolab-Type
> > header you need to rely on the folder, which you're currently
> > processing to
> > know what object to expect.
> > 
> > That is an unfortunate implementation detail, as ideally it would be
> > possible to just get the Content-Type of the attachment, see it is
> > "application/calendar+xml" and call readxCal(), which returns an
> > event/todo/journal. But instead we have to call readEvent, readTodo,
> > readJournal explicitly, which is what we need the exact type for.
> > 
> >> Do we have clients that only ever want the distribution lists out of
> >> a set
> >> of contacts, or is it merely a step client applications have gotten
> >> used to
> >> making to separately query for all distribution lists and then
> >> separately
> >> query for all (individual) contacts?
> > 
> > I'm not aware of such clients. But for contacts/dist-lists it's
> > actually not
> > even possible to rely on the folder type, so we need the X-Kolab-Type
> > definitely.
> 
> I suppose this last point on contacts/dist-lists is very much the same
> point as event/todo/task/journal, but applicable to contacts and
> distribution lists specifically as they are supposed to be contained in
> the same folder(s)?
> 
> So, solely by virtue of not being able to read in some XML, detect its
> type and act accordingly, much like there's not a single write function,
> but only type-specific write functions, we need to duplicate information
> and add headers to all Kolab objects?
> 
> What happens if, for who knows what reason, an email lists one type in
> the header, but has another (a slightly different?) type in in the
> attachment?
>

We detect the error that the xml is not as it should be during the xml schema 
based validation and set the error flag (it's not crashing or anything like 
that).

> I think it is very beneficial if we had something along the lines of:
> 
>   $object = read($xml[, $content_type])
> 
>   assert(isinstance($object, $assumed_object_type))
> 
> for this and a variety of other reasons. In fact, $object->type() could
> return some useful information at this point. Other then "difficulty",
> is there a reason this cannot technically be done?
> 

It is of course technically possible, though not necessarily with the current 
implementation approach (xsd bindings). Otherwise I would of course have 
implemented it that way ;-)

Bottomline is, while there is *some* duplication due to X-Kolab-Type, I don't 
see any problem with it, and it is IMO definitely cleaner than relying on the 
folder type. In other words, as long as we can't easily detect the type of the 
xml object, we require this property.
Technically I can detect the property, but I would have to parse the xml first 
by hand (not using the xsd bindings), and I wouldn't really see the point of 
that exercise.

> >> > > X-Kolab-Version is a duplicate of the version contained within
> >> 
> >> the XML
> >> 
> >> > > and should not be used.
> >> > 
> >> > It is not IMO, as it specifies the format of the mime message
> >> 
> >> format. I'm
> >> 
> >> > in favor of keeping it.
> >> 
> >> In that case, let's make it the full $x.$y.$z, so various version
> >> comparison
> >> routines can parse it similarly to how they (could) parse the
> >> $x.$y.$z in
> >> the XML, as opposed to an arbitrary string.
> >> 
> >> Let's suppose we don't allow the mime message format to be changed
> >> between
> >> teeny releases; only $x.$y then needs to be stored.
> >> 
> >> Also, wouldn't the version of libkolabmime be more appropriate to
> >> use here?
> > 
> > That is libkolab right now, which is stored already as User-Agent. I
> > think we need the version number of the specification there (I
> > started with
> > 3.0, as the Kolab Format 2.0 specification included the Mime Format
> > specification).
> 
> I think we have to consider the MIME structure of a message is going to
> move forward significantly slower than libkolab, and we don't want
> clients to go nuts over a version they think they may be incompatible
> with while the MIME structure hasn't actually changed.
> 

Yes, that is why we store the MIME structure version in X-Kolab-MIME-Version, 
which is then to be checked. The User-Agent header is more for debugging 
purposes, to figure out which piece of (crappy) software created the broken 
MIME message we just found on the IMAP server.

> >> > > ===
> >> > > 
> >> > > """The following headers/options MUST be used:"""
> >> > > 
> >> > > I made this a separate bullet point, and second, again I see no
> >> 
> >> reason
> >> 
> >> > > to
> >> > > restrict the encoding to quoted-printable, nor a reason to
> >> 
> >> restrict the
> >> 
> >> > > filename to kolab.xml.
> >> > 
> >> > IMO there is no reason to not enforce the encoding, and as every
> >> 
> >> client
> >> 
> >> > must be able to read all supported encoding types, I'd rather just
> >> 
> >> use
> >> 
> >> > one. I don't really care about the filename, but see no reason not
> >> 
> >> to
> >> 
> >> > just specify one, which keeps it at least consistent.
> >> 
> >> Same as the above for the encoding. The filename question is more of
> >> a "save
> >> all to disk" thing to be honest, and unless everything's hard coded
> >> to look
> >> for a particular Kolab XML attachment by detecting which attachment
> >> has
> >> filename "kolab.xml", it makes little sense to specify and restrict.
> > 
> > Agreed for the filename. For the encoding there is the same reason to
> > only
> > allow one encoding as there is for writing KEP#17 (firmer
> > specification that is
> > implementable).
> 
> I see... but I (partly) disagree. Not sure to what extent yet, though
> 
> :P
> 
> First the encoding of the attachment is restricted to quoted-printable,
> then the encoding of the XML attachment itself is set to utf-8.
> 
> Honestly, mail clients require some level of compatibility already,
> including a variety of encodings, such as iso-8859-1 or what is it
> again, and it doesn't really seem impossible to just allow "whatever is
> most convenient for the client writing it out".
> 
> That is to say, libkolabmime / libkolab could of course just only ever
> writing out quoted-printable. That's not the problem. I just think it's
> best to not restrict what anything else might write out and have
> libkolabmime / libkolab act accordingly upon reading the MIME attachment
> - even though part of the exercise is to have "our clients" use
> libkolab*.
> 

My point is, that if we allow *any* encoding, we have to support *any* 
encoding in libkolab, and so does every other kolab client, which is in fact 
impossible, unless there are some restrictions on the allowed encodings in the 
MIME RFC. Given that a kolab client needs to make that decision anyways (in 
which encoding to write out the xml), I don't see why it couldn't just use 
quoted-printable, unless the given platform would indeed lack an 
implementation of quoted-printable and implementing it would be too 
cumbersome, which is unlikely given its simplicity. So overall I just see 
allowing multiple encodings complicating matters without any benefits in 
return.

I do see your argument about using utf-8 for the xml and then using quoted 
printable to encode it again. We can allow utf-8 and quoted printable, if you 
think it's worth the extra effort, but I'm really not in favor of allowing just 
everything, because that just means we have to implement every encoding some 
client implements because we don't comply to our own spec otherwise.

Cheers,
Christian

> Kind regards,
> 
> Jeroen van Meeuwen
-- 
Christian Mollekopf
Software Engineer

Kolab Systems AG
Zürich, Switzerland

e: mollekopf at kolabsys.com
w: http://kolabsys.com

pgp: EA657400 Christian Mollekopf
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part.
URL: <http://lists.kolab.org/pipermail/devel/attachments/20120517/ce9854cf/attachment.sig>