[Kolab-devel] Kolab 3.0: Mime Message Storage

[Jeroen van Meeuwen]

On Wednesday, May 16, 2012 09:58:57 PM Christian Mollekopf wrote:
> On Wednesday 16 May 2012 16.55:24 Jeroen van Meeuwen wrote:
> > I suggest re-phrasing the entire section so that it is clear that;
> > 
> > - All folders with Kolab Groupware content MUST be annotated, while
> > folders
> > with mail contents MAY exist but without metadata, it is assumed folders
> > without metadata are mail folders, and clients SHOULD set metadata for
> > folders that have some special sort of mail (Drafts, Trash, Sent, etc...).
> > 
> > - the difference between ANNOTATEMORE (the draft) and METADATA (the RFC)
> > doesn't matter for the verbiage of this part of the format definition,
> > 
> > - The shared namespace (/vendor/kolab/ value.shared in ANNOTATEMORE,
> > /shared/vendor/kolab/ in METADATA) is good for groupware types, while sub-
> > types are usually best set in the private namespace,
> > 
> > """All annotation MUST be set during the initial creation of a folder and
> > cannot be altered afterwards"""
> > 
> > They MUST be set alright, but SHOULD NOT (or even MUST NOT) be altered
> > afterwards. It's not like they "cannot be changed".
> > 
> > Also, consider that this would apply to the type only, and not (equally
> > well) to the sub-type.
> > 
> > Folders without any metadata MUST (not SHOULD) be treated as (not assumed
> > to be) mail folders.
> > 
> > ===
> > 
> > """All "default" Kolab-Folders MUST be located within the personal IMAP
> > namespace. """
> > 
> > I don't think that MUST holds true, and I don't think we need to restrict
> > '.default' sub-types to only ever be allowed in the personal IMAP
> > namespace.
> > 
> > What must be considered is any personal IMAP namespace '.default' folder
> > takes precendence over any other, and that there may only be one (or less,
> > depending on opinions on the aforementioned) such personal IMAP namespace
> > '.default' folder.
> > 
> > I would also consider adding that, if multiple .default folders exist,
> > either "in the personal namespace" or "none in personal but multiple in
> > the
> > shared namespace" clients SHOULD pretend nothing has been marked as
> > '.default' and behave accordingly (i.e. always pop up a "save to what
> > folder?" type of dialog?).
> 
> I changed the draft accordingly. If I understand correctly, that means the
> annotations are not really backwards compatible thought.
> 

It might not be backwards compatible, but nor is the format. If a client fails 
to update its routines wrt. folder metadata, it's not necessarily 
prohibitively incompatible.

> However, I agree that it makes sense to have the groupware type as shared
> annotations and the rest as private annotation, and now is the time to make
> the changes.
> 

It's been one of those things that had to be refined it its definition.

> We might want to also allow all annotations (also default, inbox, draft,
> etc.) as shared annotations, with private annotations taking precedence
> over shared ones. This would allow for defaults in the folders in the
> shared namespace.
> 

I'm not sure what a shared folder marked as INBOX establishes exactly, but 
'spam' and 'ham' come to mind as good use-cases for mail folder-type metadata 
in the shared namespace.

> > ===
> > 
> > """TODO: we probably should have a bit a firmer specification how multiple
> > folders SHOULD be interpreted for each type to get a consistent user
> > experience."""
> > 
> > This remark doesn't make much sense, perhaps you care to elaborate?
> 
> I mean we should maybe give a better specification/guideline how folders
> should be interpreted.
> 
> In the Akonadi Resource it's not that interesting because akondi allows us
> to map the imap folders 1:1 to the calendar folders. (We can have
> subfolders for calendars, notes, and addressbooks, and we can even create
> calendar folders for journals only).
> 
> I don't know if that is possible for all systems, because an ical calendar
> is typically for events, todos and journals. So it might be that not all
> systems provide such fine-grained control (especially when it comes to
> mobile devices), and could make use of some guidline. e.g. display all
> folders containing calendar items as a single calendar and store new items
> to the folder  which was marked as default.
> 
> But since it is very straight forward for our main client, I suppose we
> don't really need to think about that now and can just tackle it when we
> are facing the issue.
> 

Right... that doesn't make any more sense than it did already, sorry ;-)

Suppose we have one or more hierarchies of calendars, address books, 
notebooks, and such and so forth... which are the aspects of such hierarchies 
that could benefit from a better specification or set of guidelines, you 
reckon?

> > ===
> > 
> > As to the X-Kolab-* headers;
> > 
> > X-Kolab-Type is a duplicate of the Content-Type header to the attachment.
> > I
> > don't really see what purpose this serves, as in addition to the
> > duplication, the object is also only supposed to be stored in a folder
> > marked as containing rfc822 encapsulated payload of a particular type.
> 
> The Content-Type header is the same for all xcal/xcard headers and doesn't
> give us this differentiation. And doing that by the xml is painful to say
> the least.
> 
> Since we have separate folders for the content types, it would be fairly
> easy to change our implementation, it could be a problem for other clients
> though, depending on how the code is written. If we remove the X-Kolab-Type
> header it is also no longer to automatically convert a single mime message
> (without the context of the folder), without specifying the type manually.
> 
> Generally I would rather keep it, as it IMO is a useful hint for client
> implementors which don't have the foldertype available right away and allows
> to double check the type. Also I'm not that fond of relying on the folder
> type to "guess" the kolab object type.
> 

I understand where it comes from, but I doubt whether it's needed.

When a client without any (current) caches opens up a folder that contains the 
same MIME content-type attachments, it does need to read all attachments in 
order to get to the display name, summary, duration and whatnot, right? What 
use does a X-Kolab-Type header have if all attachments need to be read 
regardless?

Do we have clients that only ever want the distribution lists out of a set of 
contacts, or is it merely a step client applications have gotten used to 
making to separately query for all distribution lists and then separately 
query for all (individual) contacts?

> > X-Kolab-Version is a duplicate of the version contained within the XML and
> > should not be used.
> 
> It is not IMO, as it specifies the format of the mime message format. I'm in
> favor of keeping it.
> 

In that case, let's make it the full $x.$y.$z, so various version comparison 
routines can parse it similarly to how they (could) parse the $x.$y.$z in the 
XML, as opposed to an arbitrary string.

Let's suppose we don't allow the mime message format to be changed between 
teeny releases; only $x.$y then needs to be stored.

Also, wouldn't the version of libkolabmime be more appropriate to use here?

Perhaps, also, it's best to slap a name on the header that actually represents 
the fact it's about the mime message structure. X-Kolab-Mime-Version if you 
will.

> > ===
> > 
> > """The following headers/options MUST be used:"""
> > 
> > I made this a separate bullet point, and second, I see no reason to
> > restrict the charset or encoding.
> > 
> > This brings me back to the bullet point just before this one, too. I
> > suppose a localized version of said text could be allowed, as long as the
> > English version is somewhere?
> 
> I thought that ascii-us is somewhat the industry standard for emails, I
> might be wrong though. In any case I'm ok with allowing also other charsets
> as it's not a functional but puerly informal part anyways.
> 

Right. My point was to not superfluously define things we shouldn't actually 
care all that much about.

> > ===
> > 
> > """The following headers/options MUST be used:"""
> > 
> > I made this a separate bullet point, and second, again I see no reason to
> > restrict the encoding to quoted-printable, nor a reason to restrict the
> > filename to kolab.xml.
> 
> IMO there is no reason to not enforce the encoding, and as every client must
> be able to read all supported encoding types, I'd rather just use one. I
> don't really care about the filename, but see no reason not to just specify
> one, which keeps it at least consistent.
> 

Same as the above for the encoding. The filename question is more of a "save 
all to disk" thing to be honest, and unless everything's hard coded to look 
for a particular Kolab XML attachment by detecting which attachment has 
filename "kolab.xml", it makes little sense to specify and restrict.

Perhaps, MUST can become SHOULD, and "here's an example" for both this one and 
the aforementioned one.

> > """MIMETYPE MUST be one of (as specified in KEP #17):"""
> > 
> > I think this is inconsistent with the aforementioned x vnd kolab headers?
> 
> No, respectively should be =)
> The X-Kolab-Type specifies the exact Kolab Object type, where the
> Content-Type only specifies the mime-type of the format with which the xml
> object object is stored. That means if we have a v2 and a v3 event they
> have the same X-Kolab- Type but different Content-Types.
> 
> > ===
> > 
> > Attachments SHOULD be stored separately but MUST be referenced? Perhaps a
> > little fine-tuning of the verbiage is justified... ;-)
> > 
> > ===
> > 
> > The example is... large.
> 
> Yes, I just copied a testfile to be honest. I
> 
> > For demonstrative sake, can we insert a "may remove
> > carriage returns and superfluous whitespace", if not simply a "base64
> > encoding allowed" perhaps (even though it is kinda nice to be able to read
> > as a human being) - just offering this one up for consideration.
> 
> I don't follow here:
> "may remove carriage returns and superfluous whitespace" where?
> 

At the end of every line and the indentation, respectively. The "markup" as it 
is only matters to human eye-balls glancing over the raw spool payload, no?

> Regarding the base64, as I'm not in favor of allowing this, is there any
> specific reason to do so?
> 

Not really.

Kind regards,

Jeroen van Meeuwen

-- 
Systems Architect, Kolab Systems AG

e: vanmeeuwen at kolabsys.com
m: +44 74 2516 3817
w: http://www.kolabsys.com

pgp: 9342 BF08
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.kolab.org/pipermail/devel/attachments/20120516/efd0d265/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part.
URL: <http://lists.kolab.org/pipermail/devel/attachments/20120516/efd0d265/attachment.sig>