From vanmeeuwen at kolabsys.com Mon Aug 1 16:38:32 2011
From: vanmeeuwen at kolabsys.com (Jeroen van Meeuwen (Kolab Systems))
Date: Mon, 1 Aug 2011 10:38:32 -0400
Subject: [KEP UPDATE] KEP #9: Storage of configuration and application
control information
In-Reply-To: <2515704.Z9MGcjsfzr@katana.lair>
References: <1727968.ZERSqOCkgA@katana.lair>
<201107061240.43506.vanmeeuwen@kolabsys.com>
<2515704.Z9MGcjsfzr@katana.lair>
Message-ID: <201108011038.34047.vanmeeuwen@kolabsys.com>
Georg C. F. Greve wrote:
> Hi all,
>
> Based on the very useful feedback by Jeroen and Alain, I have revised the
> draft of KEP 9 to go more into the direction of clarifying the various
> mechanisms of configuration. As both Jeroen and Alain pointed out, it is
> practically impossible to think of one without thinking of the other.
>
> So this KEP now gives a set of tools, and the individual configuration
> items and use cases will in their description need to define their optimal
> mix and usage of these mechanisms.
>
> The updated draft is now online at
>
> http://wiki.kolab.org/User:Greve/Drafts/KEP:9
>
Hi Georg,
overall, looks good. While it says ANNOTATEMORE though, in the abstract, it
continues with METADATA (and rightfully so).
In the public vs. private annotations[1], I think a value.shared / value.priv
both are perfectly feasible for a 'folder-config' type of annotation, for
example.
[1]
http://wiki.kolab.org/User:Greve/Drafts/KEP:9#Public_and_private_annotations
I think perhaps a better clause here is to outline that it is forbidden unless
specifically stated in the accepted KEP for such annotation...?
In the parsing order of folder-based XML, how do we outline whether any
annotations found or the XML contents of the folder should prevail?
I think it is useful to outline that the settings that are stored in
annotations are *exclusive* with what may occur in configuration XML. Then
again, I haven't given it a full load of thought yet.
Kind regards,
Jeroen van Meeuwen
--
Senior Engineer, Kolab Systems AG
e: vanmeeuwen at kolabsys.com
t: +44 144 340 9500
m: +44 74 2516 3817
w: http://www.kolabsys.com
pgp: 9342 BF08
From florian.samson at bsi.bund.de Thu Aug 11 16:17:54 2011
From: florian.samson at bsi.bund.de (Florian v. Samson)
Date: Thu, 11 Aug 2011 16:17:54 +0200
Subject: [KEP UPDATE] KEP #9: Storage of configuration and application
control information
In-Reply-To: <2515704.Z9MGcjsfzr@katana.lair>
References: <1727968.ZERSqOCkgA@katana.lair>
<201107061240.43506.vanmeeuwen@kolabsys.com>
<2515704.Z9MGcjsfzr@katana.lair>
Message-ID: <201108111617.54862.florian.samson@bsi.bund.de>
Hi Georg,
Am Freitag, 29. Juli 2011 um 15:50:01 schrieb Georg C. F. Greve:
>
> ...
>
> The updated draft is now online at
>
> http://wiki.kolab.org/User:Greve/Drafts/KEP:9
>
> and is awaiting further comments.
1. "From its inception in 2003 and up until and including Kolab Format 2.0,
the Kolab Groupware Solution has been using the ANNOTATEMORE
proposal which has been finalized in February 2009 into ..."
1a. Link is syntactically incorrect (No Wiki-syntax, but HTML-syntax).
1b. Statement is incorrect by content: It was primarily draft-5 which was
implemented.
1c. tools.ietf.org provides better views of RFC (-drafts) than the chosen
one.
Suggestion:
Change the Link to
and
its text to "Annotatemore-05 RFC-draft".
> As there is a bit of text that has been
> touch, I expect some editorial as well as conceptual thinking to be
> required to bring this into final shape.
I feel that the next sentence "The Kolab Format that results from this
changeset against the Kolab Format 2.0 will consider usage of the
ANNOTATEMORE proposal deprecated and specify RFC 4564 only." is phrased a
bit too softly, as IIRC we all agreed, that this must result in a
normative "Kolab-clients and -servers MUST adhere to RFC4564" in the Kolab
Format specification v2.0 and later. "will consider only", rather sounds
to me as if this will be just the only solution depicted there.
Clarifications:
a. General: "Kolab XML version 1.0", "XML version 1.1", "the object
version ... at 1.1", "Kolab Format 2.0" all do mean "Kolab Format
specification version x.y", right?
a.I. If so, please call them consistently "Kolab Format specification
version x.y" only, in order to cease the ongoing confusion (for years,
already). When talking about Kolab-objects specifically, please call
them "a Kolab Object complying with the Kolab Format specification version
x.y", or "according to", "adhering", etc.
a.II. If not, then the confusion is larger than I can grasp. Consequently
this wording-mishmash would be in dire need of more clearing-up and
clarification, then.
b. "There MUST NEVER be more than one folder per user with the
subtype 'default';" is under-determined IMHO. By context one can conclude
that shall state (I tried to improve wording as well):
"There MUST be at most a single folder of type 'configuration' with the
subtype 'default' per user;"
c. "The subtype of the configuration folder-type for shared folders MUST
NEVER be set to 'default';" -> (Improve wording, pull the most important
noun in front)
"Shared folders of the type 'configuration' MUST NEVER carry the
subtype 'default';"
HTH & cheers
Florian
From vanmeeuwen at kolabsys.com Thu Aug 11 19:58:13 2011
From: vanmeeuwen at kolabsys.com (Jeroen van Meeuwen (Kolab Systems))
Date: Thu, 11 Aug 2011 13:58:13 -0400
Subject: [KEP UPDATE] KEP #9: Storage of configuration and application
control information
In-Reply-To: <201108111617.54862.florian.samson@bsi.bund.de>
References: <1727968.ZERSqOCkgA@katana.lair> <2515704.Z9MGcjsfzr@katana.lair>
<201108111617.54862.florian.samson@bsi.bund.de>
Message-ID: <201108111358.14991.vanmeeuwen@kolabsys.com>
Florian v. Samson wrote:
> Hi Georg,
>
Hi Florian, Georg,
> b. "There MUST NEVER be more than one folder per user with the
> subtype 'default';" is under-determined IMHO. By context one can conclude
> that shall state (I tried to improve wording as well):
> "There MUST be at most a single folder of type 'configuration' with the
> subtype 'default' per user;"
>
>
> c. "The subtype of the configuration folder-type for shared folders MUST
> NEVER be set to 'default';" -> (Improve wording, pull the most important
> noun in front)
> "Shared folders of the type 'configuration' MUST NEVER carry the
> subtype 'default';"
>
In the realm of nitpicking, I know, but MUST NEVER is not RFC 2119 compliant.
MUST NOT and SHALL NOT are to be used to indicate "that the definition is an
absolute prohibition of the specification".
Kind regards,
Jeroen van Meeuwen
--
Senior Engineer, Kolab Systems AG
e: vanmeeuwen at kolabsys.com
t: +44 144 340 9500
m: +44 74 2516 3817
w: http://www.kolabsys.com
pgp: 9342 BF08
From florian.samson at bsi.bund.de Fri Aug 12 09:04:12 2011
From: florian.samson at bsi.bund.de (Florian v. Samson)
Date: Fri, 12 Aug 2011 09:04:12 +0200
Subject: FYI: The Robustness Principle Reconsidered (CACM)
Message-ID: <201108120904.12524.florian.samson@bsi.bund.de>
Hello all,
an interesting, short note on a meta-principle, which has some relevance for
design decisions WRT Kolab Format specification v2.
The Robustness Principle Reconsidered (CACM)
Cheers
Florian
From greve at kolabsys.com Thu Aug 18 07:41:31 2011
From: greve at kolabsys.com (Georg C. F. Greve)
Date: Thu, 18 Aug 2011 07:41:31 +0200
Subject: [GOING TO CLOSE] KEP 8: Priority for events,
matching iCalendar priority range
In-Reply-To: <201107291542.26124.vanmeeuwen@kolabsys.com>
References: <93713080.Stszm7CVXk@katana.lair>
<201107291542.26124.vanmeeuwen@kolabsys.com>
Message-ID: <2170547.02mu3MvOHe@katana.lair>
On Friday 29 July 2011 15.42:25 Jeroen van Meeuwen wrote:
> That's about all I have
Excellent. Seems noone else had anything else, as well.
So I applied your small language fix, latest version now at
http://wiki.kolab.org/User:Greve/Drafts/KEP:8
and will ask to issue the closing call.
Best regards,
Georg
--
Georg C. F. Greve
Chief Executive Officer
Kolab Systems AG
Z?rich, Switzerland
e: greve at kolabsys.com
t: +41 78 904 43 33
w: http://kolabsys.com
pgp: 86574ACA Georg C. F. Greve
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 308 bytes
Desc: This is a digitally signed message part.
URL:
From greve at kolabsys.com Thu Aug 18 08:11:41 2011
From: greve at kolabsys.com (Georg C. F. Greve)
Date: Thu, 18 Aug 2011 08:11:41 +0200
Subject: [KEP UPDATE] KEP #9: Storage of configuration and application
control information
In-Reply-To: <201108011038.34047.vanmeeuwen@kolabsys.com>
References: <1727968.ZERSqOCkgA@katana.lair> <2515704.Z9MGcjsfzr@katana.lair>
<201108011038.34047.vanmeeuwen@kolabsys.com>
Message-ID: <2877318.xOX1Zk38Iz@katana.lair>
On Monday 01 August 2011 10.38:32 Jeroen van Meeuwen wrote:
> overall, looks good. While it says ANNOTATEMORE though, in the abstract, it
> continues with METADATA (and rightfully so).
Thanks, fixed.
> In the public vs. private annotations[1], I think a value.shared /
> value.priv both are perfectly feasible for a 'folder-config' type of
> annotation, for example.
Ok, rephrased. Let me know whether this is better for you.
Best regards,
Georg
--
Georg C. F. Greve
Chief Executive Officer
Kolab Systems AG
Z?rich, Switzerland
e: greve at kolabsys.com
t: +41 78 904 43 33
w: http://kolabsys.com
pgp: 86574ACA Georg C. F. Greve
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 308 bytes
Desc: This is a digitally signed message part.
URL:
From greve at kolabsys.com Thu Aug 18 08:16:03 2011
From: greve at kolabsys.com (Georg C. F. Greve)
Date: Thu, 18 Aug 2011 08:16:03 +0200
Subject: [KEP UPDATE] KEP #9: Storage of configuration and application
control information
In-Reply-To: <201108111617.54862.florian.samson@bsi.bund.de>
References: <1727968.ZERSqOCkgA@katana.lair> <2515704.Z9MGcjsfzr@katana.lair>
<201108111617.54862.florian.samson@bsi.bund.de>
Message-ID: <3478074.M88JnKY8F6@katana.lair>
Hi Florian,
On Thursday 11 August 2011 16.17:54 Florian v. Samson wrote:
> 1c. tools.ietf.org provides better views of RFC (-drafts) than the chosen
> one.
Very true. This is primarily an issue of the KEP template in the Wiki, though,
so for Jeroen to fix. I think I asked him to change the link for the template
some time ago, but he probably did not manage to find the time.
> Suggestion:
> Change the Link to
> and
> its text to "Annotatemore-05 RFC-draft".
Clarified and modified as proposed.
> I feel that the next sentence "The Kolab Format that results from this
> changeset against the Kolab Format 2.0 will consider usage of the
> ANNOTATEMORE proposal deprecated and specify RFC 4564 only." is phrased a
> bit too softly, as IIRC we all agreed, that this must result in a
> normative "Kolab-clients and -servers MUST adhere to RFC4564" in the Kolab
> Format specification v2.0 and later. "will consider only", rather sounds
> to me as if this will be just the only solution depicted there.
Agreed. Made it stronger in language.
> a. General: "Kolab XML version 1.0", "XML version 1.1", "the object
> version ... at 1.1", "Kolab Format 2.0" all do mean "Kolab Format
> specification version x.y", right?
Yes.
This is precisely the language that Bernhard and yourself agreed to in the
discussions around KEP 2, so I reused the same naming conventions, so
Kolab Format
for the overall specification, and
Kolab XML version
for the version number of the XML objects, as the two have unfortunately been
allowed to diverge, so now we have a little bit of inherent confusion in
there.
> b. "There MUST NEVER be more than one folder per user with the
> subtype 'default';" is under-determined IMHO. By context one can conclude
> that shall state (I tried to improve wording as well):
> "There MUST be at most a single folder of type 'configuration' with the
> subtype 'default' per user;"
Agreed & applied.
> c. "The subtype of the configuration folder-type for shared folders MUST
> NEVER be set to 'default';" -> (Improve wording, pull the most important
> noun in front)
> "Shared folders of the type 'configuration' MUST NEVER carry the
> subtype 'default';"
Agreed & applied with Jeroens improvement.
Thanks for the review and input!
Best regards,
Georg
--
Georg C. F. Greve
Chief Executive Officer
Kolab Systems AG
Z?rich, Switzerland
e: greve at kolabsys.com
t: +41 78 904 43 33
w: http://kolabsys.com
pgp: 86574ACA Georg C. F. Greve
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 308 bytes
Desc: This is a digitally signed message part.
URL:
From greve at kolabsys.com Thu Aug 18 08:18:58 2011
From: greve at kolabsys.com (Georg C. F. Greve)
Date: Thu, 18 Aug 2011 08:18:58 +0200
Subject: [NEARING CLOSE] KEP #9: Storage of configuration and application
control information
Message-ID: <2864793.QO0DNb7T7G@katana.lair>
Dear all,
Thanks to the review and input from both Jeroen and Florian, KEP #9 has now
been revised and updated at
http://wiki.kolab.org/User:Greve/Drafts/KEP:9
Other than that there has been very little input recently, which means this
KEP is also nearing closing time. So if you haven't yet reviewed it and feel
that you absolutely must do so, please do so *now* or forever hold your peace.
Best regards,
Georg
--
Georg C. F. Greve
Chief Executive Officer
Kolab Systems AG
Z?rich, Switzerland
e: greve at kolabsys.com
t: +41 78 904 43 33
w: http://kolabsys.com
pgp: 86574ACA Georg C. F. Greve
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 308 bytes
Desc: This is a digitally signed message part.
URL:
From vanmeeuwen at kolabsys.com Wed Aug 24 10:00:24 2011
From: vanmeeuwen at kolabsys.com (Jeroen van Meeuwen (Kolab Systems))
Date: Wed, 24 Aug 2011 09:00:24 +0100
Subject: Closing Call: KEP #9
Message-ID: <201108240900.26528.vanmeeuwen@kolabsys.com>
Hello,
I would like to announce the closing call stage for KEP #9, a design KEP on
storage of configuration and application control information;
http://wiki.kolab.org/index.php?title=User:Greve/Drafts/KEP:9&oldid=11706
This KEP was proposed July 5th, 2011, and has gone without any comment or
feedback for the last month.
With the closing call, we now enter a two week period for final objections.
This means that unless serious objections are raised, KEP #9 defaults to being
accepted on Tuesday, September 7th, 2011.
Thank you for your continued support!
Kind regards,
Jeroen van Meeuwen
--
Senior Engineer, Kolab Systems AG
e: vanmeeuwen at kolabsys.com
t: +44 144 340 9500
m: +44 74 2516 3817
w: http://www.kolabsys.com
pgp: 9342 BF08
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part.
URL:
From vanmeeuwen at kolabsys.com Wed Aug 24 10:05:24 2011
From: vanmeeuwen at kolabsys.com (Jeroen van Meeuwen (Kolab Systems))
Date: Wed, 24 Aug 2011 09:05:24 +0100
Subject: Closing Call: KEP #8
Message-ID: <201108240905.25457.vanmeeuwen@kolabsys.com>
Hello,
I would like to announce the closing call stage for KEP #8, a design KEP on
Priority for events, matching iCalendar's priority range;
http://wiki.kolab.org/index.php?title=User:Greve/Drafts/KEP:8&oldid=11701
This KEP was proposed June 25th, 2011, and has gone without any comment or
feedback for the last month.
With the closing call, we now enter a two week period for final objections.
This means that unless serious objections are raised, KEP #8 defaults to being
accepted on Tuesday, September 7th, 2011.
Thank you for your continued support!
Kind regards,
Jeroen van Meeuwen
--
Senior Engineer, Kolab Systems AG
e: vanmeeuwen at kolabsys.com
t: +44 144 340 9500
m: +44 74 2516 3817
w: http://www.kolabsys.com
pgp: 9342 BF08
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
From vanmeeuwen at kolabsys.com Thu Aug 25 12:56:23 2011
From: vanmeeuwen at kolabsys.com (Jeroen van Meeuwen (Kolab Systems))
Date: Thu, 25 Aug 2011 11:56:23 +0100
Subject: Closing Call: KEP #9
In-Reply-To: <201108240900.26528.vanmeeuwen@kolabsys.com>
References: <201108240900.26528.vanmeeuwen@kolabsys.com>
Message-ID: <201108251156.24883.vanmeeuwen@kolabsys.com>
Jeroen van Meeuwen (Kolab Systems) wrote:
> Hello,
>
> I would like to announce the closing call stage for KEP #9, a design KEP on
> storage of configuration and application control information;
>
> http://wiki.kolab.org/index.php?title=User:Greve/Drafts/KEP:9&oldid=11706
>
Hi Georg,
small correction:
ANNOTATE is RFC 5257, not 5267.
Kind regards,
Jeroen van Meeuwen
--
Senior Engineer, Kolab Systems AG
e: vanmeeuwen at kolabsys.com
t: +44 144 340 9500
m: +44 74 2516 3817
w: http://www.kolabsys.com
pgp: 9342 BF08
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
From vanmeeuwen at kolabsys.com Thu Aug 25 13:03:29 2011
From: vanmeeuwen at kolabsys.com (Jeroen van Meeuwen (Kolab Systems))
Date: Thu, 25 Aug 2011 12:03:29 +0100
Subject: Closing Call: KEP #9
In-Reply-To: <201108240900.26528.vanmeeuwen@kolabsys.com>
References: <201108240900.26528.vanmeeuwen@kolabsys.com>
Message-ID: <201108251203.30792.vanmeeuwen@kolabsys.com>
Jeroen van Meeuwen (Kolab Systems) wrote:
> Hello,
>
> I would like to announce the closing call stage for KEP #9, a design KEP on
> storage of configuration and application control information;
>
> http://wiki.kolab.org/index.php?title=User:Greve/Drafts/KEP:9&oldid=11706
>
> This KEP was proposed July 5th, 2011, and has gone without any comment or
> feedback for the last month.
>
Language tweak, not sure if it makes the statement more clear but wanted to
suggest it anyways:
(Both kinds MAY be used (...))
"""but each annotations usage in all namespaces""" -->
"""but the use of an annotation"""
(MUST be defined)
"""for all prefixes"""
(in the KEP for said annotation, to avoid confusion and/or conflict)
Note how METADATA uses /private and /shared prefixes as opposed to
'value.priv' and 'value.shared' command parameters.
Kind regards,
Jeroen van Meeuwen
--
Senior Engineer, Kolab Systems AG
e: vanmeeuwen at kolabsys.com
t: +44 144 340 9500
m: +44 74 2516 3817
w: http://www.kolabsys.com
pgp: 9342 BF08
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
From greve at kolabsys.com Thu Aug 25 13:26:37 2011
From: greve at kolabsys.com (Georg C. F. Greve)
Date: Thu, 25 Aug 2011 13:26:37 +0200
Subject: Closing Call: KEP #9
In-Reply-To: <201108251156.24883.vanmeeuwen@kolabsys.com>
References: <201108240900.26528.vanmeeuwen@kolabsys.com>
<201108251156.24883.vanmeeuwen@kolabsys.com>
Message-ID: <73033785.fkgc5tK3vR@katana.lair>
On Thursday 25 August 2011 11.56:23 Jeroen van Meeuwen wrote:
> small correction:
> ANNOTATE is RFC 5257, not 5267.
Good catch! Although actually 5267 existed nowhere in the document. But 5256
did. Damn them RFC numbers. ;)
Fixed in latest revision:
http://wiki.kolab.org/index.php?title=User:Greve/Drafts/KEP:9&oldid=11727
Best regards,
Georg
P.S. Cross-posting for such corrections in KEPs is okay to announce minor fixes
in the final approval process for me, but the report should just go to the
authoritative list, IMHO to avoid cluttering all lists.
And I don't think any of this should go to kep-announce, which should be
purely for announcements.
--
Georg C. F. Greve
Chief Executive Officer
Kolab Systems AG
Z?rich, Switzerland
e: greve at kolabsys.com
t: +41 78 904 43 33
w: http://kolabsys.com
pgp: 86574ACA Georg C. F. Greve
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 308 bytes
Desc: This is a digitally signed message part.
URL:
From greve at kolabsys.com Thu Aug 25 13:27:13 2011
From: greve at kolabsys.com (Georg C. F. Greve)
Date: Thu, 25 Aug 2011 13:27:13 +0200
Subject: Closing Call: KEP #9
In-Reply-To: <201108251203.30792.vanmeeuwen@kolabsys.com>
References: <201108240900.26528.vanmeeuwen@kolabsys.com>
<201108251203.30792.vanmeeuwen@kolabsys.com>
Message-ID: <2269162.mbymxj6G0q@katana.lair>
On Thursday 25 August 2011 12.03:29 Jeroen van Meeuwen wrote:
> Note how METADATA uses /private and /shared prefixes as opposed to
> 'value.priv' and 'value.shared' command parameters.
ACK. Incorporated in latest revision:
http://wiki.kolab.org/index.php?title=User:Greve/Drafts/KEP:9&oldid=11727
Best regards,
Georg
--
Georg C. F. Greve
Chief Executive Officer
Kolab Systems AG
Z?rich, Switzerland
e: greve at kolabsys.com
t: +41 78 904 43 33
w: http://kolabsys.com
pgp: 86574ACA Georg C. F. Greve
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 308 bytes
Desc: This is a digitally signed message part.
URL:
From vanmeeuwen at kolabsys.com Thu Aug 25 13:32:44 2011
From: vanmeeuwen at kolabsys.com (Jeroen van Meeuwen (Kolab Systems))
Date: Thu, 25 Aug 2011 12:32:44 +0100
Subject: [KEP UPDATE] KEP #9: Storage of configuration and application
control information
In-Reply-To: <3478074.M88JnKY8F6@katana.lair>
References: <1727968.ZERSqOCkgA@katana.lair>
<201108111617.54862.florian.samson@bsi.bund.de>
<3478074.M88JnKY8F6@katana.lair>
Message-ID: <201108251232.45960.vanmeeuwen@kolabsys.com>
Georg C. F. Greve wrote:
> Hi Florian,
>
> On Thursday 11 August 2011 16.17:54 Florian v. Samson wrote:
> > 1c. tools.ietf.org provides better views of RFC (-drafts) than the chosen
> > one.
>
> Very true. This is primarily an issue of the KEP template in the Wiki,
> though, so for Jeroen to fix. I think I asked him to change the link for
> the template some time ago, but he probably did not manage to find the
> time.
>
I think I've only templated the reference to RFCs:
http://wiki.kolab.org/KEP:4#References_to_RFCs
which link to http://tools.ietf.org/html/rfc, list the RFC in the
references, for a nice and good printable KEP.
Kind regards,
Jeroen van Meeuwen
--
Senior Engineer, Kolab Systems AG
e: vanmeeuwen at kolabsys.com
t: +44 144 340 9500
m: +44 74 2516 3817
w: http://www.kolabsys.com
pgp: 9342 BF08
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
From greve at kolabsys.com Fri Aug 26 17:28:10 2011
From: greve at kolabsys.com (Georg C. F. Greve)
Date: Fri, 26 Aug 2011 17:28:10 +0200
Subject: NEW KEP: Color configuration storage for resources and categories
(KEP #12)
Message-ID: <3450478.OSfiVuoptM@katana.lair>
Dear all,
A new KEP has just been released into the public drafting & review process:
KEP #12: Color configuration storage for resources and categories
http://wiki.kolab.org/User:Greve/Drafts/KEP:12
The purpose of this KEP is to allow clients to consistently use the same
colors for resources such as calendars, address books and so on as well as
categories. The KEP has been based upon input collected since May onwards and
is therefore already fairly substantial.
There are however some judgement calls in particular around the category XML
object which this KEP defines, and your input would be very much welcome.
As this is a design KEP for the format, all comments, feedback, discussion
should take place on
kolab-format at kolab.org
It is strongly recommended that ALL CLIENT AUTHORS have a look at this.
With best regards,
Georg Greve
--
Georg C. F. Greve
Chief Executive Officer
Kolab Systems AG
Z?rich, Switzerland
e: greve at kolabsys.com
t: +41 78 904 43 33
w: http://kolabsys.com
pgp: 86574ACA Georg C. F. Greve
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 308 bytes
Desc: This is a digitally signed message part.
URL:
From greve at kolabsys.com Sat Aug 27 20:39:40 2011
From: greve at kolabsys.com (Georg C. F. Greve)
Date: Sat, 27 Aug 2011 20:39:40 +0200
Subject: Request for Input: Storing Searches
Message-ID: <2232400.u9sBjAbLUj@katana.lair>
Dear all,
Some of us have started tossing around thoughts about how to save searches in
one Kolab client in a way that they are re-usable in all others, ideally.
When giving it some brainspace, it turns out this is not a trivial issue, for
a variety of reasons, starting with there being a tradeoff decision between
being expensive for the CPU or storage, for instance. But it is a little bit
more complex than that, actually.
Allow me to higlight a couple of scenarios with advantages and disadvantages:
- Scenario 1: Storage with a new KEP 9 based XML object
One could attempt to model this as a "search" XML object that would
incorporate the fields of the object type searched, plus some special
fields, e.g. folders to search, as well as searches across multiple fields
and search logic (AND/OR etc),
These objects would live in the regular folders for resources, and would
potentially even replace the list object in functionality, as they would
then model a list of recipients as list of address book entries, which is
something that Alain once suggested. [1]
Advantages: Fairly close to existing functionality, and likely not too
hard to implement for most clients (in comparison, at least), no data
duplication anywhere.
Disadvantages: Expensive on the CPU, Does not work on all resources
because we cannot store these XML elements in email type mailboxes.
- Scenario 2: Creation of new folder type w/KEP #9 annotation for metadata,
create one folder per saved search
In this approach we'd create a new folder of the corresponding resource
folder type for each search which would be identified as a stored search
folder by existence of the /vendor/kolab/saved-search annotation which
carries the metadata for the search in an array, e.g.
{ 'saved_search':
{ 'search_locations': 'blabla',
'params': 'blabla',
'filter': 'blabla',
'fuzzyness': 'blabla',
'async': '0'
....
}
}
and the folder would be populated with the results of the search.
This DOES mean data duplication on the client, but Cyrus does allow to
deduplicate entries on the server side, so it would not affect storage
there. I am sure something similar would be possible with Dovecot, so
we can for the moment assume data gets duplicated on the client only.
Advantages: Allows clients without search functionality to use results,
can be automatically regenerated on the server if needs be, least CPU
usage, works on email.
Disadvantages: Data duplication on the client, possible data set de-
synchronization (e.g. contact gets edited in search results, same contact
in main box and other search results boxes must be updated, this may be
hard to ensure), increases folder clutter, some folder sharing questions.
- Scenario 3: Map searches with tags
As a Kolab object, each search will carry an ID. If we were to introduce
a new email header flag in storage that can carry an arbitrary number
of tags, we could tag each object with the ID of every search that it
matches.
IMAP searching for header fields should make it comparatively easy
and fast to find all objects of type X that match a certain tag Y,
especially if we ask the server to cache this header field.
This would be complemented by a KEP 9 compatible object to describe
the search, which could then be automatically applied to new objects on
the server, or performed by the client, based on the scenario.
Advantages: Low CPU & storage requirements, allows Kolab clients to apply
a tag concept over all object types including email with potential server
side tagging of incoming email
Disadvantages: New concept, some questions around shared folders, e.g.
what if a client sees a shared object tagged with an ID for a search it
does not know because it does not have access to the folder where that
search is defined?
There may be other advantages and disadvantages that I did not list.
Please help us identify them all, so we can come to a good decision.
Likewise, if you can think of a scenario that should be considered in addition
to the ones listed here, please let me know. As for the scenarios listed,
there are two questions in particular that I wonder about:
(a) Compatibility with clients, in particular: How will this integrate (or
not) with the new Nepomuk/Akonadi KDE Kontact basis
(b) Query language: How do we best formulate/store the query in these
scenarios?
Anyhow, these are my thoughts on the matter right now.
I'd be happy to start drafting on something once we've identified which
direction things should go into, but right now I still am not sure which is
the path to go. So input is VERY appreciated.
Best regards,
Georg
[1] http://kolab.org/pipermail/kolab-format/2011-July/001415.html
--
Georg C. F. Greve
Chief Executive Officer
Kolab Systems AG
Z?rich, Switzerland
e: greve at kolabsys.com
t: +41 78 904 43 33
w: http://kolabsys.com
pgp: 86574ACA Georg C. F. Greve
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 308 bytes
Desc: This is a digitally signed message part.
URL:
From thomas at koch.ro Sun Aug 28 15:08:05 2011
From: thomas at koch.ro (Thomas Koch)
Date: Sun, 28 Aug 2011 15:08:05 +0200
Subject: Request for Input: Storing Searches
In-Reply-To: <2232400.u9sBjAbLUj@katana.lair>
References: <2232400.u9sBjAbLUj@katana.lair>
Message-ID: <201108281508.05694.thomas@koch.ro>
Georg C. F. Greve:
> Dear all,
>
> Some of us have started tossing around thoughts about how to save searches
> in one Kolab client in a way that they are re-usable in all others,
> ideally.
Hi Georg,
this one catched my attention since my last project was about search. I try to
understand your mail, but it's hard since you write about different
implementation options without describing the underlying use cases/user
stories.
Who should search for what and why should what be saved? Is it about
performance (caching search results) or about sharing the query strings? What
about privacy issues with shared query strings? I think it would help if you'd
really describe it in terms of "User wants ..."
Regards,
Thomas Koch, http://www.koch.ro
From greve at kolabsys.com Sun Aug 28 18:48:29 2011
From: greve at kolabsys.com (Georg C. F. Greve)
Date: Sun, 28 Aug 2011 18:48:29 +0200
Subject: Request for Input: Storing Searches
In-Reply-To: <201108281508.05694.thomas@koch.ro>
References: <2232400.u9sBjAbLUj@katana.lair>
<201108281508.05694.thomas@koch.ro>
Message-ID: <4961404.Dgrlh7NzkC@katana.lair>
Hi Thomas,
On Sunday 28 August 2011 15.08:05 Thomas Koch wrote:
> Who should search for what and why should what be saved? Is it about
> performance (caching search results) or about sharing the query strings?
> What about privacy issues with shared query strings? I think it would help
> if you'd really describe it in terms of "User wants ..."
My apologies. Sometimes when you've spent a certain amount of time thinking
about an issue some things may seem obvious even when they're not.
The use case here is that users want to search through their data set for any
kind of object (including address books, calendars) and save their searches
for later re-use, e.g. have a saved search "Show me all calendar entries that
take place in Germany from this set of calendars."
So it is about storing what needs to be searched, potentially how it needs to
be searched, and to make the subsequent searches as efficient as possible, this
should be sharable between different clients, as well as different users if they
choose to give each other access to such saved searches.
Is it clearler now?
Best regards,
Georg
--
Georg C. F. Greve
Chief Executive Officer
Kolab Systems AG
Z?rich, Switzerland
e: greve at kolabsys.com
t: +41 78 904 43 33
w: http://kolabsys.com
pgp: 86574ACA Georg C. F. Greve
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 308 bytes
Desc: This is a digitally signed message part.
URL:
From greve at kolabsys.com Mon Aug 29 13:43:50 2011
From: greve at kolabsys.com (Georg C. F. Greve)
Date: Mon, 29 Aug 2011 13:43:50 +0200
Subject: NEW KEP: Update 'contact' object
Message-ID: <1990428.gAzyIjUGzZ@katana.lair>
Dear all,
A new KEP has just been released into public brainstorming:
KEP #13: Update 'contact' object
http://wiki.kolab.org/User:Greve/Drafts/KEP:13
This KEP is about updating the 'contact' object which has some well known
issues that we should resolve, and as these are going to be breaking changes,
of which we've had a couple lined up already, I figured it would make sense to
put them all into one changeset so we avoid having too many breaking
transitions.
The state of the KEP is thus that it outlines several of the issues, and
provides some input on how this could be better modeled on the storage level,
but it is still far from complete - in particular as it changes the underlying
way of modeling some of the aspects.
Input and drafting help is very much welcome!
All of that should go to kolab-format at kolab.org, please.
Best regards,
Georg
--
Georg C. F. Greve
Chief Executive Officer
Kolab Systems AG
Z?rich, Switzerland
e: greve at kolabsys.com
t: +41 78 904 43 33
w: http://kolabsys.com
pgp: 86574ACA Georg C. F. Greve
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 308 bytes
Desc: This is a digitally signed message part.
URL:
From shopov at kolabsys.com Mon Aug 29 13:51:44 2011
From: shopov at kolabsys.com (Bogomil "Bogo" Shopov)
Date: Mon, 29 Aug 2011 14:51:44 +0300
Subject: NEW KEP: Update 'contact' object
In-Reply-To: <1990428.gAzyIjUGzZ@katana.lair>
References: <1990428.gAzyIjUGzZ@katana.lair>
Message-ID: <4E5B7D50.20407@kolabsys.com>
On 08/29/2011 02:43 PM, Georg C. F. Greve wrote:
> Dear all,
>
> A new KEP has just been released into public brainstorming:
>
> KEP #13: Update 'contact' object
> http://wiki.kolab.org/User:Greve/Drafts/KEP:13
>
> This KEP is about updating the 'contact' object which has some well known
> issues that we should resolve, and as these are going to be breaking changes,
> of which we've had a couple lined up already, I figured it would make sense to
> put them all into one changeset so we avoid having too many breaking
> transitions.
>
> The state of the KEP is thus that it outlines several of the issues, and
> provides some input on how this could be better modeled on the storage level,
> but it is still far from complete - in particular as it changes the underlying
> way of modeling some of the aspects.
Maybe we can introduce to x-logo
*uri and/or bynari attribute
*
> Input and drafting help is very much welcome!
>
> All of that should go to kolab-format at kolab.org, please.
>
> Best regards,
> Georg
>
>
>
>
> _______________________________________________
> Kolab-format mailing list
> Kolab-format at kolab.org
> https://kolab.org/mailman/listinfo/kolab-format
--
Bogomil "Bogo" Shopov
Web and QA Ninja
Kolab Systems AG
Z?rich, Switzerland
e: shopov at kolabsys.com
t: +359 888 098 768
w: http://kolabsys.com
pgp: 0xE69A226A Bogomil Shopov
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
From alain.abbas at libertech.fr Mon Aug 29 16:27:50 2011
From: alain.abbas at libertech.fr (Alain Abbas)
Date: Mon, 29 Aug 2011 16:27:50 +0200
Subject: NEW KEP: Update 'contact' object
In-Reply-To: <1990428.gAzyIjUGzZ@katana.lair>
References: <1990428.gAzyIjUGzZ@katana.lair>
Message-ID: <4E5BA1E6.1000005@libertech.fr>
An HTML attachment was scrubbed...
URL:
From vanmeeuwen at kolabsys.com Tue Aug 30 10:11:09 2011
From: vanmeeuwen at kolabsys.com (Jeroen van Meeuwen (Kolab Systems))
Date: Tue, 30 Aug 2011 09:11:09 +0100
Subject: Request for Input: Storing Searches
In-Reply-To: <2232400.u9sBjAbLUj@katana.lair>
References: <2232400.u9sBjAbLUj@katana.lair>
Message-ID: <201108300911.11175.vanmeeuwen@kolabsys.com>
Georg C. F. Greve wrote:
> Allow me to higlight a couple of scenarios with advantages and
> disadvantages:
>
> - Scenario 2: Creation of new folder type w/KEP #9 annotation for
> metadata, create one folder per saved search
>
> In this approach we'd create a new folder of the corresponding resource
> folder type for each search which would be identified as a stored search
> folder by existence of the /vendor/kolab/saved-search annotation
Please note a unique annotation is technically not sustainable, because all
annotation keys need to be listed in a static list in a configuration file on
the server-side.
It's better to use /vendor/kolab/folder-type contact, with a
/vendor/kolab/folder-config holding the saved search data such as, for
example, the following snippet;
> which
> carries the metadata for the search in an array, e.g.
>
> { 'saved_search':
> { 'search_locations': 'blabla',
> 'params': 'blabla',
> 'filter': 'blabla',
> 'fuzzyness': 'blabla',
> 'async': '0'
> ....
> }
> }
>
> and the folder would be populated with the results of the search.
>
Please note that the folder *could* be populated with the results of the
search. I think this is an option.
> This DOES mean data duplication on the client, but Cyrus does allow to
> deduplicate entries on the server side, so it would not affect storage
> there. I am sure something similar would be possible with Dovecot, so
> we can for the moment assume data gets duplicated on the client only.
>
> Advantages: Allows clients without search functionality to use results,
> can be automatically regenerated on the server if needs be, least CPU
> usage, works on email.
>
> Disadvantages: Data duplication on the client, possible data set de-
> synchronization (e.g. contact gets edited in search results, same contact
> in main box and other search results boxes must be updated, this may be
> hard to ensure), increases folder clutter, some folder sharing questions.
>
Populating the folder with the results would indeed duplicate the
contacts/events/mail/whathaveyou. However, it would be a MUA problem to solve
editing an object from within the saved search (prevent editting, warn user
about editing this copy of the object, or replicate the change back to the
original source of the search result -note the object's UID is available in
the search results).
Perhaps the IMAP folder could be made not-writeable. This could have
implications on the ability to edit the saved search though.
> - Scenario 3: Map searches with tags
>
> As a Kolab object, each search will carry an ID. If we were to
introduce
> a new email header flag in storage that can carry an arbitrary number
> of tags, we could tag each object with the ID of every search that it
> matches.
>
There is a concept called IMAP Flags, and a concept of RFC 5257 per-message
annotation. Why would we need to insert a header into an email message?
> Likewise, if you can think of a scenario that should be considered in
> addition to the ones listed here, please let me know. As for the scenarios
> listed, there are two questions in particular that I wonder about:
>
> (a) Compatibility with clients, in particular: How will this integrate (or
> not) with the new Nepomuk/Akonadi KDE Kontact basis
>
> (b) Query language: How do we best formulate/store the query in these
> scenarios?
>
> Anyhow, these are my thoughts on the matter right now.
>
> I'd be happy to start drafting on something once we've identified which
> direction things should go into, but right now I still am not sure which is
> the path to go.
>
These paths do not have to be mutually exclusive. A contact folder-type with
folder-config annotations for a saved search does not inherently prevent a
Kolab XML object with configuration for a search based on tags/flags/other
message metadata (From, To, ...).
Kind regards,
Jeroen van Meeuwen
--
Senior Engineer, Kolab Systems AG
e: vanmeeuwen at kolabsys.com
t: +44 144 340 9500
m: +44 74 2516 3817
w: http://www.kolabsys.com
pgp: 9342 BF08
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
From vanmeeuwen at kolabsys.com Tue Aug 30 10:14:34 2011
From: vanmeeuwen at kolabsys.com (Jeroen van Meeuwen (Kolab Systems))
Date: Tue, 30 Aug 2011 09:14:34 +0100
Subject: NEW KEP: Update 'contact' object
In-Reply-To: <1990428.gAzyIjUGzZ@katana.lair>
References: <1990428.gAzyIjUGzZ@katana.lair>
Message-ID: <201108300914.35708.vanmeeuwen@kolabsys.com>
Georg C. F. Greve wrote:
> Dear all,
>
> A new KEP has just been released into public brainstorming:
>
> KEP #13: Update 'contact' object
> http://wiki.kolab.org/User:Greve/Drafts/KEP:13
>
> This KEP is about updating the 'contact' object which has some well known
> issues that we should resolve, and as these are going to be breaking
> changes, of which we've had a couple lined up already, I figured it would
> make sense to put them all into one changeset so we avoid having too many
> breaking transitions.
>
Can we include the fact the Subject/To/From headers not represent any of the
contact object's metadata? This means searches have to go through a lot more
(XML) data then that they would have to if any of such data was stored on any
of the email message headers.
Kind regards,
Jeroen van Meeuwen
--
Senior Engineer, Kolab Systems AG
e: vanmeeuwen at kolabsys.com
t: +44 144 340 9500
m: +44 74 2516 3817
w: http://www.kolabsys.com
pgp: 9342 BF08
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
From mollekopf at kolabsys.com Tue Aug 30 11:47:30 2011
From: mollekopf at kolabsys.com (Christian Mollekopf)
Date: Tue, 30 Aug 2011 11:47:30 +0200
Subject: Request for Input: Storing Searches
In-Reply-To: <2232400.u9sBjAbLUj@katana.lair>
References: <2232400.u9sBjAbLUj@katana.lair>
Message-ID: <2522707.RBIlZQlMEe@myhost>
On Saturday, August 27, 2011 08:39:40 PM Georg C. F. Greve wrote:
> Dear all,
>
> Some of us have started tossing around thoughts about how to save searches
> in one Kolab client in a way that they are re-usable in all others,
> ideally.
>
> When giving it some brainspace, it turns out this is not a trivial issue,
> for a variety of reasons, starting with there being a tradeoff decision
> between being expensive for the CPU or storage, for instance. But it is a
> little bit more complex than that, actually.
>
> Allow me to higlight a couple of scenarios with advantages and
> disadvantages:
>
> - Scenario 1: Storage with a new KEP 9 based XML object
>
> One could attempt to model this as a "search" XML object that would
> incorporate the fields of the object type searched, plus some special
> fields, e.g. folders to search, as well as searches across multiple fields
> and search logic (AND/OR etc),
>
> These objects would live in the regular folders for resources, and would
> potentially even replace the list object in functionality, as they would
> then model a list of recipients as list of address book entries, which is
> something that Alain once suggested. [1]
>
> Advantages: Fairly close to existing functionality, and likely not too
> hard to implement for most clients (in comparison, at least), no data
> duplication anywhere.
>
> Disadvantages: Expensive on the CPU, Does not work on all resources
> because we cannot store these XML elements in email type mailboxes.
>
>
> - Scenario 2: Creation of new folder type w/KEP #9 annotation for metadata,
> create one folder per saved search
>
> In this approach we'd create a new folder of the corresponding resource
> folder type for each search which would be identified as a stored search
> folder by existence of the /vendor/kolab/saved-search annotation which
> carries the metadata for the search in an array, e.g.
>
> { 'saved_search':
> { 'search_locations': 'blabla',
> 'params': 'blabla',
> 'filter': 'blabla',
> 'fuzzyness': 'blabla',
> 'async': '0'
> ....
> }
> }
>
> and the folder would be populated with the results of the search.
>
> This DOES mean data duplication on the client, but Cyrus does allow to
> deduplicate entries on the server side, so it would not affect storage
> there. I am sure something similar would be possible with Dovecot, so
> we can for the moment assume data gets duplicated on the client only.
>
> Advantages: Allows clients without search functionality to use results,
> can be automatically regenerated on the server if needs be, least CPU
> usage, works on email.
>
> Disadvantages: Data duplication on the client, possible data set de-
> synchronization (e.g. contact gets edited in search results, same contact
> in main box and other search results boxes must be updated, this may be
> hard to ensure), increases folder clutter, some folder sharing questions.
>
>
> - Scenario 3: Map searches with tags
>
> As a Kolab object, each search will carry an ID. If we were to introduce
> a new email header flag in storage that can carry an arbitrary number
> of tags, we could tag each object with the ID of every search that it
> matches.
>
> IMAP searching for header fields should make it comparatively easy
> and fast to find all objects of type X that match a certain tag Y,
> especially if we ask the server to cache this header field.
>
> This would be complemented by a KEP 9 compatible object to describe
> the search, which could then be automatically applied to new objects on
> the server, or performed by the client, based on the scenario.
>
> Advantages: Low CPU & storage requirements, allows Kolab clients to apply
> a tag concept over all object types including email with potential server
> side tagging of incoming email
>
> Disadvantages: New concept, some questions around shared folders, e.g.
> what if a client sees a shared object tagged with an ID for a search it
> does not know because it does not have access to the folder where that
> search is defined?
>
Something along the lines of tagging the search results via the ANNOTATE
Extension and using an XML object will probably gives us the most flexibility.
There are however some problems existing:
On the Akonadi side searches are afaik implemented as virtual folders which
are populated based on a nepomuk query(sparql). Searches don't belong to a
single resource however, since you can search accross various resources.
This means we would have to implement that functionality from outside of the
kolab resource I reckon.
What I think is possible is something along the lines of this:
- Add an option to search folders "Share this search", where you can choose a
resource which supports shared searches (Kolab).
- The action would then pick the results from the search which are in this
resource, and tag them via ANNOTATE and create an xml object with the search
info.
>From here were going the same path if we are consumer or creator of the
search:
- The resource creates a virtual-folder inside the resource i.e. "Shared
Searches/Last Search" (The name can be edited and is stored to the xml
object), based on the xml object.
- The resource populates the virtual folder with virtual items based on the
tags.
=> - No data duplication
- items are directly editable
- you get only the results for which you have permission.
Also we can implement most of the code in the kolab resource, and the only
additional thing which we need is the action which creates the saved search.
At a later stage we could even rerun the query inside the resource to find
additional matches which were not available for the creator of the search, or
new matches but that's not even necessarily expected from the user.
If we have a revision number in the xml object the search could even be
modified and updated.
I think this gives us a pretty flexible and powerful way of sharing folders
which is at least fully compatible with akonadi. No idea however if something
like this can also be implemented in an MS Exchange client.
Btw. we could even think of adding a readonly folder with copies for the
limited clients (along the lines of option #2). I.e. on akonadi this folder
would be hidden so you get the full functionality including editing via
virtual folders. On the smartphone (or just any other imap client) you get
only the readonly version, but at least you have something for reference. But
this, again, can be added at a later stage and is fully optional (as long as
we have a field in the xml object to store this folder).
I don't see any implications on nepomuk with either of those solutions.
With my best regards,
Christian
>
> There may be other advantages and disadvantages that I did not list.
>
> Please help us identify them all, so we can come to a good decision.
>
> Likewise, if you can think of a scenario that should be considered in
> addition to the ones listed here, please let me know. As for the scenarios
> listed, there are two questions in particular that I wonder about:
>
> (a) Compatibility with clients, in particular: How will this integrate (or
> not) with the new Nepomuk/Akonadi KDE Kontact basis
>
> (b) Query language: How do we best formulate/store the query in these
> scenarios?
>
> Anyhow, these are my thoughts on the matter right now.
>
> I'd be happy to start drafting on something once we've identified which
> direction things should go into, but right now I still am not sure which is
> the path to go. So input is VERY appreciated.
>
> Best regards,
> Georg
>
>
>
> [1] http://kolab.org/pipermail/kolab-format/2011-July/001415.html
--
Christian Mollekopf
Software Engineer
Kolab Systems AG
Z?rich, Switzerland
e: mollekopf at kolabsys.com
w: http://kolabsys.com
pgp: EA657400 Christian Mollekopf
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part.
URL:
From vanmeeuwen at kolabsys.com Tue Aug 30 15:03:29 2011
From: vanmeeuwen at kolabsys.com (Jeroen van Meeuwen (Kolab Systems))
Date: Tue, 30 Aug 2011 14:03:29 +0100
Subject: Request for Input: Storing Searches
In-Reply-To: <2522707.RBIlZQlMEe@myhost>
References: <2232400.u9sBjAbLUj@katana.lair> <2522707.RBIlZQlMEe@myhost>
Message-ID: <201108301403.42063.vanmeeuwen@kolabsys.com>
Christian Mollekopf wrote:
> Something along the lines of tagging the search results via the ANNOTATE
> Extension and using an XML object will probably gives us the most
> flexibility.
>
Searches may or may not include non-IMAP content (such as LDAP content).
Why / How would a Kolab XML object help? Why not save the information in a
"virtual" IMAP folder?
This IMAP folder can handle the ACLs, thus be shared, it shows up empty for
clients that are unable to handle KEP #9, and *at your option* could be
prepopulated with the results of the search (OPTIONAL) - though the tradeoff
would be data duplication.
> There are however some problems existing:
>
> On the Akonadi side searches are afaik implemented as virtual folders which
> are populated based on a nepomuk query(sparql). Searches don't belong to a
> single resource however, since you can search accross various resources.
This is a client-side (saved) search, that cannot be shared with any other
client, correct?
We are attempting to put the saved search on the server-side somehow.
> This means we would have to implement that functionality from outside of
> the kolab resource I reckon.
>
What do you mean by 'the kolab resource'? An account? A folder? A single Kolab
XML object? The yet-to-be-defined Kolab saved-search XML object?
> - The action would then pick the results from the search which are in this
> resource, and tag them via ANNOTATE and create an xml object with the
> search info.
>
Where would this XML object be put?
> From here were going the same path if we are consumer or creator of the
> search:
>
> - The resource creates a virtual-folder inside the resource i.e. "Shared
> Searches/Last Search" (The name can be edited and is stored to the xml
> object), based on the xml object.
>
"the resource" and "the resource" keep confusing me. Also, "Shared searches"
implies the searches are shared... I think "saved searches" are more
appropriate.
> - The resource populates the virtual folder with virtual items based on the
> tags.
>
> => - No data duplication
This has always been "optional"; a saved search folder *could* be pre-
populated.
Imagine a saved search across 5 contact folders with 10.000 contacts on
average.
When NOT pre-populating the saved search folder with the search results, you
pay the cost every time the folder is opened. Maybe this cost is not so great
for a fat client with a local cache to query
(Kontact/akonadi/nepomuk/Disconnected IMAP), but for a web-interface...
well...
When pre-populating the saved search folder with the search results, you pay
the cost in "duplicate" storage (as explained, not on the server side, perhaps
on the client side if it's not intelligent enough to de-duplicate).
Another penalty in pre-populating the saved search folder could, arguably, be
that perhaps there's results in said saved search folder that the person using
the folder would otherwise not have access to. However, this can also be
considered a feature; "Share all contacts from Vendors folder tagged with
'ict' with helpdesk personnel"
Kind regards,
Jeroen van Meeuwen
--
Senior Engineer, Kolab Systems AG
e: vanmeeuwen at kolabsys.com
t: +44 144 340 9500
m: +44 74 2516 3817
w: http://www.kolabsys.com
pgp: 9342 BF08
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
From mollekopf at kolabsys.com Tue Aug 30 16:11:49 2011
From: mollekopf at kolabsys.com (Christian Mollekopf)
Date: Tue, 30 Aug 2011 16:11:49 +0200
Subject: Request for Input: Storing Searches
In-Reply-To: <201108301403.42063.vanmeeuwen@kolabsys.com>
References: <2232400.u9sBjAbLUj@katana.lair> <2522707.RBIlZQlMEe@myhost>
<201108301403.42063.vanmeeuwen@kolabsys.com>
Message-ID: <1429331.OZmtLNvdxk@myhost>
On Tuesday, August 30, 2011 02:03:29 PM you wrote:
> Christian Mollekopf wrote:
> > Something along the lines of tagging the search results via the ANNOTATE
> > Extension and using an XML object will probably gives us the most
> > flexibility.
>
> Searches may or may not include non-IMAP content (such as LDAP content).
>
> Why / How would a Kolab XML object help? Why not save the information in a
> "virtual" IMAP folder?
>
> This IMAP folder can handle the ACLs, thus be shared, it shows up empty for
> clients that are unable to handle KEP #9, and *at your option* could be
> prepopulated with the results of the search (OPTIONAL) - though the tradeoff
> would be data duplication.
>
True, I thought by using something else than an empty folder we don't have to
hide the empty folder. Otherwise I guess an IMAP folder per search would do as
well.
> > There are however some problems existing:
> >
> > On the Akonadi side searches are afaik implemented as virtual folders
> > which are populated based on a nepomuk query(sparql). Searches don't
> > belong to a single resource however, since you can search accross
> > various resources.
> This is a client-side (saved) search, that cannot be shared with any other
> client, correct?
>
Yes, that is correct.
> We are attempting to put the saved search on the server-side somehow.
Clear.
>
> > This means we would have to implement that functionality from outside of
> > the kolab resource I reckon.
>
> What do you mean by 'the kolab resource'? An account? A folder? A single
> Kolab XML object? The yet-to-be-defined Kolab saved-search XML object?
>
Sorry for not being more specific. Whenever I used the word resource I am
referring to an akonadi resource (aka agent).
The connection to kolab is in akonadi realised as a kolab-resource which
handles all the kolab specific bits and then exposes the data with
message/rfc822 messages. In a similar way there exist resources to access
email from pop3 servers etc. you get the idea.
See http://community.kde.org/KDE_PIM/Akonadi for more info.
> > - The action would then pick the results from the search which are in
> > this resource, and tag them via ANNOTATE and create an xml object with
> > the search info.
>
> Where would this XML object be put?
>
I'd imagine that we put those objects in the rootfolder or in a "Saved
Searches" folder. In the "Saved Searches" folder we could then also create the
optional server-side populated search directories.
> > From here were going the same path if we are consumer or creator of the
> > search:
> >
> > - The resource creates a virtual-folder inside the resource i.e. "Shared
> > Searches/Last Search" (The name can be edited and is stored to the xml
> > object), based on the xml object.
>
> "the resource" and "the resource" keep confusing me. Also, "Shared searches"
> implies the searches are shared... I think "saved searches" are more
> appropriate.
Yeah, my "shared" was more referring to shared among different machines,
Saved searches are probably more appropriate.
>
> > - The resource populates the virtual folder with virtual items based on
> > the tags.
> >
> > => - No data duplication
>
> This has always been "optional"; a saved search folder *could* be pre-
> populated.
>
> Imagine a saved search across 5 contact folders with 10.000 contacts on
> average.
>
> When NOT pre-populating the saved search folder with the search results, you
> pay the cost every time the folder is opened. Maybe this cost is not so
> great for a fat client with a local cache to query
> (Kontact/akonadi/nepomuk/Disconnected IMAP), but for a web-interface...
> well...
Yes, since akonadi can create virtual folders it only has to populate them
once AFAIK, and then result is then cached.
For a webclient I guess you're right. But if we have the dataduplication and
it should also be writable it looks somewhat error prone to me.
Especially if we have to implement that for every client.
I reckon using akonadi as a cache for the webinterface would solve that
problem?
>
> When pre-populating the saved search folder with the search results, you pay
> the cost in "duplicate" storage (as explained, not on the server side,
> perhaps on the client side if it's not intelligent enough to de-duplicate).
>
Well, the argument for doing it server side would be that it is available on
any client (i.e. smartphone), but then read-only is the only option i see.
If it is on the client side, this ends up to be essentially the same as the
akonadi virtual folders.
> Another penalty in pre-populating the saved search folder could, arguably,
> be that perhaps there's results in said saved search folder that the person
> using the folder would otherwise not have access to. However, this can also
> be considered a feature; "Share all contacts from Vendors folder tagged
> with 'ict' with helpdesk personnel"
>
If it is being populated on the client side I don't see how you could get
access to items you shouldn't have access to.
With my best regards,
Christian
> Kind regards,
>
> Jeroen van Meeuwen
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part.
URL:
From vanmeeuwen at kolabsys.com Wed Aug 31 14:13:24 2011
From: vanmeeuwen at kolabsys.com (Jeroen van Meeuwen (Kolab Systems))
Date: Wed, 31 Aug 2011 13:13:24 +0100
Subject: Request for Input: Storing Searches
In-Reply-To: <1429331.OZmtLNvdxk@myhost>
References: <2232400.u9sBjAbLUj@katana.lair>
<201108301403.42063.vanmeeuwen@kolabsys.com>
<1429331.OZmtLNvdxk@myhost>
Message-ID: <201108311313.25074.vanmeeuwen@kolabsys.com>
Christian Mollekopf wrote:
> > > - The action would then pick the results from the search which are in
> > > this resource, and tag them via ANNOTATE and create an xml object with
> > > the search info.
> >
> > Where would this XML object be put?
>
> I'd imagine that we put those objects in the rootfolder or in a "Saved
> Searches" folder. In the "Saved Searches" folder we could then also create
> the optional server-side populated search directories.
>
There's no such thing as a 'root folder'.
If a 'Saved Searches' folder were to be used, all 'saved search' Kolab XML
objects would go into that one folder.
Sharing any particular saved search now becomes a problem,
Clients not compatible with KEP #9 are now helpless, since a top-level folder
of unknown type is encountered, but they have to descend in order to get to
any sub-folder,
The saved searches folder(s) cannot be pre-populated unless the Kolab XML
object for saved searches also states where any pre-populating should go out
to, which naturally is subject to too much change,
Keeping 'reference' objects in a 'saved search' folder creates the same
'subject to change' problem... if a reference object where to say,
user/john.doe/Contacts at example.org?uid=blabla, renaming the Contacts folder to
Kontacten would create a reference issue; any reference should be completely
referential (OLAP),
etc.
> > > - The resource populates the virtual folder with virtual items based on
> > > the tags.
> > >
> > > => - No data duplication
> >
> > This has always been "optional"; a saved search folder *could* be pre-
> > populated.
> >
> > Imagine a saved search across 5 contact folders with 10.000 contacts on
> > average.
> >
> > When NOT pre-populating the saved search folder with the search results,
> > you pay the cost every time the folder is opened. Maybe this cost is not
> > so great for a fat client with a local cache to query
> > (Kontact/akonadi/nepomuk/Disconnected IMAP), but for a web-interface...
> > well...
>
> Yes, since akonadi can create virtual folders it only has to populate them
> once AFAIK, and then result is then cached.
> For a webclient I guess you're right. But if we have the dataduplication
Again, the data duplication is *at one's option*. For web-interface, I say one
should pre-populate the search folder. For clients like Kontact (with client-
side, local caches), perhaps it's feasible to allow them to ignore the cached
results and go with a real-time search.
> and it should also be writable it looks somewhat error prone to me.
A clause in the KEP for these types of folders can be, that the content of the
folder SHOULD NOT be made writeable for any event other then 'update saved
search'.
> Especially if we have to implement that for every client.
>
We have to implement everything and anything for every client, in case you
haven't noticed.
> I reckon using akonadi as a cache for the webinterface would solve that
> problem?
>
No, the caching layer is moot unless you also consider all clients use
akonadi. One cache (in one location) to rule them all being akonadi is not
necessarily the best way to go with this. This, however, is a different topic,
and we should talk about caching separately from the saved searches topic.
> > When pre-populating the saved search folder with the search results, you
> > pay the cost in "duplicate" storage (as explained, not on the server
> > side, perhaps on the client side if it's not intelligent enough to
> > de-duplicate).
>
> Well, the argument for doing it server side would be that it is available
> on any client (i.e. smartphone), but then read-only is the only option i
> see. If it is on the client side, this ends up to be essentially the same
> as the akonadi virtual folders.
>
Note that the "problem" or "difficulty" is not the editing of an object from
within the saved search, not for a client and not for a user.
It is the occurence of said object twice or more times in or across all
readable folders that is the first problem.
> > Another penalty in pre-populating the saved search folder could,
> > arguably, be that perhaps there's results in said saved search folder
> > that the person using the folder would otherwise not have access to.
> > However, this can also be considered a feature; "Share all contacts from
> > Vendors folder tagged with 'ict' with helpdesk personnel"
>
> If it is being populated on the client side I don't see how you could get
> access to items you shouldn't have access to.
>
You want to avoid your 350.000 clients from each having to iterate and re-
iterate against most of your infrastructure components themselves, just to
pre-populate and update their saved (contact) searches cache(s), if you can do
so periodically on the server-side, under your own control.
Please note that the folders of type 'contact' that the user has access to are
not the only things that are subject to change. Please also note that these
folders and other resources can be *huge*.
Saved searches are often not the smallest set of search results. They often
include a type of query that is a little less specific then
(mail=john.doe at example.org), and inherently can include a lot of attributes
that are not (cannot be?) indexed anywhere.
As such, saved searches are *hugely* expensive to execute.
When they happen on the client side, and on one particular client only,
configurable per client, then we're all fine. Kontact for instance can do
saved searches in a reasonable fashion because of Akonadi. A web interface
such as Horde or Roundcube however cannot. Please note a saved search for
these interfaces will have to;
1) execute within the PHP execution timeout (30 seconds),
2) stay within the memory_limit (64/128/192 MB),
3) do not use the user's credentials / privileges to query resources other
then IMAP
Furthermore, interesting saved searches discussions can be held over
particular types of Kolab clients such as Z-Push (and its clients). Not all
Kolab-consuming applications have a need for, an interface for, or a place for
saved searches or server-side caching of said saved searches.
Similarly, not all clients have a need for, or place for, a server-side cache
-such as Disconnected IMAP in Kontact.
That said, a folder clearly configured as a 'saved_search' folder can be
ignored (Z-Push?) or the contents thereof can be ignored (Kontact?), while the
contents may be the periodically updated search results, or the folder be
empty. Both cases serve all clients well, KEP #9 compatible or not.
Kind regards,
Jeroen van Meeuwen
--
Senior Engineer, Kolab Systems AG
e: vanmeeuwen at kolabsys.com
t: +44 144 340 9500
m: +44 74 2516 3817
w: http://www.kolabsys.com
pgp: 9342 BF08
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
From vkrause at kde.org Wed Aug 31 15:25:40 2011
From: vkrause at kde.org (Volker Krause)
Date: Wed, 31 Aug 2011 15:25:40 +0200
Subject: Request for Input: Storing Searches
In-Reply-To: <2232400.u9sBjAbLUj@katana.lair>
References: <2232400.u9sBjAbLUj@katana.lair>
Message-ID: <2016124.HU9QVM6cYj@vkpc9>
Hi,
Christian already pointed out some implications on Kontact/Akonadi, but I have
some additional remarks on that as well.
On Saturday 27 August 2011 20:39:40 Georg C. F. Greve wrote:
> Some of us have started tossing around thoughts about how to save searches
> in one Kolab client in a way that they are re-usable in all others,
> ideally.
>
> When giving it some brainspace, it turns out this is not a trivial issue,
> for a variety of reasons, starting with there being a tradeoff decision
> between being expensive for the CPU or storage, for instance. But it is a
> little bit more complex than that, actually.
>
> Allow me to higlight a couple of scenarios with advantages and
> disadvantages:
>
> - Scenario 1: Storage with a new KEP 9 based XML object
>
> One could attempt to model this as a "search" XML object that would
> incorporate the fields of the object type searched, plus some special
> fields, e.g. folders to search, as well as searches across multiple fields
> and search logic (AND/OR etc),
Here you have the first challenge already (we run in to the same issue during
the work on Akonadi, and it's still not fully solved there): You need to
define the exact semantics of the query language and make sure all clients can
actually implement this. Seems easy at first ("person's name equals 'Georg'"),
but you quickly end up with ugly details:
- what kind of Boolean logic do you want to support, and in what nesting
depth?
- what kind of comparison operators do you want to support for strings? equal,
contains, matches regexp, case sensitive vs. insensitive, etc
- what comparison operators do you want to support for other types (numbers,
date/time)? equal, greater/less than, occurs at, etc. (keep in mind the extra
fun with DST/time zones and recurrences), also note the complexity
implications of recurrence-related queries with negations (e.g. "event does
never occur on a Friday 13th")
- on what fields are you operating? Just the ones defined in the Kolab format
spec, or is that too low-level for users and you want higher level composite
fields? Eg. a person's name maps to a bunch of internal fields.
- next to the low-level search operators, are you considering higher-level
ones that depend on external context (eg. something along the lines of KMail's
"mail from someone in my address book" or "mail from someone tagged as
'Friend'" operators, or your example of "events in Germany")?
If you want client-side search with consistent results, this needs to be
specified in detail. Unlike the format specification where we only display the
most of content without interpretation (except most date/time fields), this
actually requires agreement on exact semantics.
> These objects would live in the regular folders for resources, and would
> potentially even replace the list object in functionality, as they would
> then model a list of recipients as list of address book entries, which is
> something that Alain once suggested. [1]
>
> Advantages: Fairly close to existing functionality, and likely not too
> hard to implement for most clients (in comparison, at least), no data
> duplication anywhere.
implementation complexity for us would mainly depend on the complexity of the
query language and how well it maps to our existing search support.
> Disadvantages: Expensive on the CPU, Does not work on all resources
> because we cannot store these XML elements in email type mailboxes.
what resources are you referring to here?
> - Scenario 2: Creation of new folder type w/KEP #9 annotation for metadata,
> create one folder per saved search
>
> In this approach we'd create a new folder of the corresponding resource
> folder type for each search which would be identified as a stored search
> folder by existence of the /vendor/kolab/saved-search annotation which
> carries the metadata for the search in an array, e.g.
>
> { 'saved_search':
> { 'search_locations': 'blabla',
> 'params': 'blabla',
> 'filter': 'blabla',
> 'fuzzyness': 'blabla',
> 'async': '0'
> ....
> }
> }
>
> and the folder would be populated with the results of the search.
>
> This DOES mean data duplication on the client, but Cyrus does allow to
> deduplicate entries on the server side, so it would not affect storage
> there. I am sure something similar would be possible with Dovecot, so
> we can for the moment assume data gets duplicated on the client only.
This is actually very close to the Akonadi approach for searching. We also
have dedicated folders for search results ("virtual folders"), which contain
only references to the actual object (sort of like symlinks). This avoid
duplication and de-synchronization issues etc.. If the Kolab server would
allow us to retrieve information about the original location of an object in a
pre-populated search folder, we could symlink that on the client as well.
There are some pros and cons of the server-side search approach though:
+ there's a single implementation, so same semantics and expressive power
everywhere
+ minimal client changes required
- no search possible when offline
- increasing de-synchronization between results and reality while being
offline
> Advantages: Allows clients without search functionality to use results,
> can be automatically regenerated on the server if needs be, least CPU
> usage, works on email.
>
> Disadvantages: Data duplication on the client, possible data set de-
> synchronization (e.g. contact gets edited in search results, same contact
> in main box and other search results boxes must be updated, this may be
> hard to ensure), increases folder clutter, some folder sharing questions.
>
>
> - Scenario 3: Map searches with tags
>
> As a Kolab object, each search will carry an ID. If we were to introduce
> a new email header flag in storage that can carry an arbitrary number
> of tags, we could tag each object with the ID of every search that it
> matches.
>
> IMAP searching for header fields should make it comparatively easy
> and fast to find all objects of type X that match a certain tag Y,
> especially if we ask the server to cache this header field.
>
> This would be complemented by a KEP 9 compatible object to describe
> the search, which could then be automatically applied to new objects on
> the server, or performed by the client, based on the scenario.
This assumes there is agreement on the exact semantics, otherwise you could
end up with nasty tag ping-pong games between multiple clients and/or the
server.
> Advantages: Low CPU & storage requirements, allows Kolab clients to apply
> a tag concept over all object types including email with potential server
> side tagging of incoming email
>
> Disadvantages: New concept, some questions around shared folders, e.g.
> what if a client sees a shared object tagged with an ID for a search it
> does not know because it does not have access to the folder where that
> search is defined?
Unlike the other two option, this doesn't really fit well into the Akonadi way
of dealing with searches (searching by tags is easy, but we are not prepared
to tag and write back based on queries yet), so this would be the most
expensive one for us to implement I think.
> There may be other advantages and disadvantages that I did not list.
>
> Please help us identify them all, so we can come to a good decision.
>
> Likewise, if you can think of a scenario that should be considered in
> addition to the ones listed here, please let me know. As for the scenarios
> listed, there are two questions in particular that I wonder about:
>
> (a) Compatibility with clients, in particular: How will this integrate (or
> not) with the new Nepomuk/Akonadi KDE Kontact basis
The biggest challenge for Kontact would be the fact that these searches are
limited to just one Kolab account (and would most likely be context-free).
Both are limitations we have been working on getting rid of. Ie. we would like
the user to see his data as one set, without having to worry where it's coming
from and what capabilities follow from that. Also, we are moving to more
higher-level context-aware query/search concepts (e.g. "persons near my
current location"). We already have a similar problem with KMail local filters
and server-side Sieve filters, the latter basically having the same
limitations as a per-account cross-client saved search would have.
Of course, if you look at this from the web client POV, you don't really have
these kind of problems.
regards
Volker
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 190 bytes
Desc: This is a digitally signed message part.
URL:
From greve at kolabsys.com Wed Aug 31 20:10:02 2011
From: greve at kolabsys.com (Georg C. F. Greve)
Date: Wed, 31 Aug 2011 20:10:02 +0200
Subject: NEW KEP: KEP #14: Non-conflicting edits of RFC5228/Sieve scripts by
multiple editors
Message-ID: <2367431.zDd980amMF@katana.lair>
Dear all,
Based on the initial idea circulated in May on kolab-devel, please be informed
of a new Design KEP to enable non-conflicting mail filter edits on the server by
multiple clients:
KEP 14: Non-conflicting edits of RFC5228/Sieve scripts by multiple editors
http://wiki.kolab.org/User:Greve/Drafts/KEP:14
This is more on the convention/application behaviour side and not really a
format modification, so all comments/discussion should take place on
kolab-devel at kolab.org
for this particular KEP.
Best regards,
Georg
--
Georg C. F. Greve
Chief Executive Officer
Kolab Systems AG
Z?rich, Switzerland
e: greve at kolabsys.com
t: +41 78 904 43 33
w: http://kolabsys.com
pgp: 86574ACA Georg C. F. Greve
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 308 bytes
Desc: This is a digitally signed message part.
URL: