KEP-0002.txt

Georg Greve greve at kolabsys.com
Fri May 20 16:34:23 CEST 2011


 KEP-0002.txt |   76 +++++++++++++++++++++++++++--------------------------------
 1 file changed, 36 insertions(+), 40 deletions(-)

New commits:
commit 9426db607f9053f9b0b76d171b3b643e15ab9c14
Author: Georg Greve <greve at kolabsys.com>
Date:   Fri May 20 16:35:25 2011 +0200

    One more round of feedback from Florian von Samson & Bernhard Reiter

diff --git a/KEP-0002.txt b/KEP-0002.txt
index c1410d7..46d7756 100644
--- a/KEP-0002.txt
+++ b/KEP-0002.txt
@@ -1,5 +1,3 @@
-{{warning|Overview Discussion Page|This KEP has an overview of the discussion and summary of options and considerations at [[KEP_2_Arguments]].}}
-
 {{kep
  |number=2
  |ticketnumber=
@@ -16,19 +14,19 @@
 __TOC__
 == Abstract ==
 
-Kolab used to store all times in UTC and did not allow for time zone information. This was too simplistic an approach to solve the complex issues caused by time zones and their DST rules, as explained in the [[#Description_of_the_issue | background section]] below. 
+This is a Kolab Enhancement Proposal (KEP)<ref name"kep">[[KEP:1]]</ref> to update how the Kolab Format version 2.0 (XML version 1.0) stored date/time fields. All times were stored in UTC and did not allow for time zone information. This was too simplistic an approach to solve the complex issues caused by time zones and their DST rules, as explained in the [[#Description_of_the_issue | background section]] below. 
 
-It is primarily two issues that are being addressed by this KEP, one essential and technical, the other related to usability. The usability related issue is that users sometimes specifically set time zones for datetime fields and expects this explicit selection to be preserved across sessions and clients. Without storage of this information clients cannot meet that user expectation.
+This KEP addresses issues of usability as well as function. The usability related issue is that users sometimes specifically set time zones for datetime fields and expect this explicit selection to be preserved across sessions and clients. Without storage of this information clients cannot meet that user expectation.
 
-The functional issue is the more important of the two: For non-recurring events there can be errors in the display of an event's time if DST rules have changed since the event was made. For recurring events in parts of the world with DST regimes it was impossible to define a recurrence that takes place at the same local time all year and is correctly displayed by clients in other time zones. Both issues are accentuated by the fact that DST rules are subject to political decisions taken in the future, and consequently unknown today.
+The functional issue is the more important of the two: For non-recurring events there can be errors in the display of an event's time if DST rules have changed since the event was made. For recurring events in parts of the world with DST regimes it was impossible to define a recurrence that takes place at the same local time all year and is correctly displayed by clients in time zones with different DST rules, in particular different times of switching. Both issues are accentuated by the fact that DST rules are subject to political decisions taken in the future, and consequently unknown today.
 
 In order to achieve a recurring event that retains its local time across DST transitions, a client must know which time zone to use. The implicit assumption of older clients to always use local time zone is problematic, as explained in subsection "Description of current client behaviour" below. So enabling time zone information for datetime fields is essential.
 
-Some reference for background was provided on the Kolab format list <ref name="kolab-format-list-background">Georg Greve, http://kolab.org/pipermail/kolab-format/2010-October/001004.html</ref> and there have been various proposals for resolution of the issue, including adding time zone information in a separate XML tag, along with DST details <ref name="dst-details">Henrik Helwich, http://kolab.org/pipermail/kolab-format/2010-October/000999.html</ref>. This document is based upon those discussions and their followup throughout October, November and December 2010.
+Some reference for background was provided on the Kolab format list <ref name="kolab-format-list-background">Georg Greve, http://kolab.org/pipermail/kolab-format/2010-October/001004.html</ref> and there have been various proposals for resolution of the issue, including adding time zone information in a separate XML tag, along with DST details <ref name="dst-details">Henrik Helwich, http://kolab.org/pipermail/kolab-format/2010-October/000999.html</ref>. This document is based upon those discussions and their followup throughout October 2010 through May 2011.
 
-== Update to the XML Format ==
+== Update to the Kolab Format ==
 
-All Kolab object types hold datetime in the form of creation and modification times. Several other object types also hold other datetime fields. This KEP describes the canonical format for all datetime fields across all Kolab object types. This will ensure consistency and is part of the changeset for version 1.1 of all object types.
+All Kolab object types hold datetime in the form of creation and modification times. Several other object types also hold other datetime fields. This KEP describes the canonical format for all datetime fields across all Kolab object types. This will ensure consistency and is part of a changeset applied against version 1.0 of all object types of the Kolab Format 2.0.
 
 Change of type: '''datetime'''
 
@@ -60,11 +58,11 @@ The "Z" to specify a time in UTC '''MUST''' only be used for times in UTC and ''
 
 Per ABNF, ISO8601 and RFC3339, the "T" and "Z" characters in this syntax are explicitly defined as the upper-case letters, usage of the lower-case letters is explicitly forbidden.
 
-{{note|Usage of time-secfrag|time-secfrag usage is regulated on a per object/per field basis, and '''SHALL''' be explicitly forbidden unless required.}}
+{{note|Usage of time-secfrag|time-secfrag usage is regulated on a per object/per field basis.}}
 
 ===== Examples =====
 
-Valid date-time fields according to the above specification are
+Valid date-time fields according to the above definition are
 
   2010-01-31T11:27:21Z
   2005-12-19T02:55:23.437689098765
@@ -72,7 +70,7 @@ Valid date-time fields according to the above specification are
   2005-12-19T02:55:23.43Z
   2011-05-01
 
-==== Kolab XML Datetime Type ====
+==== Kolab Format Datetime Type ====
 
 A field of "datetime" type '''MUST''' be compliant to the Kolab ISO8601 profile.
 
@@ -82,9 +80,9 @@ Where the definition of a field explicitly specifies storage in UTC only, the "t
 
 Where the definition of a field explicitly allows storage in local time, the "tz" attribute '''MUST''' be used in all cases, including for storage of UTC.
 
-{{note|Storage of local time|Storage of local time and consequently usage of the tz attribute are regulated on a per-field basis, and '''SHOULD''' be explicitly required or forbidden.}}
+{{note|Storage of local time|Storage of local time and consequently usage of the tz attribute are regulated on a per-field basis.}}
 
-==== Kolab XML Date Type ====
+==== Kolab Format Date Type ====
 
 The Kolab Date Type is defined as full-date.
 
@@ -92,7 +90,7 @@ Where a field explicitly refers to a full-date '''ONLY''', the "tz" attribute ''
 
 ===== Examples =====
 
-Valid date-time fields according to the above specification are
+Valid date-time fields according to the above definition are
 
   <field>2010-01-31T11:27:21Z</field>
   <field tz="Europe/Berlin">2005-12-19T02:55:23.437689098765</field>
@@ -104,7 +102,7 @@ Valid date-time fields according to the above specification are
 
 * Clients '''MUST''' store all date and/or datetime fields not based on user interaction/fields that are automatically generated '''-- AND --'''  carry values in the past or present in '''UTC only'''. This explicitly includes the following fields: 'creation-date', 'last-modification-date' of all Kolab object types.
 * Clients '''MUST''' store all date and/or datetime fields based on direct user interaction '''-- OR ---''' fields that may carry values which are '''NOT''' limited to UTC only in local time using the "tz" attribute. This explicitly includes the following fields: 'start-date', 'end-date' of all Kolab object types.
-* Clients '''MUST NOT''' use fractions of seconds (time-secfrag) for any datetime field in any Kolab object type unless the definition of that field and object specifically permit or require time-secfrag, which '''SHOULD''' always be done in a way to specify the maximum number of digits. Fractions of sections '''MUST NOT''' be used for any field in object types: 'note', 'contact', 'distribution-list', 'journal', 'event', 'task'. 
+* Clients '''MUST NOT''' use fractions of seconds (time-secfrag) for any datetime field in any Kolab object type unless the definition of that field and object specifically permit or require time-secfrag, which '''MUST''' always be done in a way to specify the maximum number of digits. Fractions of sections '''MUST NOT''' be used for any field in object types: 'note', 'contact', 'distribution-list', 'journal', 'event', 'task'. 
 * Clients '''MUST''' be capable of reading date and datetime fields that comply with the writing rules of this KEP and subsequent definitions of the Kolab object types they process.
 * Clients '''MUST''' preserve user preference and selection in the "tz" attribute to the maximum extent possible.
 * Clients '''SHOULD''' check if a new update of the Olson database or the authoritative database used by the system is available and get that update at least once every three months, '''OR''' suggest update policies for their respective operating systems that ensure the time zone data database gets updated regularly. As far as is currently known, all commonly used and supported GNU/Linux distributions do this already.
@@ -115,40 +113,40 @@ Valid date-time fields according to the above specification are
 * When creating a new object with time zone sensitive fields, clients '''SHOULD''' default to the local time zone of the user, but '''SHOULD''' allow the user to select the time zone for storage and consequently recurrence calculation;
 * When modifying existing objects, clients '''MUST''' use the value of the 'tz' attribute of the respective fields to set the default/preselected value for the editing of the fields, where applicable. For instance the 'start-date' and 'end-date' time zone defaults if presented to the user by the client '''MUST''' match those stored in the 'tz' attribute. The time zone stored in the 'tz' attribute '''SHOULD''' only be changed based upon user interaction;
 * When calculating recurrences, a client '''MUST''' calculate in a way that keeps the event at the same local time in the time zone stored in the 'tz' attribute. Clients '''MUST''' then use the result as the time from which to calculate the time of the event at the client's time zone. For more detail see [[#Notes_for_client_implementors | Notes for client implementors]] below);
-* When receiving iTip invitations, a client '''MUST''' treat the time zone id in the VTIMEZONE object as authoritative and, if it is not a valid Olson database time zone identifier, translate it using the translation matrix provided by Kolab Systems in cooperation with the Kolab ecosystem. If the time zone id in the VTIMEZONE element does not exist in the matrix, clients '''MAY''' attempt to map the time zone based on its rules to a currently used time zone -- '''AND/OR''' -- allow the user to select an appropriate time zone for event storage;
+* When receiving iTip invitations, a client '''MUST''' treat the time zone id in the VTIMEZONE object as authoritative and, if it is not a valid Olson database time zone identifier, translate it using the tzid mapping table provided by the Kolab community. If the time zone id in the VTIMEZONE element does not exist in the tzid mapping table, clients '''MAY''' attempt to map the time zone based on its rules to a currently used time zone -- '''AND/OR''' -- allow the user to select an appropriate time zone for storing an event;
 * For recurrence calculation: When tz is specified as 'UTC', a client '''MUST''' calculate recurrences strictly according to UTC;
-* For recurrence calculation: Where tz is missing although the specification required it, a client '''SHOULD''' calculate recurrences strictly according to UTC;
-* When clients encounter deviations from the schema, e.g. parsing datetime objects that do not match the writing conventions, or a missing 'tz' attribute for start-date or end-date in an event using version 1.1 of the XML schema, clients '''SHOULD''' inform the user of a potential issue, using the 'product-id' to help the user identify clients that might be broken also in ways that could corrupt other data. There will likely be an explicit KEP on this issue at a later point in time. This mechanism '''MAY''' also be used for the update strategy, see below.
+* For recurrence calculation: Where tz is missing although the Kolab Format required it, a client '''SHOULD''' calculate recurrences strictly according to UTC;
+* When clients encounter deviations from the schema, e.g. parsing datetime objects that do not match the writing conventions, or a missing 'tz' attribute for start-date or end-date in an event using a version of the Kolab Format based on this KEP, clients '''SHOULD''' inform the user of a potential issue, using the 'product-id' to help the user identify clients that might be broken. There will likely be an explicit KEP on this issue at a later point in time. This mechanism '''MAY''' also be used for the update strategy, see below.
 
 === Examples ===
 
-Examples of valid 'start-date' fields using datetime structures according to the above specification are
+Examples of valid fields using datetime structures according to the above definition are
 
  <start-date tz="Europe/Rome">2011-05-01</start-date>
  <start-date tz="UTC">2010-01-31T11:27:21Z</start-date>
  <start-date tz="America/Los_Angeles">2005-12-19T02:55:23</start-date>
  <start-date tz="Europe/Brussels">2001-06-19T11:01:23</start-date>
  <last-modified>2011-04-01T01:02:33Z</last-modified>
- <future-high-precision-timestamp tz="America/Sao_Paulo">2001-06-19T16:39:57.1229853</future-high-precision-timestamp>
+ <hypothetical-high-precision-timestamp tz="America/Sao_Paulo">2001-06-19T16:39:57.1229853</hypothetical-high-precision-timestamp>
 
-{{note|Strict writing - loose parsing|The rules above have been carefully designed to address all the issues while tightening the writing rules as much as sensible to keep time stamps as consistent as possible. This is not to say that clients should not also understand times stamps that are slightly incorrect, but still consistent and readable. In such cases, clients should do their best to parse the time stamp correctly, but warn the user or administrator of a client that is not compliant with this KEP so the incorrect client can be fixed. Clients '''MUST NEVER''' rely on other clients' lax parsing.}}
+{{note|Strict writing - loose parsing|The rules above have been carefully designed to address all the issues while tightening the writing rules as much as sensible to keep time stamps as consistent as possible. This is not to say that clients should not also understand times stamps that are formally incorrect, but still are consistent and readable. In such cases, clients should do their best to parse the time stamp correctly, but warn the user or administrator of a client that is not compliant with this KEP so the incorrect client can be fixed. Clients '''MUST NEVER''' rely on other clients' lax parsing.}}
 
 === Upgrade Path ===
 
-The previous date time format that was used by Kolab XML formats up to and including version 1.0 based on strict UTC Zulu notation continues to be the authoritative form of writing all the automatically/software generated fields which carry information that is in the past or present. So all old data can be parsed by new parsers, and old clients will continue to understand at least some of the fields written in the new format. 
+The previous date time type that was used by the Kolab Formats up to and including version 2.0 based on strict UTC Zulu notation continues to be the authoritative form of writing all the automatically/software generated fields which carry information that is in the past or present. So all old data can be parsed by new parsers, and old clients will continue to understand at least some of the fields written according to future Kolab Formats based upon this KEP. 
 
 There will '''NOT''' be backwards compatibility for all types of newer objects, however. While in principle all clients should already preserve tags and attributes they do not understand, not all older clients properly guarantee this at the current point in time. So the newly introduced 'tz' attribute would be in peril of being stripped out by older clients. Older clients are also likely to falsely interpret data written by newer clients, potentially corrupting it upon write.
 
-And finally it is unavoidable that older clients will continue to behave as they did thus far, continuing to display recurrence times incorrectly.
+And finally it is unavoidable that older clients will continue to behave as they did thus far, continuing to display some recurrence times incorrectly.
 
 === Smart Upgrade Option ===
 
-When encountering data written to the old specification prior to this KEP, clients '''MAY''' choose to
+When encountering data written to the Kolab Format prior to this KEP, clients '''MAY''' choose to
 # continue to display the old data sets as the old clients did, leaving the old data unchanged;
-# bring up a dialogue informing the user of the change in data format, and suggest an update which should usually be done by the proper editing dialogue (especially for events) so users can provide the data that was absent in the old format;
-# silently update the data to the new data format, based on the assumptions that older clients made when interpreting this data, thus maximally preserving client behavior.
+# bring up a dialogue informing the user of a change of Kolab Format version and suggesting an update which should usually be carried out in the respective editing dialogue (especially for events) so users can provide the data that was absent in the old format;
+# silently update the data to the newer Kolab Format, based on the assumptions that older clients made when interpreting this data, thus maximally preserving client behavior.
 
-The first is the '''recommended''' approach for non-recurring events, the second is the '''recommended''' approach for recurring events. The third should only be used with extreme caution and ideally some explicit user interaction for the entire process, i.e. an "update wizard" after which only new clients will be connected to this server.
+The first option is the '''recommended''' approach for non-recurring events, the second is the '''recommended''' approach for recurring events. The third option should only be used with extreme caution and ideally some explicit user interaction for the entire process, e.g. an "update wizard".
 
 As all of these approaches include additional work for client implementers, none of these are required.
 
@@ -156,26 +154,24 @@ In any case it is '''NOT''' recommended to ever have older and newer clients coe
 
 === Notes for client implementors ===
 
-There may be future use cases for time zone storage and DST calculations, such as other Kolab object types.
-
-For events and tasks, which are the primary use case of the particular issues this KEP addresses, there are two existing uses:
+For events and tasks, there are two existing use cases this KEP addresses:
 
 ; '''Store user preference'''
-: A user typically has selected a time zone to enter a date/time. When storing without timezone information, that information is lost. So while the user might realistically expect the event to preserve the time zone they entered initially when editing it again, Kolab was unable to provide this functionality thus far.
+: A user typically has selected a time zone to enter a date/time, either implicitly or explicitly. When storing without timezone information, that information is lost. So while the user might realistically expect the event to preserve the time zone they entered initially when editing it again, Kolab was unable to provide this functionality thus far.
 
 ; '''Event time calculation'''
 : Other than for the storage of user preference, recurrences are the most important but not the only use case for this Kolab Enhancement Proposal.
 :* Where the 'tz' attribute is explicitly set to 'UTC' or missing (in which case clients '''SHOULD''' also issue a warning due to an incorrect data format), clients '''MUST''' implement strict mapping to UTC. Otherwise the time zone stored in the 'tz' attribute '''MUST''' be considered authoritative, and the value of the time stored in the datetime field '''MUST''' be considered local to this time zone;
-:* Where the authoritative time zone is a local time zone, clients '''MUST''' arrive at the corrected authoritative time by using their client- or system-wide time zone database (e.g. Olson, a.k.a. tzdata) to calculate the event, its possible recurrences, and its offset to UTC based on the database information;
-{{note|Example|An event set for time zone 'Europe/Brussels' with the value '2010-12-24T18:00:00' should be interpreted as '18:00 in Brussels on 24 December 2010', so calculated with the correct UTC offset of +01:00 for standard time which is in effect.}}
-:* For recurring events, the same methodology '''MUST''' be used for each instance of a recurring event;
-:* This corrected authoritative time can then be displayed (if the same as local time zone), be translated into the local time zone for display, or be translated to UTC for UTC based use cases.
+:* Where the 'tz' attribute is explicitly set to something other than 'UTC', clients '''MUST''' use a time zone database (e.g. Olson, a.k.a. tzdata) to calculate the event;
+{{note|Example|An event set for time zone 'Europe/Brussels' with the value '2010-12-24T18:00:00' should be interpreted as '18:00 in Brussels on 24 December 2010', so calculated with the correct UTC offset of +01:00 for standard time, which is in effect on December 24th.}}
+:* For recurring events, the same methodology '''MUST''' be used each time the event occurs;
+:* This resulting time can then be displayed (if the same as local time zone), be translated into the local time zone for display, or be translated to UTC for UTC based use cases.
 
-; '''Olson mapping database'''
-: There should be mapping between time zone identifiers from and to the Olson database locations for systems not based on the Olson time zone identifiers, i.e. Microsoft Windows. In order to achieve that, Kolab Systems will work with the various client implementers to provide a canonical database that all clients without Olson database can use for such mapping on their systems against the Olson database locations. This database will be provided in a location that is freely and publicly available.
+; '''tzid mapping table'''
+: There should be mapping between time zone identifiers from and to the Olson database locations for systems not based on the Olson time zone identifiers, i.e. Microsoft Windows. In order to achieve that, Kolab Systems will work with the various client implementers to provide a canonical mapping table that all clients without Olson database can use for such mapping on their systems against the Olson database locations. This tzid mapping table will be provided publicly under a Free Software license.
 
 ; '''Invitation handling'''
-: The mapping database should also be used for handling iTip invitations, which carry an arbitrary number of VTIMEZONE objects. When the id of the VTIMEZONE object is unknown, clients can fall back on automatic detection against the local database and/or user choice when storing the event. Clients '''SHOULD''' offer users the possibility to send an email with the VTIMEZONE object to kolab-format at kolab.org so the ID can be included into the next revision of the mapping database.
+: The tzid mapping table should also be used for handling iTip invitations, which carry an arbitrary number of VTIMEZONE objects. When the id of the VTIMEZONE object is unknown, clients can fall back on automatic detection against the local database and/or user choice when storing the event. Clients '''SHOULD''' offer users the possibility to send an email with the VTIMEZONE object to kolab-format at kolab.org so the ID can be included into the next revision of the tzid mapping table.
 
 == Description of the issue ==
 
@@ -240,7 +236,7 @@ Existing clients currently make the implicit assumption that the time was specif
 
 A weekly meeting is set for 11:00 every Wednesday in Zurich, Switzerland, starting on 23 June 2010. This gets translated stored in UTC as 2010-02-17T09:00:00Z. On Wednesday 17 February 2010 Switzerland is using standard time, the local timezone is therefore UTC+1. If strictly interpreting the stored information, the meeting should now start at 10:00. 
 
-Versions of KDE Kontact <=4.6.3 however display the meeting as scheduled for 11:00. The same is true for the Kolab web client based on Horde for versions of Kolab <= 2.2.4. This however is equivalent to 10:00 UTC. When adding another user in Sao Paulo, Brazil to the equation, the event is shown as taking place at 06:00 local time, or 08:00 UTC, due to the Brazilian summer time with an offset of UTC-3 that went into the assumption for the calculation of the recurrence. The result is that two users, while being presented with a data set that looks consistent, will miss each other.
+Versions of KDE Kontact <=4.6.3 however display the meeting as scheduled for 11:00. The same is true for the Kolab web client based on Horde for versions of Kolab Server <= 2.2.4. This however is equivalent to 10:00 UTC. When adding another user in Sao Paulo, Brazil to the equation, the event is shown as taking place at 06:00 local time, or 08:00 UTC, due to the Brazilian summer time with an offset of UTC-3 that went into the assumption for the calculation of the recurrence. The result is that two users, while being presented with a data set that looks consistent, will miss each other.
 
 Which other clients exhibit the same behavior is unclear, but it seems there is no reasonable assumption that current behavior correctly models any rational use case.
 
@@ -291,7 +287,7 @@ Choosing the Olson time zone identifiers will simplify matters for all clients o
 
 The issue of most concern is that Microsoft Windows has its own time zone identification system. A bridge is however provided by the Unicode Common Locale Data Repository (CLDR) as well as the International Components for Unicode (ICU). The CLDR also provides mapping for Microsoft Windows time zone IDs to the standard Olson names. So using references to the Olson timezone database is likely the best out of an imperfect set of choices.
 
-In order to support clients on non-Olson platforms, as well as all clients in their iTIP <ref name="rfc2446">{{rfc|2446|title=iCalendar Transport-Independent Interoperability Protocol (iTIP)}}</ref> handling, Kolab Systems shall work with all client implementors to maintain and continue to make freely and publicly available a database to match various time zone ids to Olson database locations.
+In order to support clients on non-Olson platforms, as well as all clients in their iTIP <ref name="rfc2446">{{rfc|2446|title=iCalendar Transport-Independent Interoperability Protocol (iTIP)}}</ref> handling, Kolab Systems shall work with all client implementors to maintain and continue to make freely and publicly available a tzid mapping table to match various time zone ids to Olson database locations.
 
 === iTip / VTIMEZONE completeness ===
 





More information about the commits mailing list