[Kolab-devel] How to deal with multiplied Kolab UIDs (Re: RE :Re: Z-push most common issue with duplicateUID)

[Bernhard Reiter]

Hi Georg, Gentlemen,

Am Wednesday, 4. January 2012 13:40:44 schrieb Georg C. F. Greve:
> On Wednesday 04 January 2012 09.51:56 Bernhard Reiter wrote:
> > I believe several times a week for a productive installation
> > in a 500 person organisation in the secretary scenario.
>
> This seems a statistically interesting claim.

maybe to clarify a technical thing:
I believe that the "need for coordination" situation occurs,
but most of these situations are _not_ leading to a duplicate Kolab ID
on the server. They get resolved by the second client, who notices
that the first user has changed the object while the second user also did.

> We have the scenario of an event seeing two subsequent edits. Typically
> this means the second edit *should* overwrite the first, because something
> has changed that should now be recorded in the event.

If you come from the workflow: Yes.
Technically there is no good way to determine which edit was first
or second, as far as I know.

> The exception to this rule is where the edits originated with separate
> people who were unaware of the other person editing the event and where the
> first edit contains irretrievable information different from the second
> edit that must under no circumstances be lost and cannot be re-added.

Yes.

> So, the case has the following prerequisites of occurring:
>
>  (0) A and B are both equally entitled to edit the meeting,
> 	so are either participant or organizer

Technically all people having write access to a folder can write
an object in there. Some groups of people seem to model this without strong 
participant or organizer roles assigned. 

I know you are taking a meeting as an example. In a contact object for 
instance edits are more rare one the average, but there is no 
clear "organizer".

>  (1) A is editing first, B is editing second without seeing A's edits
>
>  (2) The edits of B are not the same as those made by A
> 	(i.e. a change of rooms for the event, or a change of timing)

Yes.

>  (3) Where (2) is not true

... which would mean the edits of B are the same as those made by A?
Then the objects should have the same contents which would result
in a no need for coordination situation. So you probably aim for something
else.

>  and they edited the same property: 
> 	B did not have better information than A when editing
> 	(i.e. a subsequent change to the event details)

You mean A had the better infos in the same property, yes, possible.

>  (4) The information of A is irretrievable, so A cannot re-add it upon
> 	seeing an overwriting edit by B or has no way of noticing.

Mainly no way of noticing I believe.
If A or B would notice, they could do the necessary coordination
or one could decide.

> So no matter what the organization size, it seems unlikely more than 20
> people would ever reasonably edit a single event. 
> A more realistic number 
> would be 5, as meetings of departments would for instance be organized by
> the department head and 2-4 additional personnel.

I agree with this assumption for meetings. I am unsure for contacts
and possibly other situations where people make use of the groupware functions
in a creative way.

> The likelihood of (1) is governed by synchronization interval

True and offline cases where the client would like to sync, 
but cannot. Overall if the client are wide apard from each other network wise 
you will keep the client interactive for the user and do not have time to 
block until a sync with the other client is possible, so even in online cases
there will be some time of overlap to keep the client interactive.

In practice I've seen that sync intervals were extended to 5 minutes or
more because of the network load which can be a factor in some (not all) 
situations. Clients have been gotten better over time here as well
and there is still room for optimisation on protocol and implementation side.

> The question is: How many events are edited by 10 people within a single
> week?
>
> For there to be "several occurrences per week" of this happening, there
> would have to be hundreds of such events. And this is not even taking into
> account (2), (3) and (4) which will likely introduce a substantial
> reduction in likelihood of two conflicting edits actually causing trouble.

Yes, this sounds being in the right ballbark.
Think all units or an organisation of 5-20 people
using one "unit" group calender folder and all editing in them.
What I appreciate about your writeup is that you are showing that this
depends on a number of factors. 

You are now coming back to the concrete design problem as far as I understand
and the implications. 

> Viewing it from the other side, however, if we assume that 10 people are
> reasonably expected to edit an event within a single week, in an
> organization of active smart phone users this would mean that up to 10
> copies of this event are now on the server, all representing different
> edits of the same event, which are being hit by Kontact when its user tries
> to edit this same event.

I don't think that necessarily a single appointment will get into this 
concurrent edit situation that often per week. If it happens assuming my 
solution idea, the next user would see the appointment twice and could then
decide to edit either next or even manually resolve the need for coordination.
Many users already accept the "sync" option of smartphones to be only
a subset of the data, they would use a full (smartphone or desktop) Kolab 
client in between relying on the synced-smartphone.

> In my experience most people will just click on "get out of my face, give
> me the last copy and delete the rest" in 99% of all cases. Where that is
> not true, intelligent usage of the deleted flag & backups can save the day.

I think the IMAP deleted flag is an implementation detail as Jeroen
as pointed out, it does not help much as it is transparent to the client.
The conceptual problem is not solved by backups as clients and people don't 
notice the issue, they cannot play back what they haven't noticed.

> Incurring the "pop-up hell" usability penalty for all users is a steep
> price for an inferior solution, IMHO. And I have ever only seen it occur
> not because a second person edited something, but because there was some
> problem in the network and consequently with synchronization, and it has
> been supremely annoying every single time.

I agree the dialog is in deep need of improvement.
If we could fix clients that technical conflicts are rare, the "need for 
coordiantion" dialog should be more user centric. Something like

                Need for coordination
          Seems like two people were working on the same event lately:

   User A's version                         User B's version
   +participantC     
                                            +agenda item detail12
                       suggested merge new

                       +participantC
                       +agenda item detail12

    take                   take                   take
                          take+edit

and then let the user drag "agenda item detail12" or "participantC" into the 
suggested new version or deleted entries there. 
        
>  * We could just display author & date of the last change so people easily
>    know who overwrote their change and let people sort it out amongst
>    themselves, as there is likely a situation that requires human
> interaction;

This is quite close to what I would like to see from the user interaction. :)
We should work out how this can work technically, to my knowledge
the offline, scalability and imap cases put some limits on what the client
can know reliably.

>  * Clients that are on line can check with the server whether the event has
>     been modified before saving it, informing the user of a conflicting
> edit that has occurred in the meantime;

Yes, this is what clients more or less must do (and some correctly do). In 
some technical cases this is not to be guaranteed to work though:
The first client might just write in or slightly after the moment 
that the second one has completed the check. 
The second client might get delayed due to network or performance cases.
I've seen a bad case of this lately, the virtual machine where the client
runs gets suspended. There is no way for the client to know or to find out
when within a regular subroutine.

>  * Clients that are off line and find that the event has been modified by
>     someone else when coming back on line (and only then: so modification,
>     and by a different person) can then have the latest edit prevail, but
> email the author(s) of the second to last edit(s) with an update notice on
> the even that has overwritten their changes, attaching their edit so
> nothing is lost.

The notification idea is interesting and would preserve the data of course.
I think that the user of the new client would be interested in getting a 
chance to see the change that it might overwrite to make a choice.
And using email here might open up another channel with its own 
synchronisation issues. In some way, seeing a second objects with the same id 
is some sort of notification. 

I have a number of other wild ideas do reduce some subcases of the technical 
conflicts as well, but that would be something for another post I guess.
My aim is to bring in some of the design ideas of the past and observations
with this post and hope it is useful. Getting this all improved would be very
cool, so I appreciate anyone - and especially you Georg - who is thinking 
along and tries to integrate it all together.

Best Regards,
Bernhard
-- 
Managing Director + Owner: www.Intevation.net <- A Free Software Company
Kolabsys.com: Board Member          FSFE.org: Founding GA Member
Intevation GmbH, Osnabrück, DE; Amtsgericht Osnabrück, HRB 18998
Geschäftsführer Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part.
URL: <http://lists.kolab.org/pipermail/devel/attachments/20120109/eb28a0d8/attachment.sig>