[Kolab-devel] Improving Kontact's performance with large Kolab calendars

[Till Adam]

Folks,

some users have run into performance problems with very large calendars 
using Kontact (https://intevation.de/roundup/kolab/issue1118), and we 
need to decide on the best ways to remedy those problems both short 
term and long term. Below is a summary of the possible options, the way 
I see them. Please comment, so we can reach consensus on what is the 
best course of action.

There are two aspects of the performance problem, which can be addressed 
separately. 

1) Fully loading a large calendar from the kolab resource is slow 
because:
  a) many mails have to be parsed (mime-parsing) 
  b) they each contain XML attachments which have to be parsed
  c) the result has to be communicated via DCOP

The MIME parser is not so much of a problem, but the XML parsing using 
QXML is very slow. All three issues stem from the fact that the cache 
is not kept in KOrganizer's native format, but in mails.

2) The calendar is fully reloaded too often.


Long term approach:

Akonadi: The architecture here should take care of both 1) and 2). 
Notificiation happens with fine granularity, such that full calendar 
reloads should not be necessary. Since only what is shown or otherwise 
needed is loaded, the cache is in a relational db and the bulk transfer 
mechanism is fast, neither excessive parsing nor reloading should be a 
problem here. The time frame for a release quality version of this 
solution is months to a year, though, and it's a completely new 
architecture, which means it's not a viable short- to mid-term solution 
for out current users.


Mid- to short-term approaches:

A) Improve 1) b) by switching from QXml to libxml or similar. Since QXml 
is known to be slow, this should give a decent speed boost. I would 
imagine in the area of 10 to 20 percent, but that's pure guesswork. 
Hard to judge how much work this would be, I'm guessing a few man-days.

B) Avoid 1) a), b) and c) by putting an additional CachedResource, which 
stores the calendar info in its own cache and does uid-mapping, between 
KOrganizer and the KMail cache. This trades speed for space, since a 
full copy of the calendar on disk is made. Additional benefit is that 
it would allow the calendar to load without a running KMail. On disk 
cache would be ical, which is faster to parse, so the speed would be on 
par with a large ical file calendar. Somewhat of a hack, and a 
non-trival amount of work.

C) Improve 2) by increasing the granularity of notifications, thereby 
hopefully avoiding full reloads in most cases. This would mean an 
extension of the resources framework, and a difficult to judge amount 
of work in KOrganizer's view layer. This approach might bring corner 
case regressions, since currently a lot of things rely on the full 
calendar being reparsed in various situations.


Since I believe 2) to be the actual usability problem, meaning that 
users should be ok with a slow-ish initial load, provided normal 
operations afterwards are fast, I would tend towards C) as the most 
promising option. It's fairly intrusive on the KOrganizer side, though, 
so I'd especially like Reinhold's input on this. A) would be fighting 
symptoms, and might not gain as much, and B) probably appeals to me 
because it decouples KOrganizer and KMail better, which is not 
something that is of much concern to most of our users.

It should be noted that work on Akonadi is of course going on, and I 
fully support it as the only real solution to these problems (doh, I 
was much involved with its inception), but that we need something to 
improve the experience of our users within the current codebase 
nonetheless.

I'd appreciate any additional options and ideas that you guys see, and 
any input on the above.

Cheers,

Till

-- 
Till Adam -- till at kdab.net, adam at kde.org
Klarälvdalens Datakonsult AB, Platform-independent software solutions
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: <http://lists.kolab.org/pipermail/devel/attachments/20060703/d7e4b149/attachment.sig>