[Kolab-devel] [issue1093] optimization of kolabd (LDAP.pm) activities would reduce high load on kolab restart

[Fabio Pietrosanti / Khamsa SA]

New submission from Fabio Pietrosanti / Khamsa SA <kolab at khamsa.ch>:

Under a high number of users (>50.000) kolabd on startup take a lot of cpu and a lot of memory for 
more than 15 minutes, keeping the server startup very slow.

Looking at the kolabd code in LDAP.pm there is the function SyncBasic which is the one which seems to 
be the cause of the problem because of the infamous filter=* (or at least with the same results in term 
of data).

The following check attempt fetch the whoole ldap making the memory used by kolabd really big and 
taking it on job checking for quota and other minor check on all the mailboxes.
If the mailbox base is bigger, the memory and cpu problem are to be considered seriously because 
kolabd do this activity at every restart and/or syncronization.

    # Now check that all objects in LDAP have corresponding mailboxes
        # This also resurrects any missing users, if neccessary
        $ldapmesg = $ldap->search(
            base    => $dn,
            scope   => 'sub',
            filter  => '(&(objectClass=' . $Kolab::config{$p . '_object_class'} . ")$add)",
            attrs   => [
                '*',
                $Kolab::config{$p . '_field_guid'},
                $Kolab::config{$p . '_field_modified'},
                $Kolab::config{$p . '_field_quota'},
                $Kolab::config{$p . '_field_deleted'},

I have not commented out anything here because this functionality seems to me necessary and the 
performance problem could not be solved simply removing it.

----------
messages: 6392
nosy: khamsa
priority: urgent
status: unread
title: optimization of kolabd (LDAP.pm) activities would reduce high load on kolab restart
________________________________________________
Kolab issue tracker <kolab-issues at intevation.de>
<https://intevation.de/roundup/kolab/issue1093>
________________________________________________