[Kolab-devel] Other kolab scalability issues

[Fabio Pietrosanti]

Gunnar Wrobel wrote:
> Fabio Pietrosanti <lists at infosecurity.ch> writes:
>
>   
>> Dear all,
>>
>> i noticed one bad scalability issue of Kolab while you handle a lot of
>> virtual domains.
>>
>> The main issues are the 'static' configuration that could be made
>> dynamic of imapd.conf of Cyrus and main.cf of Postfix.
>>
>> For Cyrus imapd.conf the "loginrealms:" include the list of all virtual
>> domains and it's generated by kolabd.
>> If the list of loginrealms became too much long because you are using a
>> lot of virtual domains bad things could appens.
>>     
>
> What kind of limits are there and at which point are we going to hit
> these limits? What would cyrus and postfix do if this limit gets
> exceeded?
>   
I still doesn't know BUT what i found is that loginrealms: is not
required to have kolab working (at least on my kolab 2.1 installation).

Could we consider removing it or do you think it's required?

>> Solution would be to add something like this loginrealms dynamic lookup
>> described here:
>> http://www.irbs.net/internet/info-cyrus/0601/0244.html
>>
>> or i am wondering which is the real reasons for which we are requiring
>> loginrealms, because here they removed it and it worked
>> http://www.nabble.com/Multidomain-support-in-Cyrus-eGroupware-with-LDAP--solved--t3665722s3741.html
>>
>> Now on my multiple-domains (a lot of domains) email servers, i commented
>> out the loginrealms and it works fine, i can create users, change
>> password, login, change permissions, etc, etc . It seems to work.
>>
>>
>> For postfix's main.conf it's the same for the following configuration
>> line that contains the list of domains:
>> mydestination
>> masquerade_domains
>>
>> Imho we hould map this lists to dynamic lookups to the ldap directory in
>> order to scale better with a large number of virtual domains.
>>     
>
> Would the lookup only happen when the services are started? Or would
> the lookup occur associated with certain user actions? The second
> scenario would slow things down so it might be better if this is
> configurable or gets only activated if the server has more than a
> defined number of virtual domains managed.
>   

I think that i found a good way to manage both situation in one shot.

The problem of dynamic lookups is that it would require a high number of
queries under high email load.
The ldap client of postfix doesn't support lookup caching and require
for each new query a new connection.

Still, the proxymap allow a single permanent connection to be
established to the ldap server and reused by all lookups.

So, generally speaking, if we apply the proxymap instead of the ldap
direct lookup to all the postfix lookup we can gain the advantage of
connection caching for ALL the ldap lookups removing a great overhead.

http://archives.neohapsis.com/archives/postfix/2004-03/0806.html

Imho this could give a 'good' performance improvement and allow the
implementation of FULL DYNAMIC lookup of all postfix tables (also
because openldap has it's own cache).

What do you think about those two proposals?


I am just evaluating how to remove what could be somehow a scalability
issue and find out a "not in the choice of the sysadmin" solution.

Fabio
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.kolab.org/pipermail/devel/attachments/20071023/e7383d09/attachment.html>