[Kolab-devel] Hosting multiple domains

[Jeroen van Meeuwen]

On Wednesday, April 03, 2013 01:15:18 PM Timotheus Pokorra wrote:
> Hello Jeroen,
> 
> Thank you for your detailed reply!
> 

You're more than welcome! ;-)

> > What I would like us to consider is how code can be made adaptive (so no
> > "<?php $domain_base_dn='dc=foo,dc=bar';?>" code is needed on a per domain
> > name space basis), and attempt to avoid to need to write out
> > configuration files just to be able to automatically serve up a new
> > domain name space (that was just created). I think this would both serve
> > the single vhost and multiple vhost scenario best.
> 
> I assume you are refering to the entries in kolab.conf?
> 

Either through kolab.conf, though I would love that to be a fallback path;

The domain name spaces you host have an entry in cn=kolab,cn=config. These 
entries can be made to hold an inetdomainbasedn attribute, which can point to 
a root DN.

For example; Creating a domain name space for example.com allows the root DN 
to be something along the lines of o=example,c=uk, by setting the 
inetdomainbasedn attribute to that value, and otherwise the web admin panel 
(its API side actually) should be creating what we call a "standard" root DN 
of dc=example,dc=com.

Code *should* use a search against cn=kolab,cn=config to retrieve the domain 
name space details, which *should* either;

  - contain an inetdomainbasedn value, or

  - lead to the attribute value used as the RDN, the "primary domain name 
space"^1, from which the "standard" root dn can therefore be derived^2.

^1: Example scenario:

  associateddomain=example.com,cn=kolab,cn=config
  objectclass: top
  objectclass: domainrelatedobject
  associateddomain: example.com
  associateddomain: example.net

The "standard" root dn for somebody logging in as user at example.net will be 
"dc=example,dc=org";

  1) Find the domain entry:

    (&(objectclass=domainrelatedobject)(assocateddomain=example.net))

  2) Get the first attribute value for the result attribute:

    domain = domain['associateddomain'][0]
    print domain => 'example.com'

  3) Get the "standard" root DN for such domain:

    domain_components = domain.split('.')

    print domain_components => [ 'example', 'com' ]

    root_dn = 'dc=' + ',dc='.join(domain_components)

    print root_dn => 'dc=example,dc=com'

> I am wondering that we still need the entries in kolab.conf for each
> domain, eg. for the autocreate_folders and primary email address etc,
> not just for the base_dn.

Since these are domain name space specific settings, you would want them to 
exist.

We could possibly work on making sure there is a section for global defaults 
as well.

> In Kolab2, I was told, the kolabd did check the LDAP for new domains,
> and wrote to the config files. Is that correct?

It could have, but IIRC, it created new domains within the single root_dn, and 
could therefore avoid specifying the necessary logic that derived the root dn 
from the domain name space used, and search within multiple root dns. In other 
words, there was no multi-domain in Kolab v2 if you consider one domain should 
be completely separate and isolated from another domain.

The goal with Kolab 3 is to not require configuration files to be written out 
upon adding a domain name space (or upon editing, moving or deleting it, for 
that matter).

I suppose this means adaptive code, for which of course I cannot be completely 
certain I've covered all use-cases. I think at the core, all of it is there.

> Torsten Grote talked about the idea of having a cronjob checking for
> new domains that have been added in the Web Administration Panel, and
> then modify the kolab.conf from that cronjob.
> Could this be a way to automate the adding of new domains, without the
> need of manually running a script like my adddomain.sh? [1]
> 
> Could this be a way forward? Should I convert the bash script to python?
> 

Don't get me wrong, but I'm tempted to decline - but not because of your 
script or because of your use-case, though.

I suppose for (semi-)automated domain name space configuration like this is to 
really take of, we have to consider doing the following (alternative outlined 
below);

  - Allow the specification of some sort of a domain_include_dir in [kolab] in 
kolab.conf(5). Let's say this will be /etc/kolab/domains.d/, or possibly 
/var/lib/kolab/domains.d/.

    It'll be up to deployments themselves to allow this directory be writable 
for the apache user, or the kolab user for that matter.

  - Extend the Kolab Web Administration Panel Configuration file parsing to 
follow this new "domain_include_dir" setting and continue parsing these 
additional files.

  - Extend pykolab.conf to use "domain_include_dir" appropriately as well.

  - Extend the Kolab Web Administration Panel API to include Kolab settings 
among the capabilities in "add" and "edit" domain.

    This might be an additional 'settings' JSON value in the 'add'/'edit' 
service method inquiries for fields, which now include form fields for LDAP 
attributes.

    This might be easier to implement using a 'domain_settings' service rather 
then extending the methods for the manipulation of LDAP objects.

  - Extend the Kolab Web Administration Panel client interface to use this new 
"settings", which will then take a set of form fields that one could specify.

  - Extend the APIs validate_ functions,

  - Include the capability to the API to write out a new configuration file to 
whichever "domain_include_dir",

  - Modify the Kolab daemon to (re)load the configuration whenever it does a 
domain name space thing.

  - For situations where the webserver user is not allowed to write out to 
"domain_include_dir", one could imagine we write the proverbial "kolab --act-
accordingly" for cron jobs or personnel to execute.

Alternatively, we should create a proverbial "kolab --check list-domains", 
with a "kolab --apply-configuration list-domains". For two examples, please 
see:

  - list-domains [1]

  - list-mailboxes (for parsing command-line switches and options) [2]

> Just for my own clarification: the Web Administration Panel calls
> _domain_add_new($domain) in [2] for adding a new domain.
> The Kolab Python Utils [3] only use the wap_client which is basically
> a wrapper around the Web Administration Panel.
> 
> What is currently the task of kolabd? [4]
> 

The kolabd daemon is a daemon you most commonly run at one location throughout 
your infrastructure (per Kolab deployment, that is).

It creates either a persistent search (389 DS), or makes itself a slave 
replica (syncrepl in OpenLDAP) of the LDAP server, for each of the domain name 
spaces it can initially find.

Using said persistent search or slave replica connection, any further changes 
(search), or any pending changes (replica) in LDAP will be automatically 
reflected in IMAP (quota, mailboxes).

It repeats its search for domain name spaces every 10 minutes if during the 
previous iteration no changes were found. If changes were found, it searches 
for domain name space changes once more immediately.

Kind regards,

Jeroen van Meeuwen

[1] http://git.kolab.org/pykolab/tree/pykolab/cli/cmd_list_domains.py
[2] http://git.kolab.org/pykolab/tree/pykolab/cli/cmd_list_mailboxes.py

-- 
Systems Architect, Kolab Systems AG

e: vanmeeuwen at kolabsys.com
m: +44 74 2516 3817
w: http://www.kolabsys.com

pgp: 9342 BF08