[Kolab-devel] OpenLDAP problem

[Buchan Milne]

Thomas Lotterer wrote:
> Re Martin,
> 
> On Mon, Oct 04, 2004, Martin Konold wrote:
> 
>>I have seen now multiple times an OpenLDAP server [...]
>>
> 
> we recently discovered stability problems with OpenLDAP on Solaris.
> While tracking down those issues I learned that a huge number of bugs
> related to the Berkeley-DB library interface were fixed in OpenLDAP
> between 2.2.14 (OpenPKG 2.1) and 2.2.17 (OpenPKG CURRENT). With OpenLDAP
> 2.2.14 using Solaris threads simple stress testing failed miserably with
> slapd segfaulting and leaving a corrupted db behind. Because slurpd
> requires threading we decided to enforce use GNU pth to ensure usage
> of a consistent non-preemptive threading model across all supported
> platforms. The OpenPKG package needed a bit tweaking [1][2][3]. The
> combination
> 
>     db-4.2.52.2-20040630
>     openldap-2.2.17-20040923
> 
> was stress tested on FreeBSD, Linux and Solaris without failures. The
> test included adding, searching and deleting ~1500 entries with ~30
> attributes each for ~500 times. The loglevel was set to 256 and fsl was
> used producing some hundred MBs of log files. The adding and deleting
> steps were done with ~25 parallel processes causing a high system load.
> Data availability was verified after each step. The whole testing was
> successful.
> 
> There were other lessons learned from the test above.
> 
> 1.) It was executed using a configured replica which was not available.
> In such a case slurpd not only createad a 400-600MB replica log but
> also seems to keep information in memory. In the end, it exhaused all
> available system memory on our 2GB RAM Solaris box (dunno whether the
> problem is Solaris specific or the other boxes just had more memory)
> causing all kinds of bad consequences.
> 
> 2.) OpenLDAP does not care about the Berkeley-DB (transaction) logs. The
> administrator must clean up the system manually. [4]
> 
> I higly recommend that Kolab and anyone having openldap stability
> problems to swith to the openldap package from OpenPKG CURRENT as
> listed above or later and adds the GNU pth package. It seems that using
> "db" and "pth" from OpenPKG 2.1 release is ok. There are no plans to
> merge these improvements into the 2.1 release as backporting would be
> nothing more than patching in all differences to openldap-2.2.17 and the
> enforced addition of GNU pth would break almost all automated update
> systems and violates my pesonal policy of keeping updates very small and
> specific.
> 
> [1] see August-September 2004 openldap.spec timeline at
>     http://cvs.openpkg.org/rlog?f=openpkg-src/openldap/openldap.spec
> 
> [2] "GNU Pth initialized too late causing segfault"
>     http://www.openldap.org/its/index.cgi/Build?id=3344
> 
> [3] "GNU Pth signal(3C) redefinition on Solaris"
>     http://www.openldap.org/its/index.cgi/Build?id=3345
> 
> [4] "BDB log.xxx Files"
>     http://www.openldap.org/lists/openldap-software/200302/msg00926.html
> 

The 'db_archive -l |xargs rm' suggested in this mail is not a good 
solution ... if you have an old backup (made before one of the 
non-active transaction log files), after removing it, you will no longer 
be able to restore beyond the old backup.

I wrote some scripts (for 2.1.x) to handle this better (and allow fast 
restores on slaves), which can be found in Mandrake CVS:

http://cvs.mandrakesoft.com/cgi-bin/cvsweb.cgi/SPECS/openldap/ldap-common
http://cvs.mandrakesoft.com/cgi-bin/cvsweb.cgi/SPECS/openldap/ldap-hot-db-backup
http://cvs.mandrakesoft.com/cgi-bin/cvsweb.cgi/SPECS/openldap/ldap-reinitialise-slave

Regards,
Buchan

-- 
Buchan Milne                      Senior Support Technician
Obsidian Systems                  http://www.obsidian.co.za
B.Eng                                RHCE (803004789010797)