More 3.4 calendar woes

Thu Aug 20 02:55:44 CEST 2015

This is awesome! Thanks for sharing this information!

R

-----Original Message-----
From: users-bounces at lists.kolab.org [mailto:users-bounces at lists.kolab.org] On Behalf Of Homer Dokes
Sent: Wednesday, August 19, 2015 7:49 PM
To: users at lists.kolab.org
Subject: Re: Re: Re: More 3.4 calendar woes

Hello all,

1st let me say thank you to dsp3 for his response to my conundrum.  I appreciate your response tho I must say that it was unclear what to 'remove'.

2nd let me broadcast to the world that I have found a site that I did not know existed nor could I find any reference to it in the past so I want to include it here for other 'would be' administrators of the Kolab environment.  The web site, http://users.kolab.narkive.com/, looks to actually consolidate all the emails associated with users at lists.kolab.org in a 'subject' organized manner providing a single source of kolab specific issues/solutions with an ability to search through them for specific information.  I wished I had found this 6 months ago.

3rd, I am happy to say I have discovered success for this issue.

Long story short, my discoveries have revealed that roundcubemail derives it's information for display in the web client not from the mysql tables but rather from the cyrus.cache and cyrus.index files that exist under each user imap folder.

i.e. to display user contacts in the web client under the contacts menu selection, the data within is drawn from two files located in /var/spool/imap/domain/<1st initial of domain name>/<domain name>/<1st initial of mail user>/user/<username>/Contacts which are cyrus.cache and cyrus.index.

I have not been able to determine what purpose the mysql table 'kolab_cache_contact' has but I'm confident it doesn't play a part in the normal operation emails.  If I'm wrong here someone please 
correct.   I have determined however that the records in 'kolab_folders' 
does play a role to the extent that it keeps track of what folders have been created and how many objects reside within.  I am having a separate issue where this objectcount is resetting to 0 but that's for another topic.

So the primary symptom here is that contacts and events disappeared for one user.  All the contact files and events existed in the associated imap user's folder... it wasn't as tho they were physically missing.  
Once I realized that roundcubemail derived it's information from the two files, cyrus.cache and cyrus.index, I had to assume they were corrupted in this case. These are binary files so there isn't any real way, that I have been able to determine, that they were or were not corrupted. Once I accepted that they must be I set out to try to determine if and how they could be reconstructed.  I began concentrating on cyrus and not the 'kolab' umbrella.

Turns out there is a command that when executed from the command line in this format:

  'reconstruct -rf user.testuser'

If you have  the 'unixhierarchysep' parameter in the /etc/imapd.conf folder set to yes than you will need to use this format of the command: 
'reconstruct -rf user/testuser'.

It should also be noted that the format of 'testuser' is the actual user name with the domain reference, i.e. testuser at domain.com.

The initiation of this command line will reconstruct the cyrus.cache and cyrus.index files.

Now here is the kicker... it is important that you remove these two files BEFORE executing that command line as it will not reconstruct them if they already exists.  It only does so if the two files are missing.  
It should also be noted that all this assumes that the original (or
recovered) individual files exist in the respective directories.  So this means you can 'choose' which folders, i.e. contacts, calendar, tasks, etc, have their cyrus.cache and cyrus.index rebuilt by only removing the cyrus.cache and cyrus.index you want rebuilt.  The others will be left alone.  This is important if you want to only effect a sub folder but leave the parent folders as they are.  The -r parameter in the command is recursive.

Running that command line recreates the cyrus.cache and cyrus.index files with information taken from the actual content files within the respective directory.  Once I had done this the contacts and calendar events reappeared in the users roundcubemail client.  Can you say.... 
HaLLelUiAh!!!!

Now... the one unfavorable, I still don't know what caused the original corruption of these two files for this one user.  All other users have been operating without issue.  This is disconcerting as I cannot insure it won't happen again.  It isn't as tho the user had direct access to these files.  The server is well protected against power issues and the like.

I hope this information is of use for anyone who may find that their roundcubemail content has suddenly vanished.

Thank you for indulging me.

hdokes

On 8/18/2015 1:45 PM, dsp3 wrote:
> Hello hdokes,
>
> Having been through a similar scenario, I believe the issue related to 
> having a corrupt calendar event/events in a users calendar directory.
> Removing this/these resulted in all calendar items showing again as 
> expected.
>
> Sorry I can't be more specific, but I'm 99% certain this is how I 
> resolved the issue.
>
> On Tue, 2015-08-18 at 12:11 -0400, Homer Dokes wrote:
>> Hi Timotheus,
>>
>> Thank you so much for your prompt reply.
>>
>> I have enabled debugging from the initial installations (has been 
>> running for about 6 months now).  I actually monitor the log files in 
>> real time 24/7 for two configured kolab servers via ssh and tail 
>> configured in multiple windows through tmux.  This allows us to 
>> 'catch'
>> some of the issues as they occur and address them right away.
>> Unfortunately, I have been unable to determine why this particular 
>> users contacts and calendar events stopped showing in the 
>> roundcubemail web client.  As indicated before, any new entries do 
>> show... just not all the entries prior to it's hiccup.
>>
>> I appreciate the sources of information you have provided.  We will 
>> spend some time and sift through them to know what is where and how 
>> to best utilize it.
>>
>> What I really need right now is to know where the mysql tables 
>> 'kolab_cache_contact' and 'kolab_cache_calendar' get their records 
>> from.  I had rebuilt them with the command below but that only 
>> brought back the newer entries after the hiccup.  The older entries 
>> do not show up in these tables even tho the actual 'files' exist in 
>> the users respective Calendar and Contacts directories.
>>
>> Command used to rebuild tables
>> /⁠usr/⁠share/⁠roundcubemail/plugins/libkolab/bin/modcache.sh clear 
>> -⁠u <user at domain.org> -h localhost
>>
>> Within these directories are binary files that are named cyrus_cache, 
>> cyrus.header, and cyrus.index.  For this particular users contact 
>> directory, the cyrus.cache file size is 1.6mb, the cyrus index is 
>> 155kb, and the cyrus.header are 208 bytes.  I am making the 
>> assumption that these files contain reference information to the user 
>> files within the directory but I can't be sure as the cache and index 
>> files are binary and not reviewable. Judging by the size of the files 
>> I have to also believe that they contain information related to all 
>> the user contact files within that directory as I doubt the cache 
>> file would be 1.6mb in size if it only contained information on 18 
>> contacts.  Do the respective mysql table for contacts derive their 
>> content (records) from these cache files?  If so, can these cache 
>> files be rebuilt from the user contact files contained within the 
>> directory.  If not, how does one include all the contact files into 
>> the mysql kolab_cache_contact' table?  The command line above does 
>> not include them all... only the newer files that were added after 
>> the hiccup.  Should that command have included all the files from 1. 
>> to 1458. that exist within the contacts directory for that user?
>>
>> For clarification.... definition of a 'hiccup': The point at which 
>> the user realized something was amiss...  not ALWAYS consistent with 
>> when the symptom actually manifested itself. :-)
>>
>> I have tried to provide as much information here as I can think is 
>> pertinent.  If additional information would help please let me know 
>> what you would need to get a better understanding of the issues.
>>
>> Again, Thanks Tim for your reply, and to anyone else that may be able 
>> to shed some light on this issue.  In the mean time, We'll keep 
>> plugging away for a viable solution.
>>
>> hdokes
>>
>>
>> _______________________________________________
>> users mailing list
>> users at lists.kolab.org
>> https://lists.kolab.org/mailman/listinfo/users

_______________________________________________
users mailing list
users at lists.kolab.org
https://lists.kolab.org/mailman/listinfo/users