Roundcube performance

Mon Sep 15 14:03:41 CEST 2014

Well the first time you go into a folder the cyrus db has some
reconstruction or automatic update, as described here
http://docs.kolab.org/administrator-guide/upgrading-from-kolab-2.3.html#migrate-the-data-through-copying

At least I suppose. That said, if I understand correctly, it is not
critical if I don't perform that step. The conversion will be performed the
first time the user access the folder (in fact the first day was rather
funny with the conversion taking place every time I was accessing a new
folder). But this upgrade took place 1 month ago now so I doubt it is still
converting. Checking doesn't hurt, will see how to enable debug in cyrus
imap and see if I can get some useful information out of it. Still I don't
think the index is the problem... would be slow even with the imap proxy.
The proxy doesn't cache data, it only keeps open the connection to the IMAP
backend, so roundcube only opens a new one with the proxy itself.

I think I should also do another test. Using imapproxy I also remove the
requirement for using TLS when communicating with the imap server from
roundcube. For sure roundcube must do at least 2-3 connection for every
action, like refresh or opening a message. Opening a connection is
expensive, but opening TLS connection (considering also I enforce only very
strong ciphers) is way more expensive. There should be a way of configuring
multiple IMAP services in cyrus (I already to this for ipv4 and ipv6) with
different config files (and I have to discover if this is possible, but I
assume it is). I can configure one listening on localhost only not
enforcing TLS and see what's the performance change.

It might also be possible I forgot something else during my upgrade from
kolab 3.1 to 3.2. I reconfigured kolab.conf from scratch (too may
differences from the previous one) then I reconfigured, via setup-kolab,
cyrus, mta, roundcube. It was not a very smooth process, realizing I forgot
something here and there during the process. I think it all went ok in the
end, but cannot be 100% sure.

Will report if I discover something

Cheers
Enrico

On 15 September 2014 13:48, Daniel Hoffend <dh at dotlan.net> wrote:

> I realized it that mysql can't be the issue the moment I pressed "sent"
> cause you spoke about disabled the cacheing. I'm not sure where to tweak
> cyrus and the tweaking guide on docs.kolab.org doesn't say anything about
> performance either.
>
> Maybe some imap inidzies are broken and could be reconstructed, etc. Maybe
> check cyrus debugging if you can find anything there.
>
> --
> Daniel
>
> ------ Originalnachricht ------
> Von: "Enrico Tagliavini" <enrico.tagliavini at gmail.com>
> An: "Daniel Hoffend" <dh at dotlan.net>
> Cc: "users at lists.kolab.org" <users at lists.kolab.org>
> Gesendet: 15.09.2014 13:22:37
> Betreff: Re: Roundcube performance
>
>
>  Hi Daniel,
>>
>> no I didn't. MySQL doesn't seems to be the bottleneck to me. I tried to
>> disable the cache entirely, or move to APC for imap_cache, while keeping
>> the imap proxy and performance is very good, if I ignore the issue about
>> unread messages. If I disable the imap proxy roundcube goes back to being
>> slow. I can't explain this with a MySQL issue since it is unchanged between
>> my two config... to my knowledge at least. Also this is basically a system
>> with one or two users active at the same time at most.... it is not loaded
>> or the like. That said if I'm not able to explain it, it doesn't matter
>> this is actually true.
>>
>> That said I'm very happy to try, but have not much idea how can I improve
>> MySQL performance. The server is running on the same VM, this is a single
>> host installation. Do you have some advise about how to improve it?
>>
>> FTR: I've php-pecl-apcu and php-pecl-zendopcache installed and enabled, I
>> forgot to include this information in the previous email. Dunno if it
>> changes anything.
>>
>> Thank you
>>
>> Enrico
>>
>> On 15 September 2014 10:22, Daniel Hoffend <dh at dotlan.net> wrote:
>>
>>> Did you tried to improve the MySQL server which is used by roundcube as
>>> cache?
>>>
>>> --
>>> mfg
>>> Daniel Hoffend
>>>
>>> > Am 15.09.2014 um 09:59 schrieb Enrico Tagliavini <
>>> enrico.tagliavini at gmail.com>:
>>> >
>>> > Hello everyone,
>>> >
>>> > I was wondering if anybody has some suggestion on how to make
>>> roundcube in kolab 3.2 (and 3.3 as well since I plan to upgrade before
>>> Christmas I hope) a little bit faster.
>>> >
>>> > Prologue: in kolab 3.0 and 3.1 roundcube was not very fast.... but
>>> still acceptable. Updated to kolab 3.2 and, while roundcube interface is
>>> better, prettier than before it is also slower (no idea if updat of cyrus
>>> imap can be blamed as well but I don't think so). Slower to the point of
>>> being annoying.
>>> >
>>> > My config: this is a 3 users system, nothing big. spool/imap is about
>>> 118 MB with about 12000 files. I first installed this with kolab 3.0 a
>>> couple of years ago and updated it as new releases where coming out. The
>>> system runs on a VM, with 2 vcores and 3 GB of ram. The host is under my
>>> direct control and fr the reference it is a SYS from OVH. Host processor is
>>> a Xeon E3-1225 with 32 GB of ram, with an average use of cpu time under 5%
>>> according to munin. The 2 hard drives are in software raid one on the host,
>>> and the disk image for the kolab VM is an LVM logical volume passed
>>> directly to the VM. The hypervisor is KVM, OS is CentOS 6.5 both on host
>>> and guest. The file system for the imap folders is the same has /, with
>>> stock ext4 options. For such a small spool I don't think separating it
>>> would make much difference, but feel free to correct me and suggest better
>>> settings.
>>> >
>>> > To quantify slow in roundcube I would say that opening an uncached
>>> message takes an average time of 2-4 seconds (according to firefox network
>>> console / firebug), refresh can take even more sometimes. Switching folder
>>> is the same. In some rare cases some operation can reach 10 seconds, but
>>> this is quite rare. This is not network latency since it happens also from
>>> work networks. From work I have access to two networks, one is a 100 mbit
>>> eth, the second is 1gbit eth. Average latency to my server is about 13ms ±
>>> 0.2, measured with both ping (average in 50 samplings) and mtr.
>>> >
>>> > I have no clue what could be the bottleneck, the only symptom I can
>>> see, other than the slow speed, is a lot of I/O wait in the kolab VM,
>>> likely from imapd. Munin confirms that from kolab 3.1 to kolab 3.2 the cpu
>>> I/O wait time increased quite a bit. I would roughly say it is doubled,
>>> from ~2% to 3-4%. But it is hard to say since the values are small. Anyway
>>> I can see the I/O wait clearly with htop while refreshing or opening
>>> uncached messages. I can't be sure if this I/O is from disk or from
>>> network. For both I use virtio. For the disk I switched to virtio SCSI just
>>> to see if it was making any difference.
>>> >
>>> > So far I've tried to add imapproxy between roundcube and cyrus-imap. I
>>> had to do some odd config here since roundcube will refuse to connect to
>>> anything else that localhost:143, ignoring whatever I put into the config
>>> file (I tried to put imapproxy on 144 at the beginning, this bug has been
>>> reported in roundcube in the past already). So I moved cyrus-imap and bound
>>> it to the external IP only and got imapproxy on 127.0.0.1 only.
>>> >
>>> > With imapproxy the roundcube performance increase dramatically, but
>>> unfortunately the unread messages display is broken. The unread message
>>> cunt on the left is correct, but I have to force the view to unread
>>> messages only to actually be able to see them. Just opening a folder with
>>> uncached unread messages lists only the cached one (what you already read)
>>> :(. I've tried to disable caching in roudcube, both for messages and imap,
>>> the result seem the same oddly. I'll keep it disabled for a while and see
>>> if the behaviour changes. I saw an old bug report for roundcube describing
>>> this behaviour, but can't find it again right now... sorry.
>>> >
>>> > So my question is what can I do to speed up roundcube a bit without
>>> breaking it.
>>> >
>>> > Also, as a suggestion to Kolab developers: I think investigating the
>>> possibility of adding imap proxy caching to kolab would be very worth it,
>>> if you didn't already. Performance gain is massive. Provided it doesn't
>>> break roundcube itself like in my case of course. I'm not aware of any
>>> other imap proxy other than the squirrel mail one.... tried with nginx but
>>> it want to do fancy stuff for the authentication so I didn't even tried it.
>>> >
>>> > Thank you.
>>> > Best regards
>>> >
>>> > Enrico
>>> > _______________________________________________
>>> > users mailing list
>>> > users at lists.kolab.org
>>> > https://lists.kolab.org/mailman/listinfo/users
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.kolab.org/pipermail/users/attachments/20140915/e45ec7b9/attachment.html>