[Kolab-devel] Thoughts about an optional alternative to Guam

[Aaron J. Seigo]

Hi Timothy,

thanks for the thoughtful reply. it certainly helps me out :)

On Saturday, November 5, 2016 07.42:22 Timotheus Pokorra wrote:
> Perhaps we need a blog post explaining how to get started with
> debugging Guam, or even developing for it.

https://docs.kolab.org/about/guam/troubleshooting.html

I'm sure more can be added, but there's a start.

> I did make a start a while ago:
> https://github.com/TBits/KolabScripts/wiki/Debugging-Guam

Some nice things there .. would be great to merge them into the doc above. Do 
you want to do that, or should I?

> that knows a little bit of PHP and Python and packaging.
> I wonder if those people from the community should need to learn Erlang.

One can dream ;) But, no, I don't think it is entirely necessary.

On that note: While we would like to get rid of PHP in the long term, Python 
will not be going away. There will be more Elixir in the future, however. 
Thankfully that is MUCH more like the Python/Ruby world in terms of syntax and 
overall form, so the learning curve is less (though not entirely zero, of 
course).

The CloudSuite bridge has been written in Elixir, for instance.

(Yes, I wrote "has been" ... we have some exciting things to share with all of 
you this month ..)

> >> But for the open source user that just wants to manage his family
> >> email, it is overkill.
> > 
> > I would say that's true of a lot of things in Kolab, though ...
> 
> Perhaps we (as a community) should collect a basic set of features
> that is expected from Kolab.

Regardless of whether this results in a minimal edition or whatever, I feel 
this would have great value in that it would help Kolab Systems understand 
better what the *community* needs and wants. 

In a dream world this would be a broad community survey helping us all 
understand where and how people use Kolab at home or in their work.

> And then work on a minimal edition of Kolab?

Hopefully that isn't necessary; every variance is a complication for everyone.

> I just think, if we have trouble getting a release Kolab 16.1 out,
> perhaps we should focus on the core?

That isn't the core issue behind release problems ... the core issues are 
rooted in priorities and resource management, something we've made huge 
strides forward in at Kolab Systems this past year. We have a few more things 
to get sorted out, but I expect 2017 to be a lot smoother in general here.

As to focusing on the core, I agree that there is more complexity in places 
that makes it feel like there must be a "kernel" of Kolab  in there somewhere. 
That is something we need to address in the coming year(s). Personally, I 
would like to see each release of Kolab become less complex and more 
predictable, and that does indeed require a focus on "the core" as you put it, 
and thinking about the design of the pieces a bit more with our "big picture" 
glasses on.

.. but more on that in other threads in other days ...

> I also say that because packaging Kolab for Distributions like Fedora,
> it would be less work, and more likely to happen.
> Sometimes it is better to have something than nothing at all?

IME, having variants increases work loads in key areas that ends up reducing 
quality overall. It would be nice to avoid multiple versions, and instead 
focus on how we can simplify what we have to the point that it is attainable.

> >> Too many dependancies on Erlang and such.
> > 
> > .. that is really its ONLY dependency. Everything else, like openssl, is
> > already required for Kolab. So .. I don't understand "too many"
> > dependencies .. unless what you mean is "Erlang is a new dependency, and
> > any new dependency is undesired"?
> > 
> > If so, can you explain why?
> 
> Sorry, my words "and such" was not appropriate to what I wanted to say.
> What I meant: Looking at
> https://obs.kolabsys.com/project/monitor?project=Kolab%3AWinterfell
> There are 64 packages with a name starting with erlang.

Yeah, that's because Fedora/CentOS/RHEL split erlang all to hell and back 
coupled with the fact that we are doing non-vendorized builds of the 
erlang/elixir apps.

Having the core erlang distribution chopped up the way Fedora does is very 
annoying. Other distributions respect the upstream intentions of the erlang 
project much better and ship a single package for the core erlang OTP. Ah, 
well, it is what it is.

As for the non-vendorized builds, that means we ship packages for each of the 
erlang libraries used. When it is just guam, that looks like a fair number. 
But as the number BEAM apps we ship grows, those packages get shared as they 
will be re-used from project to project. We don't need more than one logging 
library, after all :)

> Perhaps if erlang was uptodate in the mainstream distributions (it
> seems it is in Debian, but not in the Redhat Universe), and Kolab
> would not need to package it, then I would not complain. It just looks

We'd still have the erlang libraries we use that distros don't package 
(because 99% of erlang apps vendorize..)

But yes, as the distros catch up with newer Erlang OTP versions, the number of 
packages will drop. As you note, some distros already do a better job of that 
than others. Debian and Suse are two examples.
 
> > There are very good reasons for using Erlang here, and we are producing
> > more code using BEAM (the Erlang equivalent of Java's JVM) languages ...
> > so I'd like to understand this.
> 
> Perhaps you need to communicate more what you want to do with it.
> I read
> https://exote.ch/blogs/aseigo/2015/12/10/guam-an-imap-session-filterproxy/
> and came to the conclusion that only one scenario (filtering folders) has
> been implemented and makes sense to a home user or small business. For huge
> enterprises, I understand that you need Erlang.
> If I see more practical use of Erlang that also benefits smaller
> users, I am ok with it.

It allows us to use the resources of large enterprises to bring cool things to 
the home users. That is a pretty compelling benefit imho.

The other benefit is that once an application is mature, it is far easier to 
make it bullet-proof, meaning lowered maintenance and troubleshooting because 
something keeps falling over. We've had so many issues with even quite small 
installations and the reliability of things like kolabd .. and yes, those 
could be sorted out without using Erlang, but it's a lot more work and lot 
easier to get wrong. A system that keeps working as intended even in the face 
of occasional error conditions is quite a blessing for small installations.

> > So of course you can do it in the imap server. Only you are now dealing
> > with C, which is not only lower productivity but a greater security risk
> > and rather more work to get to be performant.
> > 
> >> What do people think?
> > 
> > It really is not a question of "can you" (one can do almost anything with
> > enough effort, after all), but what the effort required and results out
> > the
> > other side are.
> > 
> > I would be interested in seeing your patches to cyrus for this.
> 
> Thank you for the challenge. This motivated me to look at it.

:)

> I have attached a patch.

Cool; would be nice to have this put in for review somewhere to have one of 
the cyrus devs look at it. Even if it can not be upstreamed, polishing it 
can't hurt.

> compared to
> https://cgit.kolab.org/guam/tree/apps/kolab_guam/src/rules/kolab_guam_rule_f
> ilter_groupware.erl the patch is quite small.

Certainly, because it doesn't have to filter anything and the folder metadata 
is already locally available.

> Does this make the Cyrus server slower than guam?

Probably not, no.

> We probably cannot submit the patch upstream, 

Not without making it rather more generic, no.

> but maintaining it
> locally is still less work than the guam package with its extra
> service.

it may well be. maintaining downstream patches never quite leaves a good 
feeling.

> Until there are more scenarios implemented by guam that make
> sense for home users and small offices.

BTW .. a couple of odd uses for guam: 

 * proxying multiple imap servers to just one local server over different 
ports. I've actually already used this myself to be able to swap the 
configuration of the destination IMAP server without having to touch client 
config.

 * securing stupid clients that can't do security right. I've already helped 
set it up in one place where some users were dead set on continuing to use 
their squirrel mail installation (!!!) and thanks to guam they could connect 
insecurely to guam on the local network (same machine, even) and guam would 
then do the TLS to the IMAP server. instant security improvement.

 * troublshooting IMAP .. ok, this is more of a developer thing, but being 
able to see ALL the IMAP traffic in both directions and even inject data into 
that is handy in circumstances. All things you can do with enough wireshark 
voodoo, but rather easier with an actual IMAP proxy at hand.

... and that's without even doing any fancy rulesets :)

THAT said .. if people do have needs / desires for such rulesets, I'd be very 
happy to get them as tasks on git.kolab.org so we could get around to 
implementing some of them.

-- 
Aaron Seigo
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: This is a digitally signed message part.
URL: <http://lists.kolab.org/pipermail/devel/attachments/20161107/c94e7c21/attachment.sig>