[Kolab-devel] kolab and real-time communication (chat, etc)

[Aaron Seigo]

Hello,

Hot on the heels of WOPI and CloudSuite support, we have begun work on 
bringing instant messaging features to Kolab.

Usual project/git repo links:

   https://git.kolab.org/project/view/209/
   https://git.kolab.org/diffusion/KC/

They are both VERY empty at the moment. That is because we completed 
initial requirements definitions this past week and will begin 
development proper this week. We have spent a lot of time looking at the 
various options, doing some small prototypes, and decided on what I 
consider to be a brave, but correct, course.

And I'd love your feedback!

Looking at the various XMPP servers out there, we realized a handful of 
things:

  a) they were big and complex beasts ... scalable and very competent, 
yes, but not simple things. The simpler options were not scalable / 
manageable enough for our needs.
  b) integrating IM type features into the various Kolab clients 
(Roundcube, Kube, Kontact, ..) was going to be non-trivial

So we looked at alternative concepts. One that rose to the top quickly 
was the Phoenix Framework's PubSub which includes channels (named 
pub/sub entities that can be authenticated, filtered, etc.) and presence 
out of the box in a very sophisticated and scalable way. The developers 
of it have managed to get 2.000.000 simultaneous connections actively 
chatting on *one* physical server with normal performance (e.g. actually 
usable for chat). Which is pretty insane. It is also clusterable 
(including self-healing on split/rejoin), supports presence across 
multiple devices (or web browser tabs) correctly, etc.

But while Phoenix PubSub solves the hard parts we need for chat, it has 
(obviously) zero client support as it is intended to be used by 
applications internally. We really want XMPP support.

So we spent time scouring the XMPP RFCs. Turns out that for basic XMPP 
support (auth, roster and chat), that's actually a very manageable 
project in terms of complexity to add on top of Phoneix PubSub.

With such a solution we would be able to:

  * have perfect integration with Kolab's auth system (today and in 
future)
  * provide a chat system with as close to "zero config" as possible
  * provide the integration we really want to see

So .. what do I mean by "integration we really want to see"?

In the immediate, we want to provide the ability to have not just 
one-to-one and group text chat, but also WebRTC video chat (one-to-one, 
to begin with). Slightly longer term, we want to bring proper end-to-end 
encryption for the chat (reusing prior work done in e.g. libsignal), 
well integrated file send/share, chat alongside document editing, chat 
links from contacts and identities (e.g. From: labels in emails) in 
Kolab clients, chat room scheduling from calendar events, etc.

Our current development roadmap for this is broken into three stages, 
with the third stage being a bit open-ended (I expect us to break it out 
into further milestones once we get through stage II):

I.

Feature set:
- authenticate against Kolab
    - Basic http auth
    - think about generic component for Kolab auth
- setting user presence status and receiving other users status
- notifications
- person-to-person chats 
(https://github.com/chrismccord/phoenix_chat_example)
- single page web application that shows contacts list and you can start 
chats
    - How to start a chat, set presence, display presence?
    - How to define/start a group chat? invitations?
    - "Chat window" layout
- group chats
- webRTC

II.

Feature set:
- Roundcube integration
    - Current user presence indicator in the toolbar, with possibility to 
set status
    - A way to start a chat by clicking a user name/email address/status 
icon in the email view and addressbook
    - Other users status on contacts list
    - How to start a chat from UI?
    - How to show the chat "window"?
- Jabber bridge (internal organization communication)
- Kube integration
- UI customization


III.

- Server-side chat state (e.g. which group rooms you are in)
- Files exchange
- End-to-end Encryption
- Multi-user video chats
- Federation (communication between Kolab servers)
- Server-side chat history


Our goal is to make stage I a xmas present for ourselves :) It will 
certainly be a "minimum viable product" type result, but that will allow 
us to vet the tools we have chosen further, test our assumptions, and 
give us a springboard for the bigger goals in milestone II.

Participation is welcome, as always. If you want to learn some Elixir, 
help with front-end web development, be a bleeding-edge tester ... don't 
hesitate. The project repo and workboard will be shaping up throughout 
the week, and we'll be here to discuss as we go.

-- 
Aaron Seigo