[Kolab-devel] Native Packaging, Issue Tracking & Restructuring the SCM

[Jeroen van Meeuwen (Kolab Systems)]

Hello there,

I wanted to open up the discussion on Source Code Management, in relation to a 
product (component) road map and development processes -such as testing and 
release engineering, which gets us to Issue Tracking as well, and native 
packaging efforts.

Since I feel many things are interconnected at many levels, I threw them all 
in this one thread and I hope you can all draw from it what you think is 
important and nice; If you have any questions regarding anything in 
particular, please ask!

I would propose that the various components that are all in the server 
repository now, would need to become separate components.

E.g., for example, the following become separate, standalone repositories:

- kolab-cyrus-imapd/
- kolab-daemon/
- kolab-perl/
- kolab-utils/
- kolab-webadmin/
- .../

Note that it doesn't have to be separate repositories if a proper branching 
and tagging policy is set, which I'll get to later. For now though, to 
illustrate the advantages of separate component repositories and/or 
directories, please just bear with me while I explain some of the advantages 
of separate repositories;

This would help accommodate the following;

- The components can be developed as Kolab *components*, meaning;

  - they become optional components (even though they may be required 
components by other components),

  - they can have a different road map -and different milestones to complete 
for such road map,

  - they could even have a different (group of) contributors,

  - the component may have a different branch and/or tag policy or release 
cycle.

  - they can be released with a separate 'make dist', possibly including their 
own versioning schema (while adhering to some sort of standard, of course).

- Each component that is forcefully tied to another component (such as a 
kolab-daemon or kolab-utils component and a kolab-perl component) inherently 
also forces us to work with proper, verifiable APIs,

- Proper, verifiable APIs get us to reasonable product versioning, where you 
can only bump so much at a certain time, and you are required to bump the X or 
Y or Z in a X.Y.Z product versioning schema given the inclusion of certain API 
changes.

- 3.0 development version of one component could be *separately* tested more 
easily in a 2.x kolab* environment (goes to upgrade paths),

Note that, either way we look at it, the components end up on a consumer 
system using native packages *as separate components*, possibly on separate 
systems.

Right now, the opposite is the case;

- one group of contributors contributes to all components,

- the releases are tied in together - an update for one component forces us to 
mark the other components as well (from an SCM perspective), which holds us 
back from actually releasing anything until it's worth the effort, which poses 
a (perceived) risk to consumers who may think bug- and security fixes are 
being held back until the bunch of them justify releasing, which is easier to 
do with separate components,

- Kolab expects certain sets of components to be installed on the local 
machine, which does not always make sense in different existing 
infrastructures, or more sensible architectures thereof.

You can find a few examples I've put up for the purpose of visualizing such 
structures on;

http://git.kolabsys.com/kolab-perl.git/
http://git.kolabsys.com/kolab-utils.git/
http://git.kolabsys.com/kolab-webadmin.git/

Please note that I'm not expecting any comments wrt. the exact conversion 
method used.

Again, whether it be in one big repository or not is besides the point I'm 
trying to make as well. However, if it is in one big repository, consider the 
following;

Suppose the webadmin component is refactored, and needs to branch off for such 
purpose as to not break master (which may have a milestone approaching and may 
need to be released before any merging back happens). Given one bug 
repository, the new branch name would require a "component prefix", a target 
milestone and the topic which is being worked on; Example, given we are on 2.2 
now, and two topics on the roadmap for a version of the product yet to be 
determined (future feature development):

git checkout -f master
git checkout -b webadmin/2.3-dev-refactor

This could just be mutually exclusive with;

git checkout -b webmaster/2.3-zpush-admin

So pragmatic source code managements says, use;

git checkout -b webmaster/<owner>-<target>-<topic>

Maybe, though, we have already named the milestones against the 2.3 series of 
products (a proper issue tracker would know). Suppose the goals for 2.3 are;

- isp patches
- z-push

Then frankly, 'master' should have branched off to kolab-2.3-testing, and 
further development can continue on both master as well as towards the 2.3 
milestone. The regrettable aspect of one big repository is; you require 
branching off all components and will need to merge bug-fixes that may occur 
from either branch to the other.

Right now, this results in an insurmountable set of changes being in the 
master branch, versus what is in the 2.2 branch; some things are in the 2.2 
branch but have not been merged back in with the master branch leaving us with 
fixes for 2.2 that will not be in whatever we branch off of master.

Anyway, I feel like I may have started off rambling for a little so I'll stop 
making the argument and allow for some feedback ;-)

Summarizing;

- strong preference for separate repositories for Kolab components,
- proper product component versioning,
- branching/tagging policy,
- adjacent features in the issue tracker (roadmap, milestones),
- guidelines for handling issues in tracker,
- native packaging from tarball releases.

Kind regards,

-- 
Jeroen van Meeuwen
Senior Engineer, Kolab Systems AG

e: vanmeeuwen at kolabsys.com
t: +316 42 801 403
w: http://www.kolabsys.com

pgp: 9342 BF08