[Kolab-devel] (fwd) how to deal with quotas / suggested feature "monthly quota increase"

[Jeroen van Meeuwen (Kolab Systems)]

Gavin McCullagh wrote:
> On Thu, 02 Sep 2010, Jeroen van Meeuwen (Kolab Systems) wrote:
> 
> > In my experience, there's no difference between a user hitting their quota 
and 
> > a user hitting their quota. The user can have its quota increased 
(possibly, 
> > there's additional cost?) or clean up its mailbox -like, most of the time 
it's 
> > the Trash/Junk folder filling up, haha ;-)
> 
> In my experience, it's usually people accumulating email over time.  A lot
> of people argue that they need to store their historic email so that they
> can go back and look at it.  Our company, if anything, agrees.  If they do
> go through and delete some stuff, they then hit the quota again some time
> later but this time cleaning up is probably not possible.
> 

Yes, people to accumulate email over time. Whether it be users not emptying 
their Trash/Junk folders or genuine archiving users doesn't even matter.

I also agree there must be some form of historic data, preferably available 
"live" -without grabbing the proverbial backup tape, if you will. I myself can 
go back till late '99 -and I'm one of those that does not purge anything.

Hitting quota is, inherently, also a recurring event.

Your suggested solution is to increase the quota on the primary mailbox 
storage. While perfectly feasible with something similar to the scripts I 
linked in one of the other posts in this thread, here is where Kolab (as in 
"the core groupware application") does not have to provide such functionality 
-although arguably, a set of working scripts to such purpose may or may not be 
included in our version control and/or releases as utils or tools. My point 
here being that the implementation of the functionality can easily be 
performed stand-alone; no modifications to Kolab itself are required.

For a running, live, large environment however -where quota is both a MUST-
HAVE and PITA in terms of workload, I think maybe a concept such as 
Hierarchical Storage Management (HSM) would be better suited. I'll try to 
explain why;

With HSM, you basically have more then one location to store certain types of 
data; In this case, a nice denominator would be "age" vs. "availability". 
Suppose you have two types of storage;

- One nifty, nice, fast, shiny 

Let's call it "primary" storage, for all new/recent data in mailboxes.

- One dusty, large, slow

Let's call it "archive" storage, for all data older then X months/years.

You periodically, automatically, expire "old" data from the primary storage, 
keeping what users use most available on the quickest storage -without slowing 
it down by making the spool contents larger and larger in size. What you 
expire, you store on the archive storage.

This could be as simple as creating a Archive/ hierarchy in the users' mailbox 
on a different backend server / imap spool.

One could prevent Archive/* from counting towards the primary quota, and stick 
to a static quota for the primary quota.

There's two downsides;

- Users will have to move data into the Archive/* folders even though such 
could be done automatically,

- There's some development effort to be made to make it feasible/possible.

> > Having said that, maybe I'm misunderstanding your workload. If so, please 
> > enlighten me!
> 
> I guess the most important question what does one try to achieve in using
> quotas.  I daresay it will depend on the organisation but it's probably one
> of:
> 
> 1. To enforce an absolute limit on users' disk usage regardless of
>    circumstances such as how long they are in the organisation.
> 2. To keep people aware that while they may accumulate email over time,
>    there are limits and they should delete mail which is not important.
> 3. No real purpose.
> 
> Obviously if [3], you don't set quotas.  
> 
> If [1], you may be making life very difficult for your users.  A hard quota
> like that might as well be saying "you can store mail for the past X years,
> and then you must delete anything older than that" (assuming they receive a
> constant flow of email).  If you compromise on [1] and start upping
> individual quotas, you quite quickly end up trying to do [2] or in reality
> doing [3].
> 

In an environment of 1k users with 1GB quota, which I've been running on the 
side for 5 years, the average usage is 30%; There's 6 users with a quota 
increase -one of which is to 4GB.

If I remember correctly -this is some time ago, before that the environment of 
450k users had quota set to 100MB by default. The number of support issues 
this caused in our little group of 13k users was about 6-10/month. Naturally, 
such an environment had the means to facilitate delegation properly and such 
and so forth, so someone in 1st level support could bump the number.

Like I suggested; maybe I'm not understanding what your workload is like wrt. 
quota maintenance. Could you share some numbers maybe? Don't get me wrong, I'm 
merely trying to grasp the problem and get a better understanding of it, and 
I'm also not arguing against such functionality - I'm trying to get a better 
understanding so that we can come up with the best solution.

Kind regards,

-- 
Jeroen van Meeuwen
Senior Engineer, Kolab Systems AG

e: vanmeeuwen at kolabsys.com
t: +316 42 801 403
w: http://www.kolabsys.com

pgp: 9342 BF08