[Kolab-devel] pykolab/format

[Jeroen van Meeuwen (Kolab Systems)]

On 2012-03-01 15:50, Christian Mollekopf wrote:
> On Friday 24 February 2012 12.12:31 Jeroen van Meeuwen wrote:
>> On 2012-02-24 10:58, Jeroen van Meeuwen (Kolab Systems) wrote:
>> > On 2012-02-24 8:55, Christian Mollekopf wrote:
>> >> pykolab/format/tests/test-thread_safety.py |   68
>> >> +++++++++--------------------
>> >>
>> >>  1 file changed, 22 insertions(+), 46 deletions(-)
>> >>
>> >> New commits:
>> >> commit cb63be089fce18dd9899526455022dad10f8e2ac
>> >> Author: Christian Mollekopf <mollekopf at kolabsys.com>
>> >> Date:   Fri Feb 24 09:54:15 2012 +0100
>> >>
>> >>     Actually call the function from different threads instead of 
>> the
>> >>
>> >> main thread.
>> >
>> > Hi Christian,
>> >
>> > I'm reverting this change, renaming the resulting test (the
>> > original),
>> > and applying your version of the test as the test for thread 
>> safety.
>>
>> Let me rephrase, because maybe I'm not going to do any of that... 
>> Not
>> Sure Yet(TM)
>>
>> If I understand correctly, getSerializedUID() is called from within 
>> the
>> child thread, even though the reason for it to be called is coming 
>> from
>> the main thread's <thread>.uid() call.  It seems though the most
>> upper-level thread causing the function to be called is the thread 
>> under
>> which it runs - and thus getSerializedUID() returns that thread 
>> local
>> value?
>>
>
> Hey Jeroen,
>
> In test-thread_safety.py  there shouldn't be any call to 
> getSerializedUID()
> from the main thread from what I see.

Yes, my statement confirms that for the *current* version, I was 
speaking about what (apparently) happened in the old version, before 
your latest commit that fixed the test.

>> Now, I've attached a parallel_threads test, which succeeds, but does 
>> a
>> full mesh of comparisons. Note I do not have to time.sleep() - but I
>> added some tests that sleep anyway.
>>
>> Could you please examine the output (including what I am assuming 
>> are
>> the thread identifiers spit out to stdout/stderr by libkolabxml), 
>> and
>> confirm/deny whether or not these tests are appropriate / do what we
>> expect them to do?
>>
>
> Yes, the debug output (which you have to activate in utils.cpp, 
> ThreadLocal
> class), are the thread-identifiers.
>
> It seems to work sometimes, but sometimes also not. I suspect that
> threads are being reused because you run the threads sequentially 
> instead of in parallel
> (you sleep first).
>

Well... I've established an error in my reasoning so you're helping me.

Here's what I'm thinking I'm doing (with this new attachment); poke a 
hole in this reasoning if you will, please;

1) I create two threads, that do not yet start.
1a) As part of their __init__ though, they each create an in-memory 
representation of a freshly generated Event object.

     I don't really care about these objects, but I do care about when 
the get_uids() function is being called, whether that happens 
synchronously or asynchronously, and whether or not the main thread 
continues running (creating and running that other thread) while the 
individual threads are going through their motions.

2) I start thread #1,
2a) meanwhile the main thread continues,
2b) thread #1 is serializing the event and getting its hands on a uid,
2c) here it should sleep() for longer than it takes thread #2 to be 
started and serialize it's event for the first time,

    This wasn't the case in the version of my ramblings you were looking 
at before, BTW. I think I've fixed it now though.

3) the result of get_uids() is two variables (self.uid1, self.uid2, 
supposed to be specific to the thread itself) containing a <uid/>, which 
we;

4) after joining back the threads in to the main thread,

5) store and compare against one another using the full mesh of tests.

Furthermore I'm trying to push the timings to their extremes (by 
sleeping a little, and then later on, not at all).

I think especially the mixture of tests succeeding as well as the debug 
output shown, are particularly peculiar, as sometimes without sleeping 
it seems like libkolabxml is indicating it is using the same(?) 
thread-local(?) storage?

> It's also not quite clear what you want to test with this test,

I've grown to become suspicious of things I do not completely 
understand, if you will ;-)

To phrase it differently, my gut feeling says something is wrong, or 
not quite right yet, or any variety of either - my gut feeling doesn't 
speak Dutch or English, it's hard to understand the insensible grunts.

Kind regards,

Jeroen van Meeuwen

-- 
Systems Architect, Kolab Systems AG

e: vanmeeuwen at kolabsys.com
m: +44 74 2516 3817
w: http://www.kolabsys.com

pgp: 9342 BF08