[omniORB-dev] Re: [omniORB] omniORBpy deadlock / termination problem.

[Teemu Torma]

On Wednesday 12 November 2003 13:26, Duncan Grisby wrote:
> > 1. A shutdown method that is registered with Python's atexit
> > module.   This is required to properly close pyThreadCache, which
> > needs to run Python code, so python still needs to be present.  
> > Otherwise it can be that  when pyThreadCache is destructed at
> > omniORB cleanup during process exit, the python interpreter has
> > already gone.
>
> How can that happen?  The thread cache is only destroyed if
> orb->destroy() is called. As long as that is called before process
> exit time, I don't think there can be a problem. Can you explain a
> situation where it is an issue?
>
> Also, the atexit module was added in Python 2.0. omniORB needs to
> support Python 1.5.2, so it can't be a general solution.
>
> > 2. pyThreadCache join needs to be in python's ALLOW_THREADS macros,
> > otherwise pyThreadCache cannot be shut down from a python method.  
> > (And the previous atexit change should ensure it is always called
> > from a python method).
>
> The thread cache is killed as part of orb->destroy(). The caller of
> orb->destroy() should not be holding the Python interpreter lock.
>
> > 3. pyThreadCache table mutex was split from thread kill mutex to
> > optimize the thread killing, the former mutex being too long lived.
>
> I don't understand that. Can you explain?
>
> I'm afraid I still don't understand the problem. As far as I can see,
> the problem you sent me is just that you are calling orb->destroy()
> from a thread that is holding the Python interpreter lock.

The patches are not the latest ones we are using, but in general, we are 
using python with omniORB with multiple paths:

- python using omniORBpy.
- python using omniORBpy and emebedding our code using omniORB from C++
- our C++ omniORB code embedding python which uses omniORBpy.

All this leads to one tricky situation--the embedded python is destroyed 
before the application destroys omniORB.  In this case, pyThreadCache 
has to go when python goes.  Currently our patches use atexit to make 
sure that this happens.

Another problem applies to servants written in python, and when they are 
released by poa.  In this case we there is no clear control who 
triggered the destruction of the poa, and whether the interpreter lock 
is held or not. 

Now, I am not sure if you want to include patches we have to handle all 
possible situations, since they are currently quite python 2 specific 
(they use atexit and PyGILState_Ensure, but it would not be impossible 
to replicate the functionality for older pythons).  

I can no longer be sure about the separate mutex for shutdown signal, 
but I remember seeing deadlocks with interpreter lock and 
pyThreadCache::guard when running a test application which was short 
lived, and effectively was done before the thread scavenger had change 
to even start.  Separate signal mutex and shorter living 
pyThreadCache::guard solved that particular problem.

Teemu

-------------- next part --------------
A non-text attachment was scrubbed...
Name: omniorbpy.patch
Type: text/x-diff
Size: 6084 bytes
Desc: not available
Url : http://www.omniorb-support.com/pipermail/omniorb-dev/attachments/20031112/daef5fb4/omniorbpy.bin