[omniORB] Core dump omniORB/omniNotify

Robert E. Gruber gruber@research.att.com
Thu Apr 3 19:24:02 2003


Your dump says that omni_mutex::~omni_mutex is being called at last line of the
constructor (appended below), but I don't think an unlock() call would destroy a mutex, so
I am wondering if perhaps an exception was thrown and the mutex member (_oplock) is being
destroyed as part of unwinding the constructor from the stack.   Destroying a locked mutex
tends to cause core dumps.

(For robustness purposes, it is tempting to change the mutex destructor so that it unlocks
any locked mutex prior to actually destroying it.  This could be folded into the
thread/mutex/cv wrappers that are already used in omniNotify.)

The approved method of lock/unlock a mutex is of course to use a local wrapper that will
also get unwound when an exception is thrown and unlock the mutex.  omniNotify does not do
this everywhere -- something that must really be fixed at some point.

Using the RDI thread wrappers, I should really have written

  RDI_MutexLock lock(_oplock);

Instead of 

  _oplock.lock();

and then there is no need to do the 

  _oplock.unlock(); 

line at the end.

-- Bob


RDI_NotifyConsumer::RDI_NotifyConsumer(unsigned int numThreads) :
  _oplock(),  _nonempty(&_oplock), _threads(0), 
  _nactive(0), _ndeleted(0), _terminate(0), 
  _entries(0), _last_one(0)
{
  _oplock.lock();
  if ( numThreads ) {
    _threads = new RDI_Thread * [ numThreads ];
    RDI_AssertAllocThrowNo(_threads, "Memory allocation failed -- RDI_Thread\n");
  }
  for (unsigned int i = 0; i < numThreads; i++) {
    // _threads[i] = new RDI_NotifyWorker(this, &RDI_NotifyConsumer::notify);
    _threads[i] = new RDI_NotifyBoundWorker(this, &RDI_NotifyConsumer::notify);
    RDI_AssertAllocThrowNo(_threads[i], "Failed to create new thread\n");
    // If we use unbound threads, we should start the thread using
    // _threads[i]->start();
    RDIDbgChanLog("New push consumer thread: " << _threads[i]->id() << '\n');
  }
  _nactive = numThreads;
  _oplock.unlock();
}

-----Original Message-----
From: omniorb-list-admin@omniorb-support.com
[mailto:omniorb-list-admin@omniorb-support.com] On Behalf Of Thomas Amsler
Sent: Thursday, April 03, 2003 11:51 AM
To: omniorb-list@omniorb-support.com; Robert E. Gruber
Subject: [omniORB] Core dump omniORB/omniNotify


I experience every so often a server crash, core dump, in my 
application. I am using omniORB/omniNotify on the server side and JacORB 
on the client side.  Here is the core dump back trace:

openrct1# gdb /opt/omni/bin/i586_linux_2.0_glibc2.1/notifd core.18830
GNU gdb Red Hat Linux (5.2.1-4)
Copyright 2002 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain 
conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-redhat-linux"...
Core was generated by `notifd'.
Program terminated with signal 6, Aborted.
Reading symbols from 
/opt/omni/lib/i586_linux_2.0_glibc2.1/libCOSNotify4.so.0...done.
Loaded symbols for /opt/omni/lib/i586_linux_2.0_glibc2.1/libCOSNotify4.so.0
Reading symbols from 
/opt/omni/lib/i586_linux_2.0_glibc2.1/libAttNotification4.so.0...done.
Loaded symbols for 
/opt/omni/lib/i586_linux_2.0_glibc2.1/libAttNotification4.so.0
Reading symbols from 
/opt/omni/lib/i586_linux_2.0_glibc2.1/libCOS4.so.0...done.
Loaded symbols for /opt/omni/lib/i586_linux_2.0_glibc2.1/libCOS4.so.0
Reading symbols from 
/opt/omni/lib/i586_linux_2.0_glibc2.1/libCOSDynamic4.so.0...done.
Loaded symbols for /opt/omni/lib/i586_linux_2.0_glibc2.1/libCOSDynamic4.so.0
Reading symbols from 
/opt/omni/lib/i586_linux_2.0_glibc2.1/libomniORB4.so.0...done.
Loaded symbols for /opt/omni/lib/i586_linux_2.0_glibc2.1/libomniORB4.so.0
Reading symbols from 
/opt/omni/lib/i586_linux_2.0_glibc2.1/libomniDynamic4.so.0...done.
Loaded symbols for 
/opt/omni/lib/i586_linux_2.0_glibc2.1/libomniDynamic4.so.0
Reading symbols from 
/opt/omni/lib/i586_linux_2.0_glibc2.1/libomnithread.so.3...done.
Loaded symbols for /opt/omni/lib/i586_linux_2.0_glibc2.1/libomnithread.so.3
Reading symbols from /lib/i686/libpthread.so.0...done.
Loaded symbols for /lib/i686/libpthread.so.0
Reading symbols from /usr/lib/libstdc++.so.5...done.
Loaded symbols for /usr/lib/libstdc++.so.5
Reading symbols from /lib/i686/libm.so.6...done.
Loaded symbols for /lib/i686/libm.so.6
Reading symbols from /lib/libgcc_s.so.1...done.
Loaded symbols for /lib/libgcc_s.so.1
Reading symbols from /lib/i686/libc.so.6...done.
Loaded symbols for /lib/i686/libc.so.6
Reading symbols from /lib/ld-linux.so.2...done.
Loaded symbols for /lib/ld-linux.so.2
#0  0x42028cc1 in kill () from /lib/i686/libc.so.6
(gdb) bt
#0  0x42028cc1 in kill () from /lib/i686/libc.so.6
#1  0x408df07d in raise () from /lib/i686/libpthread.so.0
#2  0x4202a019 in abort () from /lib/i686/libc.so.6
#3  0x40990fb7 in __cxxabiv1::__terminate(void (*)()) 
(handler=0x42029e80 <abort>) at 
../../../../libstdc++-v3/libsupc++/eh_terminate.cc:47
#4  0x40990e7e in __gxx_personality_v0 () at 
../../../../libstdc++-v3/libsupc++/eh_personality.cc:406
#5  0x409ddefd in _Unwind_RaiseException_Phase2 (exc=0x86fa730, 
context=0x434ff02c) at ../../gcc/unwind.inc:57
#6  0x409de02e in _Unwind_RaiseException (exc=0x86fa730) at 
../../gcc/unwind.inc:124
#7  0x40991169 in __cxa_throw (obj=0x86fa730, tinfo=0x8765e48, 
dest=0x8765e48) at ../../../../libstdc++-v3/libsupc++/eh_throw.cc:72
#8  0x408c3389 in omni_mutex::~omni_mutex() () from 
/opt/omni/lib/i586_linux_2.0_glibc2.1/libomnithread.so.3
#9  0x401a7617 in RDI_NotifyConsumer (this=0x86eb088, numThreads=4) at 
RDIChannelUtil.cc:651
#10 0x40124f33 in EventChannel_i (this=0x879cf70, cfactory=0x806ace0, 
ffactory=0x806a990, init_qos=@0x87505b0, init_adm=@0x879cd10, 
s_qos=0x8069ae8,     myserial=@0x434ff6b0) at EventChannel_i.cc:218
#11 0x40133355 in EventChannelFactory_i::_create_channel(long&) 
(this=0x806ace0, myID=@0x434ff6b0) at EventChannel_i.cc:1666
#12 0x40131cb0 in 
EventChannelFactory_i::create_channel(CosNotification::PropertySeq 
const&, CosNotification::PropertySeq const&, long&) (this=0x806ace0,
    qosP=@0x8607e50, admP=@0x876eb68, myID=@0x434ff6b0) at 
EventChannel_i.cc:1504
#13 0x40139b49 in virtual thunk to 
EventChannelFactory_i::create_channel(CosNotification::PropertySeq 
const&, CosNotification::PropertySeq const&, long&) ()
    at 
../../../../src/services/omniNotify/include/CosNotifyChannelAdmin_i.h:1898
#14 0x404ab4c0 in 
_0RL_lcfn_9043e11a9721dab3_a6000000(omniCallDescriptor*, omniServant*) 
() from /opt/omni/lib/i586_linux_2.0_glibc2.1/libCOS4.so.0
#15 0x4062e3bf in omniCallHandle::upcall(omniServant*, 
omniCallDescriptor&) () from 
/opt/omni/lib/i586_linux_2.0_glibc2.1/libomniORB4.so.0
#16 0x404ac1aa in 
CosNotifyChannelAdmin::_impl_EventChannelFactory::_dispatch(omniCallHandle&) 
() from /opt/omni/lib/i586_linux_2.0_glibc2.1/libCOS4.so.0
#17 0x40331256 in 
AttNotification::_impl_EventChannelFactory::_dispatch(omniCallHandle&) ()
   from /opt/omni/lib/i586_linux_2.0_glibc2.1/libAttNotification4.so.0
#18 0x40341bb0 in virtual thunk to 
AttNotification::_impl_EventChannelFactory::_dispatch(omniCallHandle&) ()
   from /opt/omni/lib/i586_linux_2.0_glibc2.1/libAttNotification4.so.0
#19 0x4061f3e2 in omni::omniOrbPOA::dispatch(omniCallHandle&, 
omniLocalIdentity*) () from 
/opt/omni/lib/i586_linux_2.0_glibc2.1/libomniORB4.so.0
#20 0x40605b54 in omniLocalIdentity::dispatch(omniCallHandle&) () from 
/opt/omni/lib/i586_linux_2.0_glibc2.1/libomniORB4.so.0
#21 0x40649c75 in omni::GIOP_S::handleRequest() () from 
/opt/omni/lib/i586_linux_2.0_glibc2.1/libomniORB4.so.0
#22 0x4064999d in omni::GIOP_S::dispatcher() () from 
/opt/omni/lib/i586_linux_2.0_glibc2.1/libomniORB4.so.0
#23 0x40646fac in omni::giopWorker::real_execute() () from 
/opt/omni/lib/i586_linux_2.0_glibc2.1/libomniORB4.so.0
#24 0x40646e4f in omni::giopWorkerInfo::run() () from 
/opt/omni/lib/i586_linux_2.0_glibc2.1/libomniORB4.so.0
#25 0x40646f3d in omni::giopWorker::execute() () from 
/opt/omni/lib/i586_linux_2.0_glibc2.1/libomniORB4.so.0
#26 0x405feb47 in omniAsyncWorker::real_run() () from 
/opt/omni/lib/i586_linux_2.0_glibc2.1/libomniORB4.so.0
#27 0x405fddef in omniAsyncWorkerInfo::run() () from 
/opt/omni/lib/i586_linux_2.0_glibc2.1/libomniORB4.so.0
#28 0x405fea8d in omniAsyncWorker::run(void*) () from 
/opt/omni/lib/i586_linux_2.0_glibc2.1/libomniORB4.so.0
#29 0x408c3b96 in omni_thread_wrapper () from 
/opt/omni/lib/i586_linux_2.0_glibc2.1/libomnithread.so.3
#30 0x408dc941 in pthread_start_thread () from /lib/i686/libpthread.so.0

It seems that something goes wrong in notifd. Did anybody ever 
experiance similar problems? Is this a bug?
I am using the latest CVS code for both omniORB and omniNotify and RH8.0 
with all the latest updates.
Thank you.

-- 
Thomas Amsler
W: (530) 754-5198
M: (916) 549-4192
amsler@cs.ucdavis.edu
http://www.openrct.org/thomas

"Imagination is more important than knowledge."
	--Albert Einstein