[omniORB] Threads hanging in OMNI ORB 4.2.1 with mutex lock

[Agarwal, Shelendra]

Thanks Serguei,
We are working on RHEL 7.3 and according to the link provided it seems that on 7.3 the highlighted issue should be resolved.
We checked this by running the following command:
rpm -q --changelog kernel-`uname -r` | grep futex | grep ref
- [kernel] futex: Mention key referencing differences between shared and private futexes (Larry Woodman) [1205862]
- [kernel] futex: Ensure get_futex_key_refs() always implies a barrier (Larry Woodman) [1205862]


With Best Regards,

Shelendra Agarwal
Telco Big Data Analytics
Communications & Media Solutions
Hewlett Packard Enterprise
+91-9945056319
shelendra.agarwal at hpe.com<mailto:shelendra.agarwal at hpe.com>

From: kolos [mailto:serguei.kolos at cern.ch]
Sent: Sunday, December 09, 2018 8:55 PM
To: Agarwal, Shelendra <shelendra.agarwal at hpe.com>
Cc: omniorb-list at omniorb-support.com; Cassis, Ivano <ivano.cassis at hpe.com>
Subject: Re: [omniORB] Threads hanging in OMNI ORB 4.2.1 with mutex lock

Hi

I have seen similar problem with our SW a couple of years ago which turned out to be caused by a bug
in the linux kernel used by RHEL 6. According to the following article the RHEL 7.x systems are also
affected by the same issue:
https://www.infoq.com/news/2015/05/redhat-futex

Cheers,
Serguei


On 7 Dec 2018, at 18:32, Agarwal, Shelendra via omniORB-list <omniorb-list at omniorb-support.com<mailto:omniorb-list at omniorb-support.com>> wrote:

Hi,
We are using c++ implementation of OMNI ORB 4.2.1 on RHEL 7.x.
We have a process that sometime hangs and stops responding to other processes for CORBA calls. We collected the gstack of the process and from the thread dumps we see most of the threads are in omni_mutex_lock::omni_mutex_lock condition.

We collected multiple snaps of gstack and we see that there is no changes happening over a period of time. Attached are 2 snaps taken 10 minutes apart.

We understand that threads are waiting for some resources but we are unable to understand which resource. May be they are in kind of deadlock state waiting for each other or something else, can you please throw some lights?

We noted that omniTransportLock is often involved in many mutex locks in the thread dump. Are there any details about this omniTransportLock not being released available?

Thanks in advance for your help!!


With Best Regards,

Shelendra Agarwal
Telco Big Data Analytics
Communications & Media Solutions
Hewlett Packard Enterprise
+91-9945056319
shelendra.agarwal at hpe.com<mailto:shelendra.agarwal at hpe.com>

<2018-12-06-17-50-00_GSTACK.log><2018-12-06-18-00-47_GSTACK.log>_______________________________________________
omniORB-list mailing list
omniORB-list at omniorb-support.com<mailto:omniORB-list at omniorb-support.com>
http://www.omniorb-support.com/mailman/listinfo/omniorb-list

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.omniorb-support.com/pipermail/omniorb-list/attachments/20181209/7471b473/attachment-0001.html>