[omniORB] Troubleshooting a server that is becoming inaccessible during the night.

[Mike Bendickson]

Ok, it happened again today.  I am not able to connect to that port using
telnet when this happens.  I was able to connect with telnet ealier and saw
a "GIOP" message before getting kicked off.

So, something has trashed the socket then, or the thread is not sitting on
accept?

Is there something else I should look for?  Has anyone else experienced this
with 2.8.0 on NT?

Thanks,
-Mike

-----Original Message-----
From: sll@uk.research.att.com [mailto:sll@uk.research.att.com]On Behalf
Of Sai-Lai Lo
Sent: Tuesday, November 07, 2000 12:57 PM
To: Mike Bendickson
Subject: Re: [omniORB] Troubleshooting a server that is becoming
inaccessible during the night.

The trace looks normal. The message you pointed out just means that the
ORB is garbage collecting a client connection after it has been idle for
some time.

I suggest you try to determine whether you can connect at the tcp level to
your server. Try telnet to the port which the ORB is listening on.
To find out the port no., simply catior on one of the object reference the
server is giving out. The rendezvouser thread in the ORB should kick into
action and spawn a worker.

Sai-Lai


>>>>> Mike Bendickson writes:

> I have a server on one system that becomes inaccessible every night.  Both
> network and local clients are unable to send messages to it.  If I restart
> the server, the clients are able to connect again.

> Looking at the trace output, the only thing that sticks out to me is:

> 	tcpSocketMTfactory real_shutdown: fd no. 324

> I haven't seen this message on our other (working) systems.

> I believe that tcpSocketStrand::real_shutdown() is the function that is
> printing this message.

> I'm using 2.8.0 on NT 4.0.

> Here's the trace output:

> gateKeeper is not installed. All incoming are accepted.
> strand Rope::incrRefCount: old value = 0
> strand Ripper: start.
> scavenger : start.
> tcpSocketMTfactory Rendezvouser: start.
> tcpSocketMTfactory Rendezvouser: block on accept()
> scavenger : scanning connections
> scavenger : scanning connections
> scavenger : scanning connections
> .
> .
> .
> scavenger : scanning connections
> scavenger : scanning connections
> scavenger : scanning connections
> tcpSocketMTfactory Rendezvouser: unblock from accept()
> tcpSocketMTfactory Rendezvouser: accept new strand.
> tcpSocketMTfactory Rendezvouser: block on accept()
> tcpSocketMTfactory Worker: start.
> scavenger : scanning connections
> scavenger : scanning connections
> scavenger : scanning connections
> scavenger : scanning connections
> scavenger : scanning connections
> scavenger : scanning connections
> scavenger : scanning connections
> tcpSocketMTfactory Rendezvouser: unblock from accept()
> tcpSocketMTfactory Rendezvouser: accept new strand.
> tcpSocketMTfactory Rendezvouser: block on accept()
> tcpSocketMTfactory Worker: start.
> scavenger : scanning connections
> scavenger : scanning connections
> scavenger : scanning connections
> .
> .
> .
> scavenger : scanning connections
> scavenger : scanning connections
> scavenger : scanning connections
> tcpSocketMTfactory Worker: #### Communication failure. Connection closed.
> tcpSocketMTfactory Worker: exit.
> tcpSocketMTfactory ~tcpScoketStrand: close socket no. 300
> tcpSocketMTfactory Rendezvouser: unblock from accept()
> tcpSocketMTfactory Rendezvouser: accept new strand.
> tcpSocketMTfactory Rendezvouser: block on accept()
> tcpSocketMTfactory Worker: start.
> tcpSocketMTfactory Worker: #### Communication failure. Connection closed.
> tcpSocketMTfactory Worker: exit.
> tcpSocketMTfactory ~tcpScoketStrand: close socket no. 352
> scavenger : scanning connections
> scavenger : scanning connections
> scavenger : scanning connections
> .
> .
> .
> scavenger : scanning connections
> scavenger : scanning connections
> scavenger : scanning connections
> tcpSocketMTfactory real_shutdown: fd no. 324
> scavenger : scanning connections
> scavenger : scanning connections
> scavenger : scanning connections
> .
> .
> .
> scavenger : scanning connections
> scavenger : scanning connections
> scavenger : scanning connections
> tcpSocketMTfactory Worker: #### Communication failure. Connection closed.
> tcpSocketMTfactory Worker: exit.
> tcpSocketMTfactory ~tcpScoketStrand: close socket no. 324
> scavenger : scanning connections
> scavenger : scanning connections
> scavenger : scanning connections
> .
> .
> .

> Nothing but "scavenger : scanning connections" after this, even when a
> client tries to make a request.

> Troubleshooting tips are very welcome.

> Thanks,
> -Mike





--
Sai-Lai Lo                                   S.Lo@uk.research.att.com
AT&T Laboratories Cambridge           WWW:   http://www.uk.research.att.com
24a Trumpington Street                Tel:   +44 1223 343000
Cambridge CB2 1QA                     Fax:   +44 1223 313542
ENGLAND