[omniORB] SocketCollection bug

[Duncan Grisby]

On Tue, 2010-01-19 at 08:46 -0800, Kevin Bailey wrote:

> Forgive me for jumping in, but I'd like to make sure I
> understand omniorb thread model. select() doesn't
> consume but won't it typically be followed by its own
> recv() and, if the other thread exhausted the socket,
> wouldn't this one block ?

That's not what's going on here. The threading and connection management
model is rather complex, due to the semantics of GIOP.

Imagine we're in the default thread-per-connection model (thread pool is
similar, but it's easier to explain in thread-per-connection). When no
call is happening, the connection's thread is blocked in recv() waiting
for an incoming call. Now, a call comes in so the recv() returns the
data, and the thread unmarshals the arguments and processes the up-call.
Once the call returns, the same thread marshals the return values and
goes back to blocking in recv() for the next call. Simple, right?

Unfortunately for this simple picture, the GIOP specification allows
another concurrent call to be sent on the same connection while the
first call is being processed. So, before the connection's dedicated
thread starts the up-call, it marks the connection as "selectable".
Periodically, a thread looks at all the selectable connections and uses
select (or poll) to watch them. If data arrives while the dedicated
thread is still in its upcall, a new thread from the pool is triggered
to handle the concurrent upcall.

When the first call returns, the connection is marked so it is no longer
selectable. However, it is expensive to stop the thread that's blocked
in select immediately, so it is allowed to keep watching the connection
until the next time it rescans the connections. If a new call comes in
while the select thread is still watching, it can wake up both the
select thread and the dedicated thread (which is back to blocking in
recv). However, the select thread can see that the connection is no
longer selectable, so it ignores the condition.

You can see all this in action in SocketCollection.cc and giopServer.cc
if you're interested...

Cheers,

Duncan.

-- 
 -- Duncan Grisby         --
  -- duncan at grisby.org     --
   -- http://www.grisby.org --