[omniORB] thread pool vs. thread per connection

Tue Nov 26 17:06:01 2002

The primary purpose of the research is not omniNames itself but rather our
own service, so the concurrency semantics of omniNames aren't important.
Also, it appeared that omniNames uses a readersWriter lock, so that client
could access the data concurrently.

What I'm afraid of with the connection watching is a multithreaded Java
client making concurrent calls and multiplexing the requests on a single
connection.  Assume two Java client threads make calls over a single
connection, each 2 ms apart.  Further assume the duration of each upcall is
between 10 and 200 ms.  With the 50-100ms delay in socket monitoring, the
average response time the client threads see would be impacted in a big
way, or so I theorize.  The first call would start right away.  If it takes
40ms to complete, the second call will have to wait that whole time because
the select() hasn't been restarted yet, so the same thread will see the new
request during it's watch time period.  If that second call takes 20ms,
then the response time seen is 60ms, a 3x slowdown as seen by the second
java client thread.  This is just a thought experiment at this point.

I don't think I have the ability to extend the thought experiment to dozens
of concurrent client threads and more than one worker thread doing
connection watches on the same connection.  Then there's the "data in
buffer" file descriptors which don't appear to trigger thread creation (but
perhaps should) unless select() returns nready > 0.  So I'd like to run
tests.  In this Java -> C++ environment I've always configured for thread
pool, with no connection watching to attempt to avoid the above scenario (I
thought it would be most like the Orbix2000 thread pooling used prior to
the adoption of omniORB).  I didn't realize from the wiki page or omniORB
docs that the 50-100ms delay would still impact us.  Not until the huge
change in name service performance that put me on this path of research (I
let all orb processes share a single configuration)...

By the way, the state diagrams in giopServer.cc were very helpful.  An
additional comment about the delay in connection monitoring state would
have made things crystal clear without any reading of source code.  As it
is the diagrams caused me to initially believe that the connection would be
immediately monitored upon the notifyWkPreUpCall event.

Thanks,
Kendall

                    Duncan Grisby                                                                              
                    <duncan@grisby       To:     baileyk@schneider.com                                         
                    .org>                cc:     omniorb-list@omniorb-support.com                              
                    Sent by:             Fax to:                                                               
                    dpg1@grisby.or       Subject:     Re: [omniORB] thread pool vs. thread per connection      
                    g                                                                                          

                    11/26/2002                                                                                 
                    10:24 AM                                                                                   

On Tuesday 26 November, baileyk@schneider.com wrote:

>      I found that the omniAsyncInvoker has a 10 second idle timeout for
the
> threads in the pool.  Meaning that any thread not used for 10 seconds
will
> die.  This explains the excessive thread creation.  I'd run a test, say 5
> concurrent calls, and see 5 threads created.  Then I'd look over the
> results (for more than 10 seconds) and run again, and again see 5 threads
> created.  Under continuous load, the thread creation may not be so bad,
but
> I think the 10 second timeout should be configurable.  In reality usage
> spikes may be expected ever few minutes, not seconds.

I agree that it's a good idea to make the 10 second timeout
configurable. I'll do it before the 4.0.1 release.

>    I think I've tracked it down to the logic in
> SocketCollection::setSelectable().  There's a comment
>
>     // XXX poke the thread doing accept to look at the fdset immediately.
>
> But no code follows this to do such a 'poke', which is evidently why
> there's an XXX in the comment.  A future planned enhancement?

Yes, that's exactly what's happening. The reason it hasn't been done
yet is that, as you have discovered, it's not at all easy to do in a
cross platform way. At some point, I intend to look into whether it's
possible to achieve the same effect without having to trigger the
thread doing select(). I'm open to suggestions...

Is there a good reason that you don't want to use the
threadPoolWatchConnection setup?  That way, sequences of calls from a
single client are all done by the same thread, without having to wait
for the select thread to get involved. The policy reduces the
concurrency between clients, but given that they are all accessing the
same data, they can't access it concurrently anyway. With something
like the Naming service, a single client isn't likely to make a very
large number of calls without a break, so it's only a transitory
effect.

Cheers,

Duncan.

--
 -- Duncan Grisby         --
  -- duncan@grisby.org     --
   -- http://www.grisby.org --