[omniORB] Latencies - Oneway vs. Roundtrip

[Helge Penne]

Woha!  I started using CORBA to get away from all this low-level stuff.  This is a bit
beside my field of competence...

I've now traced the control flow from the proxy object and all the way down to the NT
WinSock "send" function.  So far, it does not seems that omniORB does any buffering of
oneway calls.  That should indicate that the buffering is done by the Windows NT socket
implementation.

The problem would thus seem to be the WinNT sockets use buffering by default.  My guess
would be that the problem could be fixed by changing the socket options to
"TCP_NODELAY", and that this modification could probably be done by adding the
following statements to tcpSocketMTfactory.cc, line 834 (just after the "if (sock =
socket(...)...)" block):

    BOOL bNoDelay = TRUE;
    int nResult = setsockopt( sock,  IPPROTO_TCP1, TCP_NODELAY, &bNoDelay, sizeof(
bNoDelay) );
    // Code to check the return value should be added...

I still don't see why there is no delay for roundtrip calls, but then I'm not that
familiar with sockets and TCP/IP.

So, what do you all think?  Does this sound reasonable, or am I all lost here?  ;-)

- Helge

Helge Penne wrote:

> I know.  I seem to remember that it was fairly easy to turn off on UNIX platforms,
> though.  This problem could be important to us, so I think I'll dive into the
> source code to see what I can find out.  If anyone else has been down this path
> before, I'd be very happy if they would save me the time.  ;-)
>
> I gues what we're looking for here is either a global option to turn this buffering
> on or off, or some sort of "flush" command to send any contents in the buffer.
>
> - Helge
>
> Alberto Squassabia wrote:
>
> > It may not be omniORB that does the buffering.  Some TCP implementations
> > (don't know exactly about NT, though) are optimized for throughput, not
> > latency, and don't put a socket buffer on the wire until the buffer is
> > full.
> >
> > [Re: [omniORB] Latencies - Oneway vs. Roundtrip] Helge.Penne@datarespons.no
> > wrote:
> > >
> > > Jim Redman wrote:
> > >
> > > > I have a counter-intuitive latency problem and would like some input and
> > > > validation of what I think I'm seeing.  This is strictly latency, not
> > > > throughput.   Running between two NT 4.0 system with 2.7.0 I see latencies
> > > > for oneway messages that appear to approach 100ms.  When I change the same
> > > > calls to roundtrip the latencies drop below the NT measurable time (about
> > > > 10ms?).
> > > >
> > > > Has anyone seen anything similar or different, that is, can anyone confirm
> > > > or deny my numbers.
> > >
> > > I'm doing some benchmarking at this very moment, and just discovered this
> > > latency myself.  The test uses a "publish/subscribe" pattern to distribute a
> > > packet of data from a server to a client application.  I ran both programs on
> > > the same NT 4.0 computer with a packet rate of 100Hz, and saw latencies of up
> > > to 200ms when using oneway calls.  With ordinary calls, the latency is hardly
> > > ever measurable using GetTickCount.
> > >
> > > What was really revealing was that the latency would start at approx. 40ms,
> > > and then increase by 10ms for each call until it would reach 200 ms.  It
> > would
> > > then fall back to 40ms and start over.
> > >
> > > My theory is that omniORB buffers my oneway calls and sends them in larger
> > > packages every 200ms (5Hz).  This explains every symptom that I have seen,
> > and
> > > also fits in nicely with my previous experience with VisiBroker.  VisiBroker
> > > also buffers such requests by default, but there is an option in BOA_init
> > that
> > > lets you turn off this buffering and force these requests to be sent
> > > immediately.
> > >
> > > This leads up to a very important question: How do we turn off this buffering
> > > in omniORB?  This is quite an important question for my project.  Has anybody
> > > done this before?
> > >
> > > --
> > --
> > Alberto Squassabia  (970)898-7705  alsq@cnd.hp.com  HP TMD
> > "   We are Pentium of Borg. Division is futile. You      "
> > "   will be approximated. (ca. AD 1995)                  "