[omniORB] CORBA performance on InfiniBand network

[Alexandre DENIS]

On Mon, 3 Aug 2015 05:01:22 +0000
"Soundararajan, Krishna" <Krishna.Soundararajan at kla-tencor.com> wrote:

> Hi,
> I am performing a bandwidth comparison study on CORBA using
> Infiniband and 10GeT link.
> (...)
> With this results, I have following questions.
> 1)      I read in several old articles in internet that ORB kills the
> throughput because of inefficient memory copies and
> marshalling/unmarshalling. In 10Gb Ethernet and 1 Gb Ethernet, I do
> see almost  90% utilization(CORBA compared to ethernet). But IB card
> give less than 25% of network utilization. What is causing this
> overhead? 2)      Though CORBA is meant to be used for remote method
> calls, we are amazed with its superiority in serializing. Can we use
> it as alternative for socket data transfer with amnual serializing
> code? Is CORBA suitable in high bandwidth  networks for data transfer?

Hi,

I worked on CORBA for high-performance computing in the past, and
nowadays I work with InfiniBand. I can share my experience.

CORBA is not slow by itself. It depends on the ORB used. You can read
in this paper https://hal.inria.fr/inria-00000136 some past results.
omniORB is the best performing CORBA implementation, and reaches most
of the time the nominal network performance (in case of homogeneous
hosts).

The use of InfiniBand may be tricky. What may prevent to reach nominal
network speed is data alignment and memory registration. Bandwidth is
low when buffers are not aligned on words boundaries. I don't know
whether omniORB allocates aligned buffers or not. Memory registration
is needed to program the hardware from user-space, so that the NIC
knows the mapping for virtual addresses. This operation has a high
cost. MPI implementations usually have a cache in case the same buffer
is sent multiple times (which happens a lot, for MPI applications). I
guess socket layers do the same. However, with CORBA you always have
registration cache misses since memory at the receiver side is
dynamically allocated for each invocation. You can get some more
details here: https://hal.inria.fr/inria-00586015

What socket layer do you use? IPoIB or SDP?

Regards,
-Alexandre DENIS.



-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <http://www.omniorb-support.com/pipermail/omniorb-list/attachments/20150804/d287ef60/attachment.sig>