[omniORB] crash in openssl code

[Michael Teske]

Hi Duncan,

On 2/23/24 13:52, Duncan Grisby wrote:
> On Fri, 2024-02-23 at 13:10 +0100, Michael Teske via omniORB-list
> wrote:
>
>
>> unfortunately this is an omniORB problem with bidirectional and ssl.
>> By using a mutex around all calls using pd_ssl
>> in sslConnection.cc I found out that Recv snd Send are then used at
>> the same time by different threads. Recv is used
>>   sometimes in blocking mode so simply using a mutex is not the
>> solution as omniorb will than hang completely.
>>
>> The only option for us to use ssl now is to switch bidirectional
>> CORBA off.
> The whole point of bidirectional GIOP is that the same connection is
> used for both outgoing calls and for incoming callbacks. That is why
> there can be a thread blocked in recv at the same time another thread
> is doing a send.
>
> Clearly that works fine with TCP sockets, and it is possible to send
> and receive data literally simultaneously in TCP.

Yes, that's true, sockets are thread safe here.

> It has always worked in the past with OpenSSL as well. Has something
> changed with OpenSSL that means it is no longer thread safe for this?

Every reference I find tells me that this can cause severe problems and crashes. In the github issue I've opened

(https://github.com/openssl/openssl/issues/23650) they tell me this:

"You could definitely see bugs like this if SSL_*() functions are called simultaneously against a single SSL object from multiple threads without proper call serialization through 
locking."

Other references are here

https://github.com/openssl/openssl/issues/20622

and here

https://groups.google.com/g/mailing.openssl.users/c/scU_UV-VuGc/m/HIWCuoY-HzoJ


It works surprisingly well until something unexpected happens, e.g. a handshake in beween... I get the

crash almost exclusively on startup/opening of connection, to reproduce it I had to restart our system up to 100 times.

I found the problem on rhel9 with openssl 3.0.7. I don't know about older versions, since we switched on ssl/bidir only recently.


Another thing, apart from that, shouldn't

sslConnection::Peek()

lock pd_belong_to->pd_collection_lock

as well (before calling SSL_pending()) ?


Greetings,

   Michael




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.omniorb-support.com/pipermail/omniorb-list/attachments/20240223/59baced0/attachment.html>