[omniORB] Windows NT and dllimport/dllexport

[Hayes, Ted (London)]

Developers using omniORB on NT may be interested in a workaround for a
problem that can arise when building systems of DLLs (i.e. executable Z
loads DLL Y that loads DLL X).

Previous postings have discussed use of the USE_stub_in_nt_dll,
USE_core_stub_in_nt_dll and USE_dyn_stub_in_nt_dll
macros together with generation of .DEF files to export symbols when
building / using NT DLLs.  I have experienced a problem with this mechanism
(the same problem I have also observed in other packages including Rogue
Wave) when building systems of DLLs.  If DLL Y is a client of DLL X, a macro
used to flag the import of a symbol from X will cause an error if also used
to qualify a symbol within Y.  You need to use a different macro for each
DLL.  In the case of omniORB, if I implement a base interface in X and then
implement a derived interface in Y, using USE_stub_in_nt_dll to import the X
symbols fails as then Y contains illegal declarations - for example

static __declspec(dllimport) const char * _PD_repoId;

Inspection of the .hh file generated by omniidl shows that there is a
per-library macro that can be used to workaround the problem (maybe Sai-Lai
or Duncan anticipated this?).  In the above case the macros (one each for
core and dynamic stubs) are

USE_core_stub_in_nt_dll_NOT_DEFINED_X
USE_dyn_stub_in_nt_dll_NOT_DEFINED_X

To solve the problem, when building X from XSK.cc just leave the macros
undefined and generate a .DEF file as normal.  When building Y, add compiler
flags

-DUSE_core_stub_in_nt_dll
-DUSE_core_stub_in_nt_dll_NOT_DEFINED_X

Then when we compile through Y.hh, compilation starts through X.hh; since
USE_core_stub_in_nt_dll is defined the X.hh header makes no attempt to
define USE_core_stub_in_nt_dll_NOT_DEFINED_X (so no error arises) and we get
the dllimport for the X symbols as desired.  But the definition of
USE_core_stub_in_nt_dll_NOT_DEFINED_X then causes USE_core_stub_in_nt_dll to
be undefined on exit from X.hh.  The remainder of Y.hh compiles without the
problematic import specifier, and we can then generate Y.DEF to do the
export of symbols from Y..

Although I haven't tried, I suspect the approach can be extended to more
levels of interface inheritance by defining the NOT_DEFINED macro
appropriate to the final lower level header #included by the interface being
built..

As an example of the horrors that can arise without taking the correct
precautions, suppose the Y DLL exports additional functions that call
functions within the Y interface, but the Y interface itself is not
exported.  It then appears possible to build Y without using any macros, and
obtain successful linkage by including X.lib.  Although the X functions will
not be marked dllimport, the Y link process appears to find something in
X.lib that keeps it happy... (I know not what is going on in the mind of the
MS linker here).  Then, when we try to use the Y DLL and a Y object is being
created, eventually the _is_a() function in the Y proxy object factory will
attempt to access the (static const char *) IDL repository ID string in the
X interface - the address obtained is garbage and the program GPFs
immediately.  (I was actually doing this from a SWIG-ed Python wrapper
around the Y DLL, so a LoadLibrary(Y) was occurring).  Using the above
scheme fixed the problem.