Demand-driven object loading

Sai-Lai Lo S.Lo@orl.co.uk
Fri, 10 Oct 1997 14:31:28 +0100


>>>>> Randy Shoup writes:

> Sai-Lai --
>   We are looking into implementing a demand-driven approach to loading
> persistent objects in omniORB.  In looking through the maillist
> archives, I found the following references, where this issue was
> discussed:

> http://www.orl.co.uk/omniORB/archives/1997-Jun/0025.html
> http://www.orl.co.uk/omniORB/archives/1997-Jun/0027.html

>   In thinking about the issues, we were concerned that the forwarding
> approach would require us to have a potentially very large number of
> active objects -- at least a reDirector for each available object.  We
> are leaning toward the objectLoader approach, most particularly because
> it would allow us to control/minimize the number of active objects in
> the system at any one time.  If we were to end up modifying the ORB code
> to do this, what sorts of problems do you think this would cause?  How
> problematic are the concurrency issues you allude to in your note?  

I agreed the reDirector approach alone would still require one reDirector
for every available object. This is not scalable.

To answer your question, I will first describe the limitations of the
current omniORB implementation. Then I'll outline an interim extension to
the ORB to support an object loader. 


Limitations
-----------
1. Object IDs

   omniORB_2.2.0 uses fixed size object identifiers (OIDs). It treats its
   OIDs as pure names, i.e. it does not divide an OID into subfields and
   uses the whole OID for equality test. Applications can specify the OID for
   an object implementation but it can only be the same type as those
   generated by the ORB. Using fixed size OIDs allow us to do simple and
   fast hashing to locate an object. 

   With large scale persistent object stores, it is quite likely that the
   applications would like to name objects using variable/fixed-size OIDs
   that cannot fit into the OIDs used by the ORB. There is also the
   requirement of loading the objects on demand rather than statically at
   startup time. Some form of object loader is desirable.

   In general, an application may have multiple backend stores that it can
   load objects from and it may want to use different object loaders for
   these stores.

2. Object Loaders

   To satisfy these requirements, the ORB needs to support another form of 
   OID:
           <object manager ID><internal object ID>

   The application registers with the ORB an object manager ID (OMID) and
   an object loader to receive upcalls from the ORB. The object loader will
   be passed the internal object ID (IOID) and it is its job to further
   decode this ID and load the object from the persistent store. Once the
   object is instantiated, it pass its reference back to the ORB. The ORB
   can then dispatch requests to the object. Notice that this is a slow
   path to locate objects that have already been loaded. It is better for
   the objects to issue location forward replies to inform the clients of
   the fixed size OIDs the ORB uses internally to identify the
   objects. Subsequent calls to the objects from these clients will then be
   dispatched by the ORB through its normal fast path.

   As you know, this is not supported by omniORB_2.2.0. However, this is at
   the top of my to-do list and will be done once I get the coming release 
   out of the door.

Interim Extension
-----------------

(A) Matthew Newhook has added an object loader to omniORB_2.2.0. He has
    made his changes available for download:

     http://www.orl.co.uk/omniORB/archives/1997-Oct/0013.html
    
    This may be what you want if you are happy to use the fixed size
    omniORB OIDs to name your persistent objects.


(B) A more general object loader solution to support application defined
    variable length OIDs can be done with small amount of modifications. 
    I attached at the end the code to illustrate how this can be done. 

  I'm not completely happy with the solution because it only supports a
  single loader. Of course you can have a chain of loaders and try each one
  in turn until one succeeds or all fail. That is not satisfactory because
  your loaders may be called unnecessarily.
 
  To install an object loader, the application writes a function with this
  signature:

     typedef CORBA::Object_ptr (*loader)(CORBA::Char*key,CORBA::ULong keysize);

  The application instantiates the variable omniORB::objectLoader with the
  pointer to this function.

  The function should locate the object using the key argument and return
  the object implmentation it finds in its own active object table. If the
  key does not point to a valid object, it should raise a
  CORBA::OBJECT_NOT_EXIST exception. Concurrent upcalls may be dispatched
  by the ORB to this function so it has to deal with the concurrency
  appropriately. 

  If the application wants the ORB to dispatch future requests to the
  objects through its fast path, the loader should returns a reDirector
  object instead of the real implementation object. The reDirectors can
  then capture the requests and issue location forward messages to the real
  objects. To do this properly, the ORB should assume the role of the
  reDirectors and issues location forward messages internally. However,
  to do so would require more extensive changes to the ORB.

Have a look. Comments and feedbacks are welcomed.


Regards,

Sai-Lai

------- Interim extension to support application defined object loader ---

1. Add the type definition for the loader and the global variable
     omniORB::objectLoader in include/omniORB2/omniORB.h

2. Replace these two functions in src/lib/omniORB2/giopServer.cc with the 
   following:

XXXXXX One should also do similar modifications to createObjRef() in
       src/lib/omniORB2/objectRef.cc. The function is affected if an object
       reference unmarshalled from the wire points to a local object that
       should be loaded by the loader.

void
GIOP_S::HandleRequest(CORBA::Boolean byteorder)
{
  CORBA::ULong msgsize;
  CORBA::ULong keysize;

  try {
    // This try block catches any exception that might raise during
    // the processing of the request header
    msgsize <<= *this;
    if (msgsize > MaxMessageSize()) {
      SendMsgErrorMessage();
      setStrandDying();
      throw CORBA::COMM_FAILURE(0,CORBA::COMPLETED_NO);
    }

    RdMessageSize(msgsize,byteorder);  // Set the size of the message body.
  
    // XXX Do not support any service context yet, 
    // XXX For the moment, skips the service context
    CORBA::ULong svcccount;
    CORBA::ULong svcctag;
    CORBA::ULong svcctxtsize;
    svcccount <<= *this;
    while (svcccount-- > 0) {
      svcctag <<= *this;
      svcctxtsize <<= *this;
      skip(svcctxtsize);
    };

    pd_request_id <<= *this;

    pd_response_expected <<= *this;

    keysize <<= *this;
    get_char_array((CORBA::Char *)&pd_objkey,keysize);

    CORBA::ULong octetlen;
    octetlen <<= *this;
    if (octetlen > OMNIORB_GIOPDRIVER_GIOP_S_INLINE_BUF_SIZE)
      {
	// Do not blindly allocate a buffer for the operation string
	// a poison packet might set this to a huge value
	if (octetlen > RdMessageUnRead()) {
	  throw CORBA::MARSHAL(0,CORBA::COMPLETED_NO);
	}
	CORBA::Octet *p = new CORBA::Octet[octetlen];
	if (!p)
	  throw CORBA::NO_MEMORY(0,CORBA::COMPLETED_NO);
	pd_operation = p;
      }
    get_char_array((CORBA::Octet *)pd_operation,octetlen);

    octetlen <<= *this;
    if (octetlen > OMNIORB_GIOPDRIVER_GIOP_S_INLINE_BUF_SIZE)
      {
	// Do not blindly allocate a buffer for the principal octet vector
	// a poison packet might set this to a huge value
	if (octetlen > RdMessageUnRead()) {
	  throw CORBA::MARSHAL(0,CORBA::COMPLETED_NO);
	}
	CORBA::Octet *p = new CORBA::Octet[octetlen];
	if (!p)
	  throw CORBA::NO_MEMORY(0,CORBA::COMPLETED_NO);
	pd_principal = p;
      }
    get_char_array((CORBA::Octet *)pd_principal,octetlen);
  }
  catch (CORBA::MARSHAL &ex) {
    RequestReceived(1);
    SendMsgErrorMessage();
    pd_state = GIOP_S::Idle;
    return;
  }
  catch (CORBA::NO_MEMORY &ex) {
    RequestReceived(1);
    MarshallSystemException(this,SysExceptRepoID::NO_MEMORY,ex);
    return;
  }

    // Note: If this is a one way invocation, we choose to return a 
    // MessageError Message instead of returning a Reply with System Exception
    // message because the other-end says do not send me a reply!

  omniObject *obj = 0;    
  try {
    if (omniORB::objectLoader == 0) {
      if (keysize != sizeof(omniObjectKey)) {
	throw CORBA::OBJECT_NOT_EXIST(0,CORBA::COMPLETED_NO);
      }
      obj = omni::locateObject(pd_objkey);
    }
    else {
      // The application has installed an object loader
      // If we cannot locate the object in our object table, we call
      // the object loader to ask for it.
      // If it throws OBJECT_NOT_EXIST if it cannot find it either.
      // The loader should be capable of dealing with concurrent upcalls.
      try {
	if (keysize == sizeof(omniObjectKey))
	  obj = omni::locateObject(pd_objkey);
      }
      catch (...) { }
      if (!obj) {
	// Be careful with the reference count!
	// The returned object should have the reference count increment
	// by 1, like it is done in omni::locateObject()
	obj = omniORB::objectLoader(pd_objkey,keysize)->PR_getobj();
      }
    }

    if (!obj->dispatch(*this,(const char *)pd_operation,pd_response_expected))
	{
	  if (!obj->omniObject::dispatch(*this,(const char*)pd_operation,
					pd_response_expected))
	    {
	      RequestReceived(1);
	      throw CORBA::BAD_OPERATION(0,CORBA::COMPLETED_NO);
	    }
	}
  }
  catch (CORBA::OBJECT_NOT_EXIST &) {
    if (!obj) {
      RequestReceived(1);
      if (!pd_response_expected) {
	// This is a one way invocation, we choose to return a MessageError
	// Message instead of returning a Reply with System Exception
	// message because the other-end says do not send me a reply!
	SendMsgErrorMessage();
	ReplyCompleted();
      }
      else {
	MarshallSystemException(this,SysExceptRepoID::OBJECT_NOT_EXIST,ex);
      }
    }
    else {
      CHECK_AND_MAYBE_MARSHALL_SYSTEM_EXCEPTION (OBJECT_NOT_EXIST,ex);
    }      
  }
  catch (CORBA::UNKNOWN &ex) {
    CHECK_AND_MAYBE_MARSHALL_SYSTEM_EXCEPTION (UNKNOWN,ex);
  }
  catch (CORBA::BAD_PARAM &ex) {
    CHECK_AND_MAYBE_MARSHALL_SYSTEM_EXCEPTION (BAD_PARAM,ex);
  }
  catch (CORBA::NO_MEMORY &ex) {
    CHECK_AND_MAYBE_MARSHALL_SYSTEM_EXCEPTION (NO_MEMORY,ex);
  }
  catch (CORBA::IMP_LIMIT &ex) {
    CHECK_AND_MAYBE_MARSHALL_SYSTEM_EXCEPTION (IMP_LIMIT,ex);
  }
  catch (CORBA::COMM_FAILURE &ex) {
    setStrandDying();
    CHECK_AND_MAYBE_MARSHALL_SYSTEM_EXCEPTION (COMM_FAILURE,ex);
  }
  catch (CORBA::INV_OBJREF &ex) {
    CHECK_AND_MAYBE_MARSHALL_SYSTEM_EXCEPTION (INV_OBJREF,ex);
  }
  catch (CORBA::NO_PERMISSION &ex) {
    CHECK_AND_MAYBE_MARSHALL_SYSTEM_EXCEPTION (NO_PERMISSION,ex);
  }
  catch (CORBA::INTERNAL &ex) {
    CHECK_AND_MAYBE_MARSHALL_SYSTEM_EXCEPTION (INTERNAL,ex);
  }
  catch (CORBA::MARSHAL &ex) {
    CHECK_AND_MAYBE_MARSHALL_SYSTEM_EXCEPTION (MARSHAL,ex);
  }
  catch (CORBA::INITIALIZE &ex) {
    CHECK_AND_MAYBE_MARSHALL_SYSTEM_EXCEPTION (INITIALIZE,ex);
  }
  catch (CORBA::NO_IMPLEMENT &ex) {
    CHECK_AND_MAYBE_MARSHALL_SYSTEM_EXCEPTION (NO_IMPLEMENT,ex);
  }
  catch (CORBA::BAD_TYPECODE &ex) {
    CHECK_AND_MAYBE_MARSHALL_SYSTEM_EXCEPTION (BAD_TYPECODE,ex);
  }
  catch (CORBA::BAD_OPERATION &ex) {
    CHECK_AND_MAYBE_MARSHALL_SYSTEM_EXCEPTION (BAD_OPERATION,ex);
  }
  catch (CORBA::NO_RESOURCES &ex) {
    CHECK_AND_MAYBE_MARSHALL_SYSTEM_EXCEPTION (NO_RESOURCES,ex);
  }
  catch (CORBA::NO_RESPONSE &ex) {
    CHECK_AND_MAYBE_MARSHALL_SYSTEM_EXCEPTION (NO_RESPONSE,ex);
  }
  catch (CORBA::PERSIST_STORE &ex) {
    CHECK_AND_MAYBE_MARSHALL_SYSTEM_EXCEPTION (PERSIST_STORE,ex);
  }
  catch (CORBA::BAD_INV_ORDER &ex) {
    CHECK_AND_MAYBE_MARSHALL_SYSTEM_EXCEPTION (BAD_INV_ORDER,ex);
  }
  catch (CORBA::TRANSIENT &ex) {
    CHECK_AND_MAYBE_MARSHALL_SYSTEM_EXCEPTION (TRANSIENT,ex);
  }
  catch (CORBA::FREE_MEM &ex) {
    CHECK_AND_MAYBE_MARSHALL_SYSTEM_EXCEPTION (FREE_MEM,ex);
  }
  catch (CORBA::INV_IDENT &ex) {
    CHECK_AND_MAYBE_MARSHALL_SYSTEM_EXCEPTION (INV_IDENT,ex);
  }
  catch (CORBA::INV_FLAG &ex) {
    CHECK_AND_MAYBE_MARSHALL_SYSTEM_EXCEPTION (INV_FLAG,ex);
  }
  catch (CORBA::INTF_REPOS &ex) {
    CHECK_AND_MAYBE_MARSHALL_SYSTEM_EXCEPTION (INTF_REPOS,ex);
  }
  catch (CORBA::BAD_CONTEXT &ex) {
    CHECK_AND_MAYBE_MARSHALL_SYSTEM_EXCEPTION (BAD_CONTEXT,ex);
  }
  catch (CORBA::OBJ_ADAPTER &ex) {
    CHECK_AND_MAYBE_MARSHALL_SYSTEM_EXCEPTION (OBJ_ADAPTER,ex);
  }
  catch (CORBA::DATA_CONVERSION &ex) {
    CHECK_AND_MAYBE_MARSHALL_SYSTEM_EXCEPTION (DATA_CONVERSION,ex);
  }
  catch (...) {
    CORBA::UNKNOWN ex(0,CORBA::COMPLETED_MAYBE);
    CHECK_AND_MAYBE_MARSHALL_SYSTEM_EXCEPTION (UNKNOWN,ex);
  }
  if (obj)
    omni::objectRelease(obj);

  return;
}
#undef CHECK_AND_MAYBE_MARSHALL_SYSTEM_EXCEPTION

void
GIOP_S::HandleLocateRequest(CORBA::Boolean byteorder)
{
  CORBA::ULong msgsize;
  CORBA::ULong keysize;

  try {
    // This try block catches any exception that might raise during
    // the processing of the request header
    msgsize <<= *this;
    if (msgsize > MaxMessageSize()) {
      SendMsgErrorMessage();
      setStrandDying();
      throw CORBA::COMM_FAILURE(0,CORBA::COMPLETED_NO);
    }

    RdMessageSize(msgsize,byteorder);  // Set the size of the message body.

    pd_request_id <<= *this;
    keysize <<= *this;
    get_char_array((CORBA::Char *)&pd_objkey,keysize);
    RequestReceived();
  }
  catch (CORBA::MARSHAL &ex) {
    RequestReceived(1);
    SendMsgErrorMessage();
    pd_state = GIOP_S::Idle;
    return;
  }
  catch (CORBA::NO_MEMORY &ex) {
    RequestReceived(1);
    MarshallSystemException(this,SysExceptRepoID::NO_MEMORY,ex);
    return;
  }

  omniObject *obj = 0;
  GIOP::LocateStatusType status;

  try {
    if (omniORB::objectLoader == 0) {
      if (keysize != sizeof(omniObjectKey)) {
	throw CORBA::OBJECT_NOT_EXIST(0,CORBA::COMPLETED_NO);
      }
      obj = omni::locateObject(pd_objkey);
    }
    else {
      // The application has installed an object loader
      // If we cannot locate the object in our object table, we call
      // the object loader to ask for it.
      // If it throws OBJECT_NOT_EXIST if it cannot find it either.
      // The loader should be capable of dealing with concurrent upcalls.
      try {
	if (keysize == sizeof(omniObjectKey))
	  obj = omni::locateObject(pd_objkey);
      }
      catch (...) { }
      if (!obj) {
	// Be careful with the reference count!
	// The returned object should have the reference count increment
	// by 1, like it is done in omni::locateObject()
	obj = omniORB::objectLoader(pd_objkey,keysize)->PR_getobj();
      }
    }
    omni::objectRelease(obj);
    status = GIOP::OBJECT_HERE;
    // XXX what if the object is relocated. We should do GIOP::OBJECT_FORWARD
    // as well.
  }
  catch(...) {
    status = GIOP::UNKNOWN_OBJECT;
  }

  WrLock();
  pd_state = GIOP_S::ReplyIsBeingComposed;

  size_t bodysize = 8;
  WrMessageSize(0);
  put_char_array((CORBA::Char *)MessageHeader::LocateReply,
		 sizeof(MessageHeader::LocateReply));
  operator>>= ((CORBA::ULong)bodysize,*this);
  operator>>= (pd_request_id,*this);
  operator>>= ((CORBA::ULong)status,*this);

  flush();
  pd_state = GIOP_S::Idle;
  WrUnlock();

  return;
}


3. This is the hairy bit. You have to change the marshal object reference
   code so that it inserts the (variable length) persistent OID into the
   IOR instead of the fixed length internal OID generated by the ORB.
   The ORB code  assumes that if an object is not a proxy, its OID is
   always the fixed length OID, so be careful. If you are interested, I'll
   give it some thought as to how to do this properly.