Being able to call a server entity from another one (management calls)

[mathieucarbou]

From the TMC/TMS, we need to be able to make some management calls to some server entities.

A use case we have now is for example to configure which stats we want from which server entity.

At the moment, all stats are sent because we cannot make a management call to start collecting the stats we need.

Management calls are sent through the TMS Entity, and should arrive into the Ehcache server entity. So it means at some point an entity to entity communication.

Do we have that in place or it is something planned or to be planned in the future ?

[jd0-sag]

The plan is that IEntityMessenger would eventually be extended to support inter-entity communication if we had a good use-case to help us determine how to address the target from within the sender.

As things currently stand, the only way to do this is to make a client send the actual message.

We experimented with this, in the past, using a system where the recipient would register a factory to create the appropriate kind of EntityMessage for its sender. That factory could then be looked up, in IEntityMessenger by some kind of identifier agreed upon by the sender and receiver (which also means that they needed a way of distinguishing which target entity was which - since they could have multiple instances). This message would then be enqueued on that entity and replicated to the passives. Much like the other calls in IEntityMessenger, this call was asynchronous with no response to the caller.

This worked but was hard to follow since nobody explicitly knew about anything from another entity. We wondered if we might want entities to be able to register explicit factory interfaces (as @CommonComponent - directly known to the sender) and have target instances be directly resolvable. Since we had no use-case to guide that decision and the approach of "send from client" would already work and address this concern, the problem was shelved.

While we can re-open the discussion, it is a complicated issue, no matter how it is solved.

[mathieucarbou]

Adding reference to ehcache/ehcache3#1573

[mathieucarbou]

@jd0-sag : why are you saying "if we had a good use-case" ?

Currently, we have a good use case ;-)

We have an M&M features from TMC that is basically to be able to select which statistics should be collected. M&M does not have in its classpath "user" entities and cannot. It uses:

• TMS Entity (responsible to read the monitoring service)
• Management Entity (responsible to forward and receive management calls)

So if we extend what we have so far to do management calls to a targeted entity, then the management entity would need at some point to be able to communicate with the targeted entity, and also get the response from another way.

I just want to highlight that we currently have a strong use case for this, so it is not an hypothesis ;-) We currently cannot deliver this feature for november because we simply cannot, but is this there for client-side and should be there at some point for server-side also.

[jd0-sag]

The issue with use-case is understanding how you would do the resolve of the target entity, from the sending entity, on the server. Essentially, it is a question of what this should look like (most likely, you will just be moving this class path problem from the client to the server - don't sell the "from the client" possibility short).

So, there are a few matters which need to be resolved:

1. How would the look-up of the target entity be done? Is it type/name or does the target need to symbolically register that it can receive these messages?
2. How would you construct the message needed (you would need access to some kind code intimately connected to the receiver)?
3. What happens if the target is destroyed before you can send the message or before it is received (or do these create some sort of synthetic read-lock on the target, thus blocking client-originating life-cycle events)?

Additionally, if you wanted the target to be able to send a message back to you, it would need some way of knowing what message to send and to whom. We may be able to work that into the implementation, though (maybe there is some way the response from the active invoke could be wrapped as an invoke to the original sender).

[mathieucarbou]

For 1:

We know the entity type, name and consumerId.
I would be in favor of not retrieving the target entity but instead sending a message with a specific context that can be consumerId (or entity type/name pair).

For 2:

I don't get the problematic... All the classes/object we deal with are in the common classpath. So there won't be any classpath issues. Also, if you look at what we do for the management entity already, it is just a way to extend that: in the server entity, instead of using the client communicator to send a call to the client, we would just send a call to a consumerId with a message.

And the server entity would need also a way to send a call to the consumerId issuing the call, for the response.

For 3:

It is all async, exactly like current management call for client-side. We are using the client communicator.
If on TMS side we do not receive a responses after a certain amount of time, it times out (on tms).

All management calls are not synchronous: they should be async. It is the case already for client-side management calls (management calls that target clients).

Update 1:

If you need to see what we do for management calls targeting a client:

• sending a management call: https://goo.gl/xXtY46
• answering a management call: https://goo.gl/FgQxtP

we verify the target client, the origin client, we generate a unique id for the call, we also make some other checks and we are using the client communicator.

Basically, we could reproduce the exact same pattern with server entities and consumerId if we had a sort of Entity Communicator.

[jd0-sag]

ConsumerID is generally not visible to the entities, only the services, so we wouldn't want to introduce this at the entity level.

The main issue here is what the "message" is. It would need to be an EntityMessage of a type defined by the receiver. That means that either the sender needs to know about that type and construct it, directly (which is dangerous), or have access to some kind of factory exposed by the receiver. In either of these cases, the sender needs to have direct access to some receiver code (both logically but also in the sense of class path access).

If you wanted the receiver to be able to send a response, it would need access to the same kind of facility. Pretty quickly, everyone needs to know about everyone.

In our previous experiments, these problems were kept to a minimum through registering factories, but it made the API rather unwieldy to use (since every interaction required understanding of the levels of indirection involved and some top-level "name server" of sorts, to connect the dots).

Because of the degree of inter-entity knowledge and cross-talk required, this is largely why we suggest driving this from the client, where these ideas are already well-defined and well-tested, instead of just moving the problems to the server, where they become subject to other server-level complexities and isolation concerns.