Pivotal Knowledge Base

Follow

InternalGemFireError when using callbackArgument and enable-gateway

Environment

Pivotal GemFire 8.X

Symptom

Within a cluster configured to use WAN replication through the deprecated enable-gateway property in the cache.xml file, all overloaded methods of the Region class that receive a CallbackArgument fail with the exception below, and the entry is not replicated to the remote site.

com.gemstone.gemfire.InternalGemFireError
at com.gemstone.gemfire.internal.Assert.throwError(Assert.java:84)
at com.gemstone.gemfire.internal.Assert.assertTrue(Assert.java:48)
at com.gemstone.gemfire.internal.cache.WrappedCallbackArgument.setOriginalCallbackArgument(WrappedCallbackArgument.java:93)
at com.gemstone.gemfire.internal.cache.EntryEventImpl.setCallbackArgument(EntryEventImpl.java:971)
at com.gemstone.gemfire.internal.cache.wan.AbstractGatewaySender.distribute(AbstractGatewaySender.java:1012)
at com.gemstone.gemfire.internal.cache.LocalRegion.notifyGatewayHubs(LocalRegion.java:6501)
at com.gemstone.gemfire.internal.cache.BucketRegion.notifyGatewayHubs(BucketRegion.java:656)
at com.gemstone.gemfire.internal.cache.LocalRegion.basicPutPart2(LocalRegion.java:6021)
at com.gemstone.gemfire.internal.cache.BucketRegion.basicPutPart2(BucketRegion.java:641)
at com.gemstone.gemfire.internal.cache.AbstractRegionMap.basicPut(AbstractRegionMap.java:3014)
at com.gemstone.gemfire.internal.cache.BucketRegion.virtualPut(BucketRegion.java:500)
at com.gemstone.gemfire.internal.cache.PartitionedRegionDataStore.putLocally(PartitionedRegionDataStore.java:1300)
at com.gemstone.gemfire.internal.cache.PartitionedRegionDataStore.putLocally(PartitionedRegionDataStore.java:1274)
at com.gemstone.gemfire.internal.cache.PartitionedRegion.putInBucket(PartitionedRegion.java:2887)
at com.gemstone.gemfire.internal.cache.PartitionedRegion.virtualPut(PartitionedRegion.java:2086)
at com.gemstone.gemfire.internal.cache.LocalRegionDataView.putEntry(LocalRegionDataView.java:118)
at com.gemstone.gemfire.internal.cache.LocalRegion.basicUpdate(LocalRegion.java:5813)
at com.gemstone.gemfire.internal.cache.LocalRegion.basicBridgePut(LocalRegion.java:5445)
at com.gemstone.gemfire.internal.cache.tier.sockets.command.Put65.cmdExecute(Put65.java:397)
at com.gemstone.gemfire.internal.cache.tier.sockets.BaseCommand.execute(BaseCommand.java:181)
at com.gemstone.gemfire.internal.cache.tier.sockets.ServerConnection.doNormalMsg(ServerConnection.java:799)
at com.gemstone.gemfire.internal.cache.tier.sockets.ServerConnection.doOneMessage(ServerConnection.java:930)
at com.gemstone.gemfire.internal.cache.tier.sockets.ServerConnection.run(ServerConnection.java:1179)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at com.gemstone.gemfire.internal.cache.tier.sockets.AcceptorImpl$1$1.run(AcceptorImpl.java:555)
at java.lang.Thread.run(Thread.java:745)

Cause

The GemFire WAN architecture has evolved over the years and several properties and API interfaces have been deprecated along the way, replaced by newer ones which are easier to configure and use. With the goal of achieving backward compatibility as new versions are released, the older code hasn't been removed from the product and, thus, several checks have to be executed in runtime to decide which class must be instantiated according to the configuration options used.

The internal mechanism used to wrap an entry event into another type of event that can be understood by the gateway implementation, namely WrappedCallbackArgumentis not accurately executing the above check when the original event has an existing CallbackArgument already associated. This is the reason why the exception is thrown before the event is added to the dispatcher queue preventing the remote site from receiving the update.

Resolution

To prevent this, avoid the usage of the deprecated property enable-gateway, it has been totally removed in newer versions (GemFire 9.X), and replace it by the suggested configuration option for the relevant version (gateway-sender-ids).

As an example, if the cache.xml file currently looks like: 

<cache>
  <gateway-sender id="pri-to-sec" remote-distributed-system-id="2" parallel="true"/>
  <region name="test">
    <region-attributes data-policy="partition" enable-gateway="true" gateway-sender-ids="pri-to-sec"/>
  </region>
</cache>

It should be changed to:

<cache>
  <gateway-sender id="pri-to-sec" remote-distributed-system-id="2" parallel="true"/>
  <region name="test">
    <region-attributes data-policy="partition" gateway-sender-ids="pri-to-sec"/>
  </region>
</cache>

 

Comments

Powered by Zendesk