STREAM_FROM_REMOTE

The STREAM_FROM_REMOTE service type can be used to subscribe to Diffusion topics on the remote server, whose updates will then be published to the Diffusion topics in the local server. Thus, for this service type, the remote server is the source server and the local server is the target server. This service type also supports the on-demand publication feature, which can be enabled to publish to Diffusion topics in the local server only if there is a demand or a trigger.

There can be multiple instances of this service type added to the adapter to consume data from different Diffusion topics in the remote server and publish the updates to Diffusion topics in the local server using different configurations.

Complete configuration for this service consists of framework required configuration for streaming service and this service type specific configuration.

Example 1. Sample configuration for a service of type STREAM_FROM_REMOTE

    {
      "serviceName": "fxConsumerReplicated",
      "serviceType": "STREAM_FROM_REMOTE",
      "description": "Replicates topics from remote topic and publishes to local",
      "config": {
        "sharedConfigName": "remoteDiffusionServer",
        "framework": {
          "topicProperties": {
            "persistencePolicy": "SERVER",
            "timeSeries": true,
            "timeSeriesRetainedRange": "last 1H",
            "doNotRetainValue": false
          }
        },
        "application": {
          "topicMappingFunction": "fromRemote/replicated/<path(0)>",
          "diffusionTopicSelector": "?fx//",
          "updateMode": "STREAMING",
          "replicateTopicType": true
        }
      }
    }

Framework configuration

Details about supported configuration parameters for framework configuration for Streaming service can be found here.

Application configuration

The supported application configuration parameters for this service are defined in the table below:

STREAM_FROM_REMOTE service application configuration
Name	Type	Description	Mandatory	Default value
diffusionTopicSelector	String	The Diffusion topic selector to subscribe to on the remote server.	Only if `onDemandPublication` is not specified	N/A
topicMappingFunction	String	A function to map the Diffusion topic path on the remote server from which an update is received, to construct a topic path used to publish updates on the local server.	No	<path(0)>
updateMode	UpdateMode	Specifies the update mode to be used for topics published by the service. This could be: - `SIMPLE` mode: sends a full value to the server on every update. A series of updates are sent asynchronously to the Diffusion topic. - `SIMPLE_SYNC` mode: Similar to the `SIMPLE` mode, however, here the updates are sent synchronously. - `STREAMING` mode: takes advantage of the Diffusion `delta streaming` capabilities, such that only differences in the value are sent, thus reducing bandwidth usage. A series of deltas are sent asynchronously to the Diffusion topic.	No	STREAMING
replicateTopicType	boolean	A boolean value to specify whether to replicate the topic type of the source topic from the remote server to the local server. Setting this to false means that the default topic type or that specified in the framework configuration will be used.	No	true
removeStaleTopic	boolean	A boolean value to specify whether to remove the topic created in the local server, if the source topic in the remote server becomes unavailable.	No	true
onDemandPublication	onDemandPublicationConfig	Configuration to specify on-demand topic publication.	No	null

Apart from the above configuration parameters, it also requires the remote server connection details. These can be supplied as part of the service configuration or specified as shared configuration and referred to in the service configuration as illustrated above. See here for details on supported Diffusion server configuration parameters. Below is a sample configuration that contains the remote server connection details within the service configuration instead of specifying it in the shared configuration block:

    {
      "serviceName": "fxConsumerReplicated",
      "serviceType": "STREAM_FROM_REMOTE",
      "description": "Replicates topics from remote topic and publishes to local",
      "config": {
        "framework": {
          "topicProperties": {
            "persistencePolicy": "SERVER",
            "timeSeries": true,
            "timeSeriesRetainedRange": "last 1H",
            "doNotRetainValue": false
          }
        },
        "application": {
          "diffusion": {
            "url": "ws://localhost:7080",
            "principal": "admin",
            "password": "password",
            "reconnectIntervalMs": 5000
          },
          "topicMappingFunction": "fromRemote/replicated/<path(0)>",
          "diffusionTopicSelector": "?fx//",
          "updateMode": "STREAMING",
          "replicateTopicType": true
        }
      }
    }

Data subscription, conversion and publication to local server

In the STREAM_FROM_REMOTE service type of the Diffusion adapter, the Diffusion topic selector in the remote server, defined in the diffusionTopicSelector parameter within the application configuration, will be subscribed to. Any updates to subscribed Diffusion topics are received and converted as configured and published to a Diffusion topic in the local server, which is defined with topicMappingFunction. See here for more details on how the source topic path is mapped to a target topic path using the given mapping function.

By default, the topic type of the source topic will be replicated to the target topic. If the source topic is time series, time-series-specific properties which are timeSeriesRetainedRange and timeSeriesSubscriptionRange will also be replicated. However, if the configuration for the service also configures the target topic to be time series, time-series-specific properties specified in the configuration will be used instead of those in the source topic to create the target topic. This is configurable using the replicateTopicType configuration parameter. If it is set to false, then the topic type specified via the topicType or payloadConverter parameter in the framework configuration will be used to identify the topic type to be created, and configuration in topicProperties will be used for topic specification. If neither topicType nor payloadConverter parameters are defined explicitly, JSON topics will be created. See framework required configuration for default topic properties.

For a deeper understanding of how payload converters are used, please refer here. See here for the list of all issued payload converters by the framework.

Consequently, the simple configuration below publishes updates from fx/test Diffusion topic in the remote server to Diffusion topic fromRemote/converted/fx/test in the local server in JSON format, regardless of the fx/test topic type.

    {
      "serviceName": "fxConsumerReplicated",
      "serviceType": "STREAM_FROM_REMOTE",
      "description": "Replicates topics from remote topic and publishes to local",
      "config": {
        "sharedConfigName": "remoteDiffusionServer",
        "application": {
          "topicMappingFunction": "fromRemote/converted/<path(0)>",
          "diffusionTopicSelector": "?fx//",
          "replicateTopicType": false
        }
      }
    }

See Configuring the adapter for a complete example of the configuration for the adapter with configuration for STREAM_FROM_REMOTE services.

See Diffusion session management to understand how remote Diffusion sessions are created and used for the services of STREAM_FROM_REMOTE service type.