Skip navigation links

Package com.diffusiondata.gateway.framework

Diffusion Gateway Framework API.

See: Description

Package com.diffusiondata.gateway.framework Description

Diffusion Gateway Framework API.

This API provides the ability to write applications that interact with a Diffusion server and either consume data from back end 'sources' and publish to Diffusion, or consume data from Diffusion and feed to back end 'sinks'.

In addition, it provides the ability to write applications that consume Diffusion topics and feed data directly back to other Diffusion topics. This mode of operation is a useful way to transform Diffusion topic data. In this mode, the application may have no need to interact with any back end system.

The API provides a simple framework for creating such applications without the need for any detailed knowledge of Diffusion. Such applications may be 'adapters' providing integration between Diffusion and other diverse data systems.

To use the framework, an application must implement the GatewayApplication interface and also provide service handler implementations for services that the application is to provide.

A service is defined as a 'source' service, a 'sink' service, or a 'hybrid' service.

A 'source' service consumes data from some back end data source and publishes to Diffusion topics, either by streaming data updates from the back end or by polling the back end for updates.

A 'sink' service consumes streaming data from a Diffusion topic and publishes it to a back end 'sink'.

A 'hybrid' service is a combination of a 'sink' and 'source' service in that it both consumes data from Diffusion topics and can publish back to other Diffusion topics.

An application must declare itself to the framework using the DiffusionGatewayFramework.start method. In this method, the application and the framework will be initialized and started. During the application initialization, the framework will call back on the application via its getApplicationDetails method to obtain basic details about the application and the 'service types' that it supports. There can be many instances of any service type, each with different parameters (for example, to consume from and publish to different topics). Each service type should have a JSON schema that defines the application-specific parameters that define the instance of the service.

Applications may optionally make use of SharedConfigs. A sharedConfig may be used within the configuration to define common information about the service. For example, if many service types use the same connection details to the back end (for example, credentials), then they can use a common sharedConfig definition rather than providing the details separately for each service type. If a service uses a sharedConfig, then it will be provided by the framework when adding the service to the application. Applications that use sharedConfigs must define the 'sharedConfig types' that the application supports and also return them via the GatewayApplication.getApplicationDetails() method. Each configured sharedConfig needs to name the sharedConfig type that defines it. The framework then calls the GatewayApplication#initialize method passing ApplicationContext containing application configuration details and other resources, which the application can use to initialize any resources as required.

The framework will then validate the configuration and ensure it is compatible with the service types declared by the application.

After initialization of the application, connection to the Diffusion server (using bootstrap connection parameters) will be established and the application will be registered with the server. If the application has connected previously, then the saved configuration of the application will be retrieved from the server (although there is an option to override this configuration with a locally supplied configuration if required).

It then calls the start method to perform any initial startup processing defined in the application.

For each configured service, the framework will call the appropriate factory method on the application to add the service, and the application must return a suitable implementation of a ServiceHandler to handle the back end access for the service.

For example, if the configuration defines a service that is a 'streaming source' service, then the framework will call the addStreamingSource method. The application only needs to provide the factory methods for the service modes that it provides.

Each handler will be started by the framework using the start method. The handler should not process any back end interactions until this has been called and must stop processing when stop is called.

Applications are monitored and controlled from the Diffusion management console. The console may be used to add new services of the supported types, and one of the 'add' methods will be called on the application if this occurs.

Additionally, the console may remove or update services. In the case of removal, the framework will first call the ServiceHandler to stop the service (if it is running). In the case of updating the service, the framework will first remove the service as above and then add it again using the appropriate 'add' method.

Author:
DiffusionData Limited
Skip navigation links

Copyright © 2024 DiffusionData Limited. All rights reserved.