Implementing the GatewayApplication interface

Prerequisites: Getting framework resources, before proceeding with this step.

GatewayApplication interface

To write a gateway application, the main class must implement the GatewayApplication interface. See the Javadoc for details of the interface.

This section uses the code snippets from CsvFileSourceApplication to explain the implementation of the GatewayApplication interface. The implementation of GatewayApplication in CsvFileSinkApplication is very similar.

Declaration of ApplicationDetails

The first step in the implementation of the GatewayApplication interface is to implement method getApplicationDetails.

final class CsvFileSourceApplication implements GatewayApplication {

    static final String POLLING_JSON_SOURCE = "POLLING_JSON_SOURCE";
    static final String STREAMING_JSON_SOURCE = "STREAMING_JSON_SOURCE";
    static final String APPLICATION_TYPE = "CSV_FILE_SOURCE";

    ...

    @Override
    public ApplicationDetails getApplicationDetails() {
        return newApplicationDetailsBuilder()
            .addServiceType(
                POLLING_JSON_SOURCE,
                ServiceMode.POLLING_SOURCE,
                "A polling source service which frequently polls the " +
                    "configured CSV file for any updates and publishes to " +
                    "Diffusion server",
                null)
            .addServiceType(
                STREAMING_JSON_SOURCE,
                ServiceMode.STREAMING_SOURCE,
                "A streaming source which listens to CSV file changes and " +
                    "publishes contents to Diffusion server.",
                null)
            .build(APPLICATION_TYPE, 1);
    }

Here, the getApplicationDetails method provides details about the application, which are, the service types it supports, application type and its configuration version. The ApplicationDetails object returned in this method is created using a builder which can be obtained using DiffusionGatewayFramework.newApplicationDetailsBuilder. The builder allows the application to specify other application details such as:

  • sharedConfig types - with addSharedConfigType method

  • schema of any global configuration that applies to the application as a whole - with globalSchema method.

An ApplicationDetails instance must define the following:

Service type

It is mandatory to define at least one service type. SharedConfig types are optional.

Application type

It is mandatory to define an application type, by which it is uniquely identified in the console.

For example, application type of a Kafka adapter could just be Kafka

Version

It is mandatory to define the version of the application defined configuration. This includes global configuration and configuration for sharedConfig and services.

This version information is used by the framework to perform necessary configuration upgrades, if an older configuration is used with newer version of Gateway application.

Declaration of service handlers

With the declaration of supported ServiceTypes, corresponding ServiceHandlers are declared using the add methods.

    @Override
    public PollingSourceHandler addPollingSource(
        ServiceDefinition serviceDefinition,
        Publisher publisher,
        StateHandler stateHandler) throws InvalidConfigurationException {

        final Map<String, Object> parameters =
            serviceDefinition.getParameters();

        final SourceConfig sourceConfig =
            sourceConfigValidator.validateAndGet(parameters);

        return new CsvPollingSourceHandler(
            sourceConfig.getFileName(),
            sourceConfig.getDiffusionTopicName(),
            publisher);
    }

    @Override
    public StreamingSourceHandler addStreamingSource(
        ServiceDefinition serviceDefinition,
        Publisher publisher,
        StateHandler stateHandler) throws InvalidConfigurationException {

        final Map<String, Object> parameters =
            serviceDefinition.getParameters();

        final SourceConfig sourceConfig =
            sourceConfigValidator.validateAndGet(parameters);

        return new CsvStreamingSourceHandler(
            sourceConfig.getFileName(),
            sourceConfig.getDiffusionTopicName(),
            stateHandler,
            publisher);
    }

Since the supported ServiceTypes in the example application are of modes POLLING_SOURCE and STREAMING_SOURCE, methods addPollingSource and addStreamingSource are implemented to return the instance of PollingSourceHandler and StreamingSourceHandler respectively. See here for details on source handler implementations.

For SINK and HYBRID modes of ServiceTypes, addSink and addHybrid methods should be implemented.

Each of these methods receives an instance of a ServiceDefinition and a StateHandler. Additionally, the source service and hybrid service methods are supplied with a Publisher. These instances can be injected during the construction of service handlers to be utilized as needed.

  • The ServiceDefinition contains configuration for the service. It contains the name of the service, the ServiceType of the service, a map of parameters (user provided service configuration) and optionally a SharedConfig instance, if required by the service type.

  • The ServiceType provides its name, mode, and the JSON schema which defines the format of the parameters. The parameters are validated by the framework against the schema. The application may perform additional validation.

  • The StateHandler is an object which the service handler may use to query the current status of the service, and to report status changes (due to errors) relating to the service. This is covered in more detail, in the Reporting service status section.

  • The Publisher provided to the Source service handler and the Hybrid service handler should be used to send updates to the framework, and onwards to the Diffusion server.

    • A polling source service should only send updates in the poll method, which is called by the framework when polling data to be published to Diffusion.

    • A hybrid service should only send updates after receiving them in update method.

  • The Publisher can also be used to register a missing topic notification handler, for a branch of the topic tree. Since `Publishers' are provided to source services, both; streaming and polling source services can register for missing topic notifications and handle them as required. See here for more details on how handlers work.

The framework calls upon methods to add ServiceHandlers in the GatewayApplication to provide a service handler for every service defined in the configuration. Hence, the instance returned by these add methods needs to be unique for each call.