Running the adapter
Prerequisites
-
Java installation (min Java 8)
-
A running Kafka cluster
-
A running Diffusion server (or server cluster)
Download the adapter bundle
Download this zip file that contains the adapter jar and other resources.
Run script
The adapter can be started with any of the following:
-
just bootstrap configurations to connect to diffusion
-
a configuration file
-
with a combination of both
Provided scripts can be used to run the application which runs the executable jar of the Kafka adapter.
Starting with bootstrap configuration only
To start the adapter with the minimum required bootstrap configuration:
-
Run the
kafka-adapter-bootstrap.sh
script in thebin
directory.
The Diffusion connection details can be updated as needed in the script. By default, it is set to use the connection details of a local Diffusion server.
./bin/kafka-adapter-bootstrap.sh
If the adapter is initiated with only the bootstrap configuration, it will establish a session with the Diffusion server. Nevertheless, to fully leverage its functionality, a service needs to be added for data consumption or publication. Therefore, after starting the adapter with the bootstrap configuration, use the Diffusion management console to add a service to commence data publication or consumption. |
Starting with a configuration file
To start the adapter with a configuration file:
-
Run the
kafka-adapter.sh
script in thebin
directory.
This script uses a sample configuration file, configuration.json
, which is located in the example
folder. The configuration file contains general Gateway Framework configuration along with configuration that is specific to the Kafka adapter - contained within the application
sections. Modify this file as needed to configure services and sharedConfigs.
./bin/kafka-adapter.sh
The JSON schema files for service types and sharedConfig type in schema
folder can be used to understand the configuration requirements in more detail.
For Windows, use the kafka-adapter.bat
script instead.
.\bin\kafka-adapter.bat
Run the docker image
To run the docker image:
-
Pull the docker image.
docker pull pushtechnology/gateway-kafka-adapter:2.1.0_RC1
-
Run the image as follows:
-
With minimal bootstrap configuration:
docker run \ -e gateway.diffusion.url=ws://90.247.210.62:8080 \ -e gateway.diffusion.principal=admin \ -e gateway.diffusion.password=password \ pushtechnology/gateway-kafka-adapter:2.1.0_RC1
-
With a configuration file:
docker run \ --mount type=bind,source=configuration.json,target=/opt/kafka-adapter/configuration.json \ -e gateway.config.file=/opt/kafka-adapter/configuration.json \ pushtechnology/gateway-kafka-adapter:2.1.0_RC1
-
Here an external config file is mounted in /opt/kafka-adapter/configuration.json path and is configured as a value for the gateway.config.file environment variable to set the configuration file for the java application.
|
The docker image allows setting the EXTRA_JAVA_OPTIONS
environment variable, which
can be set to specify any additional JVM option or override the pre-configured JVM
options for the docker container.
External libraries
Any external JAR files containing payload converters or custom publishers that are not part of the Kafka adapter can be added to the 'ext' directory available in the Kafka adapter bundle. The JARs in the 'ext' directory are configured to be loaded during startup by the scripts in the 'bin/' directory.