Just a second...

Configuring the CDC adapter

Use the configuration.json file or environment variables to configure how the adapter shares data between Diffusion™ and databases.

The adapter can be started with only bootstrap configuration, with only a configuration file or with a combination of both. Bootstrap configuration refers to the details required to connect to a Diffusion server, which are server URL, principal and password. These bootstrap configurations can be passed as system properties or environment variables, which will be used by default, if they are not set in the configuration file.

The allowed system/env properties are:
diffusion.gateway.server.url
diffusion.gateway.principal
diffusion.gateway.password
gateway.client.id

When the adapter is started, it registers itself with the server using its id (defined in the configuration file) and configuration. The configuration will be empty if a configuration file is not passed as an argument when starting the adapter.

The server persists this information in its local persistence file.

If a service is added/updated/removed, the configuration of the adapter changes during runtime and is updated in the server for the adapter ID. Hence, if the adapter is shut down and started again, there is no need to pass the configuration file again, as the adapter uses the persisted configuration from the server.

To override the configuration persisted in the server, you can pass a configuration file as an argument. The passed configuration will replace the configuration persisted in the server and will be used to set up expected services.

Mapping to Diffusion Topics

Each event (insert/update/delete/read) from the database is captured and published to the server.

The adapter supports four different ways to map these events to a Diffusion topic.

  1. Object : A table is mapped to a JSON topic, with each record being a JSON object keyed by the table's primary key. If a table does not have a primary key, updates for this table are ignored.
  2. Array : A table is mapped to a JSON topic, with all records as entries in an array. If a table does not have a primary key, updates for this table are ignored. For 'Create' event, new entries are added at the end of the JSON topic's value. For 'Delete' event, the corresponding item in the value of the JSON Diffusion topic is updated to null.
  3. Row : Each table row are mapped to individual JSON topic. The topic is created using database name, table name and primary keys. E.g: database/table/pk1,pk2. If a table does not have a primary key, and this setting is used, updates for this table are ignored.
  4. None : As per Row, but the topic contents are exactly as received from Debezium. This will include schema information, as well as the table row data before and after the change.
Note: If a table has composite primary key, the values of those keys will be escaped and concatenated together with ',' to formulate a complete primary key combination, which will be used in Object and Row topic mapping, as defined above.
The publisher service supports the following two operations, which can be invoked via the Diffusion console:
  1. Pause - will close Debezium engine for the service
  2. Resume - will restart Debezium engine for the service