Binary topics

A topic that streams binary data as bytes and uses efficient binary deltas to stream only the data that changes between updates. Binary topics are stateful: each topic stores a binary value on the Diffusion™ server .

Why use a binary topic?

You can use a binary topic to transmit any arbitrary binary data without the overhead of encoding it to a string or the risk of the binary data being incorrectly escaped.

Binary topics can use binary deltas to send only the data that has changed when this is more efficient than sending the full value.

You can use a binary topic to transmit very large strings. This enables a client to use the binary delta capability to transmit only the changed parts of a string rather than the whole value. This reduces the amount of data transmitted over the network.

Binary formats, such as Google protocol buffers, can be streamed using a binary topic.

Properties of a binary topic

When you create a binary topic you can specify the following properties in the topic specification:

PUBLISH_VALUES_ONLY

By default, delta streaming is enabled. If this property is set to true, delta streaming is disabled and all values are published in full.

If there is little or no relationship between one value published to a topic and the next, delta streams will not reduce the amount of data transmitted. For such topics, it is better to set PUBLISH_VALUES_ONLY.

DONT_RETAIN_VALUE

If set to true, the latest value of the topic is not retained by the Diffusion server or the client that publishes it. New clients that subscribe do not receive an initial value. No value will be returned for fetch operations that select the topic.

For time series topics, if DONT_RETAIN_VALUE is set to true, time series events are still retained, but the latest value is not stored separately.

The DONT_RETAIN_VALUE property is useful for applications like a feed of news items, or for values that are only valid at the moment of publication. You can combine this with VALIDATE_VALUES.

Using DONT_RETAIN_VALUE reduces the topic memory footprint, but disables delta streaming. Using an update stream will send complete values to the server, instead of deltas. Disabling delta streaming is likely to increase the bandwidth used unless subsequent values are unrelated.

This property replaces the obsolete stateless topic type which was removed in Diffusion 6.2.

TIDY_ON_UNSUBSCRIBE

If set to true, when a session unsubscribes from the topic, any updates for the topic that are still queued for the session are removed.

There is a performance overhead to using this option as the client queue must be scanned to find topic updates to remove, however it may prove useful for preventing unwanted data being sent to sessions. This property is disabled by default.

VALIDATE_VALUES

If set to true, the topic rejects updates that would create invalid instances of the topic's data type.

If set to anything other than true, no validation is performed and all values are streamed to subscribing clients whether they are valid data or not.

Validation has a performance overhead and is disabled by default.

Note: If validation is disabled and the value provided is not valid, the client might produce errors or other unexpected behavior. The exact error varies depending on the client platform. To avoid this, use the client-side validation method provided by the Diffusion API.

Binary values are always valid so setting this property has no effect.

CONFLATION

Used to set a conflation policy for the topic. Conflation can merge or discard topic updates to reduce server memory and network data usage.

The supported values are:

off
conflate
unsubscribe
always

The default is "conflate".

Note: Time series topics do not allow a conflation policy of 'conflate' or 'always'. They default to 'off'. You can set a conflation policy of 'unsubscribe'.

See Using conflation for details of the policies.

OWNER

Used to set a security principal as the owner of the topic. The principal receives READ_TOPIC, MODIFY_TOPIC and UPDATE_TOPIC privileges.

The format of the property value is:

$Principal is "name"

where name is the name of the principal.

REMOVAL

Used to set a topic removal policy.

The property is an expression which defines the conditions which will trigger automatic removal of the topic or of a set of topics specified with an optional selector.

See Removing topics automatically for details of the expression format.

PERSISTENT

Used to disable persistence for a topic.

If set to "false", the topic will not be persisted, even if persistence is enabled for the server.

Considerations when using a binary topic

Data on binary topics contains no implicit information about its structure.

Data on binary topics cannot be viewed in the console.