Using missing topic notifications with fan-out

Control client sessions can use missing topic notifications to monitor the activity of end-user client sessions. In response to subscription requests to missing topics, the control client session can choose to take an action, such as creating the missing topic.

For more information, see Handling subscriptions to missing topics.

How notification propagation works

A missing topic notification is propagated from a secondary server to a missing topic handler registered against a primary server if and only if all of the following conditions are met:

There is an active fan-out connection between the secondary server and the primary server.
The selector used for the subscription request to the secondary server intersects with one or more of the fan-out links to the primary server that are configured at the secondary server.
On the secondary server, there are no currently replicated topics that match both the fan-out link and selector used in the subscription request.
The primary server has no topics that match the selector used in the subscription request.
One or more missing topic handlers are registered against the primary server for a path that matches the selector. The following rules are used to select which missing topic handler receives the notification:
- If multiple handlers are registered for the branch, the handler for the most specific topic path is notified.
- If there is more than one handler for a path, the Diffusion™ server notifies a single handler.

Example flow

An architecture comprising a control client that connects to the primary server; a primary server with a topic tree containing topics A, A/B, and A/C; a secondary server that uses fan-out replication to replicate the a branch of the topic tree from the primary server; and an end-user client.

A control client connects to the primary server and registers a missing topic notification handler against the A branch of the topic tree.
A secondary server connects to the primary server and replicates the A branch of the topic tree.
On the secondary server the replicated part of the topic tree comprises the following topics: A, A/B and A/C.
An end-user client attempts to subscribe to A/D, which does not exist.
The topic A/D is in part of the topic tree that is matched by a fan-out link selector, so the secondary server propagates the missing topic notification to the primary server.
The topic A/D does not exist on the primary server, so the primary server sends the missing topic notification to the handler registered by the control client.

Missing topic notification handlers at both the primary and secondary servers

A single subscription can cause a missing topic notification to be sent to a handler registered against the secondary server as well as a handler registered against a primary server.

The decision about whether to notify the handlers registered against a primary server is based on the intersection of the selector used by the subscription with the selector used to configure the fan-out link. It is possible for a missing topic notification to be sent to the primary server, but not to local handlers because the selector matches other (non-replicated) topics hosted by the secondary.

In particularly complex configurations, multiple primary servers might receive the notification or there can be multiple tiers of fan-out connections.

Considerations when using missing topic notifications with fan-out

The intersection of the selector used by the subscription request with a selector used for a fan-out link is calculated based only on the path-prefix of each selector. Complex selectors that use regular expressions can produce false positive results or false negative results. We recommend that you do not use regular expressions in the selectors used to configure fan-out links.

Ensure that the principal that the secondary server uses to make the fan-out connection to the primary server has the SELECT_TOPIC permission for the path prefix of the selector that triggered the missing topic notification.

A current session must exist between the secondary server and the primary server to forward notifications. If there is no session or the session fails while the missing topic notification is in-flight, the secondary server logs a warning message and discards the notification.

The robustness of the session between the servers can be improved by configuring reconnection. Fan-out connections can have a large number of messages in flight. It might be necessary to tune the reconnection time-out and increase queue depth and recovery buffer sizes.