Release Notes | Push Technology

6.4.10 (13 Dec 2021)

Bugs cleared at release 6.4.10

C Client: Segmentation fault in add_time_series_stream (27411)

Fixed a segmentation fault that occurred when calling add_time_series_stream with a NULL session.

JavaScript Client: JSON.jsonDiff(original) does not take arbitrary objects - as documented (26942)

Previously, JSON.jsonDiff(original) did not take arbitrary plain objects as documented. This has been fixed.

JavaScript Client: int64 docs reference overflowing number (27144)

The documentation of Int64 referenced a number that would cause an overflow. It has now been replaced with a number that fits into a signed 64-bit integer.

JavaScript Client: Sessions can reconnect even if explicitly closed by another session (27700)

An issue has been resolved where the server allowed clients to reconnect during the reconnection timeout, even if they had been explicitly closed by another session. This would only occur if session replication was enabled.

JavaScript Client: Connection timeout not configurable (27929)

The JavaScript client was missing an option to specify the connection timeout on establishing a session. This has been rectified.

Security: TOPIC_OWNER topic property not respected when a session changes its principal (26953)

When a session re-authenticates using the "change principal" API, the server re-evaluates its subscriptions. In previous releases, the re-evaluation failed to take account of the TOPIC_OWNER topic property. This bug has been fixed in this release. Now a session's subscriptions can change if it re-authenticates using a different principal so that it gains or loses ownership of a topic.

Security: Upgrade log4j2 to address CVE-2021-44228 security vulnerability (27991)

The log4j2 logging library used by Diffusion has been upgraded to version 2.15.0. This addresses a critical security bug [CVS-2021-44228] in log4j2. See https://logging.apache.org/log4j/2.x/security.html for details.

6.4.9 (11 May 2021)

Bugs cleared at release 6.4.9

Apple Client: HTTP Headers content checked with case-sensitive values (26469)

Addresses an issue where the Apple client fails to establish a session due to case-sensitive mismatches on HTTP headers. This has now been resolved.

Apple Client: Apple client cannot connect securely to Diffusion server via proxy (26473)

When the Apple client connected via a secure WebSocket to a Diffusion server via a proxy, the connection would be downgraded to a non-secure WebSocket. This has now been fixed.

JavaScript Client: Deregistration callbacks return Promises but this is not reflected in the TypeScript API or documentation (26551)

Deregistration callbacks in MissingTopicHandler and SessionPropertiesListener return a Promise that resolves when the listener has been deregistered. This is now reflected in the documentation and in the TypeScript API.

JavaScript Client: DataType.writeValue can accept null but this is not reflected in the TypeScript API (26554)

writeValue for primitive and JSON datatypes can accept null as a value. This is now reflected in the TypeScript API.

Topic Views: A throttled topic view can produce an invalid delta stream (26456)

Due to a bug in previous releases, it was possible for a topic view with a throttle clause to produce a corrupt delta stream. The bug has been fixed in this release.

6.4.8 (13 Apr 2021)

Bugs cleared at release 6.4.8

Apple Client: Inconsistent HTTP header format (26252)

The HTTP header format in the Apple client was inconsistent with other clients, causing certain network proxy tools like Proxyman and LittleProxy to fail to validate headers. This is now resolved.

Java & Android Client: Header parsing during proxy authentication is not case-insensitive (26361)

Checking headers such as Keep-Alive and Connection during proxy authentication is now case-insensitive.

JavaScript Client: Time series value stream event datatype compatibility problem (24780)

Value streams are compatible with parent types. For example, a ValueStream<JSON> can deserialize values from a String topic type. ValueStream<Event<JSON>> did not propagate the values for time series topics with the String event type. This has now been fixed.

JavaScript Client: RecordModel.get should allow obtaining a record by key only (24882)

The implementation of RecordModel.get could take a single key argument but the documentation and the Typescript API did not reflect this. This has now been corrected.

JavaScript Client: messes.setRequestStream and timeseries.append should take topic types and plain JS type constructors for datatypes (24885)

The implementation of setRequestStream allows topic types and plain JS type constructors for datatypes. This is now correctly reflected in the public TypeScript API .

JavaScript Client: First call of UpdateStream.set returns a TopicCreationResult (24889)

The first call to UpdateStream.set returns a TopicCreationResult. This was not reflected in the API. This has now been rectified in the TypeScript definitions and added to the documentation.

JavaScript Client: Cannot pass true to responder.respond() (26003)

The first call to UpdateStream.set returns a TopicCreationResult. This was not reflected in the API. This has now been rectified to the TypeScript definitions and added to the documentation.

JavaScript Client: QueryResult.merge throws exception: "Error: TypeError: latestByOriginalSequence[k].sequence.toNumber is not a function (26085)

Time Series QueryResult.merge() threw an exception when the two query results had events in common. Merge now works as expected.

JavaScript Client: Request-response message handler calling responder.reject throws unexpected exception and prevents further handling (26111)

Calling responder.reject in request-response messaging caused an unexpected exception and prevented further handling. This has been fixed.

JavaScript Client: JSON deltas fail with indefinite length arrays and objects (26302)

JSON deltas failed when the JSON was encoded using CBOR with indefinite length arrays and objects. Now JSON deltas work with definite and indefinite length encoded CBOR.

Replication: Cluster routing could fail to find a handler hosted by a peer (26411)

When a server belongs in a cluster, if it has no local handler for various types of event such as messages, missing topic notifications, and authentication requests, but a peer server does, the event will be forwarded to the peer for processing. Due to a bug in previous releases, some requests were not forwarded even though a peer with a suitable handler existed. The bug has been fixed in this release.

Server: Time series topic stored byte metric not reset on sequence gap (26116)

Time series topic metrics were not reset when a sequence gap was detected. This affected topic metrics for these topics. A sequence gap may occur when a clustered topic is updated while offline with file persistence activated.

Topic Views: Failure to process topic view if an expand clause has a path pointer that does not match the expanded structure (26115)

The topic view expand clause has an optional path pointer that can be used to extract an alternative part of the expanded value to use in the derived path in place of the natural key. In previous releases, processing of topic views with expand clauses could fail in arbitrary ways if the path pointer did not match the expanded value. The bug has been fixed in this release.

6.4.7 (18 Feb 2021)

Changes made to existing features at release 6.4.7

C Client: APR Library update (24869)

Following the GLIBC update to v2.32, APR and APR-Utils libraries have been updated to 1.70 and 1.6.1 respectively.

Server: Optimise subscription re-evaluation when a session's roles are changed (24804)

When a session's roles are changed using change principal or change roles API functions, it may gain or lose READ_TOPIC permission to topic paths. The server re-evaluates the topic selections for these paths, and the session will be subscribed to new topics for which it gains READ_TOPIC permission, and unsubscribed from topics for which it no longer has READ_TOPIC permission. This re-evaluation has been optimized in this release.

Bugs cleared at release 6.4.7

C Client: Selectors with descendant qualifiers not evaluated (24491)

Addresses an issue where selector strings, containing descendant qualifiers, were not evaluated in the C client. This has now been resolved.

JavaScript Client: Buffer implementation not supported outside of NodeJS (24512)

Previously, there was no access to the Buffer API when running the client in a browser context. The Buffer API is now available through diffusion.buffer.

Server: Sessions can reconnect even if explicitly closed by another session (24872)

The server allowed client sessions to reconnect during the reconnection timeout, even if they had been explicitly closed by another session. This would only occur if session replication was enabled.

Topics: Potential failure to process time series updates (24307)

In previous releases, a rapid sequence of updates to time series topics could fail to be processed correctly, leading to data corruption. The bug has been fixed in this release.

6.4.6 (5 Nov 2020)

Bugs cleared at release 6.4.6

Java & Android Client: Flow control can stall update streams under high load (23958)

In previous releases, a bug could cause a Java client using update streams to never recover from flow control. The bug has been fixed in this release.

JavaScript Client: Cannot use numbers/int64/float in JSON constraints (24085)

Previously, int64 data was serialized inconsistently and for 32 bit integers and wrongly for negative 64 bit integers. This caused partial JSON constraints to fail for integer values. It also prevented negative 64 bit integers from being passed between clients. Integers are now serialized correctly and in line with other clients. Partial JSON constraints can be used with the Int64 datatype.

Security: Failures to process locked security resources (24069)

The API allows security roles and system authentication principals to be locked by a particular principal. Due to bugs in previous releases, processing of locked resources could fail in two ways: 1. A server could not boot against a Security.store or SystemAuthentication.store configuration file containing one or more lines that modify a locked resource. The workaround was to remove those lines. 2. In a cluster, a modification to a locked resource was not properly replicated. Instead, an error message would be logged by all servers receiving the replicated update. Both bugs have been fixed in this release. Locked security roles and system authentication principals now function as designed.

Topic Views: Multiple sessions creating the same topic view causes topic view re-evaluation (23682)

Creating an identical topic view (with the same name and specification) using the same principal (and the same roles) would cause the view to be replaced unnecessarily, leading to the removal and recreation of all reference topics derived from it. This has now been resolved.

Topic Views: Deadlock due to TopicViewEvaluationImpl.reevaluateViewForAdds conflicting with update (23694)

A deadlock could occur in the server when a re-evaluation of a topic view conflicted with a concurrent update, leading to clients being unable to connect. This has now been resolved.

Topic Views: Topic view publishes new value to old topic before removing it if scalar mapping is used in a simple (non-derived) view mapping (24055)

A topic view specification that contained a scalar path mapping but no other JSON type value mappings could result in a new value on the source topic being published to the reference topic before the removal of the reference topic due to a change to the indicated scalar value. This problem has now been resolved.

6.4.5 (11 Aug 2020)

Bugs cleared at release 6.4.5

Apple Client: NSInvalidArgumentException: (notifier) is nil. (22698)

"NSInvalidArgumentException (notifier) is nil" has been removed. When Diffusion fails to generate a notifier for the dispatcher due to an invalid stream, a warning is now logged instead.

Console: mac_addresses license field causes dashboard failure in console (22674)

The Diffusion management console could fail to display some status information when running under a license with MAC address constraints. This is no longer the case.

JavaScript Client: Wrong interpretation of connection options where the credentials field is set to an empty string (22852)

Previously, when an empty string was provided for connection credentials, it was wrongly interpreted as "no credentials provided". Now an empty string is sent to the server, allowing named principals to connect with an empty credentials string.

JavaScript Client: remoteTopics.js example - undefined variable 'datatypes' (23367)

The removeTopics.js example for the JS client has undefined variable 'datatypes'. This has been fixed.

Replication: Insufficient permission for <none> to restore topic view (22713)

An error of the form "Insufficient permission for <none> to restore topic view" could be logged if repartitioning occurred at the same time as a topic view was being created or removed. This could result in a topic view being lost from the cluster. This problem has now been resolved.

Server: Fetch response hasMore should not be affected by limitDeepBranches (22658)

In previous releases, when the limitDeepBranches() option was used, the fetch request API could erroneously return a result with hasMore=true, indicating the fetch was constrained by a first(), last(), or size() request when there were no more results. The bug has been fixed in this release.

Server: Subscription operations occasionally delayed (23139)

Due to a bug in previous releases, the result of a subscription operation would occasionally be delayed. Further, if the server was under heavy load, the bug could cause the failure of a multiplexer thread, requiring the server to be restarted. The bug has been fixed in this release.

Server: PUSH-000798|Unexpected multiplexer recursion - TopicRemovalDownstreamChecksImpl (23264)

A "PUSH-000798 Unexpected multiplexer recursion" error could be logged to the server when automatic topic removal was in use. The recursion occurred in TopicRemovalDownstreamChecksImpl. Though this was logged as an error, it is not known to have caused problems with the actual processing of topic removals. This problem has now been resolved.

Topic Views: Automatic topic removal based upon subscriptions can occur even if a reference topic that depends upon the topic has subscribers (22326)

Subscriptions to reference topics derived from source topics were not being counted towards automatic topic removal evaluation. This could result in a topic being removed even though there were still subscriptions to reference topics derived from it. This problem has now been resolved.

6.4.4 (7 Jun 2020)

Bugs cleared at release 6.4.4

Server: Some requests sent by the server are not correctly guarded by timeouts (22544)

Due to a bug, in previous releases the server didn't apply timeouts for some outbound requests to control sessions. Consequently, some failures were not retried and/or reported correctly. The bug has been fixed in this release.

Server: Server will not start if JMX management not enabled and server not part of a cluster (22639)

A bug introduced in Diffusion 6.4.3 prevented the server starting if both JMX management was not configured and the server was not configured as part of a cluster. The bug has been fixed in this release.

Topic Views: NullPointerException when creating a topic while concurrently removing a topic view (22519)

Due to a bug, in previous releases an add topic operation could fail if another session concurrently removed a topic view. A NullPointerException would be logged to the server log. The bug has been fixed in this release.

6.4.3 (18 May 2020)

Bugs cleared at release 6.4.3

.NET Client: Null reference exception in AuthenticationResponse (22381)

AuthenticationResponse will no longer throw a null reference exception under certain circumstances.

Apple Client: NSInternalInconsistencyException from PTWebSocketConnection - Frame reader not allocated (22404)

PTWebSocketConnection no longer raises NSInternalInconsistencyException "frame reader not allocated" when the frame reader is unallocated. The connection is now closed when the client receives a message that contains errors in the header and a more informative error is passed.

Apple Client: NSInternalInconsistencyException when number of bytes written is 0 (22405)

NSInternalInconsistencyException "write stream has fixed length" occurred when an error in the connection caused the total number of bytes written to be 0. A clearer error is now returned.

Apple Client: NSInternalInconsistencyException from PTDiffusionTopicUnsubscriptionReasonToString (22406)

NSInternalInconsistencyException "no delegate" is no longer returned when delegate was not present. This has been replaced with a more informative error.

C Client: Default C examples do not compile (21858)

Addresses an issue where the C client examples were not compilable due to use of deprecated functions. This has now been resolved.

Client: setSessionProperties allows fixed properties to be changed (22500)

A bug in the server side validation of the setSessionProperties service meant that it was possible to change the values of fixed session properties. This has now been resolved and changes to fixed session properties will be ignored.

Console: While adding a role, adding then removing a path permission closes dialog box (22146)

In the Diffusion management console, removing a topic permission while adding or editing a role would result in the role dialog closing. This has been resolved.

Console: Roles only referenced by principals not listed in roles table (22147)

The Diffusion management console would not allow configuration or selection of roles which were only referenced by principals in the system authentication configuration. These roles are now displayed with a note and are configurable through the console.

Console: Topic views tab in console cannot cope with very large numbers of items (22193)

The Diffusion management console now performs better on servers with many topic views.

Console: Security tab missing functionality with named principals granted admin roles (22542)

When viewing a named principal with admin permissions in the console Security tab, the roles list and the controls to modify and remove principals were not displayed. This issue has now been resolved.

Demos: Sportsbook demo fails to deploy when located in a directory path with a space in its name (22086)

Addresses an issue where the sportsbook demo failed to deploy if the working directory path contained a whitespace. This has now been resolved

Demos: Sportsbook demo displays inconsistent data across clients (22115)

Addresses an issue where different odds were shown to different clients viewing the sportsbook demo. This has now been resolved.

Java & Android Client: Diffusion threads should be daemon threads (22407)

In previous releases, Diffusion managed threads were not set to be daemon threads. This has been corrected in this release. Now a client application that creates a Diffusion session will not be kept running unnecessarily after all non-Diffusion threads have terminated.

Replication: "Unsupported service nnn requested by peer Session" warnings (22518)

Warnings of the form "Unsupported service nnn requested by peer Session:" can be reported in logs when using replication. These warnings may indicate that certain components (such as security files, or topic views) have not been replicated correctly. This problem has now been resolved.

Server: Topic view evaluation cannot handle JSON values that cannot be parsed : IllegalArgumentException (22122)

If an invalid value was passed to a JSON topic that was the source of a topic view (and the VALIDATE_VALUES property was not used), it was possible that a terminal IllegalArgumentException could occur in the server during topic view evaluation leading to unpredictable results. This issue has now been resolved. If an invalid value is encountered an error will be logged to the server and the value will be treated as if it was null (CBOR null).

Topic Views: Topic views that create reference topics with PUBLISH_VALUES_ONLY set to true still publish deltas (22447)

Due to a bug in previous releases, topics created by a topic view that set PUBLISH_VALUES_ONLY to true could still publish deltas. The bug has been corrected in this release.

Topics: Routing subscriptions not allowed to topics created by topic views (22408)

Due to a bug in previous releases, a routing subscription could not be made to a target topic created by a topic view. The bug has been resolved in this release. The target topic must exist at the time the routing is evaluated – if nothing is there, the routing subscription will fail.

Topics: TopicNotificationSelectionService fails if invalid topic selector supplied (22409)

The topic notification services now correctly return a communication error when using an invalid topic selector to select or deselect topics. This applies to all clients.

6.4.2 (27 Feb 2020)

Changes made to existing features at release 6.4.2

C Client: The legacy remove topics API is deprecated (21945)

The remove_topics() function is deprecated in this release. It will be removed from Diffusion 6.5. The C client provides two functions for removing topics: topic_removal(), which was added in Diffusion 6.0, and the older remove_topics() function. Both are defined in topic-control.h. remove_topics() ought to have been deprecated in Diffusion 5.9 and removed in Diffusion 6.0. It is less powerful than topic_removal(), and does not correctly remove replicated topics.

Installation: The Diffusion docker image now uses Java 11 (21434)

The Diffusion docker image published to DockerHub is now based on an AdoptOpenJDK distribution of Java 11.

Bugs cleared at release 6.4.2

.NET Client: Send request to filter does not return number of requests sent until all responses received (21904)

An issue has been fixed where send request to filter calls waited for all responses to arrive before returning the number of requests sent. This behavior has been amended to return the number of requests immediately.

Apple Client: Send request to filter does not return number of requests sent until all responses received (21903)

An issue has been fixed where send request to filter calls waited for all responses to arrive before returning the number of requests sent. This behaviour has been amended to return the number of requests immediately.

C Client: Send request to filter does not return number of requests sent until all responses received (21905)

Console: TypeError from console logs tab (21730)

In the Diffusion management console, skipping backward or forwards while following the server log could sometimes fail to work and cause errors in the browser console. This has been fixed.

Java & Android Client: Changes to session properties in Authenticator do not occur if applied to the input map of properties (21743)

When using an Authenticator (ControlAuthenticator) for authentication, if changes are applied to the input map of session properties which is then passed to the Callback.allow(sessionProperties) method, the changes will not take place, even though the javadoc suggests that this will work. This issue has now been resolved so that changes can be applied to the input map of properties and passed directly to the 'allow' method.

Java & Android Client: Send request to filter does not return number of requests sent until all responses received (21889)

Java & Android Client: Fetch examples in API documentation incorrect (22016)

The fetchRequest() examples given in the Java and Android API documentation have been corrected.

JavaScript Client: Errors not logged by default (21780)

By default, errors and warning are now being logged.

JavaScript Client: Javascript client maintains reference to closed stream (21826)

The JavaScript client sometimes held on to references to closed value streams, leading to a memory leak. This has now been fixed.

JavaScript Client: Send request to filter does not return number of requests sent until all responses received (21902)

JavaScript Client: 'No such conversation' errors (21934)

When the server responded to a conversation closed by the client, the resulting error could propagate all the way to the top level causing user code to fail. This has been resolved.

Replication: Evaluation of missing topic notifications propagated from fanout secondary is suboptimal if primary is in a cluster (21828)

Diffusion 6.4.0 added clustering support for missing topic notifications. This did not support an optimization to the feature where the server skipped evaluation of the selector if there were no missing topic notification handlers. This optimization has now been expanded to clustered setups, increasing performance in some configurations.

Server: Subscription operation completes before notifications of immediately resolved subscriptions and associated values (21817)

When a session subscribes using a topic selector, it will be delivered a subscription notification and initial value (if available) for each existing topic to which the session becomes subscribed. The notifications and values occur before the returned subscription operation itself completes. Due to a bug in previous releases, if the server was configured to use topic replication or the selector overlapped with a fanout link but matched no fanout topics, this ordering guarantee was not satisfied. A further consequence of this bug is that the fanout-ready connector start condition added in 6.4 did not function correctly if the primary server was configured for topic replication. The bug has been fixed in this release. The subscription operation will consistently complete after notifications and values have been delivered delivered to the session; and fanout-start conditions now work as designed.

Server: The server can stall during evaluation of automatic topic removal rules (PUSH-000798 - Unexpected multiplexer recursion) (21818)

A bug introduced in Diffusion 6.4.0 could cause a heavily loaded server to fail with a PUSH-000798 / Unexpected multiplexer recursion error when evaluating automatic topic removal rules. The bug has been rectified in this release.

Topic Views: Reference topics created by topic views can be removed using the client API (21536)

In previous releases, the server allowed reference topics created by a topic view to be removed using the cilent API. Although the reference topics would immediately be re-instated by the topic view, subscribers would receive an unnecessary [ unsubscribe, subscribe, value ] sequence for each removed topic. This problem has been fixed. The server now prevents reference topics from being removed using the API.

Topic Views: Slow server start with a large number of topic views (21709)

Very large numbers of topic views (tens of thousands) could cause the server to take an unusually long time to start (over a minute). This has now been significantly improved.

Topic Views: Topic view source selectors that are integers must be quoted (21710)

In previous releases, topic view expressions with source selectors that were integers would not be accepted unless they were quoted, escaped, or prefixed by a >. For example "map 1 to x" was rejected, but the equivalent expressions "map '1' to x", "map \1 to x", and "map >1 to x" were all accepted. Similar restrictions applied to source selectors that were keywords used in other parts of the expression (for example, "hour", "minute", "second"). These restrictions have been removed in this release. There is no need to quote source selectors in topic view expressions unless they contain white space.

Topics: The server can stall after closing a session with a routing subscription then removing the source topic (21687)

In previous releases, the server's record of subscriptions was not correctly updated when a session with routing subscriptions was closed. Removing the source topic of one of the routing subscriptions could then lead to a multiplexer thread terminating (reported as a PUSH-000335 error in the log, followed by repeated PUSH-000773 warnings). The bug has been fixed in this release.

6.4.1 (9 Jan 2020)

Bugs cleared at release 6.4.1

Console: Topic view topic selector assistant: HTML escaping of topic hints is incorrect (21564)

In 6.4.0, when adding or editing a topic view, the explanatory hints in the topic selector assistant form were incorrectly escaped more than once, leading to garbled text. This is corrected in this release; the hints are now displayed correctly.

Console: Load average graph flatlines by default (21593)

The Load Average graph shown by default in the Diffusion management console was misconfigured and constantly showed 0 load. The default configuration has been fixed. You may need to restore the default layout in order for the fix to be applied.

Console: Fetching an only-child leaf topic causes infinite fetch loop (21626)

Viewing certain topic tree structures caused the Diffusion console to repeatedly fetch data from the server. This resulted in an increase in overall server load, more noticeably when using topic views with value mapping. This issue has now been fixed.

Topic Views: Topic views expand directive causes excessive topic churn when JSON object keys inserted or removed (21577)

The initial release of topic views expansion processed objects in key order. Insertions or removals of keys within an object could cause topics to be created and deleted unnecessarily (topic churn). Expansion now considers keys and derived paths to minimize topic churn.

Topics: Errors in topic REMOVAL property ignored (21559)

Some errors in a topic REMOVAL property were ignored by the Diffusion server instead of being reported. For example, if the property was set to "when time after 'Thu, 05 Dec 2019 12:45:20 GMT' and subscriptionsZ < 1 for 10s'" the server would simply ignore the second clause (which is invalid due to using "subscriptionsZ") instead of reporting an error. This has now been fixed.

6.4.0 (5 Dec 2019)

Features introduced at release 6.4.0

Client: Partial updates for JSON topics (17945)

The new partial update feature can be used to apply a JSON Patch (RFC 6902) to a JSON topic. This means that the updating client does not need to provide the full value. The JSON Patch standard supports advanced operations such as moving and copying key/value pairs, as well as testing a value before applying the patch. The dispatch of partial updates is supported by all the client SDKs.

Client: Fetch requests can limit the results returned from topic tree branches with long paths (21149)

A fetch option has been added that limits the number of topic results returned for each branch having paths with greater than a certain number of parts. See FetchRequest.limitDeepBranches().

Console: Topic view editing assistance (20549)

Extensive editing assistance has been added to the console's topic views tab. This includes syntactic highlighting, contextual help, and context-specific "assistant" dialogs for the various parts of a topic view specification.

Federation: Fan-out support for the topic REMOVAL property (18706)

In previous releases, automatic topic removal did not take into account sessions and subscribers at secondary fan-out servers, and was therefore of limited use in fan-out configurations. From this release, all sessions and subscriptions at secondary fan-out servers are taken into account when evaluating topic removal conditions.

Federation: Fan-out readiness start condition (21250)

In this release, each connector in etc/Connectors.xml can be configured with a list of conditions that must be satisfied before the connector is started. A single type of condition is supported, "fanout-ready", which is satisfied when a named fan-out link has connected to its primary server and initially synchronized all matching topics from the primary server. This can be used to delay opening of the connector port until there is information to serve to sessions.

Installation: Improved installer (6492)

The installer has been rewritten for this release. The new installer is based on the third party izpack framework. The installation options have been simplified. In previous releases, both the installer and a separate product distribution jar file were required. In this release, there is a single installer file that should be run using Java, For example: java -jar diffusion-installer-6.4.0.jar

JavaScript Client: Session#closeSession method that returns a Promise (19240)

Added closeSession method to Session that returns a Result. The result resolves once the session has been closed.

JavaScript Client: Support for HTTP proxies on Node.js (19620)

Diffusion.connect now accepts a httpProxyAgent option when run on Node.js. This allows the client to establish connections through a proxy server.

Server: Officially support Java 11, and OpenJDK derived distributions (20542)

From this release, the Diffusion server and Java client SDK are supported on a wider range of Java implementations. Specifically any Java Virtual Machine (JVM) that meets the following requirements: 1. The JVM either passes the Java TCK or is an official AdoptOpenJDK build; 2. the JVM version is 8u131-b11 (GA) or a later minor GA release of JDK 8, or 11.0.3 (GA) or a later minor GA release of JDK 11; 3. the JVM is HotSpot based; 4. the target platform is Linux, macOS, or Windows.

Server: The COMPRESSION topic property provides control over the compression level (21475)

From this release, the COMPRESSION topic property enables different compression levels to be set to trade off the CPU cost against the compression level achieved. The property now accepts the new values "off", "low", "medium", and "high", in addition to the previous values "true" (equivalent to "medium") and "false" (equivalent to "off"). The default compression level has been reduced to "low".

Topic Views: Topic views: new 'expand' directive (20655)

At this release, a major new capability has been added to topic views. There is a new 'expand' directive which allows the content of a JSON topic to be expanded into many different reference topics. For example, each element of an array from a source topic can be expanded to produce one reference topic per array entry, each being updated automatically by updates to the source topic data. See the documentation for full details of this feature.

Topics: Topic delivery priority (15924)

A PRIORITY topic property has been added that can be used to set the delivery priority of topic updates. The supported values are "high", "default" and "low". The delivery priority affects the order of topic updates sent to a subscribed client session. When there are multiple topic updates for topics with different priorities in a session's outbound queue, updates for high priority topics will be delivered first, followed by updates for default priority topics, followed by updates for low priority topics. Topic subscription/unsubscription notifications are also delivered according to the topic delivery priority. Using different delivery priorities is most beneficial when there is a large backlog of queued updates to deliver to a client session. On lightly loaded systems, updates typically remain in the outbound queue for a few milliseconds and so there is a lower chance of topic updates being reordered based on their priority. The backlog will be larger if the topic update rate is higher; the server or the client are more heavily loaded; the client session becomes temporarily disconnected; or if there is poor network connectivity between the server and the client.

Changes made to existing features at release 6.4.0

.NET Client: NewTopicSpecification convenience method added to Diffusion factory (19688)

A NewTopicSpecification factory method has been added.

.NET Client: Deprecated topic-selector interfaces, builders, and methods (20372)

Topic selectors and the methods that take them as arguments are now deprecated. Methods that accept a topic selector expression as string should be used instead.

.NET Client: Topics.GetTopicDetails and TopicAddFailReason.USER_CODE_ERROR removed (20699)

The deprecated ITopics.GetTopicDetails methods have now been removed along with all ITopicDetails interfaces (which were only retained to support these methods). The TopicAddFailReason.USER_CODE_ERROR has also been removed.

Adapters: The JMS adapter is now only supported as a standalone client (20872)

In preparation for the removal of the Publisher API (deprecated in this release), the server edition of the JMS adapter has been removed from the product.

Android Client: Support for Android API 29 (21532)

From this release, the Android client is additionally compatible with Android API 29.

Apple Client: WebSocket channels support for compress encoding (13541)

The Apple client now supports the transfer of compressed messages over WebSocket connections.

Apple Client: PTDiffusionFetchRequest.first/last accept 0 as a value (20433)

PTDiffusionFetchRequest.first and PTDiffusionFetchRequest.last can now accept a 0 value. Fetching zero topics which match a selector can be used to detect the existence of matching topics by using PTDiffusionFetchResult.hasMore.

C Client: System principals hash map now has SYSTEM_PRINCIPAL_T as its entry value (20903)

The system principals hash map (from a SYSTEM_AUTHENTICATION_STORE_T) now has SYSTEM_PRINCIPAL_T pointer values as opposed to previously SET_T.

Client: MissingTopicNotification.cancel deprecated (20950)

The MissingTopicNotification.cancel method has been deprecated at this release. This method is only useful when using the deprecated legacy fetch mechanism and will be removed when that mechanism is removed.

Console: Server configuration for console-exposed log files location (19611)

The log file monitored in the Diffusion management console can now be changed using the new 'console-monitored-log' setting in the Diffusion logging configuration. This makes it easier to use the console with custom logging configurations.

Console: Enable/disable functionality based on permissions (20449)

Some areas of the console, including the Security view, Topic Views and Metrics Collectors tabs, are now available in read-only mode to users with the appropriate permissions. Users should now only be presented with actions that they are permitted to attempt (topic tree permissions notwithstanding).

Console: Console topics tab improvements (20633)

The topics tab in the Diffusion web console has received a number of new features and usability improvements. You can now update topic values directly from the console. JSON and binary topic values are formatted for readability. The form to add a topic now informs you if the target path is already in use and allows for multiple topics. The console now asks for confirmation when subscribing to or deleting a topic. Many minor visual issues have also been corrected.

Console: Console topic browser performance significantly improved with large topic trees (20729)

Previously, using the console's topic browser to explore large topic trees could cause the console to disconnect or become unresponsive. Following improvements in 6.4, the topic tree browser should consistently perform well with topic trees of any size.

Console: Console display of locked roles and principals (20861)

The Security tab in the console now displays information on locked roles and principals, and will prevent users without the lock from attempting to edit them.

Console: "Add and continue" button to create multiple topics quickly in the console (21166)

When creating topics using the Diffusion web console, you no longer have to reopen the topic adder to create more than one topic.

Environment: docker-diffusion: diffusion home no longer depends on version (20761)

The docker-diffusion build no longer uses a version-specific path for the Diffusion home directory. Diffusion is installed to /opt/diffusion in the Docker container's filesystem.

Installation: The ec2 tools examples have been removed (21379)

Diffusion now includes systemd unit files. Upstart init scripts have been removed.

Java & Android Client: Java client now supports Server Name Indication extension to SSL/TLS (20214)

The Java client now supports the SNI extension for SSL/TLS handshakes. This is always enabled, and uses the host name that the client is connecting to.

Java & Android Client: Topics.getTopicDetails and TopicAddFailReason.USER_CODE_ERROR removed (20514)

The deprecated Topics.getTopicDetails methods have now been removed along with all TopicDetails interfaces (which were only retained to support these methods). The TopicAddFailReason.USER_CODE_ERROR has also been removed.

Java & Android Client: Consistent behaviour of CompletableFuture exceptions (20616)

In previous releases, some methods that returned CompletableFuture could complete exceptionally with an ExecutionException or an API specific exception, rather than with a CompletionException. Also, the javadoc for some older releases indicated that methods could complete exceptionally with an ExecutionException when they would actually complete exceptionally with a CompletionException. These inconsistencies have now been resolved. All such methods will now complete exceptionally with a CompletionException as now indicated by the javadoc.

JavaScript Client: Promise replaces Result (18441)

The Result type is now a type alias for Promise.

JavaScript Client: Topic selectors are no longer pre-parsed before they are dispatched to the server (20371)

For requests such as Session.select(), topic selectors are no longer pre-parsed on the client side before they are dispatched to the server. This means that, for these requests, regular expressions are not limited by the JavaScript regular expression engine.

JavaScript Client: Improved error when failing to connect because of an incompatible protocol level (20747)

Two new CloseReason enum values have been added. PROTOCOL_VERSION_MISMATCH indicates that the sever responded to a connection request with a protocol version number different to the client's protocol version. LICENSE_EXCEEDED indicates that a connection was refused because the license limit has been reached.

Logging: Log4j is now the default logging implementation (20544)

The default logging implementation is now Log4j 2. The Diffusion logging implementation (which was the default system in previous versions) is deprecated, together with most of the configuration settings in etc/Logs.xml, and will be removed in a future release.

Product Release: The docker-diffusion image is now derived from AdoptOpenJDK (20150)

The docker-diffusion image published to Docker Hub is now based on the AdoptOpenJDK OpenJDK 8 distribution. It has been updated to jdk8u232-b09.

Publisher API: Publisher API Deprecated (20553)

The Publisher API is deprecated at this release and will be removed in a future release. All aspects of topic and client handling can be performed using the Diffusion Client API. Some remaining aspects of the server-side API, such as configuration and HTTP request handling, will be retained and will in future be known as the 'Server API'.

Replication: Routing of authentication handler requests across Diffusion server clusters (20547)

Previous releases needed an authentication handler connected to every server in a cluster. At this release, authentication requests will be routed across the cluster to a server that has a suitably named handler. Handlers can be connected to more than one server for greater efficiency, but only one is required. It is the responsibility of the application to ensure that all handlers with the same name have the same behavior. Also, at this release, it is a requirement for on-premise users that the configuration of authentication handlers (in Server.xml) is the same for every server in a cluster.

Replication: Missing topic notifications are now routed across Diffusion server clusters (20893)

Missing topic notifications are now forwarded to handlers registered on other servers in the same cluster if a suitable handler is not found locally. If the server producing the event is a fanout secondary, it will still propagate the request to the primary, but the handling server will not.

Security: Principals and roles can now be locked by statements in the corresponding stores (20596)

The server now supports more granular control over the principals and roles. Previously, the MODIFY_SECURITY permission was sufficient to modify any principal or role in the security and system authentication stores. The new behavior is to optionally associate a principal or role with a principal that can edit it. The syntax for creating a principal is: `add principal 'name' 'password' locked by 'lock_principal'`. The syntax for locking a role is: `set role 'role_name' locked by 'lock_principal'` The locking principal still requires the MODIFY_SECURITY permission to modify the store. The corresponding script builders and queries to the stores have been updated to support this functionality.

Server: Improved delta calculation performance when inputs have many differences (20818)

When there is a large number of differences found in the input values, the difference engine backs off to balance the amount of CPU required against the size of the resulting delta. Improvements to the back-off algorithm have significantly reduced the CPU cost of comparing values that have a lot of differences, without further increasing the size of the delta.

Server: The Connector MBean setQueueDefinition() operation is deprecated (21232)

The connector MBean setQueueDefinition() operation has no effect. It is deprecated and will be removed from a later version of the product.

System Monitoring/Statistics: The Diffusion Publisher and JMX Adapter have been removed (20543)

Previous releases required that a special "Diffusion" publisher was deployed to support console functionality. The publisher exposed various JMX MBeans below a Diffusion/MBean topic branch. From this release, the console no longer relies on the JMX adapter. The JMX adapter and the Diffusion publisher have been removed.

System Monitoring/Statistics: Publish product version and license information to Prometheus (20902)

The server now exports information about the Diffusion product version and license to Prometheus as diffusion_release and diffusion_license metrics.

System Monitoring/Statistics: diffusion_peak_sessions Prometheus metric now a gauge (21231)

The diffusion_peak_session metric exported to Prometheus has been changed from a counter to a gauge because it is reasonable to access the metric without using rate().

Topic Views: Topic Views : Support for value mapping in JSON time series topics (20509)

Topic views now apply value mapping clauses to JSON time series topics. Previously JSON time series topics would have been ignored by views that specified value mapping clauses.

Topics: Improve evaluation of automatic topic removal policies for topics restored from persistence (19901)

Evaluation of automatic topic removal rules for topics restored from persistence is now delayed until all topics have been restored. This ensures that all restored topics that match the removal selector will be removed.

Topics: Slave topics deprecated (20282)

Slave topics have been deprecated at this release. Use topic views instead. The effect of a slave topic can be duplicated by creating a topic view that maps a single topic to a single reference topic. Topic views add far more powerful functionality such as mapping whole branches of the topic tree.

Topics: Quoting and escaping of characters in topic view expressions has been simplified (21059)

Quoting and escaping of special characters in topic view expressions has been simplified. In previous releases, escape sequences had several limitations. They could only be used in mapping clauses that were wrapped in quotes. Only a few characters could be escaped and the characters that could be escaped varied for different parts of the expression. They could not be used to escape special characters in JSON Pointer directive parameters. In this release, these limitations have been removed. The escape sequence \x can be used to literally insert any character x, with one exception: \/ cannot be used in path fragments. As a side effect of these changes, the root JSON pointer (i.e. the empty string) can now be used in <scalar> and <value> directives, as well as the new <expand> directive.

Bugs cleared at release 6.4.0

C Client: Update stream memory allocation issues (20894)

Addresses an issue where incorrectly-sized memory allocations were enacted when creating an update stream.

C Client: SECURITY_STORE_ROLE_T not present in public API (20931)

Addresses an issue where the SECURITY_STORE_ROLE_T struct was not present in the public API (as part of security-control.h). This has now been addressed.

Configuration: SecurityConfig does not prevent duplicate control authentication handlers (20765)

When configuring an embedded Diffusion server using the SecurityConfig class, multiple control authentication handlers could have the same name, leading to later issues. This has now been resolved and authentication handler configuration is properly validated.

Console: Console log issues handling Server.log file roll over (19681)

The console's log viewer would stop updating and sometimes display errors when the underlying log file was truncated, e.g. due to log rollover. The console now automatically follows log rollovers in tailing mode, and allows the user to fetch the latest log in browse mode.

Console: Selecting uncreated topics in the dashboard produces confusing results (20815)

Previous versions of the console could show outdated values in the console dashboard when tracking topics that did not exist. This is now fixed, and a warning is provided when you select a topic which does not currently exist.

Console: Console sends unescaped topic names in selectors, causing disconnection (21208)

Topics with names containing special topic selector characters could cause visual problems and disconnections in the console. This issue has now been fixed.

Console: Add sesson metric collector button looks disabled when name missing but still works (21407)

Previously the console sometimes allowed users to add metrics collectors without a name. This is no longer possible.

JavaScript Client: Server not notified when handlers are removed because they threw an exception (16184)

Handlers registered with the server are closed if their callback methods throw an exception. When this happens, they will not continue to receive notifications from the server. However, in previous versions, the server would continue to attempt to send notifications to the handler. The server is now informed of the handler close and stops sending notifications to it.

JavaScript Client: JavaScript client cannot 'unset' topic values (21163)

The client now reliably allows unsetting a topic with a primitive datatype by passing null or undefined to topicUpdate.set()

Licensing: Issues with cluster licensing view of cluster (20158)

The cluster maintains a dynamic view of the number of connections the cluster is licensed for. Earlier implementations have been subject to race conditions that caused different servers in the cluster to report inconsistent limits. Changes have been made to ensure eventual consistency.

Licensing: Soft limit breached warning at every connection (21123)

A log message is generated when the soft connection limit is breached. Previously it was generated for each connection above the soft limit. Now a message is only logged when the soft limit is first exceeded.

Replication: Replica streams do not recover after reconnection (21303)

Previous releases had a bug where topic replication did not recover correctly following the loss and reconnection of a connection between two servers. This would prevent replication of topic updates for some of the topics on some of the servers. The bug has been fixed in this release.

Replication: Cluster fails to form correctly following split brain (21370)

In previous releases, there were situations in which the topic replication service was not informed of Hazelcast partitions being moved following resolution of a split-brain condition. Consequently, some topic operations would persistently fail and report that "the cluster is re-partitioning". In this release, additional recovery code has been added to detect and resolve this problem.

Replication: A transient stale value can be published during recovery (21375)

When a server joins or leaves a cluster using topic replication, the server responsible for coordinating updates to a topic can change. When this happens, the new server notifies its peers and provides the current value of the topic. In previous releases, a server performing this "recovery" process could incorrectly re-publish an older value before publishing the latest value, resulting in some subscribers incorrectly getting the out-of-date value before being sent the correct value. This problem has been fixed in this release.

Server: Sessions using the HTTP polling transport can fail if there is a large backlog of messages (20929)

Due to a bug, sending a large backlog of messages to a session using the HTTP polling transport could occasionally fail, resulting in the session being closed with an IO_EXCEPTION exception. The bug has been fixed in this release.

Server: NullPointerException looking up GeoIP data (20994)

A NullPointerException could be thrown when looking up location and locale information for certain IP addresses. This can happen for IP addresses that are associated with the EU rather than a country in the EU. These IP addresses are now identified and the root locale is provided.

Server: Using JMX to restart a connector causes an unnecessary warning in the log (21244)

In previous releases, if JMX was used to stop and restart a connector, the server erroneously logged a PUSH-000195 "InstanceAlreadyExistsException" message to the logs. From this release, the message will no longer be logged.

Server: When the GeoIP database is enabled, the server incorrectly assigns country name to $Country (21348)

In previous releases, when the GeoIP database was enabled, the server incorrectly assigned a country name to the $Country session property. In this release, the server has been corrected to use the country ISO code, as documented.

Topic Views: Scalar topic view directives can evaluate to an illegal path (20897)

The use of scalar directives in a topic view could result in invalid paths. For example, if the scalar directive matched an empty string, a path with an empty segment could be returned. When attempting to add a reference topic at this path, an exception would be thrown. This has been addressed by filtering invalid topic paths and preventing the attempt to create reference topics for them.

Diffusion 6.4 Release Notes