Diffusion 6.3 Release Notes

6.3.9 (31 Mar 2021)

Bugs cleared at release 6.3.9

C Client: Selectors with descendant qualifiers not evaluated (24530)

Addresses an issue where selector strings containing descendant qualifiers were not evaluated in the C client. This has now been resolved.

JavaScript Client: Buffer implementation not supported outside of NodeJS (23352)

Previously, there was no access to the Buffer API when running the client in a browser context. The Buffer API is now available through diffusion.buffer.

JavaScript Client: Time series value stream event datatype compatibility problem (24781)

Value streams are compatible with parent types. For example, a ValueStream<JSON> can deserialize values from a String topic type. ValueStream<Event<JSON>> did not propagate the values for time series topics with the String event type. This has now been fixed.

JavaScript Client: RecordModel.get should allow obtaining a record by key only (24883)

The implementation of RecordModel.get could take a single key argument, but the documentation and the Typescript API did not reflect this. This has now been corrected.

JavaScript Client: messes.setRequestStream and timeseries.append should take topic types and plain JS type constructors for datatypes (24886)

The implementation of setRequestStream allows topic types and plain JavaScript type constructors for datatypes. This is now correctly reflected in the public TypeScript API .

JavaScript Client: First call of UpdateStream.set returns a TopicCreationResult (24890)

The first call to UpdateStream.set returns a TopicCreationResult. This was not reflected in the API. This has now been rectified to the TypeScript definitions and added to the documentation.

JavaScript Client: Cannot pass true to responder.respond() (26004)

The responder in request-response messaging did not allow boolean values to be passed as a response. Booleans are now detected as valid JSON data. An optional datatype could always be passed as a second argument to responder.respond(). This has now been documented and is reflected in the TypeScript API.

JavaScript Client: QueryResult.merge throws exception: "Error: TypeError: latestByOriginalSequence[k].sequence.toNumber is not a function (26086)

Time Series QueryResult.merge() threw an exception when the two query results had events in common. Merge now works as expected.

JavaScript Client: JSON delta fails with indefinite length arrays and objects (26303)

JSON deltas failed when the JSON was encoded using CBOR with indefinite length arrays and objects. Now JSON deltas work with definite and indefinite length encoded CBOR.

Server: Sessions can reconnect even if explicitly closed by another session (24873)

A issue has been resolved where the server allowed clients to reconnect during the reconnection timeout even if they had been explicitly closed by another session. This would only occur if session replication was enabled.

6.3.8 (16 Nov 2020)

Bugs cleared at release 6.3.8

Java & Android Client: Flow control can stall update streams under high load (23959)

In previous releases, a bug could cause a Java client using update streams to never recover from flow control. This caused updates to stall. The bug has been fixed in this release.

JavaScript Client: remoteTopics.js example - undefined variable 'datatypes' (23368)

The removeTopics.js example for the JavaScript client has the undefined variable 'datatypes'. This has been fixed.

JavaScript Client: Cannot use numbers/int64/float in JSON constraints (24086)

Previously, int64 data was serialized inconsistently for 32 bit integers and wrongly for negative 64 bit integers. This caused partial JSON constraints to fail for integer values. It also prevented negative 64 bit integers from being passed between clients. Integers are now serialized correctly and in line with other clients. Partial JSON constraints can be used with the Int64 datatype.

Topic Views: Multiple sessions creating the same topic view causes topic view re-evaluation (23683)

Creating an identical topic view using the same principal (with the same roles) would unnecessarily cause the view to be replaced, leading to the removal and recreation of all reference topics derived from it. This has now been resolved.

Topics: Potential failure to process time series updates (24308)

In previous releases, a rapid sequence of updates to time series topics could fail to be processed correctly, leading to data corruption. This bug has been fixed in this release.

6.3.7 (30 Jul 2020)

Bugs cleared at release 6.3.7

Apple Client: NSInvalidArgumentException: (notifier) is nil. (22672)

NSInvalidArgumentException: (notifier) is nil has been removed. When Diffusion fails to generate a notifier for the dispatcher due to an invalid stream, a warning will now be logged instead.

JavaScript Client: Connection options interpreted incorrectly when credentials field is set to an empty string (22853)

Previously, when an empty string was provided for connection credentials it was wrongly interpreted as "no credentials provided". Now an empty string is sent to the server, allowing named principals to connect with an empty credentials string.

Replication: "Unsupported service nnn requested by peer session" warnings (22533)

Warnings of the form "Unsupported service nnn requested by peer Session:" can be reported in logs when using replication. These warnings can indicate that certain components (such as security files, or topic views) have not been replicated correctly. This problem has now been resolved.

Topic Views: Topic views that create reference topics with PUBLISH_VALUES_ONLY set to true still publish deltas (22506)

Due to a bug in previous releases, topics created by a topic view that set PUBLISH_VALUES_ONLY to true could still publish deltas. The bug has been corrected in this release.

Topic Views: NullPointerException when creating a topic while concurrently removing a topic view (22694)

Due to a bug, in previous releases an add topic operation could fail if another session concurrently removed a topic view. A NullPointerException would be logged to the server log. The bug has been fixed in this release.

6.3.6 (27 Apr 2020)

Bugs cleared at release 6.3.6

Apple Client: NSInternalInconsistencyException when number of bytes written 0 (22236)

NSInternalInconsistencyException "write stream has fixed length" occurred when an error in the connection caused the total number of bytes written to be 0. A clearer error is now returned.

Apple Client: NSInternalInconsistencyException from PTWebSocketConnection - Frame reader not allocated (22311)

PTWebSocketConnection no longer raises NSInternalInconsistencyException "frame reader not allocated" when the frame reader is unallocated. The connection is now closed when the client receives a message that contains errors in the header and a more informative error is passed.

Apple Client: NSInternalInconsistencyException from PTDiffusionTopicUnsubscriptionReasonToString (22376)

NSInternalInconsistencyException "no delegate" is no longer returned when delegate was not present. This has been replaced with a more informative error.

Java & Android Client: Send request to filter does not return number of requests sent until all responses received (21906)

Send request to filter calls waited for all responses to arrive before returning the number of requests sent. This behavior has been amended to return the number of requests immediately.

Java & Android Client: Fetch examples in API documentation are incorrect (22021)

The fetchRequest() example given in the Java and Android API documentation has been corrected.

JavaScript Client: Javascript client maintains reference to closed stream (21833)

The JavaScript client sometimes held on to references to closed value streams, leading to a memory leak. This has now been fixed.

JavaScript Client: Send request to filter does not return number of requests sent until all responses received (21993)

Send request to filter calls waited for all responses to arrive before returning the number of requests sent. This behavior has been amended to return the number of requests immediately.

Topics: The server can stall after closing a session with a routing subscription then removing the source topic (21691)

In previous releases, the server's record of subscriptions was not correctly updated when a session with routing subscriptions was closed. Removing the source topic of one of the routing subscriptions could then lead to a multiplexer thread terminating (reported as a PUSH-000335 error in the log, followed by repeated PUSH-000773 warnings). The bug has been fixed in this release.

6.3.5 (9 Dec 2019)

Bugs cleared at release 6.3.5

Console: $Country not available in proposed session properties (21465)

Using the console to add a new metric collector, the "GROUP-BY" section has the auto completion function to introduce new properties. The property $Country has been added for this function because it was omitted before.

Replication: Possible deadlock when using topic replication and automatic topic removal (21520)

In previous releases, when using automatic topic removal and topic replication, a lock priority bug could result in a server deadlock. If this happened to a server, it would become unresponsive to some sessions. The problem can be confirmed by taking a Java thread dump of the server, which will report a deadlock. The bug has been fixed in this release.

6.3.4 (21 Nov 2019)

Bugs cleared at release 6.3.4

.NET Client: Topic cache not invalidated before reconnection (21277)

In rare circumstances after session reconnection, a race condition could occur that produced an incorrect value in the local topic cache, resulting in a client exception and unexpected behavior. This has now been fixed.

.NET Client: Increased memory usage when subscribing to many small topics (21403)

Subscribing to a lot of topics with relatively small values caused a session to consume a lot more memory than necessary. This has now been fixed.

Console: Creating a session metric collector works when no collector name has been specified (21409)

Previously, you could add a session metric collector via the console without specifying a name. The "add" button used to create a collector appeared disabled if no name was specified, but submitting the form still worked. This issue is now resolved.

Federation: IllegalStateException : Invalid update of PUBLISHER topic by FANOUT (21293)

When using fan-out, if there was a topic on the primary server selected for fan-out with the same path as an existing publisher-created topic on the secondary server, then a failure could occur at the secondary server reporting an IllegalStateException with the message "Invalid update of PUBLISHER topic by FANOUT". This problem has now been resolved.

JavaScript Client: JSON stream incorrectly decodes some sequences of values from string topics (21270)

In some cases involving time series topics containing int64 events, delta changes were applied to a buffer that was different to the buffer originally sent to the client. This caused the delta to be processed incorrectly, resulting in the client being unable to read any value greater than 24. The original buffer is now always used for applying a delta operation.

Replication: A transient stale value can be published during cluster recovery (21274)

When a server joins or leaves a cluster using topic replication, the server responsible for coordinating updates to a topic can change. When this happens, the new server notifies its peers and provides the current value of the topic. In previous releases, a server performing this "recovery" process could incorrectly re-publish on older value before publishing the latest value, so some subscribers could incorrectly be sent this out-of-date value before being sent the correct value. This problem has been fixed in this release.

Replication: Replica streams do not recover after reconnection (21376)

Previous releases had a bug where topic replication did not recover correctly following the loss and reconnection of a connection between two servers. This would prevent replication of topic updates for some of the topics on some of the servers. The bug has been fixed in this release.

Replication: Cluster fails to form correctly following split brain (21389)

In previous releases, there were situations in which the topic replication service was not informed of Hazelcast partitions being moved following resolution of a split-brain condition. Consequently, some topic operations would persistently fail and report that "the cluster is re-partitioning". In this release, additional recovery code has been added to detect and resolve this problem.

Server: The default conflation policy causes data corruption (21259)

In previous releases, a bug in the processing of the default 'conflate' conflation policy could cause data corruption. This would happen if there were multiple deltas in a session queue that was full enough to trigger conflation, and could result in incorrect values being published to subscribers, or fail on the server with a PUSH-000229 "InvalidDataException: Deltas do not compose" server log message. The bug has been fixed in this release.

Server: Exceptions during conflation due to outbound queue corruption (21314)

A bug in an optimization to internal queue structures was introduced in Diffusion 6.3.0. In rare cases, in sessions that received a high volume of updates for many different topics the bug caused data corruption in messages sent to the sessions. It resulted in exceptions that were logged as PUSH-000226 or PUSH-000229 multiplexer processing errors. The bug has been fixed in this release.

Server: 'always' conflation policy results in messages being wrongly accepted or rejected (21323)

In previous releases, internal byte counts were not updated correctly when a session queue overflowed during the processing of an update for a topic with the conflation property set to 'always. This could cause future messages for the session or sessions using the same connector to be incorrectly accepted or rejected. This bug has been fixed in this release.

Server: When the GeoIP database is enabled, the server now correctly assigns an ISO code to $Country (21353)

In previous releases, when the GeoIP database was enabled, the server incorrectly assigned a country name to the $Country session property. In this release, the server has been corrected to use the country ISO code, as documented.

Server: Prometheus service occasionally fails with ConcurrentModificationException logged (21424)

In previous releases, the Prometheus polling service could occasionally fail to return statistics, resulting in a ConcurrentModificationException in the server log. The bug has been fixed in this release.

Topic Views: Topic views not reevaluated for removed topics if prefix is not terminated by separator (21332)

If a topic view had a path mapping prefix which did not have a path separator terminator (for example, map a/b/c to prefix<scalar(/1)>), upon removal of topics that were included in the view, the view would not be reevaluated. This potentially resulted in reference topics not being created. This problem has now been resolved.

Topics: Null updates to int64, double, and string topics that have DONT_RETAIN_VALUE=true are not published (21279)

Previous releases had a bug where subscribers would not be notified if a null value was sent to an int64, double, or string topic with a null value, and the topic had its DONT_RETAIN_VALUE property set to true. The bug has been fixed in this release.

6.3.3 (16 Oct 2019)

Changes made to existing features at release 6.3.3

Federation: Unnecessary delta recalculation when a fanout client or topic replication stream receives a value (21262)

In previous releases, if a fan-out server received a value update, and it already had a value for the topic, it calculated a delta. This is always more expensive than sending the value, since otherwise the primary would have sent a delta update. From this release, if a fan-out secondary receives a topic value, it trusts that the primary has determined that a delta is too expensive and does not repeat the delta calculation. A similar optimization has been made for topic replication streams.

Bugs cleared at release 6.3.3

.NET Client: ISecurityControl.Script.ToScript() wrongly quotes role permissions (21154)

Calling ISecurityControl.Script.ToScript() incorrectly put role permissions in single quotes, so the security store did not update. This has now been fixed.

Apple Client: Selectors containing the '|' character not parsed correctly (20794)

Selectors containing the '|' character were not handled correctly, resulting in a failure to add stream to selected topics. This issue has now been resolved.

Client: Incorrect API documentation for the topic view <path()> directive. (21112)

The topic views API documentation for the optional second parameter of the <path()> directive has been corrected. The second parameter specifies the number of path segments to copy from the source path, not the index of the end segment.

Console: Console log unable to handle Server.log file rollover (20880)

The console's log viewer would stop updating and sometimes display errors when the underlying log file was truncated due to log rollover. The console now automatically follows log rollovers in tailing mode, and allows the user to fetch the latest log in browse mode.

Console: Selecting uncreated topics in the dashboard produces confusing results (21194)

Previous versions of the console could show outdated values in the console dashboard when tracking topics that did not exist. This is now fixed and a warning is provided when the user selects a topic which does not currently exist.

Console: Special characters in topic names cause display issues and disconnection (21209)

Topics with names containing special characters such as {, }, %, * and ? could cause visual problems and disconnections in the console. This was due to the characters not being escaped correctly, leading to the console treating them as part of a topic selector. This issue has now been fixed.

Console: Topic view list incorrect after editing a topic view (21241)

In previous releases, if a topic view was edited in the console the table of topic views could appear corrupted until the view was refreshed by switching to another tab and back. The bug has been fixed in this release.

JavaScript Client: Time series topic int64 event values are delivered incorrectly if they are greater than 24 (20908)

In some cases involving time series topics containing int64 events, delta changes were applied to a buffer that was different to the buffer originally sent to the client. This caused the delta to be processed incorrectly, resulting in the client being unable to read any value greater than 24. The original buffer is now always used for applying a delta operation.

Replication: Issues when handling cluster peer connection failure (21214)

When a server detects a connection failure to another server in the cluster, it must cancel all pending requests to that server. In previous releases, this was done in-line, and could stall multiplexer threads and affect outbound communication to other sessions. In this release, the failure handling is now offloaded to a lower priority thread.

Replication: Cluster connections no longer enforce maximum message size (21246)

When using topic replication in previous releases, the server maximum message size had to be larger than the largest topic value (plus a few tens of bytes of message framing). Otherwise the topic value could not be replicated to peers, and the cluster would fail. The maximum message size would also prevent such a value being sent to the server via the update API, and this was not readily apparent. The situation only arose in rare cases, for example, if maximum message size was reduced, the server was restarted, and a large topic value was recovered from disk; or if a client session incrementally increased the size of a value, using an update stream to send the changes as delta. From this release, the server's maximum message size is not applied to intra-cluster connections.

Server: IllegalStateException in TopicPathSelectionCache.verify() (21222)

An internal subscription index could become corrupted, producing a PUSH-00029 log message for an IllegalStateException in TopicPathSelectionCache.verify. The underlying bug is now fixed.

Server: A topic view with a path mapping containing an empty path was incorrectly accepted (21258)

In previous releases, a topic view with a path mapping containing a literal empty part such as "map x to a//b" was accepted, but would never generate reference topics. From this release, such topic views will be rejected by the server. In previous releases, if a topic view generated a path containing a leading or trailing empty part, through either a literal expression (now rejected as above) or the scalar path directive, the resulting view would fail during evaluation. The bug has been fixed in this release.

Topics: Rare update failure when using the TopicView <scalar> directive (21197)

In previous releases, there is a possible race condition in the processing of the TopicView <scalar> directive. Under rare circumstances, updates to a source topic that change the subvalue extracted by the scalar directive could fail. In this case, the server would log a PUSH-000664 message with an InvalidDataException and close the session that requested the update. Despite the transient failure, the topic view would be evaluated correctly. The issue has been fixed in this release.

Topics: Invalid pattern topic selector expressions not rejected. (21220)

In previous releases, split path pattern selector and full path pattern selector expressions that incorrectly contain an empty part (for example, ?a//b) were not rejected. This could mislead a user into thinking an expression was valid. Additionally, it could lead to other failures. For example, adding a topic view that used such an expression for the source topic selector would cause the requesting session to fail. The bug has been fixed in this release.

6.3.2 (4 Sep 2019)

Changes made to existing features at release 6.3.2

Installation: The zlib-bundle.js file is now included in the installation (20800)

Diffusion client uses zlib to support message compression. Since Diffusion 6.1, zlib is no longer included in the Diffusion JavaScript client library to reduce its size. Support for zlib is detected automatically, and compression is enabled or disabled accordingly. Applications using the JavaScript client in a browser must separately include zlib to enable message compression. This can be achieved at build time by using browserify to package the browserify-zlib npm module into the application library. In this release, a separate browserify-zlib-0.2.0.js file is included in the clients/js installation directory. An application can package this file or include it as a runtime dependency, instead of using browserify at build time. The browserify-zlib-0.2.0.js file is also available for download at https://www.pushtechnology.com/developers/releases/6.3/#js-sdk No extra step is required to enable compression when using Diffusion in node.js environments, since zlib is a default node.js module.

Server: Improved delta calculation performance when inputs have lots of differences (20870)

When there is a large number of differences found in the input values, the difference engine backs-off to balance the amount of CPU required against the size of the resulting delta. Improvements to the back-off algorithm have significantly reduced the CPU cost of comparing values that have a lot of differences without further increasing the size of the delta.

Bugs cleared at release 6.3.2

.NET Client: ReconnectionAttempt.abort does not abort (20887)

Calling ReconnectionAttempt.Abort() in IReconnectionStrategy.PerformReconnection() will now correctly abort any further reconnection attempts.

.NET Client: Invalid IUpdateStream throws SessionException instead of InvalidUpdateStreamException (20890)

An invalid IUpdateStream would throw a SessionException instead of an InvalidUpdateStreamException when calling any methods on it. This has now been corrected.

.NET Client: ISecurityControl.GetSecurityAsync always times out (20924)

The Diffusion server sent an incorrectly formatted response to ISecurityControl.GetSecurityAsync, resulting in every call timing out. The server now returns the correct response.

.NET Client: ISession.Close() logs ObjectDisposedException (20925)

Calling ISession.Close() multiple times would cause an ObjectDisposedException to be thrown internally. This would be wrongly logged as ERROR in the client log. This has now been fixed.

.NET Client: SessionClosedException stack trace growing on every async method call (20935)

The SessionClosedException stack trace would grow on every async method call. This has now been fixed and the stack trace will no longer grow.

C Client: Update stream memory allocation issues (20909)

Addresses an issue with incorrectly sized memory allocations when creating an update stream.

C Client: SECURITY_STORE_ROLE_T not present in public API (20932)

The SECURITY_STORE_ROLE_T struct was not present in the public API (as part of security-control.h). This has now been fixed.

Console: Topic browser does not update correctly after child topic removal (20742)

In some cases, when a child topic which had previously been expanded was removed, the console topic browser did not update correctly. This issue is now resolved.

Console: Console not showing full long values in the Subscription tab (20817)

The console showed only the first 80 characters of the value of a subscribed topic in the Subscriptions tab. Clicking on a row in the Subscriptions tab now displays the full topic value.

Console: Console displays UNKNOWN_GLOBAL_PERMISSION (20927)

When editing the permissions of a role in the security tab of the console, two invalid permissions were displayed. Attempting to assign them to a role caused problems. These options have now been removed.

Publisher API: Slave topics created by a publisher are not correctly removed when the master is removed (20954)

Due to a bug introduced in 6.3, slave topics created by a publisher were not correctly removed when their master topic was removed. The bug has been fixed in this release.

Replication: Topics that replace reference topics when replication is in use are created with incorrect state (21068)

Topics created on the same path as an existing reference topic (created by a topic view) could be set up with an incorrect state. This problem has now been resolved.

Security: Topic view permissions are now correctly presented to older clients as UNKNOWN_GLOBAL_PERMISSION (20926)

Clients of version 6.2 or earlier were disconnected when attempting to retrieve security details from a 6.3 server, due to a bug in serializing newer topic permissions. These permissions are now correctly displayed as UNKNOWN_GLOBAL_PERMISSION for older clients.

Server: Sessions using the HTTP polling transport could fail if there is a large backlog of messages (20941)

Sending a large backlog of messages to a session using the HTTP polling transport could occasionally fail, resulting in the session being closed with an IO_EXCEPTION exception. This has been fixed in this release.

Server: NullPointerException looking up GeoIP data (21077)

A NullPointerException could be thrown when looking up location and locale information for certain IP addresses. This can happen for IP addresses that are associated with the EU, rather than a country in the EU. These IP addresses are now identified and the root locale is provided.

Topic Views: Scalar topic view directives can evaluate to an illegal path (20917)

The use of scalar directives in a topic view could result in invalid paths. For example, if the directive matched an empty string, a path with an empty segment was returned. Trying to add a reference topic at this path would then lead to an exception being thrown. This has been addressed by filtering invalid topic paths and preventing the attempt to create reference topics for them.

6.3.1 (26 Jun 2019)

Changes made to existing features at release 6.3.1

Java & Android Client: Java client now supports Server Name Indication extension to SSL/TLS (20774)

The Java client now supports the SNI extension for SSL/TLS handshakes. This is always enabled, and uses the host name that the client is connecting to.

Bugs cleared at release 6.3.1

Android Client: Update streams do not work on Android (20511)

Addresses an issue where the Android client was unable to update topics using update streams or validate update streams.

Client: Incorrect topic view examples in API documentation (20766)

Incorrect topic view examples in API documentation have been corrected.

Configuration: Default setting for GeoLite City 2 database incorrect (20537)

An issue where the default setting for the GeoLite2 city database was pointing to 'data/GeoLiteCity.dat' rather than 'data/GeoLite2-City.mmdb' has been resolved.

Console: "Cannot read property 'specification' of undefined" when showing details of a time series topic (20485)

Time series topic values can now be viewed in the topic details view.

Console: Add new principal does not work (20523)

The [+] button to add a principal in the Security tab did not work. This issue has been resolved.

Console: Overflow when displaying large topic values (20564)

Large topic values no longer overflow the topic tree details view. Instead, values are broken up into multiple lines.

Console: Incorrect sorting of topic names (20635)

Topic node entries are now correctly sorted when being displayed.

Console: Console lags or crashes when expanding wide topic tree (20641)

The topic tree browser suffered from lags or crashes with wide topic trees. Performance has been improved by applying caching and paging mechanisms. Google Chrome may still have issues with wider topic trees. As a workaround, use the console in Firefox or Safari.

Console: Add new role does not work (20713)

The [+] button to add a role in the Security tab did not work. This issue has been resolved.

Console: Session metric collector 'Remove metrics when no sessions match' field not displayed correctly in console (20749)

A bug in the console session metric collector tab meant that the "Remove metrics when no sessions match" option could appear inactive, even if it was active. This has been corrected in this release.

Demos: ValueStream emits error after it is closed (20570)

A ValueStream would sometimes emitted an error after it was closed. This has now been fixed.

Java & Android Client: Javadoc is incorrect regarding the exceptions returned from CompletableFuture API calls (20722)

The Java API documentation for methods that return a CompletableFuture incorrectly stated that an ExecutionException is returned, when in fact a CompletionException is returned. This has now been corrected. Some methods do inconsistently return ExecutionException and these have been highlighted in the documentation and will be corrected in a future release.

Licensing: Issues with cluster licensing view of cluster (20789)

Due to a race, the cluster soft and hard connection limits were not being consistent for all the servers in the cluster. This issue has now been solved.

Persistence: File persistence service fails to restore - topic has different specification (20628)

When using both persistence and replication together a server could fail with the message "The file persistence service has failed to restore from existing files" indicating that a subsequent ADD_TOPIC record has a different specification. This made it impossible to restore from affected files. This issue has now been resolved.

Security: Roles with no permissions and no included roles are now reliably removed from the security configuration (20266)

In previous releases, if multiple roles were updated to have no included roles, redundant "set role 'X' includes []" statements could be left in the security store. If these roles had no associated global or topic permissions, they would incorrectly be included in the result of SecurityConfiguration.getRoles(). The bug has been fixed in this release.

System Monitoring/Statistics: Session metric collectors: session filter fails to match user session properties that have not yet been used (20748)

In Diffusion 6.3.0, a bug in the session property index affected the evaluation of session metrics collectors. If a session metrics collector had a session filter that referred to a user session property not previously used by a session, it would fail to match new sessions with that property. The bug has been fixed in this release.

Topic Views: Using topic views causes NullPointerException in TopicManagerImpl.createReferenceTopic (20575)

When using topic views, some circumstances could cause a NullPointerException in TopicManagerImpl.createReferenceTopic. This problem has now been resolved.

Topics: Slave topics do not rebind after removal and recreation of master (20657)

Slave topics did not bind to the master topic if it was removed and then recreated. This has now been resolved.

6.3.0 (6 May 2019)

Features introduced at release 6.3.0

.NET Client: New operations to query the permissions of the current session (20272)

The new GetGlobalPermissionsAsync and GetPathPermissionsAsync methods in the ISecurity feature allow sessions to query the permissions that have been assigned to them.

Apple Client: Session locks (18751)

Adds support for per-session locks. To request session locks, use the lockWithName:completionHandler: and lockWithName:scope:completionHandler: methods on PTDiffusionSession instances.

Apple Client: Client proposed session properties (19416)

It is now possible for Apple clients to propose user-defined session properties when opening a session. These properties may be accepted, changed or vetoed by authentication handlers that implement the new Authenticator interface.

Apple Client: New update API (19498)

The Apple client now features the improved update API, with more flexible options when updating topics. It adds conditional updates, non-exclusive update streams and the creation of missing topics through the update API.

Apple Client: New operations to query the permissions of the current session (20274)

New 'get global' and 'get topic' permissions methods have been added to the PTDiffusionSecurityFeature, allowing sessions to query the permissions that have been assigned to them.

C Client: Subscribe/unsubscribe other sessions using session properties filters (12790)

The C Client is now able to subscribe/unsubscribe a number of sessions which match a given session properties filter to/from topics. This functionality is located in subscription-control.h.

C Client: Set session properties of other clients (17354)

The C client is now able to set session properties for an individual session or number of sessions matching a session properties filter. This functionality is located in client-control.h.

C Client: Time series topics (17501)

The C client now supports time series topics. Functions and information relating to time series topics can be found in time-series.h and range-query.h.

C Client: Change roles of other clients (19394)

Client control feature (located in client-control.h) has two new API functions: diffusion_change_roles_with_session_id and diffusion_change_roles_with_filter. These functions allow clients with sufficient permissions to change the security roles of other clients.

C Client: New topic notifications feature. (19619)

Introduces the Topic Notification feature (located in topic-notifications.h). Users may select topic paths, for which notifications of the presence, addition or removal of matching topics will be received. This allows users to build a view of the topic model held within the server.

C Client: New operations to query the permissions of the current session (20273)

The new diffusion_get_global_permissions and diffusion_get_path_permissions functions in security.h allow sessions to query the permissions that have been assigned to them.

Client: Session properties filters now support an 'all' clause (20200)

The session properties filter language has been extended to include an "all" clause that matches all sessions.

Console: New Diffusion console features (19805)

The web-based monitoring console has been expanded with new features, including: a new topic tree view; a new subscription view; the ability to view topic permissions; management of topics, metric collectors and topic views; improved logs tab with syntax highlighting and tooltips; counts of log messages by code and severity.

Java & Android Client: New operations to query the permissions of the current session (20159)

The new getGlobalPermissions and getPathPermissions methods in the Security feature allow sessions to query the permissions that have been assigned to them.

JavaScript Client: New TypeScript API (17983)

The JavaScript client has been completely re-written with TypeScript. All type definitions are now automatically generated.

JavaScript Client: New Authenticator interface supersedes AuthenticationHandler (19230)

A new Authenticator API has been implemented in the JavaScript client. The AuthenticationHandler has been deprecated.

JavaScript Client: Change roles (19395)

The JavaScript client now includes the change roles operation, enabling client sessions with sufficient permissions to change the security roles of other client sessions.

JavaScript Client: New operations to query the permissions of the current session (20160)

The new methods getGlobalPermissions and getPathPermissions on Session.security allow sessions to query the permissions assigned to them.

System Monitoring/Statistics: Server metrics redesign (18452)

The set of metrics published by the server has been completely redesigned. Highlights include: new metrics for the number of connected sessions, session-related messages and bytes (inbound and outbound), total network bytes (inbound and outbound), topic data size, topic subscriptions, topic subscribers, topic updates, topic subscriber updates. Log event metrics are now broken down by log level and code. Please refer to the product documentation for full details. The new metrics are published to the developer console, JMX, and the Prometheus gateway.

System Monitoring/Statistics: Metric Collectors allow focussed aggregation of topic and session metrics (19913)

Session metric collectors can be configured to record metrics for subsets of the sessions. Each session metric collector specifies the target sessions using a session filter. The metrics can optionally be further partitioned by an arbitrary list of session properties. Topic metric collectors can be configured to record metrics for subsets of the topics. Each topic metric collector specifies the target topics using a topic selector. The metrics can optionally be further partitioned by topic type. Like server-level metrics, collected metrics are published to the console, JMX, and (optionaly) the Prometheus gateway. Metric collectors can be defined using the console or JMX. The server will be persist the configuration, so the metric collectors will be recreated if the server is restarted. If the server belongs to a cluster, the configuration will be replicated across all members of the cluster.

Topic Views: New Topic Views Feature (9832)

Topic views can be used to automatically derive topics from other topics. The dynamically managed topics, generated by topic views are called reference topics. Reference topics are derived based on the path, specification and value of the source topics. Reference topics appear as ordinary topics to clients. Topic views allow Diffusion to separate how it represents topics internally and how it presents them to clients. Topic views can modify the topic properties, apply mappings to the value published and throttle the publication rate of reference topics. Topic views are specified using a new DSL. The management of topic views is supported in all client APIs.

Changes made to existing features at release 6.3.0

.NET Client: Deprecated exceptions within PushTechnology.DiffusionCore.Exceptions removed (19004)

The previously deprecated exception types have been removed: DecryptException, InboundMessageException, InboundProtocolException, InvalidTopicException, NotCompressedException, TopicInvalidException.

.NET Client: Deprecated TopicDetails types removed (19073)

The previously deprecated TopicDetails types have been removed. The list of the deprecated types includes: ISchemaId, ITopicDetailsSchema, ITopicDetailsBuilder, ITopicDetailsBuilder<TD>, ITopicDetailsBuilder<TB, TD>, IStatelessTopicDetailsSchema, IStatelessTopicDetailsBuilder, IStatelessTopicDetailsAttributes, ISlaveTopicDetailsBuilder, ISlaveTopicDetailsSchema, IRoutingTopicDetailsSchema, IUniversalTopicDetailsSchema, IUniversalTopicDetailsBuilder, ITopicDetailsBuilderFactory, IRoutingTopicDetailsBuilder and ICacheSchemaHandler<TC>.

.NET Client: Deprecated SecurityControl methods removed. (19096)

The previously deprecated ISecurityControl.ScriptBuilder() and IScriptBuilder.Script() methods have been removed.

.NET Client: Improved memory profile during connection loss (19948)

Previously, a connection loss under high load could cause a significant increase in memory usage due to the exceptions thrown by asynchronous methods. The behavior has now been changed so that the garbage collector has an easier time cleaning up memory during a connection loss.

.NET Client: IFetchRequest.First and IFetchRequest.Last now allows 0 to be supplied as an argument. (20432)

IFetchRequest.First and IFetchRequest.Last now allow zero as a parameter. Such a request can used to discover the existence of topics IFetchResult.HasMore.

.NET Client: New FetchResult.Count and FetchResult.IsEmpty properties (20440)

IFetchResult now provides Count and IsEmpty properties for direct access to the values in the Results property.

Apple Client: Fetch operations for time series topics (19615)

Adds support for fetch operations on time series topics.

Apple Client: Conform enum descriptions (20386)

Return values for description methods on classes inheriting from PTDiffusionEnumeration have been conformed to match the equivalent values in the Java API, as returned by the toString() method.

C Client: Fetch first/last now allows 0 to be specified. (20435)

diffusion_fetch_request_first and diffusion_fetch_request_last can now accept a 0 value. Fetching zero topics which match a selector can be used to detect the existence of matching topics by using diffusion_fetch_result_has_more.

C Client: New fetch_result_size and fetch_result_is_empty functions (20438)

diffusion_fetch_result_size and diffusion_fetch_result_is_empty functions have been added for fetch requests for convenience when processing result sets.

Client: Relaxed naming rules for session property keys (19265)

User proposed session property keys were previously restricted to strings of alphanumeric characters. This restriction has been relaxed to allow all Unicode characters with the exception of: ' ', '\t', '\r', '\n', '"', ''', '(', ')'.

Client: Per-session throttling has been deprecated (20335)

The client control methods that allow individual client sessions to be throttled, together with the throttling related methods and types of MessageQueuePolicy have been deprecated and will be removed in a future release. Applications that need to control the rate of data flow should prefer the per-topic throttling capabilities of topic views. Per-topic throttling has a far lower performance impact and is easier to use than per-session throttling.

Configuration: Persistence store-directory deprecated in favor of persistence-home (20149)

The Server.xml persistence.store-directory configuration item has been deprecated in favor of the new persistence-home item. This is also reflected in PersistenceConfig and ServerConfig classes in the Publisher configuration API. This change is because the 'persistence' directory is now used to hold the security store files as well as topic persistence files.

Installation: Non-versioned versions of the client libraries are no longer included in the installation (19822)

Previously the product installation included two copies of each client library. One copy had the product version in the name and one did not, but the files were otherwise identical. The unversioned copies are no longer included.

Java & Android Client: New FetchResult.size and FetchResult.isEmpty methods (19656)

FetchResult now provides FetchResult.size and FetchResult.isEmpty methods for direct access to the values returned by FetchResult.results.

Java & Android Client: New DataType.binaryDeltaType() method (19804)

The DataType interface now provides a binaryDeltaType() convenience method. This method is equivalent to calling deltaType(BinaryDelta.class), except it returns null if the data type does not support binary deltas, rather than throwing IllegalArgumentException. Diffusion implementation code has been changed to use the new method to avoid using exceptions for flow control when working with the long and double data types.

Java & Android Client: FetchRequest.first and FetchRequest.last now allow zero to be supplied as an argument (19862)

FetchRequest.first and FetchRequest.last can now accept a 0 value. Fetching zero topics which match a selector can be used to detect the existence of matching topics by using FetchResult.hasMore.

Java Client: Prevent blocking CompletableFuture calls from Diffusion threads (19505)

In previous releases, if a client application called a blocking CompletableFuture method, such as get() or join(), from a Diffusion thread would cause the client to deadlock because the same thread is used to deliver responses to the session. This situation typically happens when chaining CompletableFutures. For example: messaging.sendMessage("x", "m1", String.class, String.class) .thenAccept(r1 -> { // This will run in the Diffusion input thread, when the result of the first call is received. CompletableFuture<String> cf = messaging.sendMessage("y", r1, String.class, String.class); String r2 = cf.get(); // Before 6.3, this blocks the input thread so the response can't be delivered. After 6.3, it will cause r2 to complete exceptionally. LOG.info("Result {}", r2); }); From this release, the Java client will detect the use of a blocking method from a Diffusion thread, and complete the CompletableFuture exceptionally with a UnsupportedOperationException. To avoid the problem, when calling a second Diffusion method from a handler attached to a Diffusion-supplied CompletableFuture, only use non-blocking methods such as thenAccept() on the result returned by the second method. For example: messaging.sendMessage("x", "m1", String.class, String.class) .thenAccept(r1 -> { // This will run in the Diffusion input thread, when the result of the first call is received. CompletableFuture<String> cf = messaging.sendMessage("y", r1, String.class, String.class); cf.thenAccept(r2 -> { LOG.info("Result {}", r2); }); });

JavaScript Client: 'UNKNOWN' added to some API enums to allow for future extension (20229)

To improve interoperability between this version of the client and future versions of the server, new "unknown" values have been added to several enumerations. The additions are: an UNKNOWN_TOPIC_TYPE topic type; UNKNOWN_GLOBAL_PERMISSION and UNKNOWN_TOPIC_PERMISSION permission types; and an UNKNOWN_UNSUBSCRIBE_REASON unsubscription reason.

JavaScript Client: Fetch first/last now allows 0 to be specified. (20434)

FetchQuery.first and FetchQuery.last can now accept a 0 value. Fetching zero topics which match a selector can be used to detect the existence of matching topics by using FetchResults.hasMore.

JavaScript Client: New FetchResult.size() and FetchResult.isEmpty() methods (20439)

The convenience methods FetchResult.size() and FetchResult.isEmpty() have been added for easier processing of result sets.

Logging: The server will not start if the ConnectionCount statistics file cannot be created (19943)

The server logs daily connection count statistics to a ConnectionCount file. The file path is determined by the default-log-directory element of the Logs.xml configuration file, evaluated relative to the Diffusion installation directory. In previous releases, if the configured log directory was not writable when the server was started, a temporary directory would be created and a warning logged. From this release, if a writable ConnectionCount file in the configured log directory does not exist and cannot be created, the server will log an error and fail to start.

Logging: Log metrics are now partitioned by code and level (20181)

The server records event counts log messages that have a PUSH-XXXXX message code, grouped by the code. From this release, these metrics are also grouped by the the log level (trace, debug, info, warn, error). Downstream systems which consume these metrics using JMX or Prometheus will need to be updated to account for this change.

Publisher API: The unused HTTPCache interface has been removed (19920)

Since Diffusion 6.1, the HTTPCache interface has not been used. It has been removed from the product in this release.

Publisher API: Per-session throttling has been deprecated (20340)

The methods in com.pushtechnology.diffusion.api.publisher.Client which allow individual client sessions to be throttled have been deprecated, together with the ClientThrottlerType enum, and will be removed in a future release. Applications which need to control the rate of data flow should prefer the per-topic throttling capabilities of topic views. Per-topic throttling has a far lower performance impact and is easier to use than per-session throttling.

Replication: Cluster-aware system authentication and security control APIs (18957)

In previous releases, the system authentication and security APIs would only update the security stores on the server that the client session was attached to. This meant that in a clustered environment, servers would be out of step, unless the same changes were made to each server. From this release, all changes to the system authentication or security stores done via the API will be properly replicated across the cluster, regardless of which server the client session was attached to.

Replication: Hazelcast configuration file now held in the etc directory. (20069)

The hazelcast.xml configuration file for configuring Hazelcast has been moved to the etc directory with the other configuration files. It can still be resolved from the data directory, but a file in etc will take precedence.

Replication: New fallback for failures when compacting the partition log (20112)

When compaction of the partition log fails, a second attempt will now be made using a new compactor, without any state carried over from previous compactions.

Replication: Replication of Configuration Items (20297)

At this release if replication is enabled then changes made to security stores using the client API will automatically be replicated to all members of the cluster. Topic Views and Metric Collectors (both new at this release) will also be replicated across the cluster if either topic or session replication is enabled. It is also possible to replicate such items across a cluster even if neither topic nor session replication is enabled using a new configurationReplication element in the Replication.xml file.

Security: The SSLv2Hello TLS handshake protocol is now disabled by default (20051)

The SSLv2Hello handshake is no longer enabled by default. It can be enabled by using the diffusion.tls.protocols Java system property.

Security: Security store files are now held in the persistence directory (20070)

In previous releases, the security store files (Security.store and SystemAuthentication.store) were held in the 'etc' directory under the Diffusion home directory. From this release, these files are held in the same directory as the persistence log files - by default, in 'persistence' under the Diffusion home directory. The sample files issued in the 'etc' directory are copied to the 'persistence' directory when a server first starts if there are no such files there already, but from then on will be maintained in the 'persistence' directory. Though these files may be manually edited, this is not recommended, especially when using replication (as changes would need to be done on every server in the cluster). The Diffusion API should be used to update these stores.

Server: Improved parsing of session filter expressions (20140)

Session filter expressions can now span multiple lines. Parsing of session filters has been tightened up and and the parser will now reject expressions with additional trailing text.

Server: Topic selector performance improvements (20276)

The runtime performance of split-path pattern topic selectors and selector sets has been improved.

System Monitoring/Statistics: Multiplexer event diagnostic recording (19792)

Diagnostic recording of multiplexer events has been added, to be used at the direction of Push Technology support. Recordings can be controlled per-multiplexer using the startEventDiagnosticRecording() and stopEventDiagnosticRecording() operations of the Multiplexer JMX MBean, or for all multiplexers using the similarly named operations of the MultiplexerManager JMX MBean.

System Monitoring/Statistics: New JMX metrics MBeans (19850)

The JMX interface to metrics has been completely revised. Previously, if the JMX statistic reporter was enabled, metric data was exposed by JMX mbeans below the com.pushtechnology.diffusion.metrics tree, each delivering a single metric value. Statistic reporters, including the JMX statistic reporter, have been removed in this release, so these mbeans are no longer available. In their place, JMX mbeans have been added to the com.pushtechnology.diffusion tree. These are always available, they do not need to be enabled separately. The new mbeans include: a NetworkMetrics mbeans that provides the server network metrics; SessionMetrics mbeans that provide server and per-session metric collector session metrics; TopicMetrics mbeans that provide server and per-topic metric collector topic metrics; LogMetrics mbeans that provide a break down of log events by severity level and code; MetricsCollector mbeans that allow Session and Topic metric collectors to be created, removed, and listed.

System Monitoring/Statistics: Per-publisher metrics have been removed (19926)

Metrics are no longer accumulated on a per-publisher basis. The PublisherStatistics interface and the related methods that were deprecated in 6.1 have been removed. Publisher code can access server-wide statistics using the Publisher.getServerStatistics() method.

System Monitoring/Statistics: Client and topic instance statistics have been replaced by Metric Collectors (20218)

In previous releases, the server could be configured to enable reporting of per-topic and per-client instance metrics. Enabling either had a significant performance impact. Topic and client instance statistics have been removed in this release. The server configuration and Publisher API to enable/disable these features is deprecated and has no effect. Users wishing to monitor a subset of topic or sessions can use the new metric collectors feature. Metric collectors are superior to instance statistics in several ways. The implementation is far more efficient, and production use of metric collectors is encouraged. A richer set of metrics is provided. Metric collectors aggregate the results across a number of instances, identified using a topic selector (for topic metric collectors) or a session filter (for session metric collectors). Metric collectors can be added and removed at anytime using the console or JMX.

System Monitoring/Statistics: Statistics reporters have been removed (20271)

In previous releases, the console dashboard required a topic statistics reporter to be enabled in the server configuration; otherwise, some metric values would not be displayed. From this release, the console communicates with the server using an internal service. The topic statistics reporter has been removed. In previous releases, a JMX statistics reporter could optionally be enabled in the server configuration which would bind server metrics to JMX MBeans. In this release, server metrics have been substantially revised. Server metrics are published via new JMX MBeans which are always enabled. The JMX statistics reporter has been removed. The reporters section of Statistics.xml and the corresponding Publisher API methods are deprecated. If any reporter configuration is found in the configuration, a warning will be logged on start up.

Web Server/HTTP: The REST service has been retired and removed (19894)

Previous releases provided a simple REST service implementation that allowed topic information to be polled over HTTP. The implementation has major functional limitations. It does not provide topic values for any of the supported topic types. It does not handle topic paths containing characters that cannot be used in XML names. Additionally, it has security issues. The topic tree structure could be browsed without authenticating against the server. The query to calculate the response could be expensive, making it a possible vector for a denial of service attack. Given these issues, we have retired the REST service and the associated XMLUtils publisher API class, and both have been removed from the product. If you are upgrading a configuration, remove http-service elements referencing com.pushtechnology.diffusion.service.RESTService class from /etc.WebServer.xml to avoid a warning message when the server starts.

Bugs cleared at release 6.3.0

Android Client: Session properties listeners do not work on Android (20283)

Addresses an issue where session properties listeners failed on particular update events.

Android Client: The authenticator API does not work on Android (20344)

Addresses an issue where the authenticator API would fail when used by the Android client. This case also resolves a similar issue with the setToString utility method.

Apple Client: Primitive value updater raising exception on attempt to update with null value (20169)

Fixes a bug in the update methods on both the PTDiffusionStringValueUpdater and PTDiffusionNumberValueUpdater classes. The API states that the values are nullable (may be provided with nil pointers) but at runtime an exception was being thrown.

C Client: unsubscribe_client function does not invoke callbacks (20037)

Addresses an issue where callback functions supplied to an unsubscribe_client() operation were not called. This has now been fixed.

C Client: Double free possible when removing a value stream with remove_stream() (20092)

Addresses an issue where a double free can occur when removing a value stream.

C Client: Incomplete topic value from topics with compression enabled (20096)

Addresses an issue where topic values received from subscriptions returned incomplete values.

C Client: Invalid raw bytes from diffusion value returned from a fetch request's result (20142)

Addresses an issue where an incorrect raw bytes value (retrieved by calling diffusion_value_get_raw_bytes) is returned from the fetch result to a fetch request

C Client: Invalid delta calculated when updating with topic update streams (20193)

Addresses an issue where subsequent invalid delta updates were calculated when updating with an update stream (after the stream's first update). This has now been resolved.

C Client: Possible crash when subscribing to a topic with REMOVAL topic property (20284)

Addresses an issue where subscribing to a topic created with the REMOVAL topic property set would crash a client application. This has now been resolved.

C Client: Incorrect value returned when NULL is supplied to topic result and fetch result functions (20455)

If a NULL pointer was supplied to topic result and fetch result functions, the correct value was not returned. This has now been resolved.

Client: New user defined properties are not reported to SessionPropertiesListener (19481)

When a session properties listener is notified of an update event, any new session properties added should be represented in the old values map with null values. This was not occurring. This problem has now been resolved.

Console: Timezone when you hover over the clients graph in the console is always GMT (14296)

Area and line charts in the console Overview tab now render time popups in locale sensitive formats.

Console: The button to the console sometimes does not appear (19752)

The "Go To Monitoring Console" button would not appear if the front page was loaded before the console DAR was loaded. This issue is resolved, and the button now appears so long as the console DAR is loaded.

Federation: Fanout connections to older servers are not supported (19826)

There was an issue with automatic protocol downgrade, preventing a Diffusion server from establishing a fan-out link to a primary server running an older version. This has been fixed so that the appropriate protocol version is correctly negotiated.

Installation: NullPointerException if multiple products found (19713)

A NullPointerException would occur in the installer if multiple products were found. This has been resolved.

Installation: Installer does not warn about existing license file (19745)

If the Diffusion installer detects a license file in the file system, it will now ask whether to use that license or the default license.

Java & Android Client: The withProperties method of TopicSpecification returns a new specification not based on the current one (20191)

The withProperties method of TopicSpecification should return a new specification based on the current one but with the properties provided. Previously it would return a new specification based on the default specification. The method has been updated to base the new specification on the current specification.

Java Client: Correct IO_EXCEPTION close reason documentation (20426)

The IO_EXCEPTION close reason documentation in ClientControl referred to incorrect information. This was regarding connection closure due to the input buffer limit being exceeded - this has now been corrected.

JavaScript Client: Updater does not implement updateValue (19836)

Updater.updateValue has been implemented.

JavaScript Client: session.select() fails when given a SelectorSet topic selector (19854)

session.select() and other methods taking a string type topic selector will now also accept a TopicSelector.

JavaScript Client: There is no public method to obtain a time series event datatype (19981)

Added DataTypes.timeseries() function to obtain a time series data type.

JavaScript Client: QueryResult.merge throws an error when called (20203)

QueryResult.merge now works as documented.

JavaScript Client: TopicControl.add() function result is not correct (20215)

TopicControl.add() now correctly returns result.added==false when a topic already exists.

Replication: Configured Hazelcast map backup count ignored (19898)

Hazelcast allows the number of backups used by distributed maps to be configured. Diffusion was overriding the configuration with its own settings for the map. This discarded the number of backups. Diffusion now merges its configuration with the configuration discovered by Hazelcast.

Replication: A race condition prevents the removal of a replicated topic (19936)

A race between reading and modifying the children of an array can prevent the removal of replicated topics. The concurrency issue has been corrected.

Replication: PartitionMigrationWatcher NullPointerException when server starts (19986)

In previous releases, a server configured to use topic replication could occasionally fail to start. A NullPointerException originating from the PartitionMigrationWatcher.onNewOwner() method would be logged. The problem has been fixed in this release.

Replication: Wildcard Hazelcast map XML configuration generates invalid configuration (20042)

The use of wildcards to configure maps in the hazelcast.xml could cause clustering to fail. Diffusion modifies the configuration found in the hazelcast.xml to ensure Hazelcast is optimised for Diffusion's use case. Hazelcast uses the same configuration object for all matching wildcard configuration this results in the optimisations being merged, breaking Diffusion. This has been fixed by introducing appropriate deep copies of the configuration to prevent merging optimisations.

Replication: Hazelcast partition log compaction failure (20068)

Hazelcast was failing to compact the partition log during the recover of a partition. The partition log is divided into two parts compacted and uncompacted records. During recovery the uncompacted records are compacted. Compaction was losing track of values associated with previously compacted records. Preventing the uncompacted records being applied. Following the compaction of the log if a delta update had been published and no value update had the next attempt to recover the partition would fail. This was made possible by a recent optimisation to the logging that reduced the circumstances values were logged. Compaction now correctly tracks values of previously compacted records.

Replication: Exception generating peer credentials during shutdown (20099)

Once a local Hazelcast has been shut down, it will throw exceptions if further operations are attempted with it. Additional handling has been added so background operations using Hazelcast do not cause these exceptions during shutdown.

Replication: Value not preserved across multiple remappings (20110)

Errors maybe reported during compaction of the backup partition that indicated a delta had been found with no value or that a delta can't be applied. This caused compaction to fail and the server to shutdown. This problem has now been resolved.

Server: Session queue threshold notifications have been corrected to work with conflation and TIDY_ON_UNSUBSCRIBE (19720)

Due to a bug, threshold notifications could fail to be generated if conflation or TIDY_ON_UNSUBSCRIBE removed multiple messages from a session's queue. The bug has been corrected in this release.

Server: NullPointerException in SessionPropertiesDispatcherImpl - Race condition (20306)

A race condition led to a possible NullPointerException in SessionPropertiesDispatcherImpl being reported on the server. This has now been resolved.

Topics: VALIDATE_VALUES not respected when adding, setting and creating update stream (20210)

The VALIDATE_VALUES topic property was not respected when adding and setting a topic using an update stream. Support for the topic property has been added to this code path.

Topics: The TopicUpdate method createUpdateStream not checking the topic and value types (20211)

The TopicUpdate method createUpdateStream documented that the topic and value types were checked for consistency. This check was not performed. The implementation has been brought inline with the documentation.

Topics: IncompatibleTopicStateException performing allowed replacement using addAndSet (20235)

It should be possible to replace topics recovered from persistence with topics with a different topic specification using the 'TopicUpdate.addAndSet' method and update streams that can create topics. These operations were failing. This has been corrected so topics are replaced as expected.

Topics: Conflation causes loss of time series topic events (20341)

Conflation of time series topics causes subscribers to miss events. This works against the expectation that time series topics should provide the full history of the topic. Conflation will no longer be supported for time series topics. The default conflation policy for time series topics is now 'off'. Attempting to create time series topics with conflation policies of 'conflate' and 'always' will now fail with an invalid specification error. It is still possible to set the conflation policy to 'unsubscribe'.

Topics: Multiple deltas may be composed incorrectly during conflation (20363)

During conflation binary deltas for the same topic may be composed together. This could result in incorrect deltas. These incorrect deltas would then be published to clients. When the clients applied these deltas they might see either an error or corrupted data. The server should now compose deltas correctly.

Topics: Conflation may cause a duplicate value to be published (20393)

Conflation could cause subscribers to see duplicate values. Conflation may replace queued messages with the latest value for the topic. When this happened the server could still queue a message for that topic. This could result in the subscriber receiving a duplicate or corrupt value. The server will now correctly not queue a message after conflating queued messages to the latest value.

Topics: Split path pattern topic selectors with disjunctions and descendant qualifiers are not evaluated correctly (20475)

Due to a bug introduced in Diffusion 6.2.4, split path pattern topic selectors containing disjunctions and descendant qualifiers (e.g. ?x|y//) were not evaluated correctly. The bug has been fixed in this release.

Web Server/HTTP: The web server does not cache large resources correctly (19918)

In previous releases, the web server did not cache large resources correctly (files larger than 1 MiB). This affected the Diffusion console, which would always be downloaded from the server rather than being served from browser cache. The bug has been fixed in this release.

Known Issues

Console: Console shows fractional users connected (20482)

The concurrent clients and client connections graphs on the console can display fractional users.

JavaScript Client: Interface 'Result<R>' incorrectly extends interface 'Promise<R>' error (22566)

Compiling TypeScript code results in an error about a bad definition in diffusion.d.ts. As a workaround, you can suppress the error by adding "skipLibCheck": true to tsconfig.json. This issue has been resolved in versions 6.4 and above.

Publisher API: Rapid adds and deletes of topics can cause TopicTreeListener notifications to arrive out of order (20333)

Under some circumstances, when the same topic is added and removed very rapidly, it is possible for added and removed notifications to be delivered to the TopicTreeListener out of order. For example, several adds could arrive before a remove. Even the very rapid addition and removal of different topics may not notify in the exact order that the requests were made, although this is only likely to cause an issue where processing of branches overlap. Users should be aware of this possibility, but there is no immediate plan to resolve this issue, as the cases where it would have an impact are thought to be extremely rare.

Publisher API: TopicTreeListener notifications incorrect for topic detach/attach operations (20483)

Since release 5.9, unbound topic tree nodes (nodes without topics) can be created by the standard API. When a hierarchic topic A/B/C is created, nodes A and A/B will have no associated topic. For backwards compatibility, when such a topic is created by a publisher, A and A/B will be stateless topics. The Publisher API will see such unbound nodes as if they had a stateless topic, even though there is no usable topic at the node. For consistency, when the standard API adds a topic to an unbound node, the TopicTreeListener is notified of a remove (the notional stateless topic) followed by an add (of the newly attached topic) and when a topic is removed from a node (as opposed to the topic branch being removed) the TopicTreeListener is notified of a remove (the previous topic) followed by an add (of a notional stateless topic). The problem with this is that in both cases the remove notification suggests that the whole branch has been removed when in fact it has not. It would be more correct if the remove was notified on the TopicDeletionListener, but the fact that these are different interfaces makes this impractical. This may be resolved in a future release by rationalizing the notifications into a single notification interface.