Metrics | Documentation

Methods of accessing metrics

There are multiple ways to access the metrics. As of Diffusion 6.3, the same information is available through each access method.

Note: In previous versions of Diffusion, metrics were sometimes referred to as "statistics".

Web console metrics: The metrics are available through the Diffusion web console. This is the most convenient way to access metrics for development and testing purposes, but does not support aggregating metrics across multiple servers or recording and retrieving historical data. JMX or Prometheus access are more suitable for production systems.
MBeans for JMX: Diffusion registers MBeans with the Java Management Extensions (JMX) service. This enables monitoring of the metrics using the JMX tools that are available from a range of vendors.
Prometheus: Diffusion provides endpoints for the Prometheus monitoring system. To use Prometheus, your Diffusion server needs to have a Commercial with Scale & Availability license, or an evaluation license such as the Community Evaluation license. See License types for more information.

Accessing metrics

The metrics can be accessed in the following recommended ways:

As MBeans, using a JMX tool, such as VisualVM or JConsole. See the table below for MBean interfaces. For more information, see Using Java VisualVM or Using JConsole.
Using the Diffusion management console. For more information, see Diffusion management console.
As Prometheus endpoints at http://localhost:8080/metrics, provided you have a suitable license. If not accessing from the same machine as the Diffusion server, replace localhost with the IP address or hostname.

Collecting custom metrics using metric collectors

A metric collector is a way to collect metrics for a particular set of topics or sessions, configured by you.

You can use the Diffusion web console or JMX to define metric collectors. See Configuring metrics for details.

Collected metrics are published to the console, JMX and optionally via Prometheus.

Counters and gauges

Metrics are divided into counters and gauges.

Counter metric: A counter is a cumulative metric, which reports a value since the server was started. A counter metric will always go up over a server's lifetime. For example, the total number of bytes received by the server is a counter.
Gauge metric: A gauge is a metric which reports the current value of a metric. A gauge value can go up or down. For example, the number of connected sessions is a gauge.

Built-in metrics

This section describes the built-in metrics that are always available, aside from any metric collectors you may have created.

Metrics are not persisted between server restarts. Restarting the server will set all counter metrics back to zero.

The following is a list of all the top level statistics and their attributes.

Table 1. Metrics provided by Diffusion
Metric name	Type	Description	Prometheus export
Log metrics	LogMetrics MBean
count	Counter	Number of log events for a given ID code and severity level (levels are `error`, `warn`, `info`, `debug`, `trace`).	diffusion_log_events_count{code="PUSH-12345",level="warn"}
Network metrics	NetworkMetrics MBean
inbound_bytes	Counter	Data received from the network in bytes.	diffusion_network_inbound_bytes
outbound_bytes	Counter	Data sent to the network in bytes.	diffusion_network_outbound_bytes
Session metrics	SessionMetrics MBean
connected	Gauge	Number of connected sessions.	diffusion_sessions_connected
inbound_bytes	Counter	Session data received from the network in bytes.	diffusion_sessions_inbound_bytes
inbound_messages	Counter	Session data received from the network in messages.	diffusion_sessions_inbound_messages
open	Gauge	Number of open sessions.	diffusion_sessions_open
outbound_bytes	Counter	Session data sent to the network in bytes.	diffusion_sessions_outbound_bytes
outbound_messages	Counter	Session data sent to the network in messages.	diffusion_sessions_outbound_messages
peak	Gauge	Peak number of sessions.	diffusion_sessions_peak
total	Counter	Total sessions opened.	diffusion_sessions_total
Topic metrics	TopicMetrics MBean
count	Gauge	Current number of topics.	diffusion_topics_count
total	Counter	Total number of topics.	diffusion_topics_total
bytes	Gauge	The value data stored by the topics, in bytes.	diffusion_topics_bytes
subscriptions	Gauge	Number of direct subscriptions to the topics.	diffusion_topics_subscriptions
subscribers	Gauge	Number of sessions subscribed to one or more topics.	diffusion_topics_subscribers
subscriber_updates	Counter	Number of updates sent to subscribers.	diffusion_topics_ subscriber_updates
subscriber_update_bytes	Counter	Data sent to subscribers, before message compression, in bytes.	diffusion_topics_ subscriber_update_bytes
subscriber_update_ compressed_bytes	Counter	Data sent to subscribers, after message compression, in bytes.	diffusion_topics_subscriber _update_compressed_bytes
value_updates	Counter	Number of updates to a topic that provide a full value.	diffusion_topics_value_updates
delta_updates	Counter	Number of updates to a topic that provide a partial value.	diffusion_topics_delta_updates
value_bytes	Counter	On each change of topic value, this metric increases by the size of the new value.	diffusion_topics_value_bytes
delta_bytes	Counter	On each change of topic value, this metric increases by the size of an internal delta representing the difference the previous and new values.	diffusion_topics_delta_bytes

Delta compression ratio

value_bytes and delta_bytes can be used to capture the theoretical delta compression ratio of the application data flowing through the topics. Both the console and the JMX MBean perform this calculation. The ratio is a value between 0 and 1. The closer the ratio is to 1, the more benefit the application data will obtain from delta streaming. If value_bytes is 0, there have been no updates, so the delta compression ratio is reported as zero. Otherwise it is calculated as:

1 - delta_bytes / value_bytes

Delta streaming is enabled for subscriptions by default, but can be disabled on a per-topic basis using the PUBLISH_VALUES_ONLY topic property. If delta streaming is enabled, a stable set of subscribers remain connected, and no session has a significant backlog (so conflation is not applied), the following relationship should hold:

subscriber_update_bytes ≅ delta_updates x subscribers

Delta streaming can also be used to update topic values. If the delta compression ratio is high, but delta_updates is zero (or low, relative to value_updates), consider whether your application can use the stateful update stream API to take advantage of delta streaming.

Log metrics

Log metrics record information about server log events. Separate metrics are kept for each unique pair of log code and log severity level that has been logged.

The log severity levels are: error, warn, info, debug, trace.

A JMX MBean is created for each pair of log code and log severity that has been logged at least once.

Here is an example MBean name: com.pushtechnology.diffusion:type=LogMetrics,server="server_name",level=warncode=PUSH-12345

Session metrics versus network metrics

The network inbound_bytes and outbound_bytes metrics include bytes that are not counted by the equivalent session metrics.

The session metrics include bytes from transport framing and all session traffic (including additional HTTP traffic from long polling).

The network metrics include all bytes included in the session metrics as well as non-session bytes such as:

TLS overhead
Web server traffic (for example, browsers downloading the web console pages)
Rejected connection attempts

Metrics in the Publisher API

Publisher metrics, client instance metrics, and topic instance metrics have all been removed.

Consequently the PublisherStatistics, ClientStatistics and TopicStatistics interfaces provide no information. These interfaces are deprecated and will be removed in a future release.

Limited server metrics are still available through the Publisher API using the ServerStatistics interface.

For more information, see the Java API documentation.