Data Types¶
Data types are containers for specific formats of of topic data and messages sent between Diffusion clients and servers.
Usage¶
from diffusion import datatypes
# the next two lines are equivalent
string_type = datatypes.get("STRING")
string_type = datatypes.STRING
string_instance = string_type("message")
# a data type can be instantiated directly:
string_instance = datatypes.STRING("message")
Available Data Types¶
diffusion.datatypes.BINARY
module-attribute
¶
Binary datatype alias, points to BinaryDataType
diffusion.datatypes.DOUBLE
module-attribute
¶
Double datatype alias, points to DoubleDataType
diffusion.datatypes.INT64
module-attribute
¶
Int64 datatype alias, points to Int64DataType
diffusion.datatypes.STRING
module-attribute
¶
String datatype alias, points to StringDataType
diffusion.datatypes.JSON
module-attribute
¶
Json datatype alias, points to JsonDataType
diffusion.datatypes.TIME_SERIES
module-attribute
¶
TimeSeries datatype alias, points to TimeSeriesEventDataType
Unimplemented Data Types¶
To be added when appropriate.
diffusion.datatypes.RECORD_V2
module-attribute
¶
Record V2 datatype alias, points to RecordDataType
diffusion.datatypes.UNKNOWN
module-attribute
¶
Unknown datatype alias, points to UnknownDataType
diffusion.datatypes.complex.RecordDataType ¶
diffusion.datatypes.complex.UnknownDataType ¶
Generic base classes¶
diffusion.datatypes.foundation.datatype.DataType ¶
Generic parent class for all data types implementations.
__init__ ¶
Initialise the datatype value
Parameters:
Name | Type | Description | Default |
---|---|---|---|
value |
typing.Optional[typing.Any]
|
the value to initialise the datatype with |
required |
read_value
classmethod
¶
Read the value from a binary stream.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
stream |
io.BytesIO
|
Binary stream containing the serialised data. |
required |
Returns:
Type | Description |
---|---|
Optional['DataType']
|
An initialised instance of the DataType. |
validate ¶
Check the current value for correctness.
Raises:
Type | Description |
---|---|
`InvalidDataError`
|
If the value is invalid. By default there is no validation. |
diffusion.datatypes.foundation.abstract.AbstractDataType ¶
Bases: DataType
, typing.Generic[TS_T, ValueType, RealValue]
serialised_value
property
¶
Return the sequence of values ready to be serialised.
It is assumed that the serialisation will use the
serialised-value
serialiser.
with_properties
instance-attribute
class-attribute
¶
A class property that returns the type of this class's appropriate TopicSpecification class, ready for instantiation with the relevant parameters.
See Also
write_value ¶
Write the value into a binary stream.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
stream |
BytesIO
|
Binary stream to serialise the value into. |
required |
to_bytes ¶
Convert the value into the binary representation.
Convenience method, not to be overridden
read_value
classmethod
¶
Read the value from a binary stream.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
stream |
BytesIO
|
Binary stream containing the serialised data. |
required |
Returns:
Type | Description |
---|---|
Optional[AbstractDataType]
|
An initialised instance of the DataType. |
from_bytes
classmethod
¶
Convert a binary representation into the corresponding value.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
data |
bytes
|
Serialised binary representation of the value. |
required |
Returns:
Type | Description |
---|---|
Optional[A_T]
|
An initialised instance of the DataType. |
encode
classmethod
¶
Convert a value into the corresponding binary representation.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
value |
Any
|
Native value to be serialised |
required |
Returns:
Type | Description |
---|---|
bytes
|
Serialised binary representation of the value. |
decode
classmethod
¶
Convert a binary representation into the corresponding value.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
data |
bytes
|
Serialised binary representation of the value. |
required |
Returns:
Type | Description |
---|---|
Any
|
Deserialised value. |
set_from_bytes ¶
Convert bytes and set the corresponding value on the instance.
diffusion.datatypes.foundation.abstract.WithProperties ¶
Bases: typing.Generic[TS_T, ValueType]
Primitive datatypes¶
diffusion.datatypes.primitives ¶
RawDataTypes
module-attribute
¶
Raw data types that are used to populate Diffusion datatypes
PrimitiveDataTypes
module-attribute
¶
Primitive diffusion data types.
PrimitiveDataTypesClasses
module-attribute
¶
PrimitiveDataTypesClasses = typing.Union[typing.Type[BINARY], typing.Type[DOUBLE], typing.Type[INT64], typing.Type[STRING]]
Classes of primitive Diffusion data types.
doubledatatype ¶
DoubleDataType ¶
Bases: PrimitiveDataType[float]
Data type that supports double-precision floating point numbers.
(Eight-byte IEEE 754)
The integer value is serialized as CBOR-format binary. A serialized value can be read as a JSON instance.
int64datatype ¶
Int64DataType ¶
Bases: PrimitiveDataType[int]
Data type that supports 64-bit, signed integer values.
The integer value is serialized as CBOR-format binary. A serialized value can be read as JSON instance.
validate ¶
Check the current value for correctness.
Raises:
Type | Description |
---|---|
`InvalidDataError`
|
If the value is invalid. |
primitivedatatype ¶
PrimitiveDataType ¶
Bases: typing.Generic[T]
, AbstractDataType[TopicSpecification['PrimitiveDataType'], 'PrimitiveDataType[T]', typing.Optional[T]]
validate ¶
Check the current value for correctness.
Raises:
Type | Description |
---|---|
`InvalidDataError`
|
If the value is invalid. |
stringdatatype ¶
StringDataType ¶
Bases: PrimitiveDataType[str]
String data type.
The string value is serialized as CBOR-format binary.
binarydatatype ¶
BinaryDataType ¶
Bases: PrimitiveDataType[bytes]
Data type that supports arbitrary binary data.
decode
classmethod
¶
Convert a binary representation into the corresponding value.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
data |
bytes
|
Serialised binary representation of the value. |
required |
Returns:
Type | Description |
---|---|
bytes
|
Deserialised value. |
Complex datatypes¶
diffusion.datatypes.complex.JsonDataType ¶
Bases: AbstractDataType[TopicSpecification['JsonDataType'], 'JsonDataType', JsonTypes]
JSON data type implementation.
validate ¶
Check the current value for correctness.
Raises:
Type | Description |
---|---|
`InvalidDataError`
|
If the value is invalid. |
diffusion.datatypes.complex.ComplexDataTypes
module-attribute
¶
Complex Diffusion data types.
diffusion.datatypes.complex.ComplexDataTypesClasses
module-attribute
¶
diffusion.datatypes.complex.ComplexDataTypesClasses = typing.Union[typing.Type[JSON], typing.Type[RECORD_V2]]
Classes of complex Diffusion data types.
Time Series¶
diffusion.datatypes.timeseries ¶
TopicSpecification ¶
Bases: typing.Generic[T]
, topic_specification.TopicSpecification[T]
Time Series Topic Specification class
TIME_SERIES_EVENT_VALUE_TYPE
instance-attribute
class-attribute
¶
Specifies the event data type for a time series topic.
TIME_SERIES_RETAINED_RANGE
instance-attribute
class-attribute
¶
Key of the topic property that specifies the range of events retained by a time series topic.
When a new event is added to the time series, older events that fall outside of the range are discarded.
If the property is not specified, a time series topic will retain the ten most recent events.
Time series range expressions¶
The property value is a time series range expression string composed of one or more constraint clauses. Constraints are combined to provide a range of events from the end of the time series.
limit
constraint¶
A limit constraint specifies the maximum number of events from the end of the time series.
last
constraint¶
A last constraint specifies the maximum duration of events from the end of the time series. The duration is expressed as an integer followed by one of the following time units.
ms
- millisecondss
- secondsh
- hours
If a range expression contains multiple constraints, the constraint that selects the smallest range is used.
Property value | Meaning |
---|---|
limit 5 |
The five most recent events |
last 10s |
All events that are no more than ten seconds older than the latest event |
last 10s limit 5 |
The five most recent events that are no more than ten seconds older than the latest event |
Range expressions are not case sensitive:
is equivalent toLIMIT 5 LAST 10S
.
Since 6.8.3
TIME_SERIES_SUBSCRIPTION_RANGE
instance-attribute
class-attribute
¶
Key of the topic property that specifies the range of time series topic events to send to new subscribers.
The property value is a time series range expression, following the format used for TIME_SERIES_RETAINED_RANGE.
If the property is not specified, new subscribers will be sent the latest event if delta streams are enabled and no events if delta streams are disabled. See the description of Subscription range in the {@link TimeSeries time series feature} documentation.
Since 6.8.3
CONFLATION
instance-attribute
class-attribute
¶
TimeSeries conflation policy is restricted to the above.
TimeSeriesWithProperties ¶
Bases: typing.Generic[T]
, WithProperties[TopicSpecification[T], Event]
TimeSeriesEventDataType ¶
Bases: typing.Generic[VT]
, AbstractDataType[TopicSpecification['TimeSeriesEventDataType[VT]'], Event[VT], Event[VT]]
A data type for time series events
with_properties
instance-attribute
class-attribute
¶
Returns a Topic Specification class filled with this type and accepting the relevant parameters
TimeSeriesDataType ¶
Bases: AbstractDataType
Time series data type implementation.
of
classmethod
¶
Provide a Time Series datatype with the given Event[VT] value type.
Please use TimeSeries.of rather than this function to obtain Time Series datatypes.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
val_type |
typing.Type[VT]
|
the type of value that events will contain. |
required |
Returns:
Type | Description |
---|---|
typing.Type[TimeSeriesEventDataType[VT]]
|
The relevant Time Series data type. |
time_series_event_metadata ¶
EventMetadata ¶
Bases: MarshalledModel
Time series event metadata.
sequence
instance-attribute
¶
Sequence number identifying this event within its time series. Assigned by the server when the event is created.
Sequence numbers are unique within a time series. Each event appended to a time series is assigned a sequence number that is is equal to the sequence number of the preceding event plus one.
timestamp
instance-attribute
¶
Event timestamp. Assigned by the server when the event is created.
Events do not have unique timestamps. Events with different sequence numbers may have the same timestamp.
Subsequent events in a time series usually have timestamps that are greater or equal to the timestamps of earlier events, but this is not guaranteed due to changes to the time source used by the server.
Timestamps represent the difference, measured in milliseconds, between the time the server added the event to the time series and midnight, January 1, 1970 UTC
author
instance-attribute
¶
Server-authenticated identity of the session that created the event.
If the session that created the event was not authenticated, the author will be an empty string.
time_series_event ¶
EventType ¶
Bases: enum.IntEnum
Event type constants
Event ¶
Bases: typing.Generic[VT]
, MarshalledModel
Implementation of a time series event
metadata
instance-attribute
class-attribute
¶
Event metadata
value
instance-attribute
class-attribute
¶
Value of event payload
create
classmethod
¶
create(metadata: EventMetadata, original_event: typing.Optional[EventMetadata], value: typing.Optional[VT_other], value_type: typing.Type[VT_other]) -> Event[VT_other]
Static factory to create a new Event instance
Parameters:
Name | Type | Description | Default |
---|---|---|---|
metadata |
EventMetadata
|
the metadata of the event |
required |
original_event |
typing.Optional[EventMetadata]
|
the original event |
required |
value |
typing.Optional[VT_other]
|
the value of the event |
required |
value_type |
typing.Type[VT_other]
|
the type of the value |
required |
Returns:
Type | Description |
---|---|
Event[VT_other]
|
a new Event instance |
with_value ¶
OriginalEvent ¶
types ¶
TimeSeriesValueTypeClasses
module-attribute
¶
Possible typing.Type values for a Time Series Value
TimeSeriesValueType
module-attribute
¶
Possible types for a Time Series Value
RawComplexDataTypes
module-attribute
¶
Types that could be JSON
TimeSeriesValueTypeOrRaw
module-attribute
¶
TimeSeriesValueTypeOrRaw = typing.Optional[typing.Union[TimeSeriesValueType, RawDataTypes, RawComplexDataTypes, StrictBool]]
Time Series Value Type parameter
VT_argtype
module-attribute
¶
Time Series Value Type parameter (TypeVar)
VT
module-attribute
¶
Possible types for a Time Series Value (TypeVar)
VT_other
module-attribute
¶
Possible types for a Time Series Value conversion target