This document is a release candidate of the Common Trace Format (CTF) version 2 specification (CTF2-SPEC-2.0).
|
RFC 2119
The key words MUST, MUST NOT, REQUIRED,
SHOULD, SHOULD NOT, MAY, and OPTIONAL in this document, when
emphasized, are to be interpreted as described in
RFC 2119.
|
1. Revision history
| Document | Publication date | Changes |
|---|---|---|
CTF2‑SPECRC‑3.0 |
17 December 2021 |
Add the optional Such a minimum alignment is needed when the alignment requirement of contained array field elements wouldn’t be enough. For example, a producer could write a single 32-bit-aligned, 32-bit little-endian integer value, but use the following static-length array field class for consumers: While the producer writes a single integer value, consumers decode said datum as an array of 32 individual flags (booleans). Update the Alignment procedure section accordingly. |
CTF2‑SPECRC‑2.0 |
9 December 2021 |
|
CTF2‑SPECRC‑1.0 |
25 November 2021 |
Initial CTF 2 specification release candidate. |
2. What’s CTF 2?
The Common Trace Format version 2 is a binary trace format designed to be very fast to write without compromising great flexibility.
The intention of CTF 2 is that applications written in any programming language, and running on any system (be it Linux or bare metal, for example), can generate traces natively.
A CTF 2 trace has all its data streams described by a metadata stream. Given the rich set of supported data field types, this makes it possible for a CTF 2 producer to append data structures as is to data streams without further “data massaging”. Indeed, the size, alignment, and byte order of fixed-length fields are all configurable parameters within the metadata stream.
CTF 2 is transport agnostic: this document doesn’t specify how to transport or store CTF 2 streams. Other documents can specify such conventions, and conform CTF 2 producers and consumers may or may not adhere to them.
CTF 2 is a major revision of CTF 1, bringing many improvements, such as:
-
Using JSON text sequences for the metadata stream.
-
Adding auxiliary streams (also JSON text sequences).
-
Simplifying the metadata stream.
-
Adding new field classes.
-
Using roles instead of reserved structure member names to identify “special” fields.
and more, while remaining backward compatible at the data stream level.
3. Common definitions
Common CTF 2 definitions:
- Byte
-
A group of eight bits operated on as a unit.
The bits are indexed such that, if the byte represents an 8-bit unsigned integer, bit 0 is the least significant and bit 7 is the most significant.
- Class
-
A set of values (instances) which share common properties.
For example, a fixed-length unsigned integer field class with an 8-bit length property is the set of the all the fixed-length unsigned integer fields from binary
00000000to11111111(integers 0 to 255).This specification often states that some class describes instances. For example, an event record class describes event records.
- Consumer
-
A software or hardware system which consumes (reads) the streams of a trace.
A trace consumer is often a trace viewer or a trace analyzer.
- Namespace
-
A string of which the purpose is to avoid naming conflicts.
This document doesn’t specify the format of a namespace. A producer SHOULD use a URI, or at least include a domain name owned by the organization defining the objects under a namespace.
The stdnamespace is reserved for the CTF 2 specification. - Producer
-
A software or hardware system which produces (writes) the streams of a trace.
A trace producer is often a tracer.
- Sequence
-
A set of related items which follow each other in a particular order.
- Stream
4. Trace composition
A trace is:
-
One metadata stream.
-
One or more data streams.
-
Zero or more auxiliary streams.
As a reminder, this specification defines a stream as a sequence of bytes.
| This document doesn’t specify how to transport or store CTF 2 streams. A producer could serialize all streams as a single file on the file system, or it could send the streams over the network using TCP, to name a few examples. |
4.1. Metadata stream (overview)
A metadata stream describes trace data streams with JSON objects.
A metadata stream describes things such as:
-
The class of the data stream default clocks.
-
The names of event record classes.
-
The classes of event record fields.
Multiple traces MAY share the same metadata stream: a given trace MAY contain specific information in its own auxiliary streams.
See Metadata stream for the full metadata stream specification.
4.2. Data stream
In the metadata stream, a data stream class describes data streams.
A packet MUST contain one or more bytes of data.
Although a packet MAY contain padding (garbage data) at the end itself, from the point of view of a data stream, there’s no padding between packets. In other words, the byte following the last byte of a packet is the first byte of the next packet.
A data stream MAY have, conceptually:
- One default, monotonic clock
-
Described by a clock class in the metadata stream.
Packets and event records MAY contain snapshots, named timestamps, of the default clock of their data stream.
- One counter of discarded event records
-
Indicates the number of event records which the producer needed to discard for different reasons.
For example, a tracer could discard an event record when it doesn’t fit some buffer and there’s no other available buffer.
A packet MAY contain a snapshot of this counter.
See Data stream decoding procedure to learn how to decode a CTF 2 data stream.
4.2.1. Packet
A packet is a segment of a data stream.
A packet contains a sequence of data fields or padding (garbage data). In the metadata stream, field classes describe data fields.
A packet P, contained in a data stream S, contains, in this order:
-
OPTIONAL: A header structure field, described at the trace class level in the metadata stream, which contains, in this order:
-
OPTIONAL: A packet magic number field (0xc1fc1fc1, or 3254525889).
-
In any order:
-
OPTIONAL: A trace class UUID field.
-
OPTIONAL: One or more fields which contain the numeric ID of the class of S.
-
OPTIONAL: One or more fields which contain the numeric ID of S.
-
-
-
OPTIONAL: A context structure field, described at the data stream class level in the metadata stream, which contains, in any order:
-
OPTIONAL: A field which contains the total size of P, in bits (always a multiple of 8).
-
OPTIONAL: A field which contains the content size of P, in bits.
-
OPTIONAL: A field which contains the beginning timestamp of P.
-
OPTIONAL: A field which contains the end timestamp of P.
-
OPTIONAL: A field which contains a snapshot of the discarded event record counter of S at the end of P.
-
OPTIONAL: A field which contains the sequence number of P within S.
-
OPTIONAL: User fields.
-
-
Zero or more event records.
A packet MUST contain one or more bytes of data.
A packet MAY contain padding (garbage data) after its last event record. The size of this padding is the difference between its total size and its content size (as found in its context structure field).
Packets are independent of each other: if one removes a packet from a data stream, a consumer can still decode the whole data stream. This is why:
-
Packets MAY contain snapshots of the discarded event record counter of their data stream.
-
Packets and event records MAY contain timestamps which are snapshots of the default clock of their data stream.
If the packet context fields of the packets of a data stream contain a packet sequence number field, a consumer can recognize missing packets.
See Packet decoding procedure to learn how to decode a CTF 2 packet.
4.2.2. Event record
An event record is the result of a producer writing a record with OPTIONAL user data when an event occurs during its execution.
A packet contains zero or more event records.
An event record class describes the specific parts of event records.
An event record E, contained in a data stream S, contains, in this order:
-
OPTIONAL: A header structure field, described at the data stream class level in the metadata stream, which contains, in any order:
-
OPTIONAL: One or more fields which contain the numeric ID of the class of E which has the class of S as its parent.
-
OPTIONAL: One or more fields which contain a timestamp or a partial timestamp.
-
-
OPTIONAL: A common context structure field, described at the data stream class level in the metadata stream, which contains user fields.
-
OPTIONAL: A specific context structure field, described at the event record class level in the metadata stream, which contains user fields.
-
OPTIONAL: A payload structure field, described at the event record class level in the metadata stream, which contains user fields.
An event record MUST contain one or more bits of data.
The default clock timestamp of an event record, that is, the value of the default clock of its data stream after its header field, if any, is encoded/decoded MUST be greater than or equal to the default clock timestamp of the previous event record, if any, within the same data stream.
See Event record decoding procedure to learn how to decode a CTF 2 event record.
4.3. Auxiliary stream
An auxiliary stream is a JSON text sequence, as specified by RFC 7464, which contains extra, structured information about the trace which doesn’t fit the data stream model.
Each element of an auxiliary stream is a JSON object which has a single property:
| Name |
Namespace of the auxiliary stream. |
| Value |
A JSON value. |
my.tracer namespace.{
"my.tracer": {
"version": [1, 3, 2],
"session-name": "amqui"
}
}42.{
"328c7a2d-a959-4f60-bd22-cca74359326f": 42
}4.3.1. Trace environment
To remain backward compatible with CTF 1, a trace MAY contain an
auxiliary stream having an element with the std namespace which
contains trace environment variables under the environment property.
The trace environment variables are a single JSON object where each property is:
| Name |
Trace environment variable name. |
| Value |
Trace environment variable value (any JSON value). |
This document doesn’t specify trace environment variable names.
std auxiliary stream element with trace environment variables.{
"std": {
"environment": {
"hostname": "amqui",
"domain": "kernel",
"sysname": "Linux",
"kernel_release": "4.12.12-1-ARCH",
"kernel_version": "#1 SMP PREEMPT Sun Sep 10 09:41:14 CEST 2017",
"tracer_name": "lttng-modules",
"tracer_major": 2,
"tracer_minor": 10,
"tracer_patchlevel": 0
}
}
}5. Metadata stream
A metadata stream is a JSON text sequence, as specified by RFC 7464, of fragments.
Together, the fragments of a metadata stream contain all the information about the data streams of one or more traces.
| Name | Type | Description | Required? | Default |
|---|---|---|---|---|
|
JSON string |
Type of F. The value of this property MUST be one of:
|
Yes |
|
|
User attributes of F. |
No |
|
|
|
Extensions of F. For any fragment except a preamble fragment, any extension which exists under this property MUST also be declared in the preamble fragment of the same metadata stream. |
No |
|
The metadata stream is a JSON text sequence of fragments instead of a single JSON object containing nested objects to enable real-time, or “live”, tracing: a consumer can always decode event records having known event record classes while a producer can always add new event record classes to a data stream class by appending additional fragments to the metadata stream. Once a producer appends a fragment to a metadata stream, the fragment is considered “frozen”, in that it never needs to change.
A metadata stream:
-
MUST start with a preamble fragment.
-
MUST contain exactly one preamble fragment.
-
MAY contain one trace class fragment.
-
MUST contain one or more data stream class fragments which MUST follow the trace class fragment, if any.
-
MAY contain one or more event record class fragments which MUST follow their parent data stream class, if any.
In the sample below, the string <RS> represents a single record
separator character (U+001E) and the string [...] represents
continuation.
<RS>{
"type": "preamble",
"version": 2
}
<RS>[...]
|
This section doesn’t specify how a metadata stream translates into data stream encoding and decoding rules; it only describes objects and their properties. See Data stream decoding procedure to learn how to decode a data stream. |
5.1. UUID
Both a trace class fragment and a clock class fragment MAY have a UUID property.
Within a metadata stream, a UUID is a JSON array of 16 JSON integers which are the numeric values of the 16 bytes of the UUID.
e53e0ab8-50a1-4f0a-b710-b5f0bba9c4ac UUID.[229, 62, 10, 184, 80, 161, 79, 10, 183, 16, 181, 240, 187, 169, 196, 172]5.2. Extensions
A producer MAY add extensions to many metadata stream JSON objects.
The purpose of an extension is to add core features to CTF 2 or to modify existing core features, as specified by this document. In other words, an extension MAY alter the format itself.
This document doesn’t specify what an extension exactly is.
The preamble fragment of the metadata stream contains extension declarations:
-
Any extension in metadata stream objects MUST be declared, by namespace and name, in the preamble fragment.
Declaring an extension is said to enable it.
-
If a consumer doesn’t support any declared extension, it MUST NOT consume the data streams of the trace.
The consumer SHOULD report unsupported extensions as an error.
Extensions are a single JSON object, where each property is:
| Name | |
| Value |
| Name |
An extension name |
| Value |
A JSON value |
The metadata stream JSON objects which MAY contain extensions as their
extensions property are:
-
Any fragment.
An extension in the preamble fragment also makes it declared/enabled.
-
Any field class.
{
"my.tracer": {
"piano": {
"keys": 88,
"temperament": "equal"
},
"ramen": 23
},
"abc/xyz": {
"sax": {
"variant": "alto"
}
}
}5.3. User attributes
A producer MAY add custom user attributes to many metadata stream JSON objects.
This document doesn’t specify what a user attribute exactly is.
Unlike extensions, a consumer MUST NOT consider user attributes to decode data streams.
User attributes are a single JSON object, where each property is:
| Name | |
| Value |
A JSON value |
The metadata stream JSON objects which MAY contain user attributes
as their user-attributes property are:
-
Any fragment.
-
Any field class.
{
"my.tracer": {
"max-count": 45,
"module": "sys"
},
"abc/xyz": true
}5.4. Field classes
A field class describes fields, that is, sequences of bits as found in a data stream.
A field is a field class instance.
This document specifies the following types of field classes:
- Abstract field classes
-
One cannot use the following field classes directly: they are bases for other, concrete field classes:
- Fixed/static-length field classes
- Variable/dynamic-length field classes
- Compound field classes
-
The following field classes contain one or more field classes.
A field class is a JSON object; its properties depend on its type
property.
| Name | Type | Description | Required? | Default |
|---|---|---|---|---|
|
JSON string |
Type of F. The value of this property MUST be one of:
|
Yes |
|
|
Roles of an instance of F. See Trace class fragment and Data stream class fragment which indicate accepted within their root field classes. |
No |
|
|
|
User attributes of F. |
No |
|
|
|
Extensions of F. Any extension which exists under this property MUST also be declared in the preamble fragment of the metadata stream. |
No |
|
The following fragment properties MUST have a structure field class as their value:
- Trace class fragment
-
packet-header-field-class - Data stream class fragment
-
-
packet-context-field-class -
event-record-header-field-class -
event-record-common-context-field-class
-
- Event record class fragment
-
-
specific-context-field-class -
payload-field-class
-
5.4.1. Field location
A field location is a means for a consumer to locate a field which it needs to decode another, subsequent field.
A consumer needs to locate another field to decode instances of the following classes:
- Dynamic-length array field class
- Dynamic-length string field class
- Dynamic-length BLOB field class
-
Needs a fixed-length unsigned integer or variable-length unsigned integer length field.
- Optional field class
-
Needs a fixed-length boolean, fixed-length integer, or variable-length integer selector field.
- Variant field class
-
Needs a fixed-length integer or variable-length integer selector field.
Let T be an anteriorly decoded field which a consumer needs to decode another field S. A field location is a JSON array where, in this order:
-
The first element is the name (JSON string) of a root field from where to start the lookup of T, amongst:
"packet-header""packet-context"Context of the packet of S.
"event-record-header"Header of the event record of S.
"event-record-common-context"Common context of the event record of S.
"event-record-specific-context"Specific context of the event record of S.
"event-record-payload"Payload of the event record of S.
In other words, T MUST be in the same packet or event record as S.
-
The following elements are structure field member names (JSON strings) to follow to locate the target field.
The length of a field location MUST be greater than or equal to two.
See Field location procedure to learn how to locate a field with a field location.
5.4.2. Integer range set
An integer range set is a JSON array of integer ranges.
An integer range set MUST contain one or more integer ranges.
An integer range is a JSON array of two elements:
-
The lower bound of the range (JSON integer, included).
-
The upper bound of the range (JSON integer, included).
An integer range represents all the integer values from the lower bound of the range to its upper bound.
The upper bound of an integer range MUST be greater than or equal to its lower bound.
If both the lower and upper bounds of an integer range are equal, then the integer range represents a single integer value.
[3, 67][-45, 101][42, 42][[3, 67], [-45, 1], [42, 42]]5.4.3. Roles
Some field class instances can have roles.
A role is specific semantics attached to the fields (instances) of a
field class. For example, the packet-magic-number role of a
fixed-length integer field class indicates that the value of its instances MUST be the
packet magic number (0xc1fc1fc1).
Roles are a JSON array of role names (JSON strings).
See Trace class fragment and Data stream class fragment which indicate accepted roles within their root field classes.
5.4.4. Fixed-length bit array field class
A fixed-length bit array field class describes fixed-length bit array fields.
A fixed-length bit array field is a simple array of contiguous bits, without any attached integer type semantics.
The length, or number of bits, of a fixed-length bit array field is a property
(length) of its class.
A fixed-length bit array field class acts as a base of a fixed-length boolean field class, a fixed-length integer field class, and a fixed-length floating point number field class.
| Name | Type | Description | Required? | Default |
|---|---|---|---|---|
|
JSON string |
Type of F. The value of this property MUST be |
Yes |
|
|
JSON integer |
Number of bits of an instance of F. The value of this property MUST be greater than zero. |
Yes |
|
|
JSON string |
Byte order of an instance of F. The value of this property MUST be one of:
|
Yes |
|
|
JSON integer |
Alignment of the first bit of an instance of F relative to the beginning of the packet which contains this instance. The value of this property MUST be a positive power of two. |
No |
|
|
Roles of an instance of F. See Trace class fragment and Data stream class fragment which indicate accepted within their root field classes. |
No |
|
|
|
User attributes of F. |
No |
|
|
|
Extensions of F. Any extension which exists under this property MUST also be declared in the preamble fragment of the metadata stream. |
No |
|
{
"type": "fixed-length-bit-array",
"length": 16,
"byte-order": "little-endian"
}{
"type": "fixed-length-bit-array",
"length": 48,
"byte-order": "big-endian",
"alignment": 32
}{
"type": "fixed-length-bit-array",
"length": 16,
"byte-order": "little-endian",
"user-attributes": {
"my.tracer": {
"is-nice": true
}
}
}5.4.5. Fixed-length boolean field class
A fixed-length boolean field class is a fixed-length bit array field class which describes fixed-length boolean fields.
A fixed-length boolean field is a fixed-length bit array field which has the following semantics:
- If all the bits of the bit array field are cleared (zero)
-
The value of the fixed-length boolean field is false.
- Otherwise
-
The value of the fixed-length boolean field is true.
| Name | Type | Description | Required? | Default |
|---|---|---|---|---|
|
JSON string |
Type of F. The value of this property MUST be |
Yes |
|
|
JSON integer |
Number of bits of an instance of F. The value of this property MUST be greater than zero. Property inherited from the fixed-length bit array field class. |
Yes |
|
|
JSON string |
Byte order of an instance of F. The value of this property MUST be one of:
Property inherited from the fixed-length bit array field class. |
Yes |
|
|
JSON integer |
Alignment of the first bit of an instance of F relative to the beginning of the packet which contains this instance. The value of this property MUST be a positive power of two. Property inherited from the fixed-length bit array field class. |
No |
|
|
Roles of an instance of F. See Trace class fragment and Data stream class fragment which indicate accepted within their root field classes. |
No |
|
|
|
User attributes of F. |
No |
|
|
|
Extensions of F. Any extension which exists under this property MUST also be declared in the preamble fragment of the metadata stream. |
No |
|
{
"type": "fixed-length-boolean",
"length": 16,
"byte-order": "little-endian"
}{
"type": "fixed-length-boolean",
"length": 48,
"byte-order": "big-endian",
"alignment": 32
}{
"type": "fixed-length-boolean",
"length": 16,
"byte-order": "little-endian",
"user-attributes": {
"my.tracer": {
"is-nice": true
}
}
}5.4.6. Abstract integer field class
An abstract integer field class is a base of a fixed-length integer field class and a variable-length integer field class.
This field class is abstract in that it only exists to show the relation between different integer field classes in this document: a packet cannot contain an abstract integer field.
| Name | Type | Description | Required? | Default | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
JSON integer |
Preferred base to display the value of an instance of F. The value of this property MUST be one of:
This property exists to remain backward compatible with CTF 1: it’s not strictly needed to decode an instance of F. |
No |
|
5.4.7. Fixed-length integer field class
A fixed-length integer field class is both an abstract integer field class and a fixed-length bit array field class which describes fixed-length integer fields.
A fixed-length integer field is a fixed-length bit array field which has integer semantics.
If the value of the type property of a fixed-length integer is
"fixed-length-signed-integer", then its instances have the two’s
complement format.
A fixed-length integer field class acts as a base of a fixed-length enumeration field class.
| Name | Type | Description | Required? | Default | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
JSON string |
Type of F. The value of this property MUST be one of:
|
Yes |
|||||||||
|
JSON integer |
Number of bits of an instance of F. The value of this property MUST be greater than zero. Property inherited from the fixed-length bit array field class. |
Yes |
|||||||||
|
JSON string |
Byte order of an instance of F. The value of this property MUST be one of:
Property inherited from the fixed-length bit array field class. |
Yes |
|||||||||
|
JSON integer |
Alignment of the first bit of an instance of F relative to the beginning of the packet which contains this instance. The value of this property MUST be a positive power of two. Property inherited from the fixed-length bit array field class. |
No |
|
||||||||
|
JSON integer |
Preferred base to display the value of an instance of F. The value of this property MUST be one of:
This property exists to remain backward compatible with CTF 1: it’s not strictly needed to decode an instance of F. Property inherited from the abstract integer field class. |
No |
|
||||||||
|
Roles of an instance of F. See Trace class fragment and Data stream class fragment which indicate accepted within their root field classes. |
No |
|
|||||||||
|
User attributes of F. |
No |
|
|||||||||
|
Extensions of F. Any extension which exists under this property MUST also be declared in the preamble fragment of the metadata stream. |
No |
|
{
"type": "fixed-length-unsigned-integer",
"length": 16,
"byte-order": "little-endian"
}{
"type": "fixed-length-signed-integer",
"length": 48,
"byte-order": "big-endian",
"alignment": 32
}{
"type": "fixed-length-unsigned-integer",
"length": 48,
"byte-order": "big-endian",
"preferred-display-base": 16
}{
"type": "fixed-length-signed-integer",
"length": 16,
"byte-order": "little-endian",
"user-attributes": {
"my.tracer": {
"is-nice": true
}
}
}5.4.8. Abstract enumeration field class
An abstract enumeration field class is a base of a fixed-length enumeration field class and a variable-length enumeration field class.
This field class is abstract in that it only exists to show the relation between different enumeration field classes in this document: a packet cannot contain an abstract enumeration field.
An abstract enumeration field class is an abstract integer field class.
An enumeration field is an integer field which MAY have one or more
associated names thanks to the mappings property of its class.
| Name | Type | Description | Required? | Default | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
JSON integer |
Preferred base to display the value of an instance of F. The value of this property MUST be one of:
This property exists to remain backward compatible with CTF 1: it’s not strictly needed to decode an instance of F. Property inherited from the abstract integer field class. |
No |
|
||||||||
|
Mappings of F. The value of this property MUST contain one or more properties. |
Yes |
5.4.8.1. Enumeration field class mappings
Enumeration field class mappings map names to integer range sets.
Enumeration field class mappings are a JSON object, where each property is:
| Name |
Mapping name. |
| Value |
Mapped ranges of integers (integer range set). |
The integer ranges of two given mappings MAY overlap.
Enumeration field class mappings MUST contain one or more properties.
In this example, the fortune and building mappings overlap with the
values 4 and 5, and the building and journal mappings overlap with
the value 80.
{
"fortune": [[3, 67], [-45, 1], [84, 84]],
"building": [[4, 5], [75, 82]],
"journal": [[100, 2305], [80, 80]]
}5.4.9. Fixed-length enumeration field class
A fixed-length enumeration field class is both an abstract enumeration field class and a fixed-length integer field class which describes fixed-length enumeration fields.
A fixed-length enumeration field is a fixed-length integer field which MAY have one or more
associated names thanks to the mappings property of its class.
If the value of the type property of a fixed-length enumeration field class is
"fixed-length-signed-enumeration", then its instances have the two’s
complement format.
| Name | Type | Description | Required? | Default | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
JSON string |
Type of F. The value of this property MUST be one of:
|
Yes |
|||||||||
|
JSON integer |
Number of bits of an instance of F. The value of this property MUST be greater than zero. Property inherited from the fixed-length bit array field class. |
Yes |
|||||||||
|
JSON string |
Byte order of an instance of F. The value of this property MUST be one of:
Property inherited from the fixed-length bit array field class. |
Yes |
|||||||||
|
JSON integer |
Alignment of the first bit of an instance of F relative to the beginning of the packet which contains this instance. The value of this property MUST be a positive power of two. Property inherited from the fixed-length bit array field class. |
No |
|
||||||||
|
JSON integer |
Preferred base to display the value of an instance of F. The value of this property MUST be one of:
This property exists to remain backward compatible with CTF 1: it’s not strictly needed to decode an instance of F. Property inherited from the abstract integer field class. |
No |
|
||||||||
|
Mappings of F. The value of this property MUST contain one or more properties. Property inherited from the abstract enumeration field class. |
Yes |
||||||||||
|
Roles of an instance of F. See Trace class fragment and Data stream class fragment which indicate accepted within their root field classes. |
No |
|
|||||||||
|
User attributes of F. |
No |
|
|||||||||
|
Extensions of F. Any extension which exists under this property MUST also be declared in the preamble fragment of the metadata stream. |
No |
|
{
"type": "fixed-length-unsigned-enumeration",
"length": 16,
"byte-order": "little-endian",
"mappings": {
"apple": [[1, 19]]
}
}{
"type": "fixed-length-signed-enumeration",
"length": 48,
"byte-order": "big-endian",
"alignment": 32,
"mappings": {
"banana": [[-27399, -1882], [8, 199], [101, 101]],
"orange": [[67, 67], [43, 1534]]
}
}{
"type": "fixed-length-unsigned-enumeration",
"length": 8,
"byte-order": "big-endian",
"preferred-display-base": 16,
"mappings": {
"lime": [[3, 3]],
"kiwi": [[8, 8]],
"blueberry": [[11, 11]]
}
}{
"type": "fixed-length-signed-enumeration",
"length": 16,
"byte-order": "little-endian",
"mappings": {
"mango": [[23, 42]]
},
"user-attributes": {
"my.tracer": {
"is-nice": true
}
}
}5.4.10. Fixed-length floating point number field class
A fixed-length floating point number field class is a fixed-length bit array field class which describes fixed-length floating point number fields.
A fixed-length floating point number field is a fixed-length bit array field which has floating point number semantics.
| Name | Type | Description | Required? | Default |
|---|---|---|---|---|
|
JSON string |
Type of F. The value of this property MUST be
|
Yes |
|
|
JSON integer |
Number of bits of an instance of F. The value of this property MUST be one of:
Property inherited from the fixed-length bit array field class. |
Yes |
|
|
JSON string |
Byte order of an instance of F. The value of this property MUST be one of:
Property inherited from the fixed-length bit array field class. |
Yes |
|
|
JSON integer |
Alignment of the first bit of an instance of F relative to the beginning of the packet which contains this instance. The value of this property MUST be a positive power of two. Property inherited from the fixed-length bit array field class. |
No |
|
|
Roles of an instance of F. See Trace class fragment and Data stream class fragment which indicate accepted within their root field classes. |
No |
|
|
|
User attributes of F. |
No |
|
|
|
Extensions of F. Any extension which exists under this property MUST also be declared in the preamble fragment of the metadata stream. |
No |
|
{
"type": "fixed-length-floating-point-number",
"length": 32,
"byte-order": "little-endian"
}{
"type": "fixed-length-floating-point-number",
"length": 64,
"byte-order": "big-endian",
"alignment": 32
}{
"type": "fixed-length-floating-point-number",
"length": 192,
"byte-order": "little-endian",
"user-attributes": {
"my.tracer": {
"is-nice": true
}
}
}5.4.11. Variable-length bit array field class
A variable-length bit array field class describes variable-length bit array fields.
A variable-length bit array field is a sequence of bytes with a variable length which contains an array of bits of which the length is a multiple of 7. A variable-length bit array field is encoded as per LEB128.
A variable-length bit array field class acts as a base of a variable-length integer field class.
| Name | Type | Description | Required? | Default |
|---|---|---|---|---|
|
JSON string |
Type of F. The value of this property MUST be |
Yes |
|
|
Roles of an instance of F. See Trace class fragment and Data stream class fragment which indicate accepted within their root field classes. |
No |
|
|
|
User attributes of F. |
No |
|
|
|
Extensions of F. Any extension which exists under this property MUST also be declared in the preamble fragment of the metadata stream. |
No |
|
{
"type": "variable-length-bit-array"
}{
"type": "variable-length-bit-array",
"user-attributes": {
"my.tracer": {
"is-nice": true
}
}
}5.4.12. Variable-length integer field class
A variable-length integer field class is both an abstract integer field class and a variable-length bit array field class which describes variable-length integer fields.
A variable-length integer field is a variable-length bit array field which has integer semantics.
If the value of the type property of a variable-length integer field class is
"variable-length-signed-integer", then its instances have the two’s
complement format.
A variable-length integer field class acts as a base of a variable-length enumeration field class.
| Name | Type | Description | Required? | Default | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
JSON string |
Type of F. The value of this property MUST be one of:
|
Yes |
|||||||||
|
JSON integer |
Preferred base to display the value of an instance of F. The value of this property MUST be one of:
This property exists to remain backward compatible with CTF 1: it’s not strictly needed to decode an instance of F. Property inherited from the abstract integer field class. |
No |
|
||||||||
|
Roles of an instance of F. See Trace class fragment and Data stream class fragment which indicate accepted within their root field classes. |
No |
|
|||||||||
|
User attributes of F. |
No |
|
|||||||||
|
Extensions of F. Any extension which exists under this property MUST also be declared in the preamble fragment of the metadata stream. |
No |
|
{
"type": "variable-length-unsigned-integer"
}{
"type": "variable-length-signed-integer",
"preferred-display-base": 16
}{
"type": "variable-length-unsigned-integer",
"user-attributes": {
"my.tracer": {
"is-nice": true
}
}
}5.4.13. Variable-length enumeration field class
A variable-length enumeration field class is both an abstract enumeration field class and a variable-length integer field class which describes variable-length enumeration fields.
A variable-length enumeration field is a variable-length integer field which MAY have one or more
associated names thanks to the mappings property of its class.
If the value of the type property of a variable-length enumeration field class is
"variable-length-signed-enumeration", then its instances have the
two’s complement format.
| Name | Type | Description | Required? | Default | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
JSON string |
Type of F. The value of this property MUST be one of:
|
Yes |
|||||||||
|
JSON integer |
Preferred base to display the value of an instance of F. The value of this property MUST be one of:
This property exists to remain backward compatible with CTF 1: it’s not strictly needed to decode an instance of F. Property inherited from the abstract integer field class. |
No |
|
||||||||
|
Mappings of F. The value of this property MUST contain one or more properties. Property inherited from the abstract enumeration field class. |
Yes |
||||||||||
|
Roles of an instance of F. See Trace class fragment and Data stream class fragment which indicate accepted within their root field classes. |
No |
|
|||||||||
|
User attributes of F. |
No |
|
|||||||||
|
Extensions of F. Any extension which exists under this property MUST also be declared in the preamble fragment of the metadata stream. |
No |
|
{
"type": "variable-length-unsigned-enumeration",
"mappings": {
"apple": [[1, 19]]
}
}{
"type": "variable-length-unsigned-enumeration",
"preferred-display-base": 16,
"mappings": {
"lime": [[3, 3]],
"kiwi": [[8, 8]],
"blueberry": [[11, 11]]
}
}{
"type": "variable-length-signed-enumeration",
"mappings": {
"banana": [[-27399, -1882], [8, 199], [101, 101]],
"orange": [[67, 67], [43, 1534]]
},
"user-attributes": {
"my.tracer": {
"is-nice": true
}
}
}5.4.14. Null-terminated string field class
A null-terminated string field class describes null-terminated string fields.
A null-terminated string field is, in this order:
-
Zero or more contiguous non-null (non-zero) bytes which form a UTF-8-encoded string.
-
One null (zero) byte.
| Name | Type | Description | Required? | Default |
|---|---|---|---|---|
|
JSON string |
Type of F. The value of this property MUST be |
Yes |
|
|
Roles of an instance of F. See Trace class fragment and Data stream class fragment which indicate accepted within their root field classes. |
No |
|
|
|
User attributes of F. |
No |
|
|
|
Extensions of F. Any extension which exists under this property MUST also be declared in the preamble fragment of the metadata stream. |
No |
|
{
"type": "null-terminated-string"
}{
"type": "null-terminated-string",
"user-attributes": {
"my.tracer": {
"is-nice": true
}
}
}5.4.15. Static-length string field class
A static-length string field class describes static-length string fields.
A static-length string field is a sequence of zero or more contiguous bytes. All the bytes of a static-length string before the first null (zero) byte, if any, form a UTF-8-encoded string. All the bytes after the first null (zero) byte, if any, are padding (garbage data).
The length, or number of bytes, of a static-length string field is a property
(length) of its class.
| Name | Type | Description | Required? | Default |
|---|---|---|---|---|
|
JSON string |
Type of F. The value of this property MUST be |
Yes |
|
|
JSON integer |
Number of bytes contained in an instance of F. The value of this property MUST be greater than or equal to zero. |
Yes |
|
|
Roles of an instance of F. See Trace class fragment and Data stream class fragment which indicate accepted within their root field classes. |
No |
|
|
|
User attributes of F. |
No |
|
|
|
Extensions of F. Any extension which exists under this property MUST also be declared in the preamble fragment of the metadata stream. |
No |
|
{
"type": "static-length-string",
"length": 0
}{
"type": "static-length-string",
"length": 100
}{
"type": "static-length-string",
"length": 13,
"user-attributes": {
"my.tracer": null
}
}5.4.16. Dynamic-length string field class
A dynamic-length string field class describes dynamic-length string fields.
A dynamic-length string field is a sequence of zero or more contiguous bytes. All the bytes of a dynamic-length string before the first null (zero) byte, if any, form a UTF-8-encoded string. All the bytes after the first null (zero) byte, if any, are padding (garbage data).
The length, or number of bytes, of a dynamic-length string field is the value of
another, anterior (already encoded/decoded) length field. A
consumer can locate this length field thanks to the
length-field-location property of the dynamic-length string field class.
| Name | Type | Description | Required? | Default |
|---|---|---|---|---|
|
JSON string |
Type of F. The value of this property MUST be |
Yes |
|
|
Location of the field of which the value is the number of bytes contained in an instance of F. The class of the length field MUST be one of: |
Yes |
||
|
Roles of an instance of F. See Trace class fragment and Data stream class fragment which indicate accepted within their root field classes. |
No |
|
|
|
User attributes of F. |
No |
|
|
|
Extensions of F. Any extension which exists under this property MUST also be declared in the preamble fragment of the metadata stream. |
No |
|
{
"type": "dynamic-length-string",
"length-field-location": ["event-record-payload", "length"]
}{
"type": "dynamic-length-string",
"length-field-location": ["event-record-common-context", "name-length"],
"user-attributes": {
"my.tracer": 177
}
}5.4.17. Abstract BLOB field class
An abstract BLOB field class is a base of a static-length BLOB field class and a dynamic-length BLOB field class.
This field class is abstract in that it only exists to show the relation between different BLOB field classes in this document: a packet cannot contain an abstract BLOB field.
| Name | Type | Description | Required? | Default |
|---|---|---|---|---|
|
JSON string |
IANA media type of an instance of F. |
No |
|
5.4.18. Static-length BLOB field class
A static-length BLOB field class is an abstract BLOB field class which describes static-length BLOB fields.
A static-length BLOB field is a sequence of zero or more contiguous
bytes with an associated IANA media type (given by the media-type
property of its class).
The length, or number of bytes, of a static-length BLOB field is a property
(length) of its class.
| Name | Type | Description | Required? | Default |
|---|---|---|---|---|
|
JSON string |
Type of F. The value of this property MUST be |
Yes |
|
|
JSON integer |
Number of bytes contained in an instance of F. The value of this property MUST be greater than or equal to zero. |
Yes |
|
|
JSON string |
IANA media type of an instance of F. Property inherited from the abstract BLOB field class. |
No |
|
|
Roles of an instance of F. See Trace class fragment and Data stream class fragment which indicate accepted within their root field classes. |
No |
|
|
|
User attributes of F. |
No |
|
|
|
Extensions of F. Any extension which exists under this property MUST also be declared in the preamble fragment of the metadata stream. |
No |
|
{
"type": "static-length-blob",
"length": 0
}{
"type": "static-length-blob",
"length": 511267,
"media-type": "image/tif"
}{
"type": "static-length-blob",
"length": 2400,
"media-type": "text/csv",
"user-attributes": {
"my.tracer": {
"csv-cols": 12
}
}
}5.4.19. Dynamic-length BLOB field class
A dynamic-length BLOB field class is an abstract BLOB field class which describes dynamic-length BLOB fields.
A dynamic-length BLOB field is a sequence of zero or more contiguous bytes with an associated IANA media type.
The length, or number of bytes, of a dynamic-length BLOB field is the value of
another, anterior (already encoded/decoded) length field. A
consumer can locate this length field thanks to the
length-field-location property of the dynamic-length BLOB field class.
| Name | Type | Description | Required? | Default |
|---|---|---|---|---|
|
JSON string |
Type of F. The value of this property MUST be |
Yes |
|
|
Location of the field of which the value is the number of bytes contained in an instance of F. The class of the length field MUST be one of: |
Yes |
||
|
JSON string |
IANA media type of an instance of F. Property inherited from the abstract BLOB field class. |
No |
|
|
Roles of an instance of F. See Trace class fragment and Data stream class fragment which indicate accepted within their root field classes. |
No |
|
|
|
User attributes of F. |
No |
|
|
|
Extensions of F. Any extension which exists under this property MUST also be declared in the preamble fragment of the metadata stream. |
No |
|
{
"type": "dynamic-length-blob",
"length-field-location": ["event-record-payload", "length"]
}{
"type": "dynamic-length-blob",
"length-field-location": ["event-record-common-context", "length"],
"media-type": "image/jpeg",
"user-attributes": {
"my.tracer": {
"quality": 85
}
}
}5.4.20. Structure field class
A structure field class describes structure fields.
A structure field is a sequence of zero or more structure field members. A structure field member is a named field.
| Name | Type | Description | Required? | Default |
|---|---|---|---|---|
|
JSON string |
Type of F. The value of this property MUST be |
Yes |
|
|
JSON array of structure field member classes |
Classes of the members of an instance of F. The |
No |
|
|
JSON integer |
Minimum alignment of the first bit of an instance of F relative to the beginning of the packet which contains this instance. The value of this property MUST be a positive power of two. The effective alignment of the first bit of an instance of F MAY be greater than the value of this property. |
No |
|
|
Roles of an instance of F. See Trace class fragment and Data stream class fragment which indicate accepted within their root field classes. |
No |
|
|
|
User attributes of F. |
No |
|
|
|
Extensions of F. Any extension which exists under this property MUST also be declared in the preamble fragment of the metadata stream. |
No |
|
{
"type": "structure"
}{
"type": "structure",
"member-classes": [
{
"name": "Villeray",
"field-class": {
"type": "null-terminated-string"
}
},
{
"name": "Berri",
"field-class": {
"type": "fixed-length-unsigned-integer",
"length": 32,
"byte-order": "little-endian",
"preferred-display-base": 2
},
"user-attributes": {
"my.tracer": {
"is-mask": true
}
}
},
{
"name": "Faillon",
"field-class": {
"type": "fixed-length-boolean",
"length": 8,
"byte-order": "little-endian"
}
}
]
}{
"type": "structure",
"member-classes": [
{
"name": "St-Denis",
"field-class": {
"type": "null-terminated-string"
}
},
{
"name": "Lajeunesse",
"field-class": {
"type": "fixed-length-unsigned-integer",
"length": 32,
"byte-order": "big-endian",
"alignment": 32
}
}
],
"minimum-alignment": 64
}{
"type": "structure",
"member-classes": [
{
"name": "Henri-Julien",
"field-class": {
"type": "fixed-length-signed-integer",
"length": 48,
"byte-order": "little-endian"
}
},
{
"name": "Casgrain",
"field-class": {
"type": "static-length-string",
"length": 32
}
}
],
"user-attributes": {
"my.tracer": {
"version": 4
}
}
}5.4.20.1. Structure field member class
A structure field member class describes structure field members.
A structure field member class is a JSON object.
| Name | Type | Description | Required? | Default |
|---|---|---|---|---|
|
JSON string |
Name of M. |
Yes |
|
|
Field class of M. |
Yes |
||
|
User attributes of M. |
No |
|
|
|
Extensions of M. Any extension which exists under this property MUST also be declared in the preamble fragment of the metadata stream. |
No |
|
cat.{
"name": "cat",
"field-class": {
"type": "null-terminated-string"
}
}{
"name": "dog",
"field-class": {
"type": "variable-length-signed-integer",
"preferred-display-base": 8
},
"user-attributes": {
"my.tracer": {
"uuid": [
243, 97, 0, 184, 236, 54, 72, 97,
141, 107, 169, 214, 171, 137, 115, 201
],
"is-pid": true
}
}
}5.4.21. Abstract array field class
An abstract array field class is a base of a static-length array field class and a dynamic-length array field class.
This field class is abstract in that it only exists to show the relation between different array field classes in this document: a packet cannot contain an abstract array field.
| Name | Type | Description | Required? | Default |
|---|---|---|---|---|
|
Class of the element fields contained in an instance of F. |
Yes |
||
|
JSON integer |
Minimum alignment of the first bit of an instance of F relative to the beginning of the packet which contains this instance. The value of this property MUST be a positive power of two. The effective alignment of the first bit of an instance of F MAY be greater than the value of this property. |
No |
|
5.4.22. Static-length array field class
A static-length array field class is an abstract array field class which describes static-length array fields.
A static-length array field is a sequence of zero or more element fields.
The length, or number of element fields, of a static-length array field is a
property (length) of its class.
| Name | Type | Description | Required? | Default |
|---|---|---|---|---|
|
JSON string |
Type of F. The value of this property MUST be |
Yes |
|
|
Class of the element fields contained in an instance of F. Property inherited from the abstract array field class. |
Yes |
||
|
JSON integer |
Minimum alignment of the first bit of an instance of F relative to the beginning of the packet which contains this instance. The value of this property MUST be a positive power of two. The effective alignment of the first bit of an instance of F MAY be greater than the value of this property. Property inherited from the abstract array field class. |
No |
|
|
JSON integer |
Number of element fields contained in an instance of F. The value of this property MUST be greater than or equal to zero. |
Yes |
|
|
Roles of an instance of F. See Trace class fragment and Data stream class fragment which indicate accepted within their root field classes. |
No |
|
|
|
User attributes of F. |
No |
|
|
|
Extensions of F. Any extension which exists under this property MUST also be declared in the preamble fragment of the metadata stream. |
No |
|
{
"type": "static-length-array",
"element-field-class": {
"type": "fixed-length-signed-integer",
"length": 16,
"byte-order": "little-endian",
"alignment": 16
},
"length": 0
}{
"type": "static-length-array",
"element-field-class": {
"type": "null-terminated-string"
},
"length": 100
}{
"type": "static-length-array",
"element-field-class": {
"type": "variable-length-unsigned-integer"
},
"length": 13,
"user-attributes": {
"my.tracer": true
}
}With the following static-length array field class, a producer can write a single 32-bit-aligned, 32-bit little-endian integer value, and have consumers decode it as an array of 32 flags (booleans).
{
"type": "static-length-array",
"length": 32,
"minimum-alignment": 32,
"element-field-class": {
"type": "fixed-length-boolean",
"length": 1,
"byte-order": "little-endian"
}
}5.4.23. Dynamic-length array field class
A dynamic-length array field class is an abstract array field class which describes dynamic-length array fields.
A dynamic-length array field is a sequence of zero or more element fields.
The length, or number of element fields, of a dynamic-length array field is the
value of another, anterior (already encoded/decoded) length field. A
consumer can locate this length field thanks to the
length-field-location property of the dynamic-length array field class.
| Name | Type | Description | Required? | Default |
|---|---|---|---|---|
|
JSON string |
Type of F. The value of this property MUST be |
Yes |
|
|
Class of the element fields contained in an instance of F. Property inherited from the abstract array field class. |
Yes |
||
|
JSON integer |
Minimum alignment of the first bit of an instance of F relative to the beginning of the packet which contains this instance. The value of this property MUST be a positive power of two. The effective alignment of the first bit of an instance of F MAY be greater than the value of this property. Property inherited from the abstract array field class. |
No |
|
|
Location of the field of which the value is the number of element fields contained in an instance of F. The class of the length field MUST be one of: |
Yes |
||
|
Roles of an instance of F. See Trace class fragment and Data stream class fragment which indicate accepted within their root field classes. |
No |
|
|
|
User attributes of F. |
No |
|
|
|
Extensions of F. Any extension which exists under this property MUST also be declared in the preamble fragment of the metadata stream. |
No |
|
{
"type": "dynamic-length-array",
"element-field-class": {
"type": "fixed-length-unsigned-integer",
"length": 32,
"byte-order": "big-endian",
"alignment": 16
},
"length-field-location": ["event-record-payload", "length"]
}{
"type": "dynamic-length-array",
"element-field-class": {
"type": "variable-length-unsigned-integer"
},
"length-field-location": ["packet-context", "common-length"],
"user-attributes": {
"my.tracer": 177
}
}5.4.24. Optional field class
An optional field class describes optional fields.
An optional field is, depending on the value of another, anterior (already encoded/decoded) selector field, one of:
-
An instance of a given field class (
field-classproperty of the optional field class).In this case, the optional field is said to be enabled.
-
A zero-bit field (no field).
In this case, the optional field is said to be disabled.
A consumer can locate the selector field thanks to the
selector-field-location property of the optional field class.
| Name | Type | Description | Required? | Default |
|---|---|---|---|---|
|
JSON string |
Type of F. The value of this property MUST be |
Yes |
|
|
Class of an instance of F when it’s enabled. |
Yes |
||
|
Location of the field of which the value indicates whether or not an instance of F is enabled. A selector field S MUST be an instance of one of:
For a given instance of F, the
|
Yes |
||
|
Ranges of integers which the value of a selector field MUST be an element of to enable an instance of F. |
Yes, if the selector field is an instance of a fixed-length integer field class or a variable-length integer field class. |
None if the selector field is an instance of a fixed-length boolean field class. |
|
|
Roles of F. See Trace class fragment and Data stream class fragment which indicate accepted within their root field classes. |
No |
|
|
|
User attributes of F. |
No |
|
|
|
Extensions of F. Any extension which exists under this property MUST also be declared in the preamble fragment of the metadata stream. |
No |
|
{
"type": "optional",
"selector-field-location": ["event-record-payload", "has-ip"],
"field-class": {
"type": "static-length-array",
"element-field-class": {
"type": "fixed-length-unsigned-integer",
"length": 8,
"byte-order": "little-endian",
"alignment": 8
},
"length": 16
}
}{
"type": "optional",
"selector-field-location": ["event-record-payload", "has-ip"],
"selector-field-ranges": [[-12, -12], [-5, 0], [15, 35]],
"field-class": {
"type": "null-terminated-string"
}
}5.4.25. Variant field class
A variant field class describes variant fields.
A variant field is, depending on the value of another, anterior (already encoded/decoded) selector field, the instance of a specific, effective field class amongst one or more variant field class options.
A consumer can locate the selector field thanks to the
selector-field-location property of the variant field class.
| Name | Type | Description | Required? | Default |
|---|---|---|---|---|
|
JSON string |
Type of F. The value of this property MUST be |
Yes |
|
|
JSON array of variant field class options |
Options containing the possible effective classes of an instance of F. This array MUST contain one or more elements. The The integer ranges ( |
Yes |
|
|
Location of the field of which the value indicates which option of F contains the effective class of an instance of F. For a given instance of F, the |
Yes |
||
|
Roles of F. See Trace class fragment and Data stream class fragment which indicate accepted within their root field classes. |
No |
|
|
|
User attributes of F. |
No |
|
|
|
Extensions of F. Any extension which exists under this property MUST also be declared in the preamble fragment of the metadata stream. |
No |
|
{
"type": "variant",
"selector-field-location": ["event-record-payload", "sel"],
"options": [
{
"selector-field-ranges": [[5, 5]],
"field-class": {
"type": "null-terminated-string"
}
},
{
"selector-field-ranges": [[8, 8]],
"field-class": {
"type": "fixed-length-signed-integer",
"length": 16,
"byte-order": "little-endian",
"preferred-display-base": 8
}
}
]
}This example shows that an optional field class and a contained variant field class MAY share the same selector field location.
In this example, depending on the value of the selector field:
| 0 |
The optional field is not enabled. |
| 1 |
The optional field is enabled and is a variant field. The variant field is an instance of a null-terminated string field class (effective class). |
| 2 |
The optional field is enabled and is a variant field. The variant field is an instance of a variable-length signed integer field class (effective class). |
{
"type": "optional",
"selector-field-location": ["event-record-payload", "sel"],
"selector-field-ranges": [[1, 255]],
"field-class": {
"type": "variant",
"selector-field-location": ["event-record-payload", "sel"],
"options": [
{
"selector-field-ranges": [[1, 1]],
"field-class": {
"type": "null-terminated-string"
}
},
{
"selector-field-ranges": [[2, 2]],
"field-class": {
"type": "variable-length-signed-integer",
"preferred-display-base": 16
}
}
]
}
}{
"type": "variant",
"selector-field-location": ["event-record-specific-context", "sel"],
"options": [
{
"selector-field-ranges": [[5, 5], [10, 10], [15, 15]],
"field-class": {
"type": "static-length-string",
"length": 20
}
},
{
"selector-field-ranges": [[0, 4], [6, 9], [11, 14], [16, 127]],
"field-class": {
"type": "fixed-length-floating-point-number",
"length": 32,
"byte-order": "big-endian"
}
}
],
"user-attributes": {
"my.tracer": {
"owner": "Jimmy",
"id": 199990
}
}
}5.4.25.1. Variant field class option
A variant field class option contains a possible effective class of a variant field.
A variant field class option O also contains the ranges of
integer values (selector-field-ranges property) of which the value of
a selector field MUST be an element of for the effective class of a
variant field to be the field class of O.
A variant field class option is a JSON object.
| Name | Type | Description | Required? | Default |
|---|---|---|---|---|
|
Field class of O. |
Yes |
||
|
Ranges of integers which the value of a selector field MUST be an
element of for the effective class of an instance of F
to be the field class ( |
Yes |
||
|
JSON string |
Name of O. This property exists to remain backward compatible with CTF 1: it’s not strictly needed to decode an instance of F. |
No |
O is unnamed |
|
User attributes of O. |
No |
|
|
|
Extensions of O. Any extension which exists under this property MUST also be declared in the preamble fragment of the metadata stream. |
No |
|
{
"field-class": {
"type": "null-terminated-string"
},
"selector-field-ranges": [[3, 9]]
}{
"name": "juice",
"field-class": {
"type": "variable-length-signed-integer",
"preferred-display-base": 16
},
"selector-field-ranges": [[-4, 4], [9, 9], [100, 200]],
"user-attributes": {
"my.tracer": {
"uuid": [
243, 97, 0, 184, 236, 54, 72, 97,
141, 107, 169, 214, 171, 137, 115, 201
],
"is-did": true
}
}
}5.5. Preamble fragment
A preamble fragment indicates:
-
The CTF 2 major version (2).
CTF 2 doesn’t have a minor version: users can use user attributes and extensions to add features to, or change features of, the format which this document specifies.
-
Extension declarations.
An extension declaration is an initial extension of which the purpose is to declare that it’s enabled within the metadata stream.
Because an extension MAY alter the CTF 2 format itself, and because a preamble fragment is always the first metadata stream fragment, those extension declarations make it possible for a consumer to gracefully decline the data streams of the trace if it doesn’t support any declared extension.
The first fragment of a metadata stream MUST be a preamble fragment.
| Name | Type | Description | Required? | Default |
|---|---|---|---|---|
|
JSON string |
Type of F. The value of this property MUST be |
Yes |
|
|
JSON integer |
CTF 2 major version. The value of this property MUST be |
Yes |
|
|
User attributes of F. |
No |
|
|
|
Extension declarations of F. The name of each property is a namespace and its value is a namespaced extensions object. Within a namespaced extensions object, an extension named N is declared when it exists as a property named N, whatever the value of the property. |
No |
|
{
"type": "preamble",
"version": 2
}The following preamble fragment declares the piano and ramen
extensions under the my.tracer namespace.
{
"type": "preamble",
"version": 2,
"extensions": {
"my.tracer": {
"piano": {
"keys": 88,
"temperament": "equal"
},
"ramen": null
}
}
}5.6. Trace class fragment
A trace class describes traces.
Within a metadata stream, a trace class fragment MUST occur, if any, before any data stream class fragment.
| Name | Type | Description | Required? | Default |
|---|---|---|---|---|
|
JSON string |
Type of F. The value of this property MUST be |
Yes |
|
|
UUID of F. |
No |
F has no UUID |
|
|
Class of all the packet header fields of an instance of F. Any field class as the value of this property MUST satisfy at least one of:
|
No |
F has no packet header field class |
|
|
User attributes of F. |
No |
|
|
|
Extensions of F. Any extension which exists under this property MUST also be declared in the preamble fragment of the metadata stream. |
No |
|
5.6.1. Roles
If the packet-header-field-class property of a
trace class fragment exists, then the field classes of its
member classes MAY have the following
roles:
| Name | Description | Field class (F) constraints | Other constraints | ||
|---|---|---|---|---|---|
|
Packet magic number. The purpose of a packet magic number field is to confirm the beginning of a CTF 2 packet. |
Fixed-length unsigned integer field class with the following property value:
|
An instance of F MUST be the first member of the packet header structure field. The value of an instance of F value MUST be 0xc1fc1fc1 (3254525889). |
||
|
Trace class UUID. The purpose of a trace class UUID field is to confirm the association between a data stream and a metadata stream. |
Static-length BLOB field class with the following property value:
|
The The 16 bytes of an instance of F MUST be equal to the
16 JSON integers of the |
||
|
Data stream class ID. The purpose of a data stream class ID field is to set the current ID of the class of the data stream of the current packet. |
||||
|
Data stream ID. The purpose of a data stream ID field is to set the current ID of the data stream of the current packet. Combined with the ID of its class, such a field makes it possible to uniquely identify a data stream within a trace. |
{
"type": "trace-class",
"uuid": [
30, 201, 100, 148, 228, 2, 69, 70,
147, 219, 233, 34, 43, 238, 108, 199
],
"packet-header-field-class": {
"type": "structure",
"member-classes": [
{
"name": "the magic!",
"field-class": {
"type": "fixed-length-unsigned-integer",
"length": 32,
"byte-order": "little-endian",
"preferred-display-base": 16,
"roles": ["packet-magic-number"]
}
},
{
"name": "the UUID",
"field-class": {
"type": "static-length-blob",
"length": 16,
"roles": ["trace-class-uuid"]
}
},
{
"name": "my data stream class ID",
"field-class": {
"type": "fixed-length-unsigned-integer",
"length": 8,
"byte-order": "little-endian",
"roles": ["data-stream-class-id"]
}
},
{
"name": "my data stream ID",
"field-class": {
"type": "variable-length-unsigned-integer",
"roles": ["data-stream-id"]
}
}
]
}
}5.7. Clock class fragment
A clock class describes clocks.
A data stream MAY have a default clock.
Within a metadata stream, a clock class fragment MUST occur before any
data stream class fragment which refers to it by name with
its default-clock-class-name property.
| Name | Type | Description | Required? | Default |
|---|---|---|---|---|
|
JSON string |
Type of F. The value of this property MUST be |
Yes |
|
|
JSON integer |
Frequency of an instance of F (Hz). The value of this property MUST be greater than zero. |
Yes |
|
|
JSON string |
Name of F. |
Yes |
|
|
JSON string |
Textual description of F. This property exists to remain backward compatible with CTF 1: it’s not strictly needed to decode data streams. |
No |
F has no textual description |
|
UUID of F. This property exists to remain backward compatible with CTF 1: it’s not strictly needed to decode data streams. |
No |
F has no UUID |
|
|
JSON boolean |
Whether or not the origin of an instance of F is the Unix epoch. If the value of this property is |
No |
|
|
Offset of an instance of F relative to its origin. Let:
Then the effective offset of an instance of F, in clock cycles, is S × H + C. |
No |
|
|
|
JSON integer |
Precision of an instance of F (clock cycles). The value of this property MUST be greater than or equal to zero. Let P be the value of this property and V the value of an instance of F: the range of possible values of the instance is [V − P, V + P]. |
No |
|
|
User attributes of F. |
No |
|
|
|
Extensions of F. Any extension which exists under this property MUST also be declared in the preamble fragment of the metadata stream. |
No |
|
Within a metadata stream, two given clock class fragments MUST NOT:
-
Share the same
nameproperty value. -
Share the same
uuidproperty value.
{
"type": "clock-class",
"name": "my clock class",
"frequency": 1000000000
}{
"type": "clock-class",
"name": "my clock class",
"frequency": 1000000000,
"uuid": [
116, 210, 0, 140, 239, 255, 77, 3,
129, 99, 233, 226, 134, 106, 207, 32
]
}{
"type": "clock-class",
"name": "my clock class",
"frequency": 1000000000,
"offset": {
"seconds": 1605112699,
"cycles": 2878388
}
}{
"type": "clock-class",
"name": "my clock class",
"frequency": 8000000,
"precision": 100
}{
"type": "clock-class",
"name": "my clock class",
"frequency": 1000000000,
"origin-is-unix-epoch": false
}{
"type": "clock-class",
"name": "my clock class",
"frequency": 16000000,
"user-attributes": {
"my.tracer": {
"sys-name": "SOC23",
"bus": {
"name": "LMB5",
"index": 5
},
"propagation-delay-ps": 177
}
}
}5.7.1. Clock offset
A clock offset contains the offset of the instances of a clock class relative to their origin.
A clock offset is a JSON object.
| Name | Type | Description | Required? | Default |
|---|---|---|---|---|
|
JSON integer |
Offset, in seconds, of an instance of F relative to its origin. |
No |
|
|
JSON integer |
Offset, in cycles, of an instance of F relative to its origin. The value of this property MUST be greater than or equal to zero. The value of this property MUST be less than the value of the
|
No |
|
{}{
"seconds": 1605112699,
"cycles": 2878388
}{
"seconds": 1605111293
}This example shows that a clock offset MAY be negative, that is, before the origin of the clock.
{
"seconds": -18003,
"cycles": 11928547
}5.8. Data stream class fragment
A data stream class describes data streams.
Within a metadata stream, a data stream class fragment F MUST occur before any event record class fragment of which F is the parent.
| Name | Type | Description | Required? | Default |
|---|---|---|---|---|
|
JSON string |
Type of F. The value of this property MUST be |
Yes |
|
|
JSON integer |
Numeric ID of F. The value of this property MUST be greater than or equal to zero. |
No |
|
|
JSON string |
Name of F. The purpose of this property, combined with the |
No |
F is unnamed |
|
JSON string |
No |
F has no namespace |
|
|
JSON string |
Name of the class of the default clock of an instance of F. Within the metadata stream containing F, the clock
class fragment which has the value of this property as its |
No |
An instance of F has no default clock |
|
Class of all the packet context fields of an instance of F. |
No |
F has no packet context field class |
|
|
Class of all the event record header fields of an instance of F. Any field class as the value of this property MUST satisfy at least one of:
|
No |
F has no event record header field class |
|
|
Class of all the event record common context fields of an instance of F. |
No |
F has no event record common context field class |
|
|
User attributes of F. |
No |
|
|
|
Extensions of F. Any extension which exists under this property MUST also be declared in the preamble fragment of the metadata stream. |
No |
|
Within a metadata stream, two given data stream class fragments
MUST NOT share the same id property value.
5.8.1. Roles
If the packet-context-field-class property of a data
stream class fragment exists, then the field classes of its
member classes MAY have the following
roles:
| Name | Description | Field class (F) constraints | Other constraints |
|---|---|---|---|
|
Total size (bits) of the packet. This size includes any padding after the packet contents. |
The value of an instance of F MUST be greater than or
equal to the value of an instance of a field class having the
|
|
|
Content size (bits) of the packet. |
The value of an instance of F MUST be less than or equal
to the value of an instance of a field class having the
|
|
|
Timestamp of the default clock of the data stream when the packet begins. |
The timestamps of all the event records of the packet MUST be greater than or equal to the value of an instance of F. The value of an instance of F MUST be less than or equal to
the value of an instance of a field class having the
|
|
|
Timestamp of the default clock of the data stream when the packet ends. |
The timestamps of all the event records of the packet MUST be less than or equal to the value of an instance of F. The value of an instance of F MUST be greater than or equal
to the value of an instance of a field class having the
|
|
|
Snapshot of the discarded event record counter of the data stream when the packet ends. |
||
Sequence number of the packet within its data stream. |
If the event-record-header-field-class property of
a data stream class fragment exists, then the field classes of
its member classes MAY have the following
roles:
| Name | Description | Field class (F) constraints |
|---|---|---|
|
Event record class ID. The purpose of an event record class ID field is to set the current ID of the class of the event record within its parent data stream class. |
|
|
Current timestamp of the default clock of the data stream when the event record occurs. |
5.9. Event record class fragment
An event record class describes event records.
The data stream class fragment of which the value of the
id property matches the value of the data-stream-class-id property
of an event record class fragment F is considered the
parent of F.
| Name | Type | Description | Required? | Default |
|---|---|---|---|---|
|
JSON string |
Type of F. The value of this property MUST be |
Yes |
|
|
JSON integer |
Numeric ID of F within P. The value of this property MUST be greater than or equal to zero. |
No |
|
|
JSON integer |
Numeric ID of P. The value of this property MUST be greater than or equal to zero. Within the metadata stream, P MUST occur before F. |
No |
|
|
JSON string |
Name of F. The purpose of this property, combined with the |
No |
F is unnamed |
|
JSON string |
No |
F has no namespace |
|
|
Class of the event record specific context field of an instance of F. |
No |
F has no event record specific context field class |
|
|
Class of the event record payload field of an instance of F. |
No |
F has no event record payload field class |
|
|
User attributes of F. |
No |
|
|
|
Extensions of F. Any extension which exists under this property MUST also be declared in the preamble fragment of the metadata stream. |
No |
|
Within a metadata stream, two given event record class fragments
MUST NOT share the same id property value and the same
data-stream-class-id property value.
6. Data stream decoding procedure
This section shows how to, procedurally, decode a CTF 2 data stream.
Decoding a data stream is the responsibility of a consumer.
This document doesn’t specify how to encode a data stream, as this procedure implies much more freedom than decoding. One can deduce how to encode a data stream from the decoding procedure.
A consumer needs to keep a data stream decoding state while decoding a data stream. A data stream decoding state comprises the following variable:
| Name | Type | Description | Initial value |
|---|---|---|---|
O |
Unsigned integer |
Current decoding offset/position (bits) from the beginning of S. |
0 |
To decode a data stream S:
-
While there’s remaining data in S:
6.1. Packet decoding procedure
A consumer needs to keep a packet decoding state while decoding a packet. A packet decoding state comprises the following variables:
| Name | Type | Description | Initial value |
|---|---|---|---|
DEF_CLK_VAL |
Unsigned integer |
Current value (clock cycles) of the default clock of S, if any. |
0 |
DSC_ID |
Unsigned integer |
Current ID of the class of S. |
0 |
DSC |
Optional data stream class |
Current class of S. |
None |
DS_ID |
Optional unsigned integer |
Current ID of S. |
None |
PKT_TOTAL_SZ |
Unsigned integer |
Current total size (bits) of P. |
∞ |
PKT_CONTENT_SZ |
Unsigned integer |
Current content size (bits) of P. |
∞ |
LAST_BO |
Optional string |
Byte order of the last decoded fixed-length bit array field. |
None |
To decode a packet P within a data stream S:
-
Let PO be the current value of O.
While decoding the packet, O − PO is the current decoding offset/position (bits) from the beginning of P.
-
If the
packet-header-field-classproperty of the trace class fragment of the metadata stream exists, then decode the header field of P using this property.During the packet header field decoding procedure, after having decoded a field F having the class C with a
rolesproperty:-
If C has the role
packet-magic-number, then validate that the unsigned integer value of F is 0xc1fc1fc1 (3254525889).A consumer SHOULD report an invalid packet magic number as an error.
-
If C has the role
trace-type-uuid, then validate that the value of F matches theuuidproperty of the trace class fragment.A consumer SHOULD report a trace type UUID mismatch as an error.
-
If C has the role
data-stream-class-id, then set DSC_ID to the unsigned integer value of F. -
If C has the role
data-stream-id, then set DS_ID to the unsigned integer value of F.
After having decoded the whole packet header field, if DS_ID is set, then it’s the ID of S within its class. In other words, two data streams MAY have the same ID if they are instances of different data stream classes.
-
-
Set DSC to the data stream class having DSC_ID as the value of its
idproperty.If no data stream class has the ID DSC_ID, then report an error and abort the data stream decoding process.
-
If the
packet-context-field-classproperty of DSC exists, then decode the context field of P using this property.During the packet context field decoding procedure, after having decoded a field F having the class C with a
rolesproperty:-
If C has the role
packet-total-size, then set PKT_TOTAL_SZ to the unsigned integer value of F. -
If C has the role
packet-content-size, then set PKT_CONTENT_SZ to the unsigned integer value of F. -
If C has the role
packet-beginning-default-clock-timestamp, then set DEF_CLK_VAL to the unsigned integer value of F. -
If C has the role
packet-end-default-clock-timestamp, then the unsigned integer value of F is the value of the default clock of S at the end of P. -
If C has the role
discarded-event-record-counter-snapshot, then the unsigned integer value of F is a snapshot of the discarded event record counter of S at the end of P. -
If C has the role
packet-sequence-number, then the unsigned integer value of F is the sequence number of P within S.
A consumer SHOULD report a beginning default clock timestamp greater than an end default clock timestamp as an error.
-
-
If PKT_TOTAL_SZ is ∞ and PKT_CONTENT_SZ is not ∞, then set PKT_TOTAL_SZ to PKT_CONTENT_SZ.
-
If PKT_CONTENT_SZ is ∞ and PKT_TOTAL_SZ is not ∞, then set PKT_CONTENT_SZ to PKT_TOTAL_SZ.
-
While O < PO + PKT_CONTENT_SZ and there’s remaining data in S:
-
If PKT_TOTAL_SZ and PKT_CONTENT_SZ both are not ∞, then set O to PO + PKT_TOTAL_SZ, effectively skipping end-of-packet padding.
6.2. Event record decoding procedure
A consumer needs to keep an event record decoding state while decoding an event record. An event record decoding state comprises the following variables:
| Name | Type | Description | Initial value |
|---|---|---|---|
ERC_ID |
Unsigned integer |
Current ID of the class of E of which the parent is the class of S. |
0 |
ERC |
Optional event record class |
Current class of E. |
None |
To decode an event record E within a data stream S:
-
If the
event-record-header-field-classproperty of DSC exists, then decode the header field of E using this property.During the event record header field decoding procedure, after having decoded a field F having the class C with a
rolesproperty:-
If C has the role
event-record-class-id, then set ERC_ID to the unsigned integer value of F. -
If C has the role
default-clock-timestamp, then update DEF_CLK_VAL from F.
After having decoded the whole event record header field, DEF_CLK_VAL is the value of the default clock of S when E occurs.
-
-
Set ERC to the event record class having:
-
DSC_ID as the value of its
data-stream-class-idproperty. -
ERC_ID as the value of its
idproperty.
If no event record class has the ID ERC_ID within a data stream class having the ID DSC_ID, then report an error and abort the data stream decoding process.
-
-
If the
event-record-common-context-field-classproperty of DSC exists, then decode the common context field of E using this property. -
If the
specific-context-field-classproperty of ERC exists, then decode the specific context field of E using this property. -
If the
payload-field-classproperty of ERC exists, then decode the payload field of E using this property.
6.2.1. Clock value update procedure
To update DEF_CLK_VAL from an unsigned integer field F having the unsigned integer value V and the class C:
-
Let L be an unsigned integer initialized to, depending on the
typeproperty of C:"fixed-length-unsigned-integer""fixed-length-unsigned-enumeration"-
The value of the
lengthproperty of C. "variable-length-unsigned-integer""variable-length-unsigned-enumeration"-
S ×7, where S is the number of bytes which F occupies with the data stream.
-
Let MASK be an unsigned integer initialized to 2L − 1.
-
Let H be an unsigned integer initialized to DEF_CLK_VAL & ~MASK, where “&” is the bitwise AND operator and “~” is the bitwise NOT operator.
-
Let CUR be an unsigned integer initialized to DEF_CLK_VAL & MASK, where “&” is the bitwise AND operator.
-
Set DEF_CLK_VAL to:
- If V ≥ CUR
-
H + V
- Else
-
H + MASK + 1 + V
6.3. Field decoding procedure
The class of a field contains what’s needed to decode it as a value.
While a field is an actual sequence of bits within a data stream, a value is its conceptual interpretation with attached semantics.
| Value type | Possible values |
|---|---|
Nil |
None. |
Boolean |
True or false. |
Unsigned/signed integer |
Integral quantity. |
Real |
Continuous quantity. |
String |
|
Array |
Sequence of values. |
Structure |
Sequence of named values (members). |
To decode an instance of a field class F, depending on the
value of its type property:
Value of the type property of F |
Decoding procedure of F |
|---|---|
6.3.1. Alignment procedure
The decoding procedure of many fields require O − PO to have a specific alignment.
The alignment requirement of an instance of a field
class F is, depending on the value of its type property:
type property of F |
Alignment requirement of F |
|---|---|
The value of the |
|
8 |
|
The maximum value of:
|
|
The maximum value of:
|
|
1 |
To align O − PO to some alignment requirement A (bits):
-
Set O to PO + ((O − PO + A − 1) & −A), where “&” is the bitwise AND operator.
6.3.2. Field location procedure
To locate a previously decoded field using a field location FL:
-
Let V be, depending on the first element of FL:
"packet-header"-
The header structure of P (current packet).
"packet-context"-
The context structure of P.
"event-record-header"-
The header structure of E (current event record).
"event-record-common-context"-
The common context structure of E.
"event-record-specific-context"-
The specific context structure of E.
"event-record-payload"-
The payload structure of E.
If the consumer cannot set V because there’s no such structure or because it’s not already decoded nor currently being decoded, then report an error and abort the data stream decoding process.
-
For each element FLE of FL, starting from the second element:
-
Let V be the value of the member named FLE within V.
If no member is named FLE within V, then report an error and abort the data stream decoding process.
If the member named FLE within V is not already decoded nor currently being decoded, then report an error and abort the data stream decoding process.
-
Depending on the type of V:
- Boolean
- Signed integer
- Unsigned integer
-
If FLE is not the last element of FL, then report an error and abort the data stream decoding process.
- Structure
-
Continue.
- Array
-
While V is an array:
-
If V is not currently being decoded, then report an error and abort the data stream decoding process.
Set V to the element of V currently being decoded.
-
- Other
-
Report an error and abort the data stream decoding process.
-
V is the located field.
Assume the following JSON object is an event record payload structure field class.
{
"type": "structure",
"member-classes": [
{
"name": "corn", (3)
"field-class": {
"type": "fixed-length-unsigned-integer",
"length": 32,
"byte-order": "little-endian"
}
},
{
"name": "inside",
"field-class": {
"type": "fixed-length-unsigned-integer",
"length": 16,
"byte-order": "little-endian"
}
},
{
"name": "carbon",
"field-class": {
"type": "dynamic-length-array", (1)
"length-field-location": ["event-record-payload", "corn"], (2)
"element-field-class": {
"type": "null-terminated-string"
}
}
}
]
}| 1 | Dynamic-length array field class. |
| 2 | Length field location of the dynamic-length array field class. |
| 3 | Length member class. |
Assume the following JSON object is an event record payload structure field class.
Both the dynamic-length array field and its length field exist within the same
element of the static-length array field named nature.
{
"type": "structure",
"member-classes": [
{
"name": "norm",
"field-class": {
"type": "null-terminated-string"
}
},
{
"name": "nature",
"field-class": {
"type": "static-length-array",
"length": 43,
"element-field-class": {
"type": "structure",
"member-classes": [
{
"name": "laser", (3)
"field-class": {
"type": "variable-length-unsigned-integer"
}
},
{
"name": "joystick",
"field-class": {
"type": "dynamic-length-array", (1)
"length-field-location": [ (2)
"event-record-payload",
"nature",
"laser"
],
"element-field-class": {
"type": "null-terminated-string"
}
}
}
]
}
}
}
]
}| 1 | Dynamic-length array field class. |
| 2 | Length field location of the dynamic-length array field class. |
| 3 | Length member class. |
Assume the following JSON object is an event record payload structure field class.
Both the dynamic-length array field and its length field exist within the same
option of the variant field named clinic.
Moreover, the selector field of the clinic variant field is the
lawyer field.
{
"type": "structure",
"member-classes": [
{
"name": "lawyer", (5)
"field-class": {
"type": "fixed-length-signed-integer",
"length": 16,
"byte-order": "little-endian"
}
},
{
"name": "clinic",
"field-class": {
"type": "variant",
"selector-field-location": ["event-record-payload", "lawyer"], (4)
"options": [
{
"selector-field-ranges": [[0, 0]],
"field-class": {
"type": "null-terminated-string"
}
},
{
"selector-field-ranges": [[1, 4]],
"field-class": {
"type": "structure",
"member-classes": [
{
"name": "lemon", (3)
"field-class": {
"type": "fixed-length-unsigned-integer",
"length": 8,
"byte-order": "big-endian"
}
},
{
"name": "joystick",
"field-class": {
"type": "dynamic-length-array", (1)
"length-field-location": [ (2)
"event-record-payload",
"clinic",
"lemon"
],
"element-field-class": {
"type": "null-terminated-string"
}
}
}
]
}
},
{
"selector-field-ranges": [[5, 5], [7, 7]],
"field-class": {
"type": "fixed-length-boolean",
"length": 8,
"byte-order": "little-endian"
}
}
]
}
}
]
}| 1 | Dynamic-length array field class. |
| 2 | Length field location of the dynamic-length array field class. |
| 3 | Length member class. |
| 4 | Selector field location of the variant field class. |
| 5 | Selector member class. |
Assume the following JSON object is an event record payload structure field class.
The length field of the dynamic-length array field is a variant field: it can be an 8-bit, a 16-bit, or a 32-bit fixed-length integer field, depending on the selection of the variant field.
Moreover, the selector field of the variant field is located in another root field (event record specific context).
{
"type": "structure",
"member-classes": [
{
"name": "glass", (3)
"field-class": {
"type": "variant",
"selector-field-location": ["event-record-specific-context", "sel"],
"options": [
{
"selector-field-ranges": [[0, 0]],
"field-class": {
"type": "fixed-length-unsigned-integer", (4)
"length": 8,
"byte-order": "little-endian"
}
},
{
"selector-field-ranges": [[1, 1]],
"field-class": {
"type": "fixed-length-unsigned-integer", (4)
"length": 16,
"byte-order": "little-endian"
}
},
{
"selector-field-ranges": [[2, 2]],
"field-class": {
"type": "fixed-length-unsigned-integer", (4)
"length": 32,
"byte-order": "little-endian"
}
}
]
}
},
{
"name": "margin",
"field-class": {
"type": "dynamic-length-array", (1)
"length-field-location": ["event-record-payload", "glass"], (2)
"element-field-class": {
"type": "null-terminated-string"
}
}
}
]
}| 1 | Dynamic-length array field class. |
| 2 | Length field location of the dynamic-length array field class. |
| 3 | Length member class. |
| 4 | Possible length field class. |
Assume the following JSON object is an event record payload structure field class.
The length field of the dynamic-length array field is within a structure field which is a variant field.
Moreover:
-
The selector field of the variant field is located in another root field (event record common context).
-
The field class of the third option of the
glassvariant field class contains a dynamic-length BLOB field class (lockmember); the length field of its instance is the previous member (eagle) within the same structure field.
{
"type": "structure",
"member-classes": [
{
"name": "glass",
"field-class": {
"type": "variant",
"selector-field-location": ["event-record-common-context", "sel"],
"options": [
{
"selector-field-ranges": [[0, 0]],
"field-class": {
"type": "structure",
"member-classes": [
{
"name": "eagle",
"field-class": {
"type": "fixed-length-unsigned-integer", (3)
"length": 16,
"byte-order": "little-endian"
}
},
{
"name": "road",
"field-class": {
"type": "null-terminated-string"
}
}
]
}
},
{
"selector-field-ranges": [[32, 172]],
"field-class": {
"type": "structure",
"member-classes": [
{
"name": "nuance",
"field-class": {
"type": "null-terminated-string"
}
},
{
"name": "eagle",
"field-class": {
"type": "fixed-length-unsigned-integer", (3)
"length": 24,
"byte-order": "big-endian"
}
}
]
}
},
{
"selector-field-ranges": [[5, 5]],
"field-class": {
"type": "structure",
"member-classes": [
{
"name": "eagle", (5)
"field-class": {
"type": "variable-length-unsigned-integer" (3)
}
},
{
"name": "lock",
"field-class": {
"type": "dynamic-length-blob",
"length-field-location": [ (4)
"event-record-payload",
"glass",
"eagle"
]
}
}
]
}
}
]
}
},
{
"name": "margin",
"field-class": {
"type": "dynamic-length-array", (1)
"length-field-location": [ (2)
"event-record-payload",
"glass",
"eagle"
],
"element-field-class": {
"type": "null-terminated-string"
}
}
}
]
}| 1 | Dynamic-length array field class. |
| 2 | Length field location of the dynamic-length array field class. |
| 3 | Possible length field class. |
| 4 | Length field location of the dynamic-length BLOB field class. |
| 5 | Length field class for the dynamic-length BLOB field class. |
Note that both the dynamic-length array and dynamic-length BLOB field classes have the same length field location.
Assume the following JSON objects are the event record specific context and payload structure field classes of the same event record class.
The length field of the dynamic-length array field of the event record payload is within the event record specific context.
{
"type": "structure",
"member-classes": [
{
"name": "cook",
"field-class": {
"type": "fixed-length-floating-point-number",
"length": 64,
"byte-order": "little-endian"
}
},
{
"name": "vegetable", (1)
"field-class": {
"type": "variable-length-unsigned-integer"
}
}
]
}| 1 | Length member class. |
{
"type": "structure",
"member-classes": [
{
"name": "avenue",
"field-class": {
"type": "dynamic-length-array", (1)
"length-field-location": [ (2)
"event-record-specific-context",
"vegetable"
],
"element-field-class": {
"type": "null-terminated-string"
}
}
},
{
"name": "railroad",
"field-class": {
"type": "null-terminated-string"
}
}
]
}| 1 | Dynamic-length array field class. |
| 2 | Length field location of the dynamic-length array field class. |
6.3.3. Fixed-length bit array field decoding procedure
For this whole section about decoding an instance of a
fixed-length bit array field class F, let BO be the value of the byte-order
property of F.
-
Let:
-
BYTE_I be (O − PO) / 8 (integral division; remainder discarded).
-
BIT_I be:
BO is "big-endian"7 − ((O − PO) mod 8)
BO is "little-endian"(O − PO) mod 8
-
-
Depending on the value of the bit at the index BIT_I, where 0 is the index of the least significant bit, within the byte at the index BYTE_I from the beginning of P (the current packet being decoded):
0 The bit value is false.
1 The bit value is true.
To decode an instance of F:
-
Let:
-
L be the value of the
lengthproperty of F. -
V be an array of booleans of length L.
-
I be an unsigned integer initialized to 0.
-
-
Align O − PO according to F.
-
If ((O − PO) mod 8 ≠ 0) and LAST_BO ≠ BO, then report an error and abort the data stream decoding process.
-
While I < L:
-
Let VI be an unsigned integer initialized to:
BO is "big-endian"0
BO is "little-endian"L − I − 1
-
Set the element at the index VI of V to the current single bit value.
-
Set I to I + 1.
-
Set O to O + 1.
-
-
Set LAST_BO to BO.
V is the decoded bit array value.
To add to the decoding procedure above, note that the “reading direction” within a byte depends on BO. Assuming O − PO = 0, the following diagrams show which bit is selected by O − PO within a 16-bit fixed-length bit array field depending on BO:
"big-endian"-
"little-endian"-
This example shows the binary layout of contiguous big-endian and little-endian fixed-length bit array fields.
Assume that O − PO = 16. All the fixed-length bit array fields of this example have an implicit 1-bit aligment requirement.
Considering the following member classes of some structure field class:
[
{
"name": "green",
"field-class": {
"type": "fixed-length-bit-array",
"length": 3,
"byte-order": "big-endian"
}
},
{
"name": "blue",
"field-class": {
"type": "fixed-length-bit-array",
"length": 9,
"byte-order": "big-endian"
}
},
{
"name": "yellow",
"field-class": {
"type": "fixed-length-bit-array",
"length": 14,
"byte-order": "big-endian"
}
},
{
"name": "red",
"field-class": {
"type": "fixed-length-bit-array",
"length": 4,
"byte-order": "big-endian"
}
}
]The binary layout is as such:
Considering the following member classes of some structure field class,
the only difference with the previous sample being the values of the
byte-order property:
[
{
"name": "green",
"field-class": {
"type": "fixed-length-bit-array",
"length": 3,
"byte-order": "little-endian"
}
},
{
"name": "blue",
"field-class": {
"type": "fixed-length-bit-array",
"length": 9,
"byte-order": "little-endian"
}
},
{
"name": "yellow",
"field-class": {
"type": "fixed-length-bit-array",
"length": 14,
"byte-order": "little-endian"
}
},
{
"name": "red",
"field-class": {
"type": "fixed-length-bit-array",
"length": 4,
"byte-order": "little-endian"
}
}
]The binary layout is as such:
This example shows how the alignment requirement of a fixed-length bit array field can translate into padding bits to skip during the decoding process.
Assume that O − PO = 32.
Considering the following member classes of some structure field class:
[
{
"name": "green",
"field-class": {
"type": "fixed-length-bit-array",
"length": 5,
"byte-order": "big-endian"
}
},
{
"name": "blue",
"field-class": {
"type": "fixed-length-bit-array",
"length": 3,
"byte-order": "big-endian",
"alignment": 8
}
},
{
"name": "yellow",
"field-class": {
"type": "fixed-length-bit-array",
"length": 4,
"byte-order": "big-endian",
"alignment": 4
}
}
]The binary layout is as such:
Considering the following member classes of some structure field class,
the only difference with the previous sample being the values of the
byte-order property:
[
{
"name": "green",
"field-class": {
"type": "fixed-length-bit-array",
"length": 5,
"byte-order": "little-endian"
}
},
{
"name": "blue",
"field-class": {
"type": "fixed-length-bit-array",
"length": 3,
"byte-order": "little-endian",
"alignment": 8
}
},
{
"name": "yellow",
"field-class": {
"type": "fixed-length-bit-array",
"length": 4,
"byte-order": "little-endian",
"alignment": 4
}
}
]The binary layout is as such:
Step 3 of the decoding procedure above requires that a consumer stops the data stream decoding process if the byte order between two contiguous fixed-length bit array fields changes when O − PO is not a mutliple of 8.
In other words, a given data stream byte MUST NOT contain bits of two fixed-length bit array fields having different byte orders.
This example shows how contiguous fixed-length bit array fields may have different byte orders with correct alignment.
Assume that O − PO = 16.
Considering the following member classes of some structure field class:
[
{
"name": "green",
"field-class": {
"type": "fixed-length-bit-array",
"length": 3,
"byte-order": "big-endian"
}
},
{
"name": "blue",
"field-class": {
"type": "fixed-length-bit-array",
"length": 5,
"byte-order": "big-endian"
}
},
{
"name": "yellow",
"field-class": {
"type": "fixed-length-bit-array",
"length": 5,
"byte-order": "little-endian"
}
},
{
"name": "orange",
"field-class": {
"type": "fixed-length-bit-array",
"length": 8,
"byte-order": "little-endian"
}
},
{
"name": "red",
"field-class": {
"type": "fixed-length-bit-array",
"length": 6,
"byte-order": "big-endian",
"alignment": 8
}
}
]The binary layout is as such:
6.3.4. Fixed-length boolean field decoding procedure
To decode an instance of a fixed-length boolean field class:
-
Let VB be a boolean.
-
If all the elements of V are false, then set VB to false.
Else, set VB to true.
VB is the decoded boolean value.
6.3.5. Fixed-length unsigned integer field decoding procedure
To decode an instance of a fixed-length unsigned integer field class:
-
Let VI be an unsigned integer.
-
Set VI as the unsigned integer interpretation of V, where the first element of V is the most significant bit.
VI is the decoded unsigned integer value.
6.3.6. Fixed-length signed integer field decoding procedure
To decode an instance of a fixed-length signed integer field class:
-
Let VI be a signed integer.
-
Set VI as the signed integer interpretation, following the two’s complement format, of V, where the first element of V is the most significant bit.
VI is the decoded signed integer value.
6.3.7. Fixed-length floating point number field decoding procedure
To decode an instance of a fixed-length floating point number field class:
-
Let VR be a real value.
-
Set VR to the real number interpretation, following the IEEE 754-2008 binary interchange format, of V, where the first element of V is the most significant bit.
VR is the decoded real value.
6.3.8. Variable-length bit array field decoding procedure
To decode an instance of a variable-length bit array field class F:
-
Let V be an empty array of booleans.
-
Align O − PO according to F.
-
Read N bytes of data from S at the offset O, as many as needed following the unsigned LEB128 format, appending the decoded bits to V as booleans from the most significant to the least significant.
-
Set O to O + N × 8.
V is the decoded value.
Consider the following variable-length bit array field class F:
{
"type": "variable-length-bit-array"
}The following diagram shows the three bytes of an instance of F and the resulting bit array value.
Note that the data bits of the last byte (in red) become the first elements of the resulting bit array value because LEB128 is a little-endian encoding.
6.3.9. Variable-length unsigned integer field decoding procedure
To decode an instance of a variable-length unsigned integer field class:
-
Let VI be an unsigned integer.
-
Set VI as the unsigned integer interpretation of V, where the first element of V is the most significant bit.
VI is the decoded unsigned integer value.
6.3.10. Variable-length signed integer field decoding procedure
To decode an instance of a variable-length signed integer field class:
-
Let VI be a signed integer.
-
Set VI as the signed integer interpretation, following the two’s complement format, of V, where the first element of V is the most significant bit.
VI is the decoded signed integer value.
6.3.11. Null-terminated string field decoding procedure
To decode an instance of a null-terminated string field class F:
-
Let:
-
B be a byte.
-
A be an empty sequence of bytes.
-
V be a string.
-
-
Align O − PO according to F.
-
Read one byte of data from S at the offset O as B.
-
Set O to O + 8.
-
While B ≠ 0:
-
Append B to A.
-
Read one byte of data from S at the offset O as B.
-
Set O to O + 8.
-
-
Decode A, following UTF-8, as V.
V is the decoded string value.
Consider the following null-terminated string field class F:
{
"type": "null-terminated-string"
}The following diagram shows packet bytes including a 22-byte instance of F (in blue) and its resulting string value.
The offset of the null-terminated string field, from the beginning of the packet, is 0xfc23 bytes, which means O − PO = 516,376.
The field contains 21 UTF-8 bytes and a null terminating byte.
The resulting string value contains 19 Unicode characters.
After the field is decoded, O − PO = 516,552.
6.3.12. Static-length string field decoding procedure
To decode an instance of a static-length string field class F:
-
Let:
-
L be the
lengthproperty of F. -
I be an unsigned integer initialized to 0.
-
B be a byte.
-
R be a boolean initialized to true.
-
A be an empty sequence of bytes.
-
V be a string.
-
-
Align O − PO according to F.
-
While I < L:
-
Read one byte of data from S at the offset O as B.
-
If B = 0, then set R to false.
Else, if R is true, then append B to A.
-
Set O to O + 8.
-
Set I to I + 1.
-
-
Decode A, following UTF-8, as V.
V is the decoded string value.
Consider the following null-terminated string field class F:
{
"type": "static-length-string",
"length": 18
}The following diagram shows packet bytes including a 18-byte instance of F (in green) and its resulting string value.
The offset of the static-length string field, from the beginning of the packet, is 0x8c46 bytes, which means O − PO = 287,280.
The field contains 14 UTF-8 bytes, a null terminating byte, and three garbage data bytes to ignore.
The resulting string value contains seven Unicode characters.
After the field is decoded, O − PO = 287,424.
6.3.13. Static-length BLOB field decoding procedure
To decode an instance of a static-length BLOB field class F:
-
Let:
-
L be the
lengthproperty of F. -
V be an array of bytes of length L.
-
-
Align O − PO according to F.
-
Read L bytes of data from S at the offset O as V.
-
Set O to O + L × 8.
V is the decoded BLOB value.
6.3.14. Dynamic-length string field decoding procedure
To decode an instance of a dynamic-length string field class F:
-
Let:
-
L be the value of the previously decoded unsigned integer field of which the
length-field-locationproperty of F indicates the location. -
I be an unsigned integer initialized to 0.
-
B be a byte.
-
R be a boolean initialized to true.
-
A be an empty sequence of bytes.
-
V be a string.
-
-
Align O − PO according to F.
-
While I < L:
-
Read one byte of data from S at the offset O as B.
-
If B = 0, then set R to false.
Else, if R is true, then append B to A.
-
Set O to O + 8.
-
Set I to I + 1.
-
-
Decode A, following UTF-8, as V.
V is the decoded string value.
6.3.15. Dynamic-length BLOB field decoding procedure
To decode an instance of a dynamic-length BLOB field class F:
-
Let:
-
L be the value of the previously decoded unsigned integer field of which the
length-field-locationproperty of F indicates the location. -
V be an array of bytes of length L.
-
-
Align O − PO according to F.
-
Read L bytes of data from S at the offset O as V.
-
Set O to O + L × 8.
V is the decoded BLOB value.
6.3.16. Structure field decoding procedure
To decode an instance of a structure field class F:
-
Let:
-
M be the
member-classesproperty of F. -
V be an empty structure.
-
-
Align O − PO according to F.
-
For each member class MC of M:
-
Let:
-
MF be the
field-classproperty of MC. -
MN be the
nameproperty of MC.
-
-
Decode one instance of MF, appending the resulting value as a member of V named MN.
-
V is the decoded value.
6.3.17. Static-length array field decoding procedure
To decode an instance of a static-length array field class F:
-
Let:
-
L be the
lengthproperty of F. -
EF be the
element-field-classproperty of F. -
I be an unsigned integer initialized to 0.
-
V be an array of values of length L.
-
-
Align O − PO according to F.
-
While I < L:
-
Decode one instance of EF as element I of V.
-
Set I to I + 1.
-
V is the decoded value.
6.3.18. Dynamic-length array field decoding procedure
To decode an instance of a dynamic-length array field class F:
-
Let:
-
L be the value of the previously decoded unsigned integer field of which the
length-field-locationproperty of F indicates the location. -
EF be the
element-field-classproperty of F. -
I be an unsigned integer initialized to 0.
-
V be an array of values of length L
-
-
Align O − PO according to F.
-
While I < L:
-
Decode one instance of EF as element I of V.
-
Set I to I + 1.
-
V is the decoded value.
Consider the following event record payload structure field class F:
{
"type": "structure",
"member-classes": [
{
"name": "len",
"field-class": {
"type": "fixed-length-unsigned-integer",
"length": 16,
"byte-order": "big-endian"
}
},
{
"name": "id",
"field-class": {
"type": "null-terminated-string"
}
},
{
"name": "vals",
"field-class": {
"type":
"type": "dynamic-length-array",
"length-field-location": ["event-record-payload", "len"],
"element-field-class": {
"type": "fixed-length-unsigned-integer",
"length": 32,
"byte-order": "little-endian",
"alignment": 32
}
}
}
]
}The following diagram shows packet bytes including an instance
of F (starting in green and ending in blue) and the resulting
signed integer values of its vals member.
The offset of the len fixed-length unsigned integer field, from the beginning of the packet, is
0x42c1 bytes, which means O − PO = 136,712.
The offset of the vals dynamic-length array field, from the beginning of the
packet, is 0x42c8 bytes, which means
O − PO = 136,768.
The unsigned integer value of the len field is 5, which means the
vals field contains five fixed-length signed integer fields.
After the structure field is decoded, O − PO = 136,928.
6.3.19. Optional field decoding procedure
To decode an instance of an optional field class F:
-
Let:
-
If the class of SEL is a fixed-length boolean field class and SEL is true, then:
-
Decode one instance of OF as V.
Else, if SEL is an element of any integer range of the
selector-field-rangesproperty of F, then:-
Decode one instance of OF as V.
-
V is the decoded value.
6.3.20. Variant field decoding procedure
To decode an instance of an variant field class F:
-
Let:
-
SEL be the value of the previously decoded integer field of which the
selector-field-locationproperty of F indicates the location. -
OPTS be the
optionsproperty of F. -
OF be the
field-classproperty of the variant field class option OPT of OPTS of which SEL is an element of any integer range of theselector-field-rangesproperty of OPT. -
V be a value.
-
-
Decode one instance of OF as V.
V is the decoded value.