This document is a release candidate of the Common Trace Format (CTF) version 2 specification (CTF2-SPEC-2.0).

RFC 2119
The key words MUST, MUST NOT, REQUIRED, SHOULD, SHOULD NOT, MAY, and OPTIONAL in this document, when emphasized, are to be interpreted as described in RFC 2119.

1. Revision history

Table 1. Revision history.
Document Publication date Changes

CTF2‑SPECRC‑6.0

8 July 2022

  • Remove the variable-length bit array field concept, keeping only the variable-length integer field one.

    A variable-length bit array field isn’t flexible enough because its length must be a multiple of 7, whereas a variable-length integer field may contain a specific integral value thanks to sign extension.

  • The preamble fragment MAY have a metadata stream UUID property.

    The purpose of such a UUID is to associate data stream packets to a specific metadata stream through the metadata-stream-uuid role of a packet header static-length BLOB field.

    The metadata-stream-uuid role used to be named trace-class-uuid and had the same purpose, except that it targeted what was called the trace class UUID.

    As of CTF2‑SPECRC‑6.0, the uuid property of a trace class fragment F is the UUID of any instance (trace) of F.

  • Remove the “Trace environment data stream (backward compatibility)” section.

    The new environment property of a trace class fragment F indicates the environment of any instance (trace) of F.

CTF2‑SPECRC‑5.0

5 April 2022

  • Remove JSON text sequence auxiliary streams.

    CTF2‑SPECRC‑4.0 JSON text sequence auxiliary streams and data streams are redundant concepts considering that a CTF 2 data stream class MAY have a namespace and a name to identify its instances as containing specific/auxiliary data.

    Moreover, using data streams to transport “auxiliary” data means:

    • Simpler implementation within existing CTF 1 tools: they can already read and/or write data streams.

      This could mean faster CTF 2 adoption.

    • Simpler CTF 2 specification (JSON text sequence auxiliary stream concept is removed).

    • More compact data streams containing auxiliary data out of the box.

    • A producer may attach user attributes to a data stream through its class.

    Note that if a data stream doesn’t need a default clock, then its class simply needs to not have any default clock class. This makes it possible to have event records only containing data, without timing information, which matches what JSON text sequence auxiliary streams were intended for.

  • Because JSON text sequence auxiliary streams don’t exist anymore, the new “Trace environment data stream (backward compatibility)” section specifies a way to have trace environment data as a data stream.

  • The roles property is only available for fixed-length unsigned integer, variable-length unsigned integer, and static-length BLOB field classes.

    This property used to be common to any field class, but it’s only useful for the specific ones above.

CTF2‑SPECRC‑4.0

2 March 2022

  • During the packet and event record decoding process, make all fields having any role except packet-magic-number and trace-class-uuid update a decoding variable so that there’s no exception.

    Decoding variable interpretation and validation is now required after having decoded the packet header, packet context, and event record header.

    CTF2‑SPECRC‑3.0 only had the fields having the data-stream-class-id, data-stream-id, event-record-class-id, packet-content-size, packet-total-size, and default-clock-timestamp roles update a decoding variable.

  • Remove the packet-beginning-default-clock-timestamp role: use default-clock-timestamp in the packet context instead, like in the event record header.

    The packet decoding procedure specifies that, after having decoded the packet context, if the DEF_CLK_VAL variable is set, then it’s the value of the default data stream clock at the beginning of the packet being decoded. This remains compatible with CTF 1.8 timestamp_begin packet context fields.

  • Specify that the default-clock-timestamp and packet-end-default-clock-timestamp roles require that the data stream class has a default-clock-class-name property.

  • Use the terms “packet content length” and “packet total length” instead of “packet content size” and “packet total size” for consistency.

CTF2‑SPECRC‑3.0

17 December 2021

Add the optional minimum-alignment property to the abstract array field class.

Such a minimum alignment is needed when the alignment requirement of contained array field elements wouldn’t be enough. For example, a producer could write a single 32-bit-aligned, 32-bit little-endian integer value, but use the following static-length array field class for consumers:

{
  "type": "static-length-array",
  "length": 32,
  "minimum-alignment": 32,
  "element-field-class": {
    "type": "fixed-length-boolean",
    "length": 1,
    "byte-order": "little-endian"
  }
}

While the producer writes a single integer value, consumers decode said datum as an array of 32 individual flags (booleans).

Update the Alignment procedure section accordingly.

CTF2‑SPECRC‑2.0

9 December 2021

  • Describe how to use a field location with a procedure in a new Field location procedure section to make constraints easier to understand.

  • Rename the members property of a structure field class to member-classes as this JSON array contains structure field member classes.

  • Make an optional field require that all its possible selector fields be boolean fields, unsigned integer fields, or signed integer fields.

    Correspondingly, make a variant field require that all its possible selector fields be either unsigned integer fields or signed integer fields.

    This constraint exists to accomodate some consumer implementations, in particular the ones with limited integer types.

  • Add the “nil” decoding value type so that decoding an optional field always generates a value.

  • Specify that an “array” decoding value contains a sequence of values, whatever their types.

    Indeed, an array field MAY contain variant fields, making it possible for a resulting array value to contain values having different types.

CTF2‑SPECRC‑1.0

25 November 2021

Initial CTF 2 specification release candidate.

2. What’s CTF 2?

The Common Trace Format version 2 is a binary trace format designed to be very fast to write without compromising great flexibility.

The intention of CTF 2 is that applications written in any programming language, and running on any system (be it Linux or bare metal, for example), can generate traces natively.

A CTF 2 trace has all its data streams described by a metadata stream. Given the rich set of supported data field types, this makes it possible for a CTF 2 producer to append data structures as is to data streams without further “data massaging”. Indeed, the length, alignment, and byte order of fixed-length fields are all configurable parameters within the metadata stream.

CTF 2 is transport agnostic: this document doesn’t specify how to transport or store CTF 2 streams. Other documents can specify such conventions, and conform CTF 2 producers and consumers may or may not adhere to them.

CTF 2 is a major revision of CTF 1, bringing many improvements, such as:

  • Using JSON text sequences for the metadata stream.

  • Simplifying the metadata stream.

  • Adding new field classes.

  • Using roles instead of reserved structure member names to identify “special” fields.

and more, while remaining backward compatible at the data stream level.

3. Common definitions

Common CTF 2 definitions:

Byte

A group of eight bits operated on as a unit.

The bits are indexed such that, if the byte represents an 8-bit unsigned integer, bit 0 is the least significant and bit 7 is the most significant.

Class

A set of values (instances) which share common properties.

For example, a fixed-length unsigned integer field class with an 8-bit length property is the set of the all the fixed-length unsigned integer fields from binary 00000000 to 11111111 (integers 0 to 255).

This specification often states that some class describes instances. For example, an event record class describes event records.

Consumer

A software or hardware system which consumes (reads) the streams of a trace.

A trace consumer is often a trace viewer or a trace analyzer.

Namespace

A string of which the purpose is to avoid naming conflicts.

This document doesn’t specify the format of a namespace. A producer SHOULD use a URI, or at least include a domain name owned by the organization defining the objects under a namespace.

The std namespace is reserved for the CTF 2 specification.
Producer

A software or hardware system which produces (writes) the streams of a trace.

A trace producer is often a tracer.

Sequence

A set of related items which follow each other in a particular order.

Stream

A sequence of bytes.

4. Trace composition

A trace is:

As a reminder, this specification defines a stream as a sequence of bytes.

This document doesn’t specify how to transport or store CTF 2 streams. A producer could serialize all streams as a single file on the file system, or it could send the streams over the network using TCP, to name a few examples.

4.1. Metadata stream (overview)

A metadata stream describes trace data streams with JSON objects.

A metadata stream describes things such as:

Multiple traces MAY share the same metadata stream.

See Metadata stream for the full metadata stream specification.

4.2. Data stream

A data stream is a sequence of one or more data packets:

ctf trace all

In the metadata stream, a data stream class describes data streams.

A packet MUST contain one or more bytes of data.

Although a packet MAY contain padding (garbage data) at the end itself, from the point of view of a data stream, there’s no padding between packets. In other words, the byte following the last byte of a packet is the first byte of the next packet.

A data stream MAY have, conceptually:

One default, monotonic clock

Described by a clock class in the metadata stream.

Packets and event records MAY contain snapshots, named timestamps, of the default clock of their data stream.

One counter of discarded event records

Indicates the number of event records which the producer needed to discard for different reasons.

For example, a tracer could discard an event record when it doesn’t fit some buffer and there’s no other available buffer.

A packet MAY contain a snapshot of this counter.

See Data stream decoding procedure to learn how to decode a CTF 2 data stream.

4.2.1. Packet

A packet is a segment of a data stream.

A packet contains a sequence of data fields or padding (garbage data). In the metadata stream, field classes describe data fields.

A packet P, contained in a data stream S, contains, in this order:

  1. OPTIONAL: A header structure field, described at the trace class level in the metadata stream, which contains, in this order:

    1. OPTIONAL: A fixed-length unsigned integer field which contains the packet magic number value (0xc1fc1fc1, or 3,254,525,889).

    2. In any order:

      • OPTIONAL: One or more static-length BLOB fields which contain the metadata stream UUID.

      • OPTIONAL: One or more unsigned integer fields which contain the current numeric ID of the class of S.

      • OPTIONAL: One or more unsigned integer fields which contain the current numeric ID of S.

  2. OPTIONAL: A context structure field, described at the data stream class level in the metadata stream, which contains, in any order:

    • OPTIONAL: One or more unsigned integer fields which contain the current total length of P, in bits (always a multiple of 8).

    • OPTIONAL: One or more unsigned integer fields which contain the current content length of P, in bits.

    • OPTIONAL: One or more unsigned integer fields which contain the current beginning timestamp or partial timestamp of P.

    • OPTIONAL: One or more unsigned integer fields which contain the current end timestamp of P.

    • OPTIONAL: One or more unsigned integer fields which contain the current snapshot of the discarded event record counter of S at the end of P.

    • OPTIONAL: One or more unsigned integer fields which contain the current sequence number of P within S.

    • OPTIONAL: User fields.

  3. Zero or more event records.

A packet MUST contain one or more bytes of data.

A packet MAY contain padding (garbage data) after its last event record. The length of this padding is the difference between its total length and its content length (as found in its context structure field).

Packets are independent of each other: if one removes a packet from a data stream, a consumer can still decode the whole data stream. This is why:

If the packet context fields of the packets of a data stream contain a packet sequence number field, a consumer can recognize missing packets.

See Packet decoding procedure to learn how to decode a CTF 2 packet.

4.2.2. Event record

An event record is the result of a producer writing a record with OPTIONAL user data when an event occurs during its execution.

A packet contains zero or more event records.

An event record class describes the specific parts of event records.

An event record E, contained in a data stream S, contains, in this order:

  1. OPTIONAL: A header structure field, described at the data stream class level in the metadata stream, which contains, in any order:

    • OPTIONAL: One or more unsigned integer fields which contain the numeric ID of the class of E which has the class of S as its parent.

    • OPTIONAL: One or more unsigned integer fields which contain a timestamp or a partial timestamp.

  2. OPTIONAL: A common context structure field, described at the data stream class level in the metadata stream, which contains user fields.

  3. OPTIONAL: A specific context structure field, described at the event record class level in the metadata stream, which contains user fields.

  4. OPTIONAL: A payload structure field, described at the event record class level in the metadata stream, which contains user fields.

An event record MUST contain one or more bits of data.

The default clock timestamp of an event record, that is, the value of the default clock of its data stream after its header field, if any, is encoded/decoded MUST be greater than or equal to the default clock timestamp of the previous event record, if any, within the same data stream.

See Event record decoding procedure to learn how to decode a CTF 2 event record.

5. Metadata stream

A metadata stream is a JSON text sequence, as specified by RFC 7464, of fragments.

Together, the fragments of a metadata stream contain all the information about the data streams of one or more traces.

A fragment is a JSON object; its allowed properties depend on its type property.

Table 2. Common properties of a fragment F.
Name Type Description Required? Default

type

JSON string

Type of F.

The value of this property MUST be one of:

"preamble"

F is a preamble fragment.

"trace-class"

F is a trace class fragment.

"clock-class"

F is a clock class fragment.

"data-stream-class"

F is a data stream class fragment.

"event-record-class"

F is a event record class fragment.

Yes

user-attributes

User attributes

User attributes of F.

No

{}

extensions

Extensions

Extensions of F.

For any fragment except a preamble fragment, any extension which exists under this property MUST also be declared in the preamble fragment of the same metadata stream.

No

{}

The metadata stream is a JSON text sequence of fragments instead of a single JSON object containing nested objects to enable real-time, or “live”, tracing: a consumer can always decode event records having known event record classes while a producer can always add new event record classes to a data stream class by appending additional fragments to the metadata stream. Once a producer appends a fragment to a metadata stream, the fragment is considered “frozen”, in that it never needs to change.

A metadata stream:

Example 1. Partial metadata stream.

In the sample below, the string <RS> represents a single record separator character (U+001E) and the string [...] represents continuation.

<RS>{
  "type": "preamble",
  "version": 2
}
<RS>[...]

This section doesn’t specify how a metadata stream translates into data stream encoding and decoding rules; it only describes objects and their properties.

See Data stream decoding procedure to learn how to decode a data stream.

5.1. UUID

A preamble fragment, a trace class fragment, and a clock class fragment MAY have a UUID property.

Within a metadata stream, a UUID is a JSON array of 16 JSON integers which are the numeric values of the 16 bytes of the UUID.

Example 2. e53e0ab8-50a1-4f0a-b710-b5f0bba9c4ac UUID.
[229, 62, 10, 184, 80, 161, 79, 10, 183, 16, 181, 240, 187, 169, 196, 172]

5.2. Extensions

A producer MAY add extensions to many metadata stream JSON objects.

The purpose of an extension is to add core features to CTF 2 or to modify existing core features, as specified by this document. In other words, an extension MAY alter the format itself.

This document doesn’t specify what an extension exactly is.

The preamble fragment of the metadata stream contains extension declarations:

  • Any extension in metadata stream objects MUST be declared, by namespace and name, in the preamble fragment.

    Declaring an extension is said to enable it.

  • If a consumer doesn’t support any declared extension, it MUST NOT consume the data streams of the trace.

    The consumer SHOULD report unsupported extensions as an error.

Extensions are a single JSON object, where each property is:

Name

A namespace
Value

A namespaced extensions object

A namespaced extensions object is a JSON object, where each property is:

Name

An extension name
Value

A JSON value

The metadata stream JSON objects which MAY contain extensions as their extensions property are:

Example 3. Three extensions under two namespaces.
{
  "my.tracer": {
    "piano": {
      "keys": 88,
      "temperament": "equal"
    },
    "ramen": 23
  },
  "abc/xyz": {
    "sax": {
      "variant": "alto"
    }
  }
}

5.3. User attributes

A producer MAY add custom user attributes to many metadata stream JSON objects.

This document doesn’t specify what a user attribute exactly is.

Unlike extensions, a consumer MUST NOT consider user attributes to decode data streams.

User attributes are a single JSON object, where each property is:

Name

A namespace
Value

A JSON value

The metadata stream JSON objects which MAY contain user attributes as their user-attributes property are:

Example 4. User attributes under two namespaces.
{
  "my.tracer": {
    "max-count": 45,
    "module": "sys"
  },
  "abc/xyz": true
}

5.4. Field classes

A field class describes fields, that is, sequences of bits as found in a data stream.

A field class contains all the properties a consumer needs to decode a given field.

A field is a field class instance.

This document specifies the following types of field classes:

Abstract field classes

One cannot use the following field classes directly: they are bases for other, concrete field classes:

Fixed/static-length field classes
Variable/dynamic-length field classes
Compound field classes

The following field classes contain one or more field classes.

A field class is a JSON object; its properties depend on its type property.

Table 3. Common properties of a field class F.
Name Type Description Required? Default

type

JSON string

Type of F.

The value of this property MUST be one of:

"fixed-length-bit-array"

F is a fixed-length bit array field class.

"fixed-length-boolean"

F is a fixed-length boolean field class.

"fixed-length-unsigned-integer"
"fixed-length-signed-integer"

F is a fixed-length integer field class.

"fixed-length-unsigned-enumeration"
"fixed-length-signed-enumeration"

F is a fixed-length enumeration field class.

"fixed-length-floating-point-number"

F is a fixed-length floating point number field class.

"variable-length-unsigned-integer"
"variable-length-signed-integer"

F is a variable-length integer field class.

"variable-length-unsigned-enumeration"
"variable-length-signed-enumeration"

F is a variable-length enumeration field class.

"null-terminated-string"

F is a null-terminated string field class.

"static-length-string"

F is a static-length string field class.

"static-length-blob"

F is a static-length BLOB field class.

"dynamic-length-string"

F is a dynamic-length string field class.

"dynamic-length-blob"

F is a dynamic-length BLOB field class.

"structure"

F is a structure field class.

"static-length-array"

F is a static-length array field class.

"dynamic-length-array"

F is a dynamic-length array field class.

"optional"

F is a optional field class.

"variant"

F is a variant field class.

Yes

user-attributes

User attributes

User attributes of F.

No

{}

extensions

Extensions

Extensions of F.

Any extension which exists under this property MUST also be declared in the preamble fragment of the metadata stream.

No

{}

The following fragment properties MUST have a structure field class as their value:

Trace class fragment

packet-header-field-class

Data stream class fragment

  • packet-context-field-class

  • event-record-header-field-class

  • event-record-common-context-field-class

Event record class fragment

  • specific-context-field-class

  • payload-field-class

5.4.1. Field location

A field location is a means for a consumer to locate a field which it needs to decode another, subsequent field.

A consumer needs to locate another field to decode instances of the following classes:

Dynamic-length array field class
Dynamic-length string field class
Dynamic-length BLOB field class

Needs a fixed-length unsigned integer or variable-length unsigned integer length field.

Optional field class

Needs a fixed-length boolean, fixed-length integer, or variable-length integer selector field.

Variant field class

Needs a fixed-length integer or variable-length integer selector field.

Let T be an anteriorly decoded field which a consumer needs to decode another field S. A field location is a JSON array where, in this order:

  1. The first element is the name (JSON string) of a root field from where to start the lookup of T, amongst:

    "packet-header"

    Header of the packet of S.
    "packet-context"

    Context of the packet of S.
    "event-record-header"

    Header of the event record of S.
    "event-record-common-context"

    Common context of the event record of S.
    "event-record-specific-context"

    Specific context of the event record of S.
    "event-record-payload"

    Payload of the event record of S.

    In other words, T MUST be in the same packet or event record as S.

  2. The following elements are structure field member names (JSON strings) to follow to locate the target field.

The length of a field location MUST be greater than or equal to two.

See Field location procedure to learn how to locate a field with a field location.

5.4.2. Integer range set

An integer range set is a JSON array of integer ranges.

An integer range set MUST contain one or more integer ranges.

An integer range is a JSON array of two elements:

  1. The lower bound of the range (JSON integer, included).

  2. The upper bound of the range (JSON integer, included).

An integer range represents all the integer values from the lower bound of the range to its upper bound.

The upper bound of an integer range MUST be greater than or equal to its lower bound.

If both the lower and upper bounds of an integer range are equal, then the integer range represents a single integer value.

Example 5. Integer ranges.
[3, 67]
[-45, 101]
Single integer value.
[42, 42]
Example 6. Integer range set containing three integer ranges.
[[3, 67], [-45, 1], [42, 42]]

5.4.3. Roles

Some fixed-length unsigned integer, variable-length unsigned integer, and static-length BLOB field class instances can have roles.

A role is specific semantics attached to the fields (instances) of a field class. For example, the packet-magic-number role of a fixed-length unsigned integer field class indicates that the value of its instances MUST be the packet magic number (0xc1fc1fc1).

Roles are a JSON array of role names (JSON strings).

See Trace class fragment and Data stream class fragment which indicate accepted roles within their root field classes.

5.4.4. Fixed-length bit array field class

A fixed-length bit array field class describes fixed-length bit array fields.

A fixed-length bit array field is a simple array of contiguous bits, without any attached integer type semantics.

The length, or number of bits, of a fixed-length bit array field is a property (length) of its class.

A fixed-length bit array field class acts as a base of a fixed-length boolean field class, a fixed-length integer field class, and a fixed-length floating point number field class.

Table 4. Common properties of a fixed-length bit array field class F.
Name Type Description Required? Default

type

JSON string

Type of F.

The value of this property MUST be "fixed-length-bit-array".

Yes

length

JSON integer

Number of bits of an instance of F.

The value of this property MUST be greater than zero.

Yes

byte-order

JSON string

Byte order of an instance of F.

The value of this property MUST be one of:

"big-endian"

Big-endian.

"little-endian"

Little-endian.

Yes

alignment

JSON integer

Alignment of the first bit of an instance of F relative to the beginning of the packet which contains this instance.

The value of this property MUST be a positive power of two.

No

1

user-attributes

User attributes

User attributes of F.

No

{}

extensions

Extensions

Extensions of F.

Any extension which exists under this property MUST also be declared in the preamble fragment of the metadata stream.

No

{}
Example 7. Minimal fixed-length bit array field class.
{
  "type": "fixed-length-bit-array",
  "length": 16,
  "byte-order": "little-endian"
}
Example 8. Fixed-length bit array field class with instances aligned to 32 bits.
{
  "type": "fixed-length-bit-array",
  "length": 48,
  "byte-order": "big-endian",
  "alignment": 32
}
Example 9. Fixed-length bit array field class with user attributes.
{
  "type": "fixed-length-bit-array",
  "length": 16,
  "byte-order": "little-endian",
  "user-attributes": {
    "my.tracer": {
      "is-nice": true
    }
  }
}

5.4.5. Fixed-length boolean field class

A fixed-length boolean field class is a fixed-length bit array field class which describes fixed-length boolean fields.

A fixed-length boolean field is a fixed-length bit array field which has the following semantics:

If all the bits of the bit array field are cleared (zero)

The value of the fixed-length boolean field is false.

Otherwise

The value of the fixed-length boolean field is true.

Table 5. Properties of a fixed-length boolean field class F.
Name Type Description Required? Default

type

JSON string

Type of F.

The value of this property MUST be "fixed-length-boolean".

Yes

length

JSON integer

Number of bits of an instance of F.

The value of this property MUST be greater than zero.

Property inherited from the fixed-length bit array field class.

Yes

byte-order

JSON string

Byte order of an instance of F.

The value of this property MUST be one of:

"big-endian"

Big-endian.

"little-endian"

Little-endian.

Property inherited from the fixed-length bit array field class.

Yes

alignment

JSON integer

Alignment of the first bit of an instance of F relative to the beginning of the packet which contains this instance.

The value of this property MUST be a positive power of two.

Property inherited from the fixed-length bit array field class.

No

1

user-attributes

User attributes

User attributes of F.

No

{}

extensions

Extensions

Extensions of F.

Any extension which exists under this property MUST also be declared in the preamble fragment of the metadata stream.

No

{}
Example 10. Minimal fixed-length boolean field class.
{
  "type": "fixed-length-boolean",
  "length": 16,
  "byte-order": "little-endian"
}
Example 11. Fixed-length boolean field class with instances aligned to 32 bits.
{
  "type": "fixed-length-boolean",
  "length": 48,
  "byte-order": "big-endian",
  "alignment": 32
}
Example 12. Fixed-length boolean field class with user attributes.
{
  "type": "fixed-length-boolean",
  "length": 16,
  "byte-order": "little-endian",
  "user-attributes": {
    "my.tracer": {
      "is-nice": true
    }
  }
}

5.4.6. Abstract integer field class

An abstract integer field class is a base of a fixed-length integer field class and a variable-length integer field class.

This field class is abstract in that it only exists to show the relation between different integer field classes in this document: a packet cannot contain an abstract integer field.

Table 6. Common property of an integer field class F.
Name Type Description Required? Default

preferred-display-base

JSON integer

Preferred base to display the value of an instance of F.

The value of this property MUST be one of:

2

Binary base.
8

Octal base.
10

Decimal base.
16

Hexadecimal base.

This property exists to remain backward compatible with CTF 1: it’s not strictly needed to decode an instance of F.

No

10

5.4.7. Fixed-length integer field class

A fixed-length integer field class is both an abstract integer field class and a fixed-length bit array field class which describes fixed-length integer fields.

A fixed-length integer field is a fixed-length bit array field which has integer semantics.

If the value of the type property of a fixed-length integer is "fixed-length-signed-integer", then its instances have the two’s complement format.

A fixed-length integer field class acts as a base of a fixed-length enumeration field class.

Table 7. Common properties of a fixed-length integer field class F.
Name Type Description Required? Default

type

JSON string

Type of F.

The value of this property MUST be one of:

"fixed-length-unsigned-integer"

The instances of F are fixed-length unsigned integer fields.

"fixed-length-signed-integer"

The instances of F are fixed-length signed integer fields.

Yes

length

JSON integer

Number of bits of an instance of F.

The value of this property MUST be greater than zero.

Property inherited from the fixed-length bit array field class.

Yes

byte-order

JSON string

Byte order of an instance of F.

The value of this property MUST be one of:

"big-endian"

Big-endian.

"little-endian"

Little-endian.

Property inherited from the fixed-length bit array field class.

Yes

alignment

JSON integer

Alignment of the first bit of an instance of F relative to the beginning of the packet which contains this instance.

The value of this property MUST be a positive power of two.

Property inherited from the fixed-length bit array field class.

No

1

preferred-display-base

JSON integer

Preferred base to display the value of an instance of F.

The value of this property MUST be one of:

2

Binary base.
8

Octal base.
10

Decimal base.
16

Hexadecimal base.

This property exists to remain backward compatible with CTF 1: it’s not strictly needed to decode an instance of F.

Property inherited from the abstract integer field class.

No

10

roles

Roles

Roles of an instance of F.

See Trace class fragment and Data stream class fragment which indicate accepted within their root field classes.

This property MAY only exist when the type property of F is "fixed-length-unsigned-integer".

No

[]

user-attributes

User attributes

User attributes of F.

No

{}

extensions

Extensions

Extensions of F.

Any extension which exists under this property MUST also be declared in the preamble fragment of the metadata stream.

No

{}
Example 13. Minimal fixed-length unsigned integer field class.
{
  "type": "fixed-length-unsigned-integer",
  "length": 16,
  "byte-order": "little-endian"
}
Example 14. Fixed-length signed integer field class with instances aligned to 32 bits.
{
  "type": "fixed-length-signed-integer",
  "length": 48,
  "byte-order": "big-endian",
  "alignment": 32
}
Example 15. Fixed-length unsigned integer field class with instances to be preferably displayed with a hexadecimal base.
{
  "type": "fixed-length-unsigned-integer",
  "length": 48,
  "byte-order": "big-endian",
  "preferred-display-base": 16
}
Example 16. Fixed-length signed integer field class with user attributes.
{
  "type": "fixed-length-signed-integer",
  "length": 16,
  "byte-order": "little-endian",
  "user-attributes": {
    "my.tracer": {
      "is-nice": true
    }
  }
}

5.4.8. Abstract enumeration field class

An abstract enumeration field class is a base of a fixed-length enumeration field class and a variable-length enumeration field class.

This field class is abstract in that it only exists to show the relation between different enumeration field classes in this document: a packet cannot contain an abstract enumeration field.

An abstract enumeration field class is an abstract integer field class.

An enumeration field is an integer field which MAY have one or more associated names thanks to the mappings property of its class.

Table 8. Common property of an enumeration field class F.
Name Type Description Required? Default

preferred-display-base

JSON integer

Preferred base to display the value of an instance of F.

The value of this property MUST be one of:

2

Binary base.
8

Octal base.
10

Decimal base.
16

Hexadecimal base.

This property exists to remain backward compatible with CTF 1: it’s not strictly needed to decode an instance of F.

Property inherited from the abstract integer field class.

No

10

mappings

Enumeration field class mappings

Mappings of F.

The value of this property MUST contain one or more properties.

Yes
5.4.8.1. Enumeration field class mappings

Enumeration field class mappings map names to integer range sets.

Enumeration field class mappings are a JSON object, where each property is:

Name

Mapping name.
Value

Mapped ranges of integers (integer range set).

The integer ranges of two given mappings MAY overlap.

Enumeration field class mappings MUST contain one or more properties.

Example 17. Enumeration field class mappings with three mappings.

In this example, the fortune and building mappings overlap with the values 4 and 5, and the building and journal mappings overlap with the value 80.

{
  "fortune": [[3, 67], [-45, 1], [84, 84]],
  "building": [[4, 5], [75, 82]],
  "journal": [[100, 2305], [80, 80]]
}

5.4.9. Fixed-length enumeration field class

A fixed-length enumeration field class is both an abstract enumeration field class and a fixed-length integer field class which describes fixed-length enumeration fields.

A fixed-length enumeration field is a fixed-length integer field which MAY have one or more associated names thanks to the mappings property of its class.

If the value of the type property of a fixed-length enumeration field class is "fixed-length-signed-enumeration", then its instances have the two’s complement format.

Table 9. Properties of a fixed-length enumeration field class F.
Name Type Description Required? Default

type

JSON string

Type of F.

The value of this property MUST be one of:

"fixed-length-unsigned-enumeration"

The instances of F are fixed-length unsigned enumeration fields.

"fixed-length-signed-enumeration"

The instances of F are fixed-length signed enumeration fields.

Yes

length

JSON integer

Number of bits of an instance of F.

The value of this property MUST be greater than zero.

Property inherited from the fixed-length bit array field class.

Yes

byte-order

JSON string

Byte order of an instance of F.

The value of this property MUST be one of:

"big-endian"

Big-endian.

"little-endian"

Little-endian.

Property inherited from the fixed-length bit array field class.

Yes

alignment

JSON integer

Alignment of the first bit of an instance of F relative to the beginning of the packet which contains this instance.

The value of this property MUST be a positive power of two.

Property inherited from the fixed-length bit array field class.

No

1

preferred-display-base

JSON integer

Preferred base to display the value of an instance of F.

The value of this property MUST be one of:

2

Binary base.
8

Octal base.
10

Decimal base.
16

Hexadecimal base.

This property exists to remain backward compatible with CTF 1: it’s not strictly needed to decode an instance of F.

Property inherited from the abstract integer field class.

No

10

mappings

Enumeration field class mappings

Mappings of F.

The value of this property MUST contain one or more properties.

Property inherited from the abstract enumeration field class.

Yes

roles

Roles

Roles of an instance of F.

See Trace class fragment and Data stream class fragment which indicate accepted within their root field classes.

This property MAY only exist when the type property of F is "fixed-length-unsigned-enumeration".

No

[]

user-attributes

User attributes

User attributes of F.

No

{}

extensions

Extensions

Extensions of F.

Any extension which exists under this property MUST also be declared in the preamble fragment of the metadata stream.

No

{}
Example 18. Minimal fixed-length unsigned enumeration field class.
{
  "type": "fixed-length-unsigned-enumeration",
  "length": 16,
  "byte-order": "little-endian",
  "mappings": {
    "apple": [[1, 19]]
  }
}
Example 19. Fixed-length signed enumeration field class with instances aligned to 32 bits.
{
  "type": "fixed-length-signed-enumeration",
  "length": 48,
  "byte-order": "big-endian",
  "alignment": 32,
  "mappings": {
    "banana": [[-27399, -1882], [8, 199], [101, 101]],
    "orange": [[67, 67], [43, 1534]]
  }
}
Example 20. Fixed-length unsigned enumeration field class with instances to be preferably displayed with a hexadecimal base.
{
  "type": "fixed-length-unsigned-enumeration",
  "length": 8,
  "byte-order": "big-endian",
  "preferred-display-base": 16,
  "mappings": {
    "lime": [[3, 3]],
    "kiwi": [[8, 8]],
    "blueberry": [[11, 11]]
  }
}
Example 21. Fixed-length signed enumeration field class with user attributes.
{
  "type": "fixed-length-signed-enumeration",
  "length": 16,
  "byte-order": "little-endian",
  "mappings": {
    "mango": [[23, 42]]
  },
  "user-attributes": {
    "my.tracer": {
      "is-nice": true
    }
  }
}

5.4.10. Fixed-length floating point number field class

A fixed-length floating point number field class is a fixed-length bit array field class which describes fixed-length floating point number fields.

A fixed-length floating point number field is a fixed-length bit array field which has floating point number semantics.

Table 10. Properties of a fixed-length floating point number field class F.
Name Type Description Required? Default

type

JSON string

Type of F.

The value of this property MUST be "fixed-length-floating-point-number".

Yes

length

JSON integer

Number of bits of an instance of F.

The value of this property MUST be one of:

16

The instances of F are binary16 floating point numbers, as per the IEEE 754-2008 binary interchange format.

32

The instances of F are binary32 floating point numbers.

64

The instances of F are binary64 floating point numbers.

128

The instances of F are binary128 floating point numbers.

K, where K is greater than 128 and a multiple of 32

The instances of F are binaryK floating point numbers.

Property inherited from the fixed-length bit array field class.

Yes

byte-order

JSON string

Byte order of an instance of F.

The value of this property MUST be one of:

"big-endian"

Big-endian.

"little-endian"

Little-endian.

Property inherited from the fixed-length bit array field class.

Yes

alignment

JSON integer

Alignment of the first bit of an instance of F relative to the beginning of the packet which contains this instance.

The value of this property MUST be a positive power of two.

Property inherited from the fixed-length bit array field class.

No

1

user-attributes

User attributes

User attributes of F.

No

{}

extensions

Extensions

Extensions of F.

Any extension which exists under this property MUST also be declared in the preamble fragment of the metadata stream.

No

{}
Example 22. Minimal binary32 fixed-length floating point number field class.
{
  "type": "fixed-length-floating-point-number",
  "length": 32,
  "byte-order": "little-endian"
}
Example 23. binary64 fixed-length floating point number field class with instances aligned to 32 bits.
{
  "type": "fixed-length-floating-point-number",
  "length": 64,
  "byte-order": "big-endian",
  "alignment": 32
}
Example 24. binary192 fixed-length floating point number field class with user attributes.
{
  "type": "fixed-length-floating-point-number",
  "length": 192,
  "byte-order": "little-endian",
  "user-attributes": {
    "my.tracer": {
      "is-nice": true
    }
  }
}

5.4.11. Variable-length integer field class

A variable-length integer field class is an abstract integer field class which describes variable-length integer fields.

A variable-length integer field is a sequence of bytes with a variable length which contains a multiple of 7 effective bits encoding an unsigned or signed integer. A variable-length integer field is encoded as per LEB128.

If the value of the type property of a variable-length integer field class is "variable-length-signed-integer", then its instances have the two’s complement format.

A variable-length integer field class acts as a base of a variable-length enumeration field class.

Table 11. Common properties of a variable-length integer field class F.
Name Type Description Required? Default

type

JSON string

Type of F.

The value of this property MUST be one of:

"variable-length-unsigned-integer"

The instances of F are variable-length unsigned integer fields.

"variable-length-signed-integer"

The instances of F are variable-length signed integer fields.

Yes

preferred-display-base

JSON integer

Preferred base to display the value of an instance of F.

The value of this property MUST be one of:

2

Binary base.
8

Octal base.
10

Decimal base.
16

Hexadecimal base.

This property exists to remain backward compatible with CTF 1: it’s not strictly needed to decode an instance of F.

Property inherited from the abstract integer field class.

No

10

roles

Roles

Roles of an instance of F.

See Trace class fragment and Data stream class fragment which indicate accepted within their root field classes.

This property MAY only exist when the type property of F is "variable-length-unsigned-integer".

No

[]

user-attributes

User attributes

User attributes of F.

No

{}

extensions

Extensions

Extensions of F.

Any extension which exists under this property MUST also be declared in the preamble fragment of the metadata stream.

No

{}
Example 25. Minimal variable-length unsigned integer field class.
{
  "type": "variable-length-unsigned-integer"
}
Example 26. Variable-length signed integer field class with instances to be preferably displayed with a hexadecimal base.
{
  "type": "variable-length-signed-integer",
  "preferred-display-base": 16
}
Example 27. Variable-length unsigned integer field class with user attributes.
{
  "type": "variable-length-unsigned-integer",
  "user-attributes": {
    "my.tracer": {
      "is-nice": true
    }
  }
}

5.4.12. Variable-length enumeration field class

A variable-length enumeration field class is both an abstract enumeration field class and a variable-length integer field class which describes variable-length enumeration fields.

A variable-length enumeration field is a variable-length integer field which MAY have one or more associated names thanks to the mappings property of its class.

If the value of the type property of a variable-length enumeration field class is "variable-length-signed-enumeration", then its instances have the two’s complement format.

Table 12. Properties of a variable-length enumeration field class F.
Name Type Description Required? Default

type

JSON string

Type of F.

The value of this property MUST be one of:

"variable-length-unsigned-enumeration"

The instances of F are variable-length unsigned enumeration fields.

"variable-length-signed-enumeration"

The instances of F are variable-length signed enumeration fields.

Yes

preferred-display-base

JSON integer

Preferred base to display the value of an instance of F.

The value of this property MUST be one of:

2

Binary base.
8

Octal base.
10

Decimal base.
16

Hexadecimal base.

This property exists to remain backward compatible with CTF 1: it’s not strictly needed to decode an instance of F.

Property inherited from the abstract integer field class.

No

10

mappings

Enumeration field class mappings

Mappings of F.

The value of this property MUST contain one or more properties.

Property inherited from the abstract enumeration field class.

Yes

roles

Roles

Roles of an instance of F.

See Trace class fragment and Data stream class fragment which indicate accepted within their root field classes.

This property MAY only exist when the type property of F is "variable-length-unsigned-enumeration".

No

[]

user-attributes

User attributes

User attributes of F.

No

{}

extensions

Extensions

Extensions of F.

Any extension which exists under this property MUST also be declared in the preamble fragment of the metadata stream.

No

{}
Example 28. Minimal variable-length unsigned enumeration field class.
{
  "type": "variable-length-unsigned-enumeration",
  "mappings": {
    "apple": [[1, 19]]
  }
}
Example 29. Variable-length unsigned enumeration field class with instances to be preferably displayed with a hexadecimal base.
{
  "type": "variable-length-unsigned-enumeration",
  "preferred-display-base": 16,
  "mappings": {
    "lime": [[3, 3]],
    "kiwi": [[8, 8]],
    "blueberry": [[11, 11]]
  }
}
Example 30. Variable-length signed enumeration field class with user attributes.
{
  "type": "variable-length-signed-enumeration",
  "mappings": {
    "banana": [[-27399, -1882], [8, 199], [101, 101]],
    "orange": [[67, 67], [43, 1534]]
  },
  "user-attributes": {
    "my.tracer": {
      "is-nice": true
    }
  }
}

5.4.13. Null-terminated string field class

A null-terminated string field class describes null-terminated string fields.

A null-terminated string field is, in this order:

  1. Zero or more contiguous non-null (non-zero) bytes which form a UTF-8-encoded string.

  2. One null (zero) byte.

Table 13. Properties of a null-terminated string field class F.
Name Type Description Required? Default

type

JSON string

Type of F.

The value of this property MUST be "null-terminated-string".

Yes

user-attributes

User attributes

User attributes of F.

No

{}

extensions

Extensions

Extensions of F.

Any extension which exists under this property MUST also be declared in the preamble fragment of the metadata stream.

No

{}
Example 31. Minimal null-terminated string field class.
{
  "type": "null-terminated-string"
}
Example 32. Null-terminated string field class with user attributes.
{
  "type": "null-terminated-string",
  "user-attributes": {
    "my.tracer": {
      "is-nice": true
    }
  }
}

5.4.14. Static-length string field class

A static-length string field class describes static-length string fields.

A static-length string field is a sequence of zero or more contiguous bytes. All the bytes of a static-length string before the first null (zero) byte, if any, form a UTF-8-encoded string. All the bytes after the first null (zero) byte, if any, are padding (garbage data).

The length, or number of bytes, of a static-length string field is a property (length) of its class.

Table 14. Properties of a static-length string field class F.
Name Type Description Required? Default

type

JSON string

Type of F.

The value of this property MUST be "static-length-string".

Yes

length

JSON integer

Number of bytes contained in an instance of F.

The value of this property MUST be greater than or equal to zero.

Yes

user-attributes

User attributes

User attributes of F.

No

{}

extensions

Extensions

Extensions of F.

Any extension which exists under this property MUST also be declared in the preamble fragment of the metadata stream.

No

{}
Example 33. Empty static-length string field class.
{
  "type": "static-length-string",
  "length": 0
}
Example 34. Static-length string field class with instances having 100 bytes.
{
  "type": "static-length-string",
  "length": 100
}
Example 35. Static-length string field class with user attributes.
{
  "type": "static-length-string",
  "length": 13,
  "user-attributes": {
    "my.tracer": null
  }
}

5.4.15. Dynamic-length string field class

A dynamic-length string field class describes dynamic-length string fields.

A dynamic-length string field is a sequence of zero or more contiguous bytes. All the bytes of a dynamic-length string before the first null (zero) byte, if any, form a UTF-8-encoded string. All the bytes after the first null (zero) byte, if any, are padding (garbage data).

The length, or number of bytes, of a dynamic-length string field is the value of another, anterior (already encoded/decoded) length field. A consumer can locate this length field thanks to the length-field-location property of the dynamic-length string field class.

Table 15. Properties of a dynamic-length string field class F.
Name Type Description Required? Default

type

JSON string

Type of F.

The value of this property MUST be "dynamic-length-string".

Yes

length-field-location

Field location

Location of the field of which the value is the number of bytes contained in an instance of F.

Any length field MUST be an instance of one of:

Yes

user-attributes

User attributes

User attributes of F.

No

{}

extensions

Extensions

Extensions of F.

Any extension which exists under this property MUST also be declared in the preamble fragment of the metadata stream.

No

{}
Example 36. Dynamic-length string field class.
{
  "type": "dynamic-length-string",
  "length-field-location": ["event-record-payload", "length"]
}
Example 37. Dynamic-length string field class with user attributes.
{
  "type": "dynamic-length-string",
  "length-field-location": ["event-record-common-context", "name-length"],
  "user-attributes": {
    "my.tracer": 177
  }
}

5.4.16. Abstract BLOB field class

An abstract BLOB field class is a base of a static-length BLOB field class and a dynamic-length BLOB field class.

This field class is abstract in that it only exists to show the relation between different BLOB field classes in this document: a packet cannot contain an abstract BLOB field.

Table 16. Common properties of a BLOB field class F.
Name Type Description Required? Default

media-type

JSON string

IANA media type of an instance of F.

No

"application/octet-stream"

5.4.17. Static-length BLOB field class

A static-length BLOB field class is an abstract BLOB field class which describes static-length BLOB fields.

A static-length BLOB field is a sequence of zero or more contiguous bytes with an associated IANA media type (given by the media-type property of its class).

The length, or number of bytes, of a static-length BLOB field is a property (length) of its class.

Table 17. Properties of a static-length BLOB field class F.
Name Type Description Required? Default

type

JSON string

Type of F.

The value of this property MUST be "static-length-blob".

Yes

length

JSON integer

Number of bytes contained in an instance of F.

The value of this property MUST be greater than or equal to zero.

Yes

media-type

JSON string

IANA media type of an instance of F.

Property inherited from the abstract BLOB field class.

No

"application/octet-stream"

roles

Roles

Roles of an instance of F.

See Trace class fragment and Data stream class fragment which indicate accepted within their root field classes.

No

[]

user-attributes

User attributes

User attributes of F.

No

{}

extensions

Extensions

Extensions of F.

Any extension which exists under this property MUST also be declared in the preamble fragment of the metadata stream.

No

{}
Example 38. Empty static-length BLOB field class with instances having a default IANA media type.
{
  "type": "static-length-blob",
  "length": 0
}
Example 39. Static-length TIFF BLOB field class with instances having 511,267 bytes.
{
  "type": "static-length-blob",
  "length": 511267,
  "media-type": "image/tif"
}
Example 40. Static-length CSV BLOB field class with user attributes.
{
  "type": "static-length-blob",
  "length": 2400,
  "media-type": "text/csv",
  "user-attributes": {
    "my.tracer": {
      "csv-cols": 12
    }
  }
}

5.4.18. Dynamic-length BLOB field class

A dynamic-length BLOB field class is an abstract BLOB field class which describes dynamic-length BLOB fields.

A dynamic-length BLOB field is a sequence of zero or more contiguous bytes with an associated IANA media type.

The length, or number of bytes, of a dynamic-length BLOB field is the value of another, anterior (already encoded/decoded) length field. A consumer can locate this length field thanks to the length-field-location property of the dynamic-length BLOB field class.

Table 18. Properties of a dynamic-length BLOB field class F.
Name Type Description Required? Default

type

JSON string

Type of F.

The value of this property MUST be "dynamic-length-blob".

Yes

length-field-location

Field location

Location of the field of which the value is the number of bytes contained in an instance of F.

Any length field MUST be an instance of one of:

Yes

media-type

JSON string

IANA media type of an instance of F.

Property inherited from the abstract BLOB field class.

No

"application/octet-stream"

user-attributes

User attributes

User attributes of F.

No

{}

extensions

Extensions

Extensions of F.

Any extension which exists under this property MUST also be declared in the preamble fragment of the metadata stream.

No

{}
Example 41. Dynamic-length BLOB field class with instances having a default IANA media type.
{
  "type": "dynamic-length-blob",
  "length-field-location": ["event-record-payload", "length"]
}
Example 42. Dynamic-length JPEG BLOB field class with user attributes.
{
  "type": "dynamic-length-blob",
  "length-field-location": ["event-record-common-context", "length"],
  "media-type": "image/jpeg",
  "user-attributes": {
    "my.tracer": {
      "quality": 85
    }
  }
}

5.4.19. Structure field class

A structure field class describes structure fields.

A structure field is a sequence of zero or more structure field members. A structure field member is a named field.

Table 19. Properties of a structure field class F.
Name Type Description Required? Default

type

JSON string

Type of F.

The value of this property MUST be "structure".

Yes

member-classes

JSON array of structure field member classes

Classes of the members of an instance of F.

The name property of each member class MUST be unique within the member class names of F.

No

[]

minimum-alignment

JSON integer

Minimum alignment of the first bit of an instance of F relative to the beginning of the packet which contains this instance.

The value of this property MUST be a positive power of two.

The effective alignment of the first bit of an instance of F MAY be greater than the value of this property.

No

1

user-attributes

User attributes

User attributes of F.

No

{}

extensions

Extensions

Extensions of F.

Any extension which exists under this property MUST also be declared in the preamble fragment of the metadata stream.

No

{}
Example 43. Empty structure field class: instances have no members.
{
  "type": "structure"
}
Example 44. Structure field class with three member classes.
{
  "type": "structure",
  "member-classes": [
    {
      "name": "Villeray",
      "field-class": {
        "type": "null-terminated-string"
      }
    },
    {
      "name": "Berri",
      "field-class": {
        "type": "fixed-length-unsigned-integer",
        "length": 32,
        "byte-order": "little-endian",
        "preferred-display-base": 2
      },
      "user-attributes": {
        "my.tracer": {
          "is-mask": true
        }
      }
    },
    {
      "name": "Faillon",
      "field-class": {
        "type": "fixed-length-boolean",
        "length": 8,
        "byte-order": "little-endian"
      }
    }
  ]
}
Example 45. Structure field class with instances minimally aligned to 64 bits.
{
  "type": "structure",
  "member-classes": [
    {
      "name": "St-Denis",
      "field-class": {
        "type": "null-terminated-string"
      }
    },
    {
      "name": "Lajeunesse",
      "field-class": {
        "type": "fixed-length-unsigned-integer",
        "length": 32,
        "byte-order": "big-endian",
        "alignment": 32
      }
    }
  ],
  "minimum-alignment": 64
}
Example 46. Structure field class with user attributes.
{
  "type": "structure",
  "member-classes": [
    {
      "name": "Henri-Julien",
      "field-class": {
        "type": "fixed-length-signed-integer",
        "length": 48,
        "byte-order": "little-endian"
      }
    },
    {
      "name": "Casgrain",
      "field-class": {
        "type": "static-length-string",
        "length": 32
      }
    }
  ],
  "user-attributes": {
    "my.tracer": {
      "version": 4
    }
  }
}
5.4.19.1. Structure field member class

A structure field member class describes structure field members.

A structure field member class is a JSON object.

Table 20. Properties of a structure field member class M.
Name Type Description Required? Default

name

JSON string

Name of M.

Yes

field-class

Field class

Field class of M.

Yes

user-attributes

User attributes

User attributes of M.

No

{}

extensions

Extensions

Extensions of M.

Any extension which exists under this property MUST also be declared in the preamble fragment of the metadata stream.

No

{}
Example 47. Null-terminated string field class member class named cat.
{
  "name": "cat",
  "field-class": {
    "type": "null-terminated-string"
  }
}
Example 48. Variable-length signed integer field class member class named dog with user attributes.
{
  "name": "dog",
  "field-class": {
    "type": "variable-length-signed-integer",
    "preferred-display-base": 8
  },
  "user-attributes": {
    "my.tracer": {
      "uuid": [
        243, 97, 0, 184, 236, 54, 72, 97,
        141, 107, 169, 214, 171, 137, 115, 201
      ],
      "is-pid": true
    }
  }
}

5.4.20. Abstract array field class

An abstract array field class is a base of a static-length array field class and a dynamic-length array field class.

This field class is abstract in that it only exists to show the relation between different array field classes in this document: a packet cannot contain an abstract array field.

Table 21. Common properties of an array field class F.
Name Type Description Required? Default

element-field-class

Field class

Class of the element fields contained in an instance of F.

Yes

minimum-alignment

JSON integer

Minimum alignment of the first bit of an instance of F relative to the beginning of the packet which contains this instance.

The value of this property MUST be a positive power of two.

The effective alignment of the first bit of an instance of F MAY be greater than the value of this property.

No

1

5.4.21. Static-length array field class

A static-length array field class is an abstract array field class which describes static-length array fields.

A static-length array field is a sequence of zero or more element fields.

The length, or number of element fields, of a static-length array field is a property (length) of its class.

Table 22. Properties of a static-length array field class F.
Name Type Description Required? Default

type

JSON string

Type of F.

The value of this property MUST be "static-length-array".

Yes

element-field-class

Field class

Class of the element fields contained in an instance of F.

Property inherited from the abstract array field class.

Yes

minimum-alignment

JSON integer

Minimum alignment of the first bit of an instance of F relative to the beginning of the packet which contains this instance.

The value of this property MUST be a positive power of two.

The effective alignment of the first bit of an instance of F MAY be greater than the value of this property.

Property inherited from the abstract array field class.

No

1

length

JSON integer

Number of element fields contained in an instance of F.

The value of this property MUST be greater than or equal to zero.

Yes

user-attributes

User attributes

User attributes of F.

No

{}

extensions

Extensions

Extensions of F.

Any extension which exists under this property MUST also be declared in the preamble fragment of the metadata stream.

No

{}
Example 49. Empty static-length array field class.
{
  "type": "static-length-array",
  "element-field-class": {
    "type": "fixed-length-signed-integer",
    "length": 16,
    "byte-order": "little-endian",
    "alignment": 16
  },
  "length": 0
}
Example 50. Static-length array field class with instances having 100 null-terminated string fields.
{
  "type": "static-length-array",
  "element-field-class": {
    "type": "null-terminated-string"
  },
  "length": 100
}
Example 51. Static-length array field class with user attributes.
{
  "type": "static-length-array",
  "element-field-class": {
    "type": "variable-length-unsigned-integer"
  },
  "length": 13,
  "user-attributes": {
    "my.tracer": true
  }
}
Example 52. Static-length array field class with instances minimally aligned to 32 bits.

With the following static-length array field class, a producer can write a single 32-bit-aligned, 32-bit little-endian integer value, and have consumers decode it as an array of 32 flags (booleans).

{
  "type": "static-length-array",
  "length": 32,
  "minimum-alignment": 32,
  "element-field-class": {
    "type": "fixed-length-boolean",
    "length": 1,
    "byte-order": "little-endian"
  }
}

5.4.22. Dynamic-length array field class

A dynamic-length array field class is an abstract array field class which describes dynamic-length array fields.

A dynamic-length array field is a sequence of zero or more element fields.

The length, or number of element fields, of a dynamic-length array field is the value of another, anterior (already encoded/decoded) length field. A consumer can locate this length field thanks to the length-field-location property of the dynamic-length array field class.

Table 23. Properties of a dynamic-length array field class F.
Name Type Description Required? Default

type

JSON string

Type of F.

The value of this property MUST be "dynamic-length-array".

Yes

element-field-class

Field class

Class of the element fields contained in an instance of F.

Property inherited from the abstract array field class.

Yes

minimum-alignment

JSON integer

Minimum alignment of the first bit of an instance of F relative to the beginning of the packet which contains this instance.

The value of this property MUST be a positive power of two.

The effective alignment of the first bit of an instance of F MAY be greater than the value of this property.

Property inherited from the abstract array field class.

No

1

length-field-location

Field location

Location of the field of which the value is the number of element fields contained in an instance of F.

Any length field MUST be an instance of one of:

Yes

user-attributes

User attributes

User attributes of F.

No

{}

extensions

Extensions

Extensions of F.

Any extension which exists under this property MUST also be declared in the preamble fragment of the metadata stream.

No

{}
Example 53. Dynamic-length array field class.
{
  "type": "dynamic-length-array",
  "element-field-class": {
    "type": "fixed-length-unsigned-integer",
    "length": 32,
    "byte-order": "big-endian",
    "alignment": 16
  },
  "length-field-location": ["event-record-payload", "length"]
}
Example 54. Dynamic-length array field class with user attributes.
{
  "type": "dynamic-length-array",
  "element-field-class": {
    "type": "variable-length-unsigned-integer"
  },
  "length-field-location": ["packet-context", "common-length"],
  "user-attributes": {
    "my.tracer": 177
  }
}

5.4.23. Optional field class

An optional field class describes optional fields.

An optional field is, depending on the value of another, anterior (already encoded/decoded) selector field, one of:

  • An instance of a given field class (field-class property of the optional field class).

    In this case, the optional field is said to be enabled.

  • A zero-bit field (no field).

    In this case, the optional field is said to be disabled.

A consumer can locate the selector field thanks to the selector-field-location property of the optional field class.

Table 24. Properties of an optional field class F.
Name Type Description Required? Default

type

JSON string

Type of F.

The value of this property MUST be "optional".

Yes

field-class

Field class

Class of an instance of F when it’s enabled.

Yes

selector-field-location

Field location

Location of the field of which the value indicates whether or not an instance of F is enabled.

A selector field S MUST be an instance of one of:

Fixed-length boolean field class

An instance of F is enabled when S is true.

Fixed-length integer field class
Variable-length integer field class

An instance of F is enabled when the value of S is an element of any of the integer ranges of the selector-field-ranges property of F.

For a given instance of F, the type property of the classes of all the possible selector fields MUST be one of:

Yes

selector-field-ranges

Integer range set

Ranges of integers which the value of a selector field MUST be an element of to enable an instance of F.

Yes, if the selector field is an instance of a fixed-length integer field class or a variable-length integer field class.

None if the selector field is an instance of a fixed-length boolean field class.

user-attributes

User attributes

User attributes of F.

No

{}

extensions

Extensions

Extensions of F.

Any extension which exists under this property MUST also be declared in the preamble fragment of the metadata stream.

No

{}
Example 55. Optional static-length array field class with a boolean selector field class.
{
  "type": "optional",
  "selector-field-location": ["event-record-payload", "has-ip"],
  "field-class": {
    "type": "static-length-array",
    "element-field-class": {
      "type": "fixed-length-unsigned-integer",
      "length": 8,
      "byte-order": "little-endian",
      "alignment": 8
    },
    "length": 16
  }
}
Example 56. Optional null-terminated string with a fixed-length signed integer selector field class.
{
  "type": "optional",
  "selector-field-location": ["event-record-payload", "has-ip"],
  "selector-field-ranges": [[-12, -12], [-5, 0], [15, 35]],
  "field-class": {
    "type": "null-terminated-string"
  }
}

5.4.24. Variant field class

A variant field class describes variant fields.

A variant field is, depending on the value of another, anterior (already encoded/decoded) selector field, the instance of a specific, effective field class amongst one or more variant field class options.

A consumer can locate the selector field thanks to the selector-field-location property of the variant field class.

Table 25. Properties of a variant field class F.
Name Type Description Required? Default

type

JSON string

Type of F.

The value of this property MUST be "variant".

Yes

options

JSON array of variant field class options

Options containing the possible effective classes of an instance of F.

This array MUST contain one or more elements.

The name property of each option, if it’s set, MUST be unique within the option names of F.

The integer ranges (selector-field-ranges property) of two given options MUST NOT intersect.

Yes

selector-field-location

Field location

Location of the field of which the value indicates which option of F contains the effective class of an instance of F.

For a given instance of F, the type property of the classes of all the possible selector fields MUST be one of:

Yes

user-attributes

User attributes

User attributes of F.

No

{}

extensions

Extensions

Extensions of F.

Any extension which exists under this property MUST also be declared in the preamble fragment of the metadata stream.

No

{}
Example 57. Variant field class with two options.
{
  "type": "variant",
  "selector-field-location": ["event-record-payload", "sel"],
  "options": [
    {
      "selector-field-ranges": [[5, 5]],
      "field-class": {
        "type": "null-terminated-string"
      }
    },
    {
      "selector-field-ranges": [[8, 8]],
      "field-class": {
        "type": "fixed-length-signed-integer",
        "length": 16,
        "byte-order": "little-endian",
        "preferred-display-base": 8
      }
    }
  ]
}
Example 58. Variant field class within an optional field class which share the same selector field location.

This example shows that an optional field class and a contained variant field class MAY share the same selector field location.

In this example, depending on the value of the selector field:

0

The optional field is not enabled.
1

The optional field is enabled and is a variant field.

The variant field is an instance of a null-terminated string field class (effective class).
2

The optional field is enabled and is a variant field.

The variant field is an instance of a variable-length signed integer field class (effective class).

 
{
  "type": "optional",
  "selector-field-location": ["event-record-payload", "sel"],
  "selector-field-ranges": [[1, 255]],
  "field-class": {
    "type": "variant",
    "selector-field-location": ["event-record-payload", "sel"],
    "options": [
      {
        "selector-field-ranges": [[1, 1]],
        "field-class": {
          "type": "null-terminated-string"
        }
      },
      {
        "selector-field-ranges": [[2, 2]],
        "field-class": {
          "type": "variable-length-signed-integer",
          "preferred-display-base": 16
        }
      }
    ]
  }
}
Example 59. Variant field class with user attributes.
{
  "type": "variant",
  "selector-field-location": ["event-record-specific-context", "sel"],
  "options": [
    {
      "selector-field-ranges": [[5, 5], [10, 10], [15, 15]],
      "field-class": {
        "type": "static-length-string",
        "length": 20
      }
    },
    {
      "selector-field-ranges": [[0, 4], [6, 9], [11, 14], [16, 127]],
      "field-class": {
        "type": "fixed-length-floating-point-number",
        "length": 32,
        "byte-order": "big-endian"
      }
    }
  ],
  "user-attributes": {
    "my.tracer": {
      "owner": "Jimmy",
      "id": 199990
    }
  }
}
5.4.24.1. Variant field class option

A variant field class option contains a possible effective class of a variant field.

A variant field class option O also contains the ranges of integer values (selector-field-ranges property) of which the value of a selector field MUST be an element of for the effective class of a variant field to be the field class of O.

A variant field class option is a JSON object.

Table 26. Properties of a variant field class option O contained in a variant field class F.
Name Type Description Required? Default

field-class

Field class

Field class of O.

Yes

selector-field-ranges

Integer range set

Ranges of integers which the value of a selector field MUST be an element of for the effective class of an instance of F to be the field class (field-class property) of O.

Yes

name

JSON string

Name of O.

This property exists to remain backward compatible with CTF 1: it’s not strictly needed to decode an instance of F.

No

O is unnamed

user-attributes

User attributes

User attributes of O.

No

{}

extensions

Extensions

Extensions of O.

Any extension which exists under this property MUST also be declared in the preamble fragment of the metadata stream.

No

{}
Example 60. Unnamed null-terminated string field class option.
{
  "field-class": {
    "type": "null-terminated-string"
  },
  "selector-field-ranges": [[3, 9]]
}
Example 61. Variable-length signed integer field class option named juice with user attributes.
{
  "name": "juice",
  "field-class": {
    "type": "variable-length-signed-integer",
    "preferred-display-base": 16
  },
  "selector-field-ranges": [[-4, 4], [9, 9], [100, 200]],
  "user-attributes": {
    "my.tracer": {
      "uuid": [
        243, 97, 0, 184, 236, 54, 72, 97,
        141, 107, 169, 214, 171, 137, 115, 201
      ],
      "is-did": true
    }
  }
}

5.5. Preamble fragment

A preamble fragment indicates:

  • The CTF 2 major version (2).

    CTF 2 doesn’t have a minor version: users can use user attributes and extensions to add features to, or change features of, the format which this document specifies.

  • OPTIONAL: The UUID of the whole metadata stream.

    The purpose of such a UUID is to associate data stream packets to a specific metadata stream through the metadata-stream-uuid role of a packet header static-length BLOB field.

  • OPTIONAL: Extension declarations.

    An extension declaration is an initial extension of which the purpose is to declare that it’s enabled within the metadata stream.

    Because an extension MAY alter the CTF 2 format itself, and because a preamble fragment is always the first metadata stream fragment, those extension declarations make it possible for a consumer to gracefully decline the data streams of the trace if it doesn’t support any declared extension.

The first fragment of a metadata stream MUST be a preamble fragment.

Table 27. Properties of a preamble fragment F.
Name Type Description Required? Default

type

JSON string

Type of F.

The value of this property MUST be "preamble".

Yes

version

JSON integer

CTF 2 major version.

The value of this property MUST be 2.

Yes

uuid

UUID

UUID of the whole metadata stream.

The purpose of this UUID is to associate data stream packets to a specific metadata stream through the metadata-stream-uuid role of a packet header static-length BLOB field.

No

The metadata stream has no UUID

user-attributes

User attributes

User attributes of F.

No

{}

extensions

Extensions

Extension declarations of F.

The name of each property is a namespace and its value is a namespaced extensions object.

Within a namespaced extensions object, an extension named N is declared when it exists as a property named N, whatever the value of the property.

No

{}
Example 62. Minimal preamble fragment.
{
  "type": "preamble",
  "version": 2
}
Example 63. Preamble fragment with extension declarations.

The following preamble fragment declares the piano and ramen extensions under the my.tracer namespace.

{
  "type": "preamble",
  "version": 2,
  "extensions": {
    "my.tracer": {
      "piano": {
        "keys": 88,
        "temperament": "equal"
      },
      "ramen": null
    }
  }
}

5.6. Trace class fragment

A trace class describes traces.

Within a metadata stream, a trace class fragment MUST occur, if any, before any data stream class fragment.

Table 28. Properties of a trace class fragment F.
Name Type Description Required? Default

type

JSON string

Type of F.

The value of this property MUST be "trace-class".

Yes

uuid

UUID

UUID of an instance of F.

No

An instance of F has no UUID

environment

JSON object, where each property is:

Name

Environment entry name.

Value

Environment entry value (JSON integer or JSON string).

Environment of an instance of F.

This property exists to remain backward compatible with CTF 1: this document doesn’t specify what an environment entry exactly is.

No

{}

packet-header-field-class

Structure field class

Class of all the packet header fields of an instance of F.

Any field class as the value of this property MUST satisfy at least one of:

No

F has no packet header field class

user-attributes

User attributes

User attributes of F.

No

{}

extensions

Extensions

Extensions of F.

Any extension which exists under this property MUST also be declared in the preamble fragment of the metadata stream.

No

{}

5.6.1. Roles

If the packet-header-field-class property of a trace class fragment exists, then the field classes of its member classes MAY have the following roles:

Table 29. Roles of field classes of the member classes of a packet header field class.
Name Description Field class (F) constraints Other constraints

data-stream-class-id

Current data stream class ID.

The purpose of a data stream class ID field is to set the current ID of the class of the data stream of the current packet.

Fixed-length unsigned integer field class or variable-length unsigned integer field class.

data-stream-id

Current data stream ID.

The purpose of a data stream ID field is to set the current ID of the data stream of the current packet.

Combined with the ID of its class, such a field makes it possible to uniquely identify a data stream within a trace.

Fixed-length unsigned integer field class or variable-length unsigned integer field class.

packet-magic-number

Packet magic number.

The purpose of a packet magic number field is to confirm the beginning of a CTF 2 packet.

Fixed-length unsigned integer field class with the following property value:

length

32

An instance of F MUST be the first member of the packet header structure field.

The value of an instance of F value MUST be 0xc1fc1fc1 (3,254,525,889).

metadata-stream-uuid

Metadata stream UUID.

Static-length BLOB field class with the following property value:

length

16

The uuid property of the preamble fragment MUST exist.

The 16 bytes of an instance of F MUST be equal to the 16 JSON integers of the uuid property of the preamble fragment.
Example 64. Metadata stream fragments.
Preamble fragment.
{
  "type": "preamble",
  "version": 2,
  "uuid": [
    30, 201, 100, 148, 228, 2, 69, 70,
    147, 219, 233, 34, 43, 238, 108, 199
  ]
}
Trace class fragment.
{
  "type": "trace-class",
  "packet-header-field-class": {
    "type": "structure",
    "member-classes": [
      {
        "name": "the magic!",
        "field-class": {
          "type": "fixed-length-unsigned-integer",
          "length": 32,
          "byte-order": "little-endian",
          "preferred-display-base": 16,
          "roles": ["packet-magic-number"]
        }
      },
      {
        "name": "the UUID",
        "field-class": {
          "type": "static-length-blob",
          "length": 16,
          "roles": ["metadata-stream-uuid"]
        }
      },
      {
        "name": "my data stream class ID",
        "field-class": {
          "type": "fixed-length-unsigned-integer",
          "length": 8,
          "byte-order": "little-endian",
          "roles": ["data-stream-class-id"]
        }
      },
      {
        "name": "my data stream ID",
        "field-class": {
          "type": "variable-length-unsigned-integer",
          "roles": ["data-stream-id"]
        }
      }
    ]
  }
}

5.7. Clock class fragment

A clock class describes clocks.

A data stream MAY have a default clock.

Within a metadata stream, a clock class fragment MUST occur before any data stream class fragment which refers to it by name with its default-clock-class-name property.

Table 30. Properties of a clock class fragment F.
Name Type Description Required? Default

type

JSON string

Type of F.

The value of this property MUST be "clock-class".

Yes

frequency

JSON integer

Frequency of an instance of F (Hz).

The value of this property MUST be greater than zero.

Yes

name

JSON string

Name of F.

Yes

description

JSON string

Textual description of F.

This property exists to remain backward compatible with CTF 1: it’s not strictly needed to decode data streams.

No

F has no textual description

uuid

UUID

UUID of F.

This property exists to remain backward compatible with CTF 1: it’s not strictly needed to decode data streams.

No

F has no UUID

origin-is-unix-epoch

JSON boolean

Whether or not the origin of an instance of F is the Unix epoch.

If the value of this property is false, then the origin of an instance of F is unknown.

No

true

offset

Clock offset

Offset of an instance of F relative to its origin.

Let:

  1. H be the value of the frequency property of F.

  2. O be the value of this property.

  3. S be the value of the seconds property of O.

  4. C be the value of the cycles property of O.

Then the effective offset of an instance of F, in clock cycles, is S × H + C.

No

{"seconds": 0, "cycles": 0}

precision

JSON integer

Precision of an instance of F (clock cycles).

The value of this property MUST be greater than or equal to zero.

Let P be the value of this property and V the value of an instance of F: the range of possible values of the instance is [V − P, V + P].

No

0

user-attributes

User attributes

User attributes of F.

No

{}

extensions

Extensions

Extensions of F.

Any extension which exists under this property MUST also be declared in the preamble fragment of the metadata stream.

No

{}

Within a metadata stream, two given clock class fragments MUST NOT:

  • Share the same name property value.

  • Share the same uuid property value.

Example 65. Minimal clock class fragment with 1-GHz instances.
{
  "type": "clock-class",
  "name": "my clock class",
  "frequency": 1000000000
}
Example 66. Clock class fragment with a UUID property.
{
  "type": "clock-class",
  "name": "my clock class",
  "frequency": 1000000000,
  "uuid": [
    116, 210, 0, 140, 239, 255, 77, 3,
    129, 99, 233, 226, 134, 106, 207, 32
  ]
}
Example 67. Clock class fragment with instances having a specific offset.
{
  "type": "clock-class",
  "name": "my clock class",
  "frequency": 1000000000,
  "offset": {
    "seconds": 1605112699,
    "cycles": 2878388
  }
}
Example 68. Clock class fragment with instances having a specific precision.
{
  "type": "clock-class",
  "name": "my clock class",
  "frequency": 8000000,
  "precision": 100
}
Example 69. Clock class fragment with instances having an origin which isn’t the Unix epoch.
{
  "type": "clock-class",
  "name": "my clock class",
  "frequency": 1000000000,
  "origin-is-unix-epoch": false
}
Example 70. Clock class fragment with user attributes.
{
  "type": "clock-class",
  "name": "my clock class",
  "frequency": 16000000,
  "user-attributes": {
    "my.tracer": {
      "sys-name": "SOC23",
      "bus": {
        "name": "LMB5",
        "index": 5
      },
      "propagation-delay-ps": 177
    }
  }
}

5.7.1. Clock offset

A clock offset contains the offset of the instances of a clock class relative to their origin.

A clock offset is a JSON object.

Table 31. Properties of a clock offset contained in a clock class fragment F.
Name Type Description Required? Default

seconds

JSON integer

Offset, in seconds, of an instance of F relative to its origin.

No

0

cycles

JSON integer

Offset, in cycles, of an instance of F relative to its origin.

The value of this property MUST be greater than or equal to zero.

The value of this property MUST be less than the value of the frequency property of F.

No

0
Example 71. Minimal clock offset.
{}
Example 72. Clock offset with seconds and cycles.
{
  "seconds": 1605112699,
  "cycles": 2878388
}
Example 73. Clock offset with seconds only.
{
  "seconds": 1605111293
}
Example 74. Negative clock offset.

This example shows that a clock offset MAY be negative, that is, before the origin of the clock.

{
  "seconds": -18003,
  "cycles": 11928547
}

5.8. Data stream class fragment

A data stream class describes data streams.

Within a metadata stream, a data stream class fragment F MUST occur before any event record class fragment of which F is the parent.

Table 32. Properties of a data stream class fragment F.
Name Type Description Required? Default

type

JSON string

Type of F.

The value of this property MUST be "data-stream-class".

Yes

id

JSON integer

Numeric ID of F.

The value of this property MUST be greater than or equal to zero.

No

0

name

JSON string

Name of F.

The purpose of this property, combined with the namespace property, is to uniquely identify a data stream class amongst many producers.

No

F is unnamed

namespace

JSON string

Namespace of F.

The purpose of this property, combined with the name property, is to uniquely identify a data stream class amongst many producers.

No

F has no namespace

default-clock-class-name

JSON string

Name of the class of the default clock of an instance of F.

Within the metadata stream containing F, the clock class fragment which has the value of this property as its name property MUST occur before F.

No

An instance of F has no default clock

packet-context-field-class

Structure field class

Class of all the packet context fields of an instance of F.

No

F has no packet context field class

event-record-header-field-class

Structure field class

Class of all the event record header fields of an instance of F.

Any field class as the value of this property MUST satisfy at least one of:

No

F has no event record header field class

event-record-common-context-field-class

Structure field class

Class of all the event record common context fields of an instance of F.

No

F has no event record common context field class

user-attributes

User attributes

User attributes of F.

No

{}

extensions

Extensions

Extensions of F.

Any extension which exists under this property MUST also be declared in the preamble fragment of the metadata stream.

No

{}

Within a metadata stream, two given data stream class fragments MUST NOT share the same id property value.

5.8.1. Roles

If the packet-context-field-class property of a data stream class fragment exists, then the field classes of its member classes MAY have the following roles:

Table 33. Roles of field classes of the member classes of a packet context field class.
Name Description Field class (F) constraints Other constraints

default-clock-timestamp

Current timestamp of the default clock of the data stream when the packet begins.

Fixed-length unsigned integer field class or variable-length unsigned integer field class.

The data stream class has a default-clock-class-name property.

discarded-event-record-counter-snapshot

Current snapshot of the discarded event record counter of the data stream when the packet ends.

Fixed-length unsigned integer field class or variable-length unsigned integer field class.

packet-content-length

Current content length (bits) of the packet.

Fixed-length unsigned integer field class or variable-length unsigned integer field class.

packet-end-default-clock-timestamp

Current timestamp of the default clock of the data stream when the packet ends.

Fixed-length unsigned integer field class or variable-length unsigned integer field class.

The data stream class has a default-clock-class-name property.

packet-sequence-number

Current sequence number of the packet within its data stream.

Fixed-length unsigned integer field class or variable-length unsigned integer field class.

packet-total-length

Current total length (bits) of the packet.

Fixed-length unsigned integer field class or variable-length unsigned integer field class.

If the event-record-header-field-class property of a data stream class fragment exists, then the field classes of its member classes MAY have the following roles:

Table 34. Roles of field classes of the member classes of an event record header field class.
Name Description Field class (F) constraints Other constraints

default-clock-timestamp

Current timestamp of the default clock of the data stream when the event record occurs.

Fixed-length unsigned integer field class or variable-length unsigned integer field class.

The data stream class has a default-clock-class-name property.

event-record-class-id

Current event record class ID.

The purpose of a field having this role is to set the current ID of the class of the event record within its parent data stream class.

Fixed-length unsigned integer field class or variable-length unsigned integer field class.

5.9. Event record class fragment

An event record class describes event records.

The data stream class fragment of which the value of the id property matches the value of the data-stream-class-id property of an event record class fragment F is considered the parent of F.

Table 35. Properties of an event record class fragment F having the data stream class P as its parent.
Name Type Description Required? Default

type

JSON string

Type of F.

The value of this property MUST be "event-record-class".

Yes

id

JSON integer

Numeric ID of F within P.

The value of this property MUST be greater than or equal to zero.

No

0

data-stream-class-id

JSON integer

Numeric ID of P.

The value of this property MUST be greater than or equal to zero.

Within the metadata stream, P MUST occur before F.

No

0

name

JSON string

Name of F.

The purpose of this property, combined with the namespace property, is to uniquely identify an event record class amongst many producers.

No

F is unnamed

namespace

JSON string

Namespace of F.

The purpose of this property, combined with the name property, is to uniquely identify an event record class amongst many producers.

No

F has no namespace

specific-context-field-class

Structure field class

Class of the event record specific context field of an instance of F.

No

F has no event record specific context field class

payload-field-class

Structure field class

Class of the event record payload field of an instance of F.

No

F has no event record payload field class

user-attributes

User attributes

User attributes of F.

No

{}

extensions

Extensions

Extensions of F.

Any extension which exists under this property MUST also be declared in the preamble fragment of the metadata stream.

No

{}

Within a metadata stream, two given event record class fragments MUST NOT share the same id property value and the same data-stream-class-id property value.

6. Data stream decoding procedure

This section shows how to, procedurally, decode a CTF 2 data stream.

Decoding a data stream is the responsibility of a consumer.

This document doesn’t specify how to encode a data stream, as this procedure implies much more freedom than decoding. One can deduce how to encode a data stream from the decoding procedure.

A consumer needs to keep a data stream decoding state while decoding a data stream. A data stream decoding state comprises the following variable:

Table 36. Variable needed to decode a data stream S.
Name Type Description Initial value

O

Unsigned integer

Current decoding offset/position (bits) from the beginning of S.

0

To decode a data stream S:

6.1. Packet decoding procedure

A consumer needs to keep a packet decoding state while decoding a packet. A packet decoding state comprises the following variables:

Table 37. Variables needed to decode a packet P within a data stream S.
Name Type Description Initial value

DEF_CLK_VAL

Unsigned integer

Current value (clock cycles) of the default clock of S, if any.

0

DISC_ER_SNAP

Optional unsigned integer

Current snapshot of the discarded event record counter of S at the end of P.

None

DSC

Optional data stream class

Current class of S.

None

DSC_ID

Unsigned integer

Current ID of the class of S.

0

DS_ID

Optional unsigned integer

Current ID of S.

None

LAST_BO

Optional string

Byte order of the last decoded fixed-length bit array field.

None

PKT_CONTENT_LEN

Unsigned integer

Current content length (bits) of P.

∞

PKT_END_DEF_CLK_VAL

Optional unsigned integer

Current value (clock cycles) of the default clock of S, if any, at the end of P.

None

PKT_SEQ_NUM

Optional unsigned integer

Current sequence number of P.

None

PKT_TOTAL_LEN

Unsigned integer

Current total length (bits) of P.

∞

To decode a packet P within a data stream S:

  1. Let PO be the current value of O.

    While decoding the packet, O − PO is the current decoding offset/position (bits) from the beginning of P.

  2. If the packet-header-field-class property of the trace class fragment of the metadata stream exists, then decode the header field of P using this property.

    During the packet header field decoding procedure, after having decoded a field F as V, with F having the class C with a roles property:

    • If C has the role data-stream-class-id, then set DSC_ID to V.

    • If C has the role data-stream-id, then set DS_ID to V.

    • If C has the role packet-magic-number, then validate that V is 0xc1fc1fc1 (3,254,525,889).

      A consumer SHOULD report an invalid packet magic number as an error.

    • If C has the role metadata-stream-uuid, then validate that V matches the uuid property of the preamble fragment of the metadata stream of the trace of S.

      A consumer SHOULD report a metadata stream UUID mismatch as an error.

    After having decoded the whole packet header field, if DS_ID is set, then it’s the ID of S within its class. In other words, two data streams MAY have the same ID if they’re instances of different data stream classes.

  3. Set DSC to the data stream class having DSC_ID as the value of its id property.

    If no data stream class has the ID DSC_ID, then report an error and abort the data stream decoding process.

  4. If the packet-context-field-class property of DSC exists, then decode the context field of P using this property.

    During the packet context field decoding procedure, after having decoded a field F as V, with F having the class C with a roles property:

    • If C has the role default-clock-timestamp, then update DEF_CLK_VAL from F.

    • If C has the role discarded-event-record-counter-snapshot, then set DISC_ER_SNAP to V.

    • If C has the role packet-content-length, then set PKT_CONTENT_LEN to V.

    • If C has the role packet-end-default-clock-timestamp, then set PKT_END_DEF_CLK_VAL to V.

    • If C has the role packet-sequence-number, then set PKT_SEQ_NUM to V.

    • If C has the role packet-total-length, then set PKT_TOTAL_LEN to V.

    After having decoded the whole packet context field:

    • If set, DEF_CLK_VAL is the value of the default clock of S at the beginning of P.

    • If set, PKT_END_DEF_CLK_VAL is the value of the default clock of S at the end of P.

    • If both DEF_CLK_VAL and PKT_END_DEF_CLK_VAL are set, and if DEF_CLK_VAL > PKT_END_DEF_CLK_VAL (packet beginning timestamp is greater than packet end timestamp), then a consumer SHOULD report an error.

    • If PKT_TOTAL_LEN is ∞ and PKT_CONTENT_LEN is not ∞, then set PKT_TOTAL_LEN to PKT_CONTENT_LEN.

    • If PKT_CONTENT_LEN is ∞ and PKT_TOTAL_LEN is not ∞, then set PKT_CONTENT_LEN to PKT_TOTAL_LEN.

    • If both PKT_TOTAL_LEN and PKT_CONTENT_LEN are set, and if PKT_CONTENT_LEN > PKT_TOTAL_LEN, then report an error and abort the data stream decoding process.

    • If set, DISC_ER_SNAP is a snapshot of the discarded event record counter of S at the end of P.

    • If set, PKT_SEQ_NUM is the sequence number of P.

  5. While O < PO + PKT_CONTENT_LEN and there’s remaining data in S:

  6. If PKT_TOTAL_LEN and PKT_CONTENT_LEN both are not ∞, then set O to PO + PKT_TOTAL_LEN, effectively skipping end-of-packet padding.

6.2. Event record decoding procedure

A consumer needs to keep an event record decoding state while decoding an event record. An event record decoding state comprises the following variables:

Table 38. Variables needed to decode an event record E within a data stream S.
Name Type Description Initial value

ERC_ID

Unsigned integer

Current ID of the class of E of which the parent is the class of S.

0

ERC

Optional event record class

Current class of E.

None

To decode an event record E within a data stream S:

  1. If the event-record-header-field-class property of DSC exists, then decode the header field of E using this property.

    During the event record header field decoding procedure, after having decoded a field F as V, with F having the class C with a roles property:

    • If C has the role event-record-class-id, then set ERC_ID to V.

    • If C has the role default-clock-timestamp, then update DEF_CLK_VAL from F.

    After having decoded the whole event record header field, DEF_CLK_VAL is the value of the default clock of S when E occurs.

  2. Set ERC to the event record class having:

    • DSC_ID as the value of its data-stream-class-id property.

    • ERC_ID as the value of its id property.

    If no event record class has the ID ERC_ID within a data stream class having the ID DSC_ID, then report an error and abort the data stream decoding process.

  3. If the event-record-common-context-field-class property of DSC exists, then decode the common context field of E using this property.

  4. If the specific-context-field-class property of ERC exists, then decode the specific context field of E using this property.

  5. If the payload-field-class property of ERC exists, then decode the payload field of E using this property.

6.2.1. Clock value update procedure

To update DEF_CLK_VAL from an unsigned integer field F having the unsigned integer value V and the class C:

  1. Let L be an unsigned integer initialized to, depending on the type property of C:

    "fixed-length-unsigned-integer"
    "fixed-length-unsigned-enumeration"

    The value of the length property of C.

    "variable-length-unsigned-integer"
    "variable-length-unsigned-enumeration"

    S ×7, where S is the number of bytes which F occupies with the data stream.

  2. Let MASK be an unsigned integer initialized to 2L − 1.

  3. Let H be an unsigned integer initialized to DEF_CLK_VAL & ~MASK, where “&” is the bitwise AND operator and “~” is the bitwise NOT operator.

  4. Let CUR be an unsigned integer initialized to DEF_CLK_VAL & MASK, where “&” is the bitwise AND operator.

  5. Set DEF_CLK_VAL to:

    If V ≥ CUR

    H + V

    Else

    H + MASK + 1 + V

6.3. Field decoding procedure

The class of a field contains what’s needed to decode it as a value.

While a field is an actual sequence of bits within a data stream, a value is its conceptual interpretation with attached semantics.

The types of values are:

Value type Possible values

Nil

None.

Boolean

True or false.

Unsigned/signed integer

Integral quantity.

Real

Continuous quantity.

String

Sequence of Unicode characters.

Array

Sequence of values.

Structure

Sequence of named values (members).

To decode an instance of a field class F, depending on the value of its type property:

Value of the type property of F Decoding procedure of F

"fixed-length-bit-array"

Decode a fixed-length bit array field.

"fixed-length-boolean"

Decode a fixed-length boolean field.

Decode a fixed-length unsigned integer field.

Decode a fixed-length signed integer field.

"fixed-length-floating-point-number"

Decode a fixed-length floating point number field.

Decode a variable-length unsigned integer field.

Decode a variable-length signed integer.

"null-terminated-string"

Decode a null-terminated string field.

"static-length-string"

Decode a static-length string field.

"static-length-blob"

Decode a static-length BLOB field.

"dynamic-length-string"

Decode a dynamic-length string field.

"dynamic-length-blob"

Decode a dynamic-length BLOB field.

"structure"

Decode a structure field.

"static-length-array"

Decode a static-length array field.

"dynamic-length-array"

Decode a dynamic-length array field.

"optional"

Decode an optional field.

"variant"

Decode a variant field.

6.3.1. Alignment procedure

The decoding procedure of many fields require O − PO to have a specific alignment.

The alignment requirement of an instance of a field class F is, depending on the value of its type property:

type property of F Alignment requirement of an instance of F

The value of the alignment property of F.

8

"structure"

The maximum value of:

  • The value of the minimum-alignment property of F.

  • The alignment requirements of the instances of the field-class property of each member class of the member-classes property of F.

The maximum value of:

  • The value of the minimum-alignment property of F.

  • The alignment requirement of an instance of the element-field-class property of F.

1

To align O − PO to some alignment requirement A (bits):

  • Set O to PO + ((O − PO + A − 1) & −A), where “&” is the bitwise AND operator.

6.3.2. Field location procedure

To locate a previously decoded field using a field location FL:

  1. Let V be, depending on the first element of FL:

    "packet-header"

    The header structure of P (current packet).

    "packet-context"

    The context structure of P.

    "event-record-header"

    The header structure of E (current event record).

    "event-record-common-context"

    The common context structure of E.

    "event-record-specific-context"

    The specific context structure of E.

    "event-record-payload"

    The payload structure of E.

    If the consumer cannot set V because there’s no such structure or because it’s not already decoded nor currently being decoded, then report an error and abort the data stream decoding process.

  2. For each element FLE of FL, starting from the second element:

    1. Let V be the value of the member named FLE within V.

      If no member is named FLE within V, then report an error and abort the data stream decoding process.

      If the member named FLE within V is not already decoded nor currently being decoded, then report an error and abort the data stream decoding process.

    2. Depending on the type of V:

      Boolean
      Signed integer
      Unsigned integer

      If FLE is not the last element of FL, then report an error and abort the data stream decoding process.

      Structure

      Continue.

      Array

      While V is an array:

      • If V is not currently being decoded, then report an error and abort the data stream decoding process.

        Set V to the element of V currently being decoded.

      Other

      Report an error and abort the data stream decoding process.

V is the located field.

Example 75. Dynamic-length array field and its length field in the same root field.

Assume the following JSON object is an event record payload structure field class.

{
  "type": "structure",
  "member-classes": [
    {
      "name": "corn", (3)
      "field-class": {
        "type": "fixed-length-unsigned-integer",
        "length": 32,
        "byte-order": "little-endian"
      }
    },
    {
      "name": "inside",
      "field-class": {
        "type": "fixed-length-unsigned-integer",
        "length": 16,
        "byte-order": "little-endian"
      }
    },
    {
      "name": "carbon",
      "field-class": {
        "type": "dynamic-length-array", (1)
        "length-field-location": ["event-record-payload", "corn"], (2)
        "element-field-class": {
          "type": "null-terminated-string"
        }
      }
    }
  ]
}
1 Dynamic-length array field class.
2 Length field location of the dynamic-length array field class.
3 Length member class.
Example 76. Dynamic-length array field and its length field in the same root field, within the same array field element.

Assume the following JSON object is an event record payload structure field class.

Both the dynamic-length array field and its length field exist within the same element of the static-length array field named nature.

{
  "type": "structure",
  "member-classes": [
    {
      "name": "norm",
      "field-class": {
        "type": "null-terminated-string"
      }
    },
    {
      "name": "nature",
      "field-class": {
        "type": "static-length-array",
        "length": 43,
        "element-field-class": {
          "type": "structure",
          "member-classes": [
            {
              "name": "laser", (3)
              "field-class": {
                "type": "variable-length-unsigned-integer"
              }
            },
            {
              "name": "joystick",
              "field-class": {
                "type": "dynamic-length-array", (1)
                "length-field-location": [ (2)
                  "event-record-payload",
                  "nature",
                  "laser"
                ],
                "element-field-class": {
                  "type": "null-terminated-string"
                }
              }
            }
          ]
        }
      }
    }
  ]
}
1 Dynamic-length array field class.
2 Length field location of the dynamic-length array field class.
3 Length member class.
Example 77. Dynamic-length array and its length field in the same root field, within the same variant field.

Assume the following JSON object is an event record payload structure field class.

Both the dynamic-length array field and its length field exist within the same option of the variant field named clinic.

Moreover, the selector field of the clinic variant field is the lawyer field.

{
  "type": "structure",
  "member-classes": [
    {
      "name": "lawyer", (5)
      "field-class": {
        "type": "fixed-length-signed-integer",
        "length": 16,
        "byte-order": "little-endian"
      }
    },
    {
      "name": "clinic",
      "field-class": {
        "type": "variant",
        "selector-field-location": ["event-record-payload", "lawyer"], (4)
        "options": [
          {
            "selector-field-ranges": [[0, 0]],
            "field-class": {
              "type": "null-terminated-string"
            }
          },
          {
            "selector-field-ranges": [[1, 4]],
            "field-class": {
              "type": "structure",
              "member-classes": [
                {
                  "name": "lemon", (3)
                  "field-class": {
                    "type": "fixed-length-unsigned-integer",
                    "length": 8,
                    "byte-order": "big-endian"
                  }
                },
                {
                  "name": "joystick",
                  "field-class": {
                    "type": "dynamic-length-array", (1)
                    "length-field-location": [ (2)
                      "event-record-payload",
                      "clinic",
                      "lemon"
                    ],
                    "element-field-class": {
                      "type": "null-terminated-string"
                    }
                  }
                }
              ]
            }
          },
          {
            "selector-field-ranges": [[5, 5], [7, 7]],
            "field-class": {
              "type": "fixed-length-boolean",
              "length": 8,
              "byte-order": "little-endian"
            }
          }
        ]
      }
    }
  ]
}
1 Dynamic-length array field class.
2 Length field location of the dynamic-length array field class.
3 Length member class.
4 Selector field location of the variant field class.
5 Selector member class.
Example 78. Dynamic-length array and its length field in the same root field; length field is a variant field.

Assume the following JSON object is an event record payload structure field class.

The length field of the dynamic-length array field is a variant field: it can be an 8-bit, a 16-bit, or a 32-bit fixed-length integer field, depending on the selection of the variant field.

Moreover, the selector field of the variant field is located in another root field (event record specific context).

{
  "type": "structure",
  "member-classes": [
    {
      "name": "glass", (3)
      "field-class": {
        "type": "variant",
        "selector-field-location": ["event-record-specific-context", "sel"],
        "options": [
          {
            "selector-field-ranges": [[0, 0]],
            "field-class": {
              "type": "fixed-length-unsigned-integer", (4)
              "length": 8,
              "byte-order": "little-endian"
            }
          },
          {
            "selector-field-ranges": [[1, 1]],
            "field-class": {
              "type": "fixed-length-unsigned-integer", (4)
              "length": 16,
              "byte-order": "little-endian"
            }
          },
          {
            "selector-field-ranges": [[2, 2]],
            "field-class": {
              "type": "fixed-length-unsigned-integer", (4)
              "length": 32,
              "byte-order": "little-endian"
            }
          }
        ]
      }
    },
    {
      "name": "margin",
      "field-class": {
        "type": "dynamic-length-array", (1)
        "length-field-location": ["event-record-payload", "glass"], (2)
        "element-field-class": {
          "type": "null-terminated-string"
        }
      }
    }
  ]
}
1 Dynamic-length array field class.
2 Length field location of the dynamic-length array field class.
3 Length member class.
4 Possible length field class.
Example 79. Dynamic-length array and its length field in the same root field; structure field containing length field is a variant field.

Assume the following JSON object is an event record payload structure field class.

The length field of the dynamic-length array field is within a structure field which is a variant field.

Moreover:

  • The selector field of the variant field is located in another root field (event record common context).

  • The field class of the third option of the glass variant field class contains a dynamic-length BLOB field class (lock member); the length field of its instance is the previous member (eagle) within the same structure field.

{
  "type": "structure",
  "member-classes": [
    {
      "name": "glass",
      "field-class": {
        "type": "variant",
        "selector-field-location": ["event-record-common-context", "sel"],
        "options": [
          {
            "selector-field-ranges": [[0, 0]],
            "field-class": {
              "type": "structure",
              "member-classes": [
                {
                  "name": "eagle",
                  "field-class": {
                    "type": "fixed-length-unsigned-integer", (3)
                    "length": 16,
                    "byte-order": "little-endian"
                  }
                },
                {
                  "name": "road",
                  "field-class": {
                    "type": "null-terminated-string"
                  }
                }
              ]
            }
          },
          {
            "selector-field-ranges": [[32, 172]],
            "field-class": {
              "type": "structure",
              "member-classes": [
                {
                  "name": "nuance",
                  "field-class": {
                    "type": "null-terminated-string"
                  }
                },
                {
                  "name": "eagle",
                  "field-class": {
                    "type": "fixed-length-unsigned-integer", (3)
                    "length": 24,
                    "byte-order": "big-endian"
                  }
                }
              ]
            }
          },
          {
            "selector-field-ranges": [[5, 5]],
            "field-class": {
              "type": "structure",
              "member-classes": [
                {
                  "name": "eagle", (5)
                  "field-class": {
                    "type": "variable-length-unsigned-integer" (3)
                  }
                },
                {
                  "name": "lock",
                  "field-class": {
                    "type": "dynamic-length-blob",
                    "length-field-location": [ (4)
                      "event-record-payload",
                      "glass",
                      "eagle"
                    ]
                  }
                }
              ]
            }
          }
        ]
      }
    },
    {
      "name": "margin",
      "field-class": {
        "type": "dynamic-length-array", (1)
        "length-field-location": [ (2)
          "event-record-payload",
          "glass",
          "eagle"
        ],
        "element-field-class": {
          "type": "null-terminated-string"
        }
      }
    }
  ]
}
1 Dynamic-length array field class.
2 Length field location of the dynamic-length array field class.
3 Possible length field class.
4 Length field location of the dynamic-length BLOB field class.
5 Length field class for the dynamic-length BLOB field class.

Note that both the dynamic-length array and dynamic-length BLOB field classes have the same length field location.

Example 80. Dynamic-length array and its length field in another root field.

Assume the following JSON objects are the event record specific context and payload structure field classes of the same event record class.

The length field of the dynamic-length array field of the event record payload is within the event record specific context.

Event record specific context field class.
{
  "type": "structure",
  "member-classes": [
    {
      "name": "cook",
      "field-class": {
        "type": "fixed-length-floating-point-number",
        "length": 64,
        "byte-order": "little-endian"
      }
    },
    {
      "name": "vegetable", (1)
      "field-class": {
        "type": "variable-length-unsigned-integer"
      }
    }
  ]
}
1 Length member class.
Event record payload field class.
{
  "type": "structure",
  "member-classes": [
    {
      "name": "avenue",
      "field-class": {
        "type": "dynamic-length-array", (1)
        "length-field-location": [ (2)
          "event-record-specific-context",
          "vegetable"
        ],
        "element-field-class": {
          "type": "null-terminated-string"
        }
      }
    },
    {
      "name": "railroad",
      "field-class": {
        "type": "null-terminated-string"
      }
    }
  ]
}
1 Dynamic-length array field class.
2 Length field location of the dynamic-length array field class.

6.3.3. Fixed-length bit array field decoding procedure

For this whole section about decoding an instance of a fixed-length bit array field class F, let BO be the value of the byte-order property of F.

To read a single data stream bit from an instance of F:

  1. Let:

    • BYTE_I be (O − PO) / 8 (integral division; remainder discarded).

    • BIT_I be:

      BO is "big-endian"

      7 − ((O − PO) mod 8)
      BO is "little-endian"

      (O − PO) mod 8

  2. Depending on the value of the bit at the index BIT_I, where 0 is the index of the least significant bit, within the byte at the index BYTE_I from the beginning of P (the current packet being decoded):

    0

    The bit value is false.
    1

    The bit value is true.

To decode an instance of F:

  1. Let:

    1. L be the value of the length property of F.

    2. V be an array of booleans of length L.

    3. I be an unsigned integer initialized to 0.

  2. Align O − PO according to F.

  3. If ((O − PO) mod 8 ≠ 0) and LAST_BO ≠ BO, then report an error and abort the data stream decoding process.

  4. While I < L:

    1. Let VI be an unsigned integer initialized to:

      BO is "big-endian"

      0
      BO is "little-endian"

      L − I − 1

    2. Set the element at the index VI of V to the current single bit value.

    3. Set I to I + 1.

    4. Set O to O + 1.

  5. Set LAST_BO to BO.

V is the decoded bit array value.

To add to the decoding procedure above, note that the “reading direction” within a byte depends on BO. Assuming O − PO = 0, the following diagrams show which bit is selected by O − PO within a 16-bit fixed-length bit array field depending on BO:

"big-endian"

bit array decoding be
"little-endian"

bit array decoding le
Example 81. Contiguous fixed-length bit array fields: big-endian versus little-endian.

This example shows the binary layout of contiguous big-endian and little-endian fixed-length bit array fields.

Assume that O − PO = 16. All the fixed-length bit array fields of this example have an implicit 1-bit aligment requirement.

Considering the following member classes of some structure field class:

[
  {
    "name": "green",
    "field-class": {
      "type": "fixed-length-bit-array",
      "length": 3,
      "byte-order": "big-endian"
    }
  },
  {
    "name": "blue",
    "field-class": {
      "type": "fixed-length-bit-array",
      "length": 9,
      "byte-order": "big-endian"
    }
  },
  {
    "name": "yellow",
    "field-class": {
      "type": "fixed-length-bit-array",
      "length": 14,
      "byte-order": "big-endian"
    }
  },
  {
    "name": "red",
    "field-class": {
      "type": "fixed-length-bit-array",
      "length": 4,
      "byte-order": "big-endian"
    }
  }
]

The binary layout is as such:

bit array fields be

Considering the following member classes of some structure field class, the only difference with the previous sample being the values of the byte-order property:

[
  {
    "name": "green",
    "field-class": {
      "type": "fixed-length-bit-array",
      "length": 3,
      "byte-order": "little-endian"
    }
  },
  {
    "name": "blue",
    "field-class": {
      "type": "fixed-length-bit-array",
      "length": 9,
      "byte-order": "little-endian"
    }
  },
  {
    "name": "yellow",
    "field-class": {
      "type": "fixed-length-bit-array",
      "length": 14,
      "byte-order": "little-endian"
    }
  },
  {
    "name": "red",
    "field-class": {
      "type": "fixed-length-bit-array",
      "length": 4,
      "byte-order": "little-endian"
    }
  }
]

The binary layout is as such:

bit array fields le
Example 82. Padding between fixed-length bit array fields.

This example shows how the alignment requirement of a fixed-length bit array field can translate into padding bits to skip during the decoding process.

Assume that O − PO = 32.

Considering the following member classes of some structure field class:

[
  {
    "name": "green",
    "field-class": {
      "type": "fixed-length-bit-array",
      "length": 5,
      "byte-order": "big-endian"
    }
  },
  {
    "name": "blue",
    "field-class": {
      "type": "fixed-length-bit-array",
      "length": 3,
      "byte-order": "big-endian",
      "alignment": 8
    }
  },
  {
    "name": "yellow",
    "field-class": {
      "type": "fixed-length-bit-array",
      "length": 4,
      "byte-order": "big-endian",
      "alignment": 4
    }
  }
]

The binary layout is as such:

bit array fields padding be

Considering the following member classes of some structure field class, the only difference with the previous sample being the values of the byte-order property:

[
  {
    "name": "green",
    "field-class": {
      "type": "fixed-length-bit-array",
      "length": 5,
      "byte-order": "little-endian"
    }
  },
  {
    "name": "blue",
    "field-class": {
      "type": "fixed-length-bit-array",
      "length": 3,
      "byte-order": "little-endian",
      "alignment": 8
    }
  },
  {
    "name": "yellow",
    "field-class": {
      "type": "fixed-length-bit-array",
      "length": 4,
      "byte-order": "little-endian",
      "alignment": 4
    }
  }
]

The binary layout is as such:

bit array fields padding le
Example 83. Contiguous fixed-length bit array fields with different byte orders.

Step 3 of the decoding procedure above requires that a consumer stops the data stream decoding process if the byte order between two contiguous fixed-length bit array fields changes when O − PO is not a mutliple of 8.

In other words, a given data stream byte MUST NOT contain bits of two fixed-length bit array fields having different byte orders.

This example shows how contiguous fixed-length bit array fields may have different byte orders with correct alignment.

Assume that O − PO = 16.

Considering the following member classes of some structure field class:

[
  {
    "name": "green",
    "field-class": {
      "type": "fixed-length-bit-array",
      "length": 3,
      "byte-order": "big-endian"
    }
  },
  {
    "name": "blue",
    "field-class": {
      "type": "fixed-length-bit-array",
      "length": 5,
      "byte-order": "big-endian"
    }
  },
  {
    "name": "yellow",
    "field-class": {
      "type": "fixed-length-bit-array",
      "length": 5,
      "byte-order": "little-endian"
    }
  },
  {
    "name": "orange",
    "field-class": {
      "type": "fixed-length-bit-array",
      "length": 8,
      "byte-order": "little-endian"
    }
  },
  {
    "name": "red",
    "field-class": {
      "type": "fixed-length-bit-array",
      "length": 6,
      "byte-order": "big-endian",
      "alignment": 8
    }
  }
]

The binary layout is as such:

bit array fields be le

6.3.4. Fixed-length boolean field decoding procedure

To decode an instance of a fixed-length boolean field class:

  1. Let VB be a boolean.

  2. Decode the instance as a fixed-length bit array field.

  3. If all the elements of V are false, then set VB to false.

    Else, set VB to true.

VB is the decoded boolean value.

6.3.5. Fixed-length unsigned integer field decoding procedure

To decode an instance of a fixed-length unsigned integer field class:

  1. Let VI be an unsigned integer.

  2. Decode the instance as a fixed-length bit array field.

  3. Set VI as the unsigned integer interpretation of V, where the first element of V is the most significant bit.

VI is the decoded unsigned integer value.

6.3.6. Fixed-length signed integer field decoding procedure

To decode an instance of a fixed-length signed integer field class:

  1. Let VI be a signed integer.

  2. Decode the instance as a fixed-length bit array field.

  3. Set VI as the signed integer interpretation, following the two’s complement format, of V, where the first element of V is the most significant bit.

VI is the decoded signed integer value.

6.3.7. Fixed-length floating point number field decoding procedure

To decode an instance of a fixed-length floating point number field class:

  1. Let VR be a real value.

  2. Decode the instance as a fixed-length bit array field.

  3. Set VR to the real number interpretation, following the IEEE 754-2008 binary interchange format, of V, where the first element of V is the most significant bit.

VR is the decoded real value.

6.3.8. Variable-length unsigned integer field decoding procedure

To decode an instance of a variable-length unsigned integer field class F:

  1. Let:

    • V be an empty array of booleans.

    • VI be an unsigned integer.

  2. Align O − PO according to F.

  3. Read N bytes of data from S at the offset O, as many as needed following the unsigned LEB128 format, appending the decoded bits to V as booleans from the most significant to the least significant.

  4. Set VI as the unsigned integer interpretation of V, where the first element of V is the most significant bit.

  5. Set O to O + N × 8.

VI is the decoded unsigned integer value.

Example 84. 3-byte variable-length unsigned integer field decoding.

Consider the following variable-length unsigned integer field class F:

{
  "type": "variable-length-unsigned-integer"
}

The following diagram shows the three bytes of an instance of F and the resulting bits.

vl int

Note that the data bits of the last byte (in red) become the first elements of the resulting bits because LEB128 is a little-endian encoding.

The decoded unsigned integer value is 1,876,916.

6.3.9. Variable-length signed integer field decoding procedure

To decode an instance of a variable-length signed integer field class F:

  1. Let:

    • V be an empty array of booleans.

    • VI be a signed integer.

  2. Align O − PO according to F.

  3. Read N bytes of data from S at the offset O, as many as needed following the unsigned LEB128 format, appending the decoded bits to V as booleans from the most significant to the least significant.

  4. Set VI as the signed integer interpretation (two’s complement) of V, where the first element of V is the most significant bit.

  5. Set O to O + N × 8.

VI is the decoded signed integer value.

Example 85. 3-byte variable-length signed integer field decoding.

Consider the following variable-length signed integer field class F:

{
  "type": "variable-length-signed-integer"
}

The following diagram shows the three bytes of an instance of F and the resulting bits.

vl int

Note that the data bits of the last byte (in red) become the first elements of the resulting bits because LEB128 is a little-endian encoding.

The decoded signed integer value is −220,236.

6.3.10. Null-terminated string field decoding procedure

To decode an instance of a null-terminated string field class F:

  1. Let:

    • B be a byte.

    • A be an empty sequence of bytes.

    • V be a string.

  2. Align O − PO according to F.

  3. Read one byte of data from S at the offset O as B.

  4. Set O to O + 8.

  5. While B ≠ 0:

    1. Append B to A.

    2. Read one byte of data from S at the offset O as B.

    3. Set O to O + 8.

  6. Decode A, following UTF-8, as V.

V is the decoded string value.

Example 86. 22-byte null-terminated string field decoding.

Consider the following null-terminated string field class F:

{
  "type": "null-terminated-string"
}

The following diagram shows packet bytes including a 22-byte instance of F (in blue) and its resulting string value.

The offset of the null-terminated string field, from the beginning of the packet, is 0xfc23 bytes, which means O − PO = 516,376.

str

The field contains 21 UTF-8 bytes and a null terminating byte.

The resulting string value contains 19 Unicode characters.

After the field is decoded, O − PO = 516,552.

6.3.11. Static-length string field decoding procedure

To decode an instance of a static-length string field class F:

  1. Let:

    • L be the length property of F.

    • I be an unsigned integer initialized to 0.

    • B be a byte.

    • R be a boolean initialized to true.

    • A be an empty sequence of bytes.

    • V be a string.

  2. Align O − PO according to F.

  3. While I < L:

    1. Read one byte of data from S at the offset O as B.

    2. If B = 0, then set R to false.

      Else, if R is true, then append B to A.

    3. Set O to O + 8.

    4. Set I to I + 1.

  4. Decode A, following UTF-8, as V.

V is the decoded string value.

Example 87. 18-byte static-length string field decoding.

Consider the following null-terminated string field class F:

{
  "type": "static-length-string",
  "length": 18
}

The following diagram shows packet bytes including a 18-byte instance of F (in green) and its resulting string value.

The offset of the static-length string field, from the beginning of the packet, is 0x8c46 bytes, which means O − PO = 287,280.

sl str

The field contains 14 UTF-8 bytes, a null terminating byte, and three garbage data bytes to ignore.

The resulting string value contains seven Unicode characters.

After the field is decoded, O − PO = 287,424.

6.3.12. Static-length BLOB field decoding procedure

To decode an instance of a static-length BLOB field class F:

  1. Let:

    1. L be the length property of F.

    2. V be an array of bytes of length L.

  2. Align O − PO according to F.

  3. Read L bytes of data from S at the offset O as V.

  4. Set O to O + L × 8.

V is the decoded BLOB value.

6.3.13. Dynamic-length string field decoding procedure

To decode an instance of a dynamic-length string field class F:

  1. Let:

    • L be the value of the previously decoded unsigned integer field of which the length-field-location property of F indicates the location.

    • I be an unsigned integer initialized to 0.

    • B be a byte.

    • R be a boolean initialized to true.

    • A be an empty sequence of bytes.

    • V be a string.

  2. Align O − PO according to F.

  3. While I < L:

    1. Read one byte of data from S at the offset O as B.

    2. If B = 0, then set R to false.

      Else, if R is true, then append B to A.

    3. Set O to O + 8.

    4. Set I to I + 1.

  4. Decode A, following UTF-8, as V.

V is the decoded string value.

6.3.14. Dynamic-length BLOB field decoding procedure

To decode an instance of a dynamic-length BLOB field class F:

  1. Let:

    1. L be the value of the previously decoded unsigned integer field of which the length-field-location property of F indicates the location.

    2. V be an array of bytes of length L.

  2. Align O − PO according to F.

  3. Read L bytes of data from S at the offset O as V.

  4. Set O to O + L × 8.

V is the decoded BLOB value.

6.3.15. Structure field decoding procedure

To decode an instance of a structure field class F:

  1. Let:

    • M be the member-classes property of F.

    • V be an empty structure.

  2. Align O − PO according to F.

  3. For each member class MC of M:

    1. Let:

      • MF be the field-class property of MC.

      • MN be the name property of MC.

    2. Decode one instance of MF, appending the resulting value as a member of V named MN.

V is the decoded value.

6.3.16. Static-length array field decoding procedure

To decode an instance of a static-length array field class F:

  1. Let:

    1. L be the length property of F.

    2. EF be the element-field-class property of F.

    3. I be an unsigned integer initialized to 0.

    4. V be an array of values of length L.

  2. Align O − PO according to F.

  3. While I < L:

    1. Decode one instance of EF as element I of V.

    2. Set I to I + 1.

V is the decoded value.

6.3.17. Dynamic-length array field decoding procedure

To decode an instance of a dynamic-length array field class F:

  1. Let:

    1. L be the value of the previously decoded unsigned integer field of which the length-field-location property of F indicates the location.

    2. EF be the element-field-class property of F.

    3. I be an unsigned integer initialized to 0.

    4. V be an array of values of length L

  2. Align O − PO according to F.

  3. While I < L:

    1. Decode one instance of EF as element I of V.

    2. Set I to I + 1.

V is the decoded value.

Example 88. 5-element dynamic-length array field decoding.

Consider the following event record payload structure field class F:

{
  "type": "structure",
  "member-classes": [
    {
      "name": "len",
      "field-class": {
        "type": "fixed-length-unsigned-integer",
        "length": 16,
        "byte-order": "big-endian"
      }
    },
    {
      "name": "id",
      "field-class": {
        "type": "null-terminated-string"
      }
    },
    {
      "name": "vals",
      "field-class": {
        "type":
        "type": "dynamic-length-array",
        "length-field-location": ["event-record-payload", "len"],
        "element-field-class": {
          "type": "fixed-length-unsigned-integer",
          "length": 32,
          "byte-order": "little-endian",
          "alignment": 32
        }
      }
    }
  ]
}

The following diagram shows packet bytes including an instance of F (starting in green and ending in blue) and the resulting signed integer values of its vals member.

The offset of the len fixed-length unsigned integer field, from the beginning of the packet, is 0x42c1 bytes, which means O − PO = 136,712.

The offset of the vals dynamic-length array field, from the beginning of the packet, is 0x42c8 bytes, which means O − PO = 136,768.

dl array

The unsigned integer value of the len field is 5, which means the vals field contains five fixed-length signed integer fields.

After the structure field is decoded, O − PO = 136,928.

6.3.18. Optional field decoding procedure

To decode an instance of an optional field class F:

  1. Let:

    • SEL be the value of the previously decoded boolean or integer field of which the selector-field-location property of F indicates the location.

    • OF be the field-class property of F.

    • V be a nil value.

  2. If the class of SEL is a fixed-length boolean field class and SEL is true, then:

    • Decode one instance of OF as V.

    Else, if SEL is an element of any integer range of the selector-field-ranges property of F, then:

    • Decode one instance of OF as V.

V is the decoded value.

6.3.19. Variant field decoding procedure

To decode an instance of an variant field class F:

  1. Let:

    1. SEL be the value of the previously decoded integer field of which the selector-field-location property of F indicates the location.

    2. OPTS be the options property of F.

    3. OF be the field-class property of the variant field class option OPT of OPTS of which SEL is an element of any integer range of the selector-field-ranges property of OPT.

    4. V be a value.

  2. Decode one instance of OF as V.

V is the decoded value.