]> ZTE Corporation

No.6 Huashi Park Rd Wuhan Hubei 430223 China xiong.quan@zte.com.cn

ZTE Corporation

China liu.aihua@zte.com.cn

Cisco Systems, Inc.

Canada rgandhi@cisco.com

Beijing Jiaotong University

Beijing China dyang@bjtu.edu.cn

Routing DETNET This document discusses the specific metadata which should be carried in Enhanced Data plane (EDP), proposes the DetNet data fields and option types for EDP such as Deterministic Latency Action Option. DetNet Data-Fields for EDP can be encapsulated into a variety of protocols such as MPLS, IPv6 and SRv6 networks.

Introduction According to , Deterministic Networking (DetNet) operates at the IP layer and delivers service which provides extremely low data loss rates and bounded latency within a network domain. DetNet data planes has been specified in . The existing deterministic technologies are facing large-scale number of nodes and long-distance transmission, traffic scheduling, dynamic flows, and other controversial issues in large-scale networks. The DetNet Enhanced Data plane (EDP) is required to support a data plane method of flow identification and packet treatment. has described the enhancement requirements for DetNet enhanced data plane, such as aggregated flow identification, redundancy, explicit path selection and deterministic latency guarantees. has proposed the overall framework of DetNet enhancements for large-scale deterministic networks. The packet treatment should schedule the resources and indicate the behaviour to ensure the deterministic latency. Moreover, new functions and related metadata should be supported in enhanced DetNet. This document discusses the specific metadata which should be carried in Enhanced Data plane (EDP), proposes the DetNet data fields and option types for EDP such as Deterministic Latency Action Option. DetNet Data-Fields for EDP can be encapsulated into a variety of protocols such as MPLS, IPv6 and SRv6 networks.

Conventions used in this document

Terminology The terminology is defined as , and .

Requirements Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here. Abbreviations and definitions used in this document:

EDP:

Enhanced Data plane

SRH:

Segment Routing Header

SRv6:

Segment Routing for IPv6 forwarding plane

DLA:

Deterministic Latency Action

Specific Metadata for DetNet Enhanced Data Plane

Queuing-based Metadata As per , the queuing-based mechanisms is an important type of resource to ensure the deterministic latency. As described in , the end-to-end bounded latency depends on the value of queuing delay bound along with the queuing mechanisms. Multiple queuing mechanisms can be used to guarantee the bounded latency in DetNet. And many types of queuing mechanisms have been proposed to provide diversified deterministic service for various applications. For example, time-scheduling queuing mechanisms includes the TAS (Time Aware Shaping) [IIEEE802.1Qbv] and priority-scheduling includes the CBS (Credit-Based Shaper)[IEEE802.1Q-2014] with ATS (Asynchronous Traffic Shaping)[IEEE802.1Qcr]. The cyclic-scheduling queuing mechanism has been proposed such as CQF (Cyclic Queuing and Forwarding) in [IEEE802.1Qch] and improved as per multi-CQF , T-CQF and CSQF . The deadline-scheduling queuing mechanism has been proposed in and improved as per Deadline . The per-flow queuing mechanism includes Guaranteed-Service Integrated service (IntServ) . The asynchronous queuing mechanism includes the Asynchronous Deterministic Networking (ADN) as per and . The Packet Timeslot Mechanism is also proposed as per TQF . The functions such as the queuing mechanisms should be provided for enhanced DetNet to ensure the deterministic latency. And when queuing mechanisms used in large-scale networks, some queuing parameters should be carried for coordination between nodes so as to make appropriate packet forwarding and scheduling decisions to meet the time bounds. The DetNet forwarding nodes along the path can apply the function and the deterministic latency related information should be carried as metadata in the packet to achieve the end-to-end bounded latency.

Traffic class Metadata As per , DetNet service sub-layer may provide traffic scheduling for multiple DetNet flows to achieve the end-to-end bounded latency with differentiated DetNet QoS. The enhanced DetNet data plane may also encode the traffic class metadata in packets. The DetNet Traffic Class (DC) has been defined to indicate the DetNet traffic class as per , The traffic class information can also reuse the IP DSCP or MPLS TC field.

Data Fields for DetNet Enhanced Data Plane

DetNet Option-Types and Data-Fields The enhanced functions and related metadata for DetNet EDP should be confirmed before the encapsulations. While more than one metadata should be carried in EDP, the common DetNet header for EDP should be considered to cover all option-types and data.

DetNet Header for EDP

DetNet-Type: 8-bit unsigned integer, defining the DetNet Option-type for EDP. This document defines an Option: Deterministic Latency Action Option as defined in section 4.2. DetNet-Length: 8-bit unsigned integer, defined the Length of the DetNet Header for EDP in 4-octet units.

DetNet Deterministic Latency Action Option The DetNet Deterministic Latency Action (DLA) Option carries data that is added by the DetNet encapsulating node and interpreted by the decapsulating node. The DetNet transit nodes MAY process the data by forwarding the option data determined by option type and may modify it. The DetNet DLA Option consist of a fixed-size "DetNet DLA Option Header" and a variable-size "DetNet DLA Option Data". The Header and Data may be encapsulated continuously or separately. A Data or more than one Data in lists can be carried in packets.

DetNet DLA Option Header DetNet Deterministic Latency Action (DLA) Option header:

DLA Option header

DLA(Deterministic Latency Action) Type(16 bits): indicates the type of deterministic latency actions for DetNet metadata. The DLA Type can be divided into two parts including behaviour action type and function/queuing type. The format is 16 bits such as 0xFFFF. The DLA Type field is designed as follow:

DLA Type

DetNet DLA Behaviour Type DLA B-type(8 bits): indicates the behaviour action type of packet treatment ensuring the deterministic latency as following shown. This type can also indicate the traffic class.

Behaviour Type (B-type)

DetNet DLA Queuing Type DLA Q-type(8 bits): indicates the type of queuing-based mechanisms or functions ensuring the deterministic latency and related metadata. For example, the functions such as a particular queuing mechanism may be indicated and related parameters should be provided as section 3.1.2 shown.

Queuing/Function Action Sub-type

Data Len: 8-bit unsigned integer. Length of DLA option data, in octets. Ancillary Len: 8-bit unsigned integer. Length of DLA ancillary data, in octets. The types of Deterministic Latency functions should cover all the mechanisms ensuring the Deterministic Latency such as the existing queuing and scheduling mechanisms and other mechanisms which may be proposed in the future.

DetNet DLA Option Data DetNet Deterministic Latency Action option data MUST be aligned by 4 octets:

DLA Option Data Field

DLA option data: Variable-length field. It provides function-based or queuing-based information for a node to forward a DetNet flow. The data of which is determined by the DLA Q-type. The examples of different types of queuing-based data is as following sections shown. DLA ancillary data: Variable-length field. It provides additional information for a node to forward a DetNet flow. The data of which is determined by the DLA type. The DetNet option data and Ancillary data can be provided one time or in list.

Cycle Queuing Data When the Sub-type is set to 0x0001, indicates the Multiple Cyclic Queuing mechanism as defined in and . The Cycle Queuing Data may be carried and designed as following shown:

Cycle Queuing Data

Cycle Profile ID (32bits): indicates the profile ID which the cyclic queue applied at a node. Cycle ID (32bits): indicates the Cycle ID for a node to forward a DetNet flow.

Deadline Queuing Data When the Sub-type is set to 0x0002, indicates the deadline mechanisms as defined in . The Deadline Queuing Data may be carried and designed as follow:

Deadline Queuing Data

Planned and deadline Deviation has been provided as defined in .

Local Deadline Queuing Data When the Sub-type is set to 0x0003, indicates the local deadline mechanisms as defined in . The Local Deadline Queuing Data may be carried and designed as follow:

Local Deadline Queuing Data

Local Deadline: indicates the local deadline as defined in .

Timeslot Queuing Data When the Sub-type is set to 0x0004, indicates the local deadline mechanisms as defined in . The time-slot information may be carried and designed as follow:

Timeslot Queuing Data

Timeslot ID: indicates the identifier of the timeslot as defined in .

Encapsulation Considerations for DetNet Enhanced Data Plane

Metadata for DetNet Enhanced Data Plane The packet treatment should indicate the behaviour action ensuring the deterministic latency at DetNet nodes such as queuing-based mechanisms. The deterministic latency action type and related parameters such as queuing-based information should be carried as metadta in data plane. And the definitions may follow these polices. The data plane enhancement must be generic and the format must be applied to all functions and queuing mechanisms. The metadata and definitions should be common among different candidate queuing solutions. Information and metadata MUST be simplified and limited to be carried in DetNet packets for provided deterministic latency related scheduling along the forwarding path. For example, the queuing-based information should be carried in metadata for coordination between nodes. The requirement of the flow or service may be not suitable to be carried explicitly in DetNet data plane. The packet treatment should schedule the resources and indicate the behaviour to ensure the deterministic latency in forwarding sub-layer. So the queuing mechanisms could be viewed as a type of deterministic resources. The resources type and queuing type should be explicitly indicated.

Encoding for DetNet Enhanced Data Plane Reusing the DSCP or existing field is reasonable and simple to define and easy to standardize. For example, in IPv4 and traditional MPLS networks, it is not suitable to carry new metadata and it is suggested to reuse the original bits such as DSCP . The mapping from DSCP and the metadata such as queuing information MUST be provided in the controller plane. DSCP value may be not sufficient and hard to distinguish between the original DiffServ service and the deterministic service. The DetNet-specific metadata can also be encoded as a common data fields and the definition of data fields is independent from the encapsulating protocols. The data fields could be encapsulated into a variety of protocols, such as MPLS 2.0 , IPv6 , SRv6 and so on.

Security Considerations TBA

IANA Considerations TBA

Acknowledgements TBA

Normative References In many standards track documents several words are used to signify the requirements in the specification. These words are often capitalized. This document defines these words as they should be interpreted in IETF documents. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements. RFC 2119 specifies common key words that may be used in protocol specifications. This document aims to reduce the ambiguity by clarifying that only UPPERCASE usage of the key words have the defined special meanings. This document provides the overall architecture for Deterministic Networking (DetNet), which provides a capability to carry specified unicast or multicast data flows for real-time applications with extremely low data loss rates and bounded latency within a network domain. Techniques used include 1) reserving data-plane resources for individual (or aggregated) DetNet flows in some or all of the intermediate nodes along the path of the flow, 2) providing explicit routes for DetNet flows that do not immediately change with the network topology, and 3) distributing data from DetNet flow packets over time and/or space to ensure delivery of each packet's data in spite of the loss of a path. DetNet operates at the IP layer and delivers service over lower-layer technologies such as MPLS and Time- Sensitive Networking (TSN) as defined by IEEE 802.1. This document provides an overall framework for the Deterministic Networking (DetNet) data plane. It covers concepts and considerations that are generally common to any DetNet data plane specification. It describes related Controller Plane considerations as well. This memo describes the network element behavior required to deliver a guaranteed service (guaranteed delay and bandwidth) in the Internet. [STANDARDS-TRACK] This document presents a timing model for sources, destinations, and Deterministic Networking (DetNet) transit nodes. Using the model, it provides a methodology to compute end-to-end latency and backlog bounds for various queuing methods. The methodology can be used by the management and control planes and by resource reservation algorithms to provide bounded latency and zero congestion loss for the DetNet service. ZTE Corporation CAICT The enhanced Deterministic Networking (DetNet) is required to provide packet treatment for data plane to achieve the DetNet QoS such as deterministic latency. New functions and related metadata should be supported and DetNet data fields and option types such as Deterministic Latency Action (DLA) option should be carried in enhanced DetNet. This document defines how DetNet data fields are encapsulated in IPv6 option. ZTE Corp. ZTE Corp. Cisco Systems, Inc. This document specifies formats and principals for the MPLS header which contains the Deterministic Latency Action (DLA) option, designed for use over a DetNet network with MPLS data plane. It enables guaranteed latency support and makes scheduling decisions for time-sensitive service running on DetNet nodes that operate within a constrained network domain. Futurewei Technologies USA Huawei Technologies University of Surrey ICS Malis Consulting ETRI China Mobile Huawei Technologies Huawei Technologies Huawei Technologies This memo specifies a forwarding method for bounded latency and bounded jitter for Deterministic Networks and is a variant of the IEEE TSN Cyclic Queuing and Forwarding (CQF) method. Tagged CQF (TCQF) supports more than 2 cycles and indicates the cycle number via an existing or new packet header field called the tag to replace the cycle mapping in CQF which is based purely on synchronized reception clock. This memo standardizes TCQF as a mechanism independent of the tagging method used. It also specifies tagging via the (1) the existing MPLS packet Traffic Class (TC) field for MPLS packets, (2) the IP/IPv6 DSCP field for IP/IPv6 packets, and (3) a new TCQF Option header for IPv6 packets. Target benefits of TCQF include low end-to-end jitter, ease of high- speed hardware implementation, optional ability to support large number of flow in large networks via DiffServ style aggregation by applying TCQF to the DetNet aggregate instead of each DetNet flow individually, and support of wide-area DetNet networks with arbitrary link latencies and latency variations as well as low accuracy clock synchronization. Huawei Huawei This document presents a queuing mechanism with multiple cyclic buffers. Sangmyung University ETRI ETRI Huawei China Mobile This document describes an overall framework of Asynchronous Deterministic Networking (ADN) for large-scale networks. It specifies the functional architecture and requirements for providing latency and jitter bounds to high priority traffic, without strict time-synchronization of network nodes. ZTE Corporation China Mobile Oxford Brookes University New H3C Technologies Beijing Jiaotong University This document describes a deterministic forwarding mechanism to IP/ MPLS network, as well as corresponding resource reservation, admission control, policing, etc, to provide guaranteed latency. Especially, latency compensation with core stateless is discussed to replace reshaping and also achieve low jitter. ZTE China Mobile Oxford Brookes University ZTE Beijing Jiaotong University Beijing University of Posts and Telecommunications IP/MPLS networks use packet switching (with the feature store-and- forward) and are based on statistical multiplexing. Statistical multiplexing is essentially a variant of time division multiplexing, which refers to the asynchronous and dynamic allocation of link timeslot resources. In this case, the service flow does not occupy a fixed timeslot, and the length of the timeslot is not fixed, but depends on the size of the packet. Statistical multiplexing has certain challenges and complexity in meeting deterministic QoS, and its delay performance is dependent on the the used queueing mechanism. This document further describes a generic time division multiplexing scheme in IP/MPLS networks, which we call timeslot queueing and forwarding (TQF) mechanism. It aims to bring timeslot resources to layer-3, to make it easier for the control plane to calculate the delay performance based on the deterministic resources, and also make it easier for the data plane to create more flexible timeslot mapping. RAD Routers perform two distinct user-plane functionalities, namely forwarding (where the packet should be sent) and scheduling (when the packet should be sent). One forwarding paradigm is segment routing, in which forwarding instructions are encoded in the packet in a stack data structure, rather than programmed into the routers. Time Sensitive Networking and Deterministic Networking provide several mechanisms for scheduling under the assumption that routers are time synchronized. The most effective mechanisms for delay minimization involve per-flow resource allocation. SRTSN is a unified approach to forwarding and scheduling that uses a single stack data structure. Each stack entry consists of a forwarding portion (e.g., IP addresses or suffixes) and a scheduling portion (deadline by which the packet must exit the router). SRTSN thus fully implements network programming for time sensitive flows, by prescribing to each router both to-where and by-when each packet should be sent. China Mobile Huawei Futurewei Technologies USA ZTE Corporation ETRI New H3C Technologies Huawei Aiming at scaling deterministic networks, this document describes the technical and operational requirements when the network has large variation in latency among hops, great number of flows and/or multiple domains without the same time source. Different deterministic levels of applications co-exist and are transported in such a network. This document also describes the corresponding Deterministic Networking (DetNet) data plane enhancement requirements. ZTE Corporation China Mobile CAICT Beijing Jiaotong University The Enhanced Deterministic Networking (EDN) is required to provide the enhancement of flow identification and packet treatment for Deterministic Networking (DetNet) to achieve the DetNet QoS in large- scale networks. This document proposes the enhancement of packet treatment to support the functions and metadata for Enhanced DetNet Data plane (EDP). It describes related enhanced controller plane considerations as well. ZTE Corporation ZTE Corporation China Mobile This document introduces new IPv6 options for Hop-by-Hop Options header, to carry deadline related information for deterministic flows. ZTE Corporation ZTE Corporation Beijing Jiaotong University This document defines how DetNet data fields are encapsulated as part of the Segment Routing with IPv6 data plane (SRv6) header [RFC8754]. Huawei Huawei China Mobile Sangmyung University ETRI One of the goals of DetNet is to provide bounded end-to-end latency for critical flows. This document defines how to leverage Segment Routing (SR) to implement bounded latency. Specifically, the SR Identifier (SID) is used to specify transmission time (cycles) of a packet. When forwarding devices along the path follow the instructions carried in the packet, the bounded latency is achieved. This is called Cycle Specified Queuing and Forwarding (CSQF) in this document. Since SR is a source routing technology, no per-flow state is maintained at intermediate and egress nodes, SR-based CSQF naturally supports flow aggregation that is deemed to be a key capability to allow DetNet to scale to large networks. Sangmyung University ETRI ETRI Huawei China Mobile This document specifies the framework and the operational procedure for deterministic networks with work conserving packet schedulers that guarantees end-to-end latency bounds to flows. Schedulers in core nodes of the network do not need to maintain flow states. Instead, the entrance node of a flow marks an ideal service completion time, called Finish Time (FT), of a packet in the packet header. The subsequent core nodes update FT by adding a delay factor, which is a function of the flow and upstream nodes. The packets in the queue are served in the ascending order of the FT. This technique is called fair queuing. The result is that flows are isolated from each other almost perfectly. The latency bound of a flow depends only on the flow's intrinsic parameters, except the maximum packet length among other flows sharing each output link with the flow. As per [I-D.ietf-teas-rfc3272bis], DetNet can also be seen as a specialized branch of TE. As it is required to provide enhancements for data plane in scaling networks, this document proposes a set of extensions for traffic engineering to achieve the differentiated DetNet QoS in enhanced DetNet.