Precise Network Measurement Protocol

The precision of some conventional ways used to measure the one-way or round-trip delay and jitter, including ICMP (ping command) and Iperf, a measurement tool, is highly dependent on NTPprecision which is between millisecond and second. As 5G has arisen and it is still continuously evolving, many industrial scenarios, such as internet of vehicles, and other time sensitive services have new requirements for time precision which is in microsecond and even in nanosecond. With the growing support of Precision Time Protocol (PTP) , in many industrial scenarios, such as industrial control network and video transmission network, devices can be synchronized in very high precision that is in sub-microsecond. Although TWAMP has already supported PTP timestamp, as is stated in , the current protocol doesn't allow customizing the length and priorities of packets. Since packets of actual services have different length and priorities, which MAY lead to different time delay, the measurement packets need to be designed to meet such requirements. Moreover, when there are many different paths between source and destination, ECMP, equal cost multi-path algorithm is used to balance the load in different paths. In order to make measurement packets transmitted in the same path as the packet of monitored services, they MUST contain the same 5-tuple elements when computing the hash algorithm. The document defines PNMP by introducing an extendable header over IP header, which could make ECMP algorithm treats the measurement packets and the monitored packets as the same.

NTP Network Time Protocol PTP Precision Time Protocol TWAMP Two-Way Active Measurement Protocol DSCP Differentiated Services Code Point ICMP Internet Control Message Protocol ECMP Equal Cost Multi Path PNMP Precise Network Measurement Protocol

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here.

PNMP needs to modify the IP header and adds an extendable layer between layer 3 and layer 4. The added layer records the information copied from the monitored packet in order to compute the hash algorithm, and additionally, it serves as a sign to tell switches or routers at each hop that the packet is used for network measurement. The major purpose of the definition of PNMP is to ensure that the measurement packet can be treated as much likely the same as the monitored packet. In this way, the out-of-band measurement can approximate the in-band network measurement to a great extent.

Before introducing the extendable PNMP header, some updates in the IP header needs to be declared. Such updates have been shown in the figures below in both IPv4 and IPv6 header format. The protocol fields in both IPv4 and IPv6 headers are updated to represent that the extendable PNMP header is added over layer 3.

0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |Version| IHL | DS | Total Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Identification |Flags| Fragment Offset | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Time to Live | Protocol=PNMP | Header Checksum | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Source IPv4 Address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Destination IPv4 Address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |Version| Traffic Class | Flow Label | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Payload Length |Next Hdr = PNMP| Hop Limit | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | + + | | + Source IPv6 Address + | | + + | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | + + | | + Destination IPv6 Address + | | + + | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

PNMP header format is shown in figure below. The extendable header is a 64-bit header which contains several fields. *Version. This field represents the version number of the protocol. Since the protocol is first defined in this document, the version number is 0 here. *Next Header. Since PNMP header is inserted between layer 3 and layer 4, the next header field needs to record the followed layer 4 header, UDP or TCP. *Source Port. This source port MUST be clarified, because it is not the source port copied from layer 4 of this packet, but from the monitored packet. When using ECMP algorithm to compute the hash value of the chosen 5-tuple elements that contains the source and destination IP address, source and destination port, and the layer 4 protocol, UDP or TCP, PNMP MUST ensure that the measurement packet has the same hash value as the monitored packet. According to this principle, the source port field in PNMP header MUST be the same as the source port in layer 4 header of the monitored packet. *Destination Port. Similarly, this field is the same as the destination port of the monitored packet. *Pre-allocated field. This field is used for some specific purposes.

Another feature of PNMP is that the length and priorities of packets can be set manually in order to get close to the messages of monitored services, and this is crucial for some time sensitive services. Customization of message length and priority can be done in adjustments below.

The complete PTP event or general message is composed by three major parts, header, body, and suffix, as described in PTPv2 . The specification allows the suffix to be zero length if the message does not carry any information other than its timestamp. To simulate the transmission of messages of monitored services, the suffix can be filled with extra bytes, and in this way, the total length of this PTP messages can be the same as the actual ones. Thereby, in the figure below, the suffix is labeled as 'customized'.

0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ~ ~ | header (34 octets) | ~ ~ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | | Timestamp (10 octets) | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ~ ~ | suffix (customized) | ~ ~ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Priorities of packets are set in the DS field of IP header which is defined in . The format of IP header is shown in the figure below where the DS field occupies the second octet. The first 6 bits of the DS field is named as DSCP, differentiated services code point, which are used to represent maximum 64 priorities.

0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |Version| IHL | DS | Total Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Identification |Flags| Fragment Offset | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Time to Live | Protocol=PNMP | Header Checksum | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Source Address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Destination Address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ The complete encapsulation of PTP messages by the UDP/TCP header, PNMP header, IP header, and Mac header is shown in the figure below, with their length and priorities customized.

0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ~ ~ | Ethernet header (14 octets) | ~ ~ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ~ ~ | IP header (20 octets) | ~ ~ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | PNMP header | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | UDP/TCP header | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ~ ~ | PTP Message in Payload (more than 44 octets) | ~ ~ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | FCS | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

With application of PNMP, there are three types of nodes: source node, intermediate node, and tail node. On the source node, we need to enable PNMP based on characteristics of IP packets, IPSA, IPDA, IP Protocol, L4 Source Port, L4 Dest Port.On the intermediate node, the processing of PNMP includes three aspects: first to parse and identify PNMP protocol packet, the secondary is to update timestamp, the third is to perform load balancing forwarding based on the header of PNMP packet under the ECMP routing. The HASH field of PNMP packet is consistent with the original packet to ensure the same forwarding path. under ECMP routing. On tail node, when receiving PNMP packet，the forwarding delay of the path is calculated according to the timestamp carried in the PNMP packet.

* First of all, the network to which both source and destination are connected needs to be synchronized globally. * Before measuring the time delay and jitter between source and destination, measurement mode needs to be enabled and every switch or router MUST support the ability to distinguish packets encapsulated by PNMP header. * At each hop, every monitored packet needs to know the next hop it will go to, so as the measurement packet. Apart from updating the source and destination address in IP header, the PNMP header should be updated too. The source and destination port of monitored packets MUST be recorded first and pasted on the source port and destination port of PNMP header respectively. In this way, when there are multiple paths between two consecutive hops, the measurement packets can be transmitted together with the monitored packets.

TBD.

As is regularized in IANA, the source port or destination port 319 and 320 in UDP/TCP header are defined to represent PTP event message and PTP general message respectively, the source port and destination port in PNMP header MUST not cover 319 or 320.