The data networks evolution entails an increase in the volume of transmitted traffic, which requires the quality of service policy usage. The policy implementation will allow classifying network traffic and distributing network resources between traffic classes.
In packet networks traffic is transmitted from the sending node to the receiving node through communication channels and intermediate devices. Generally a data packet is processed by each intermediate device independently. Let's look at the example of a data packet processing by an intermediate network device (Figure 1):
Note, in the modern network devices, network interfaces mostly are combined and can operate as both incoming and outgoing.
Figure 1 - A network device traffic passing scheme |
A network device can be intermediate for several pairs of nodes, and each can transmit data of several services (Figure 2a). Let's look at the scheme there the Network device is intermediate for the traffic of nodes pairs Node-1 - Node-4, Node-2 - Node-5 and Node-3 - Node-6. The first pair transmits data of three services, the second of two, the third of one. If there are no QoS settings, the data of all services get through the general queue in the order they are received on the Network device and in the same order they will be transferred from the queue to outgoing interfaces.
With QoS configured, each of the incoming traffic flows can be classified by its type (for example) and map a separate queue to each class (Figure 2b). Each packet queue can be assigned a priority, which will be taken into account during packets extraction from queues, and will guarantee quality indicators. The traffic flows classification can be performed not only by the services used, but according to other criteria. For example, each pair of nodes can be assigned a separate message queue (Figure 2c).
Figure 2a - Queuing for various services without QoS Figure 2b - Queuing for various services with QoS Figure 2b - Queuing for various users with QoS |
Keep in mind that several intermediate network devices can be located on the data path between the source and the receiver, with independent packets queues on each, i.e. effective QoS policy implementation will require the configuration of all network nodes.
The main conclusions from the previous section, which will be used to define quality metrics:
There are three main quality metrics:
Let's look at the metrics on the example: Node-2 transmits three data packets to Node-5, the data source and recipient are connected to an intermediate Network device, the packets are transmitted within the same service, i.e. their key service fields are the same.
During a data stream transmission, some packets may not be received, or may be received with errors. This process is called as data loss, and is measured as the number of received packets to the number of transmit packets ratio. In the example (Figure 3), Node-2 transmits packets with identifiers 1,2 and 3, however, Node-5 receives only packets 1 and 3, i.e. packet with identifier 2 was lost. There are network mechanisms which allows retransmitting lost data. For example, such mechanisms are the TCP and ARQ protocols.
The causes of data loss can be divided into the following groups:
Figure 3 - Data packet loss example |
Losses affect two indicators: throughput and packet performance, which are not belong to basic.
One of the main indicator practically used is a throughput, which value depends on losses. Throughput is defined by the physical channel capabilities and by the intermediate network devices ability to process the data stream. The link throughput is defined as the maximum amount of data that can be transmitted from a source to a receiver per unit time.
The parameter that affects a throughput and the queues state is a packet performance of the device. Packet performance is a maximum data packets amount of a given length that a device is capable to transmit per unit of time.
A real throughput depends on both packet performance and interface characteristics, therefore, at the network design stage, pay attention to these parameters coherence so avoid the situation when one of them becomes a bottleneck for a link and network segment.
The packet performance is defined by the hardware capabilities of the central processor and the internal memory amount. Network devices process multiple traffic streams with different L2 frame sizes, so the following Ethernet frame size values are used for a performance testing:
Due to the limited internal memory amount, better packet performance is achieved for a minimum frame size. Minimal size frames using assume a large overhead amount: each data frames has an service header, which size does not depend on the size of the frame itself.
For example, the service header length for frames 64 bytes long (Figure 4b) and 156 bytes (Figure 4c) will be the same, but the user data amount will be different. To transmit 138 bytes of user data, three frames 64 bytes long or one frame 156 bytes long will be required, so in the first case 192 bytes are needed, in the second - 156 bytes. If link has the same throughput, large frames will increase efficiency by rising the useful throughput of the system. The Infinet devices performance values in various conditions is shown in the "Performance of the InfiNet Wireless devices" document.
Figure 4 - Examples of various lengths Ethernet frame structure |
Delay is the data packet transmission time from a source to a receiver. The delay value consists of the following parts:
The delay is often measured, as a round-trip time (RTT), i.e. the time it takes for the data packet to be transmitted from a source to a destination and backward. For example, this value is used in the ping command results. The state of intermediate network devices during processing the data packets forward and backward may differ, therefore, usually the round-trip time is not equal to two one-way delays.
Figure 5 - Example of data transfer delay |
CPU loading and the packets queues status on intermediate network devices are frequently changing, so the delay during data packets transmission can change. In the example (Figure 6), the transmission time of packages with identifiers 1 and 2 is different. The difference between the maximum and average delay values is called jitter.
Figure 6 - Example of floating delay in data transfer |
In a redundant network infrastructure data between the source and the receiver can be transmitted in various ways, it will also lead to the jitter appearance. Sometimes the difference between the delays in the link may become so large that the transmitted data packets order will change on the receiving side (Figure 7). In the example, packets with identifiers were received in a different order.
The effect depends on the service characteristics and the ability to restore the original sequence by higher levels network protocols. For example, if the traffic of different services is transmitted through different paths, then it will not affect the disorder of the received data.
Figure 7 - Example of unordered data delivery |
Each of the data transfer services has a requirements set for quality indicators. The RFC 4594 document includes the following service types:
|
|
The various services traffic transmission is performed on a single network infrastructure, which has limited resources, therefore, mechanisms should be provided for the resources distribution between services.
Let's look at the example (Figure 8), Node-2 generates several services traffic with a total speed of 1 Gbit/s, Environment-2 allows to transfer this data stream to an intermediate network device, however, the maximum link throughput between network device and Node-5 is 500 Mbps. Obviously, the data stream cannot be processed completely and part of this stream must be dropped. The QoS task is to make this drops manageable to provide the end services the required metric values. Of course, it is impossible to provide the required performance for all services, as links throughputs do not match, therefore, the QoS policy implementation involves that critical services traffic should be processed first.
Figure 8 - Example of inconsistency in incoming traffic amount and links throughputs |
The example above allows shows two main methods used in the QoS policy implementation:
Let's look at the example above, and add the second intermediate device to the data distribution scheme (Figure 9a). The packets distribution scheme has the following steps:
Each intermediate network device without traffic prioritization settings will increase the data transmission delay, and the delay amount is unpredictable. Thus, a large number of intermediate devices will make real-time services operation impossible because of the quality indicators unattainability, i.e. traffic prioritization must be done along the entire network traffic transmission path (Figure 9b).
Keep in mind that implementing QoS policies is only one component to ensure quality metrics. For maximum effect, the QoS configuration should be synchronized with other settings. For example, using TDMA technology instead of Polling on InfiLINK 2x2 and InfiMAN 2x2 family devices reduces jitter by stabilizing the delay value (see TDMA and Polling: Application features).
Figure 9a - Example of data distribution with partly implemented QoS policy Figure 9b - Example of data distribution with implemented QoS policy |
Traffic transmission path from the management point of view in the network can be described in two ways (Figure 10a, b):
Figure 10a - White-box structure example Figure 10b - Black-box structure example |
To solve the described problem for the black-box network structure, labeling the packet headers can be performed: the priority required for the packet processing is set in a header field and keeped over the whole path. In this case, all intermediate devices can put incoming data in the queue in accordance with the fields values in which the priority is indicated. It will require the standard protocols development and their implementation by equipment manufacturers.
Keep in mind, usually equipment located in an external responsibility zone does not support data prioritization in accordance with the priority values in the service headers. Traffic priority coordination at the responsibility zones border should be performed at the administrative level.
A processing priority for a packet can be set by the service fields of various network protocols. This article describes the use of Ethernet and IPv4 protocol headers.
The Ethernet frame header includes the "User Priority" service field, which is used to prioritize data frames. The field has a 3 bits size, which allows to select 8 traffic classes: 0 class - the lowest priority, 7 class - the highest priority.Keep in mind that the "User Priority" field is only in 802.1q frames, i.e. tagged with the VLAN tag.
Figure 11 - Frame prioritization service field in Ethernet header |
IP protocol has three historical stages in the development of the service field responsible for packets prioritization:
Thus, ToS allows to distinguish 8 traffic classes: 0 - the lowest priority, 7 - the highest priority, and DSCP - 64 classes: 0 - the lowest priority, 63 - the highest priority.
Figure 12a - ToS service field in IP packet header Figure 12b - DSCP service field in IP packet header |
Many end nodes on the network do not support manipulation with service headers: can not set or remove the priority, so this functionality should be implemented on the intermediate network devices.
Let's look at the example of data transmission from Node-1 to Node-2 through a DS-domain and a third-party telecom operator network (Figures 13a-c). The DS domain includes three devices, two of them for the domain are borderline and one is intermediate. Lets look at the steps of processing data in a network using an Ethernet frame transmission (the basic principles discussed in the example are applicable for an IP packet or other protocol that supports data prioritization):
Figure 13a - Example of the Ethernet frame priority changing during transmission through two network segments (priority in segments is coordinated) Figure 13b - Example of the Ethernet frame priority changing during transmission through two network segments (priority in segments is coordinated, but should be changed) Figure 13c - Example of the Ethernet frame priority changing during transmission through two network segments (priority in segments is not coordinated) |
For a device the process of analyzing priority in service headers and processing data in accordance with these priorities is not a simple task due to the following reasons:
The tables below show data on the queues internal architecture, the priorities managing possibilities and the relation between the protocol and internal priorities values.
Note the queuing architectural feature of Infinet devices: all queues share a single memory buffer. In case the traffic falls into one queue, its size will be equal to the size of the buffer, if there will be several queues, the size of the memory buffer will be evenly divided between them.
Table of packets internal queuing
Correspondence table of protocols and internal priorities for InfiLINK 2x2, InfiMAN 2x2 family devices
Correspondence table of protocols and internal priorities for InfiLINK XG, InfiLINK XG 1000, Quanta 5, Quanta 70 family devices
|
Prioritization assumes the use of several packes queues, which content must be transmitted to outgoing interfaces through a common bus. Infinet devices support two mechanisms for packets transmission from queues to the bus: strict and weighted scheduling.
The strict prioritization mechanism assumes sequential queues emptying in accordance with priority values. Packets with priority 2 will only be sent after all packets with priority 1 will be transferred to the bus (Figure 14). After packets with priorities 1 and 2 are sent, the device will start sending packets with priority 3.
The lack of this mechanism is that resources will not be allocated to low-priority traffic if there are packets in higher priority queues, it will lead to the complete inaccessibility of some network services.
Figure 14 - Strict packets scheduling |
Weighted scheduling doesn't have disadvantages of strict scheduling. Weighted scheduling assumes the resources allocation between all queues in accordance with weighting factors that correspond to priority values. If there are three queues (Figure 15), weighted factors can be distributed in the following way:
When using weighted scheduling, each queue will receive resources, i.e. there will be no situation with the complete inaccessibility of some network service.
Figure 15 - Weighted packets scheduling |
Universal recommendations for configuration of the traffic prioritization mechanisms:
The network resources distribution between traffic flows can be performed not only by prioritization, but also using the throughput limitation mechanism. In this case, the stream bitrate cannot exceed the threshold level set by the network administrator.
The throughput limitation principle is to constantly measure the data stream intensity and apply the restrictions if the intensity value exceeds the set threshold (Figure 16a,b). The throughput limitation in Infinet devices is performed in accordance to the Token Bucket algorithm, all data packets above the throughput threshold are discarded. As a result the losses described above appears.
Figure 16a - Graph of unlimited data flow rate Figure 16b - Graph of limited data flow rate |
There are logical buffer for each speed limit rule containing allowed for transfer data amount. Usually, the buffer size is larger than the limitation size. Each unit of time to such buffer is allocated a data size equal to the set threshold of the bitrate limit.
In the example (video 1), the speed limit is 3 data units, the buffer size is 12 data units. The buffer is constantly replenished in accordance with the threshold, however, it cannot be filled over its own volume.
Video 1 - Resource allocation to speed limit buffer |
Data received by the device inbound interface will be processed only if the buffer contains resources for their processing (video 2). Thus, the passing data empties the buffer resource. If the buffer is empty at the time of data arrival, the data will be discarded.
Video 2 - Dedicated resources usage for data processing |
Keep in mind that the resources allocating to the buffer and data processing are performed simultaneously (video 3).
The data flows intensity in packet networks is inconsistent, it allows to demonstrate the advantages of the Token Bucket algorithm. Time intervals in which data are not transmitted allow to accumulate resources in the buffer, and then process the data amount that exceeds the threshold. A wide band will be allocated to pulse data streams, such as web traffic, to ensure quick web pages loading, and increase the end user comfort level.
Despite the described advantage of the Token Bucket algorithm, the average throughput will fit to the set threshold, as in the long time period, the resources amount will be determined not by the size of the buffer, but by the intensity of its filling, which is equal to the throughput threshold.
Video 3 - Data processing by the speed limit buffer |
The Token Bucket algorithm can be applied to separated traffic flows, in this case, the speed limit buffer will be allocated for each flow (video 4).
In this example, two speed limit rules are implemented: for vlan 161 traffic - 3 data units per time block, for vlan 162 traffic - 2 data units. The buffer size for each traffic flow contains 4 time intervals, i.e. 12 data units for vlan 161 traffic and 8 data units for vlan 162 traffic. Totally 5 data units are allocated to the buffers in each time interval, then the allocated resources are distributed between the buffers. Since the buffers size is limited, resources which exceeds their size cannot be used.
Video 4 - Resources allocation for two speed limit buffers |
The each buffer resources can only be used for traffic of the corresponding service (video 5). Thus, to handle vlan 161 traffic, a resources of buffer for vlan 161 traffic are used. Similarly, buffer resources are used for vlan 162 traffic.
Video 5 - Dedicated resources for data processing using |
There are ways to connect resource buffers with each other. For example, on Infinet devices, allocated resource buffers can be connected via classes (see below). If one resource buffer is full (video 6), its resources can be provided to another buffer.
In the example, the buffer for vlan 162 traffic is full, it allows to fill in the vlan 161 traffic buffer with 5 selected data units, instead of 3. In this case, the vlan 161 service throughput will increase. But when vlan 162 traffic resource buffer will have free space, resource allocation will return to normal mode: for vlan 161 traffic - 3 data units, for vlan 162 traffic - 2 data units.
Video 6 - Allocated resources redistribution between various services traffic limitation buffers |
The throughput limitation principle described above is implemented in Infinet devices in two ways:
Infinet devices allow to configure hierarchical throughput allocation structures. Two objects types are used to perform this: a logical channel and a class, which are connected by a child-parent relationship. The class has throughput value distributed between the child logical channels, and the channel has guaranteed and maximum throughput values - CIR and MIR.
Let's look at the example of two services traffic transmission associated with vlan id's 161 and 162, between Master and Slave (Figure 17a). Totally the service traffic should not exceed 9 Mbps.
The Master device configuration can be performed in a following way (Figure 17b):
Figure 17a - Example throughput limit for traffic with vlan-id's 161, 162 Figure 17b - Hierarchical channel structure of throughput limits for vlan's 161 and 162 traffic |
The throughput limitation capabilities of all Infinet devices families are shown in the table below:
Table of throughput limitation capabilities in Infinet devices
|
Use the following recommendations during data throughput limitation configuring: