November 28, 2014

When you are looking to test TCP performance, the IETF standard RFC 6349 is a great place to start. However, while it provides some interesting metrics, such as transfer time ratio (actual TCP transfer time ration vs. ideal TCP transfer time ratio) and TCP efficiency (percentage of bytes that has not been retransmitted), it doesn't say anything about how to test QoS-enabled connections using TCP.

This blog post provides some brief information about RFC 6349, as well as some insights on its shortcomings for QoS-enabled connections, and finally how typical traffic engineering and QoS are designed for different classes in such cases and how to verify such connections.

RFC 6349

In short, RFC 6349 is a framework published in 2011 that describes a practical methodology for measuring end-to-end TCP throughput in a managed IP network. The goal is to provide a better indication in regard to user experience.

One of the most important parts of the RFC is to specify a test that uses a real stateful TCP measurement, and not stateless (sometimes hardware-based test instruments use UDP-type of tests, but with a TCP protocol type number in the header of the packets). This because it is difficult to emulate a real TCP stack in a hardware-based instrument. This is also mentioned in the RFC as a reason, as "Stateful TCP test traffic means that the test host MUST fully implement a TCP/IP stack; this is generally a comment aimed at dedicated communications test equipment that sometimes "blasts" packets with TCP headers."

Besides this basic and most important fact, the RFC provides some interesting metrics in order to assist the user in understanding how close to “ideal” the performace is. The metrics that should be calculated from the measurement are:

  • Transfer time ratio - the ratio between the Actual TCP Transfer Time versus the Ideal TCP Transfer Time
  • TCP Efficiency - the percentage of Bytes that were not retransmitted
  • Buffer delay - represents the increase in RTT during a TCP Throughput test versus the inherent or baseline RTT

The metrics are all valid and relevant. What this RFC lacks is however a specification of how to test that the Quality of Service works.

Traffic Engineering for TCP QoS

In general it is more complex to configure with QoS than without, and it is therefore much more common with mistakes and errors in the configuration. This is especially true for “off net” deliveries where more than one service provider is involved in the end-to-end connection.

A typical way of engineering QoS is to define how many percent of the bandwidth of a connection that is reserved for each particular QoS class. Let me give you an example.

Let us assume that there are three QoS classes:

  • Best Effort (BE with DSCP0)
  • Business (BC with DSCP 24)
  • Real Time (RT with DSCP 46).

In this example we further assume that BE has a reservation of 50% of the bandwidth, BC 30%, and RT 20%, which adds up to 100% in total. So if the connection is loaded with traffic from all three classes, the distribution according to above should be maintained.

Assume that there is only traffic in two of the classes, BE and BC as an example. What should the distribution look like in that case?

First you need to normalize to 100%, because in this case those two classes will fill all the bandwidth of the link. In other words, BE should get (50/(50+30))100=62.5% and BC 30/ (50+30)100 = 37.5%. In total 100% of the bandwidth.

Normally the RT class is used for UDP-based services such as VoIP and video conference, but from a QoS perspective it is still relevant to test the distribution.

Multi TCP QoS Testing

Once you have determined what the distribution should look like, it is of high interest to verify that this is actually correctly configured end-to-end. To test this, you should run one or several TCP sessions in each QoS class and measure throughput to verify that the total throughput is according to expectations, but also that the distribution of bandwidth between the classes is according to agreement.

In practice, the distribution might differ a few percent, and it should be considered good enough, but if it deviates more than say 10% from the expected distribution, it is not ok.

Using Netrounds

If you are using Netrounds to test bandwidth distribution between queues, you can quickly and easily set:

  • Number of TCP streams per class
  • DSCP per class
  • How many classes to run traffic in
  • Expected distribution in %
  • Maximum deviation from expected distribution

If the deviation is within the limits you expect, the test will pass, and otherwise it will fail. Below is an example of how to start a test, and the corresponding result from such a test.

Setting up the test:




Corresponding result:



Comment: As you can see, the QoS does not work in this case since the distribution is even between the classes, which was not expected.

Summary and Conclusion

While RFC 6349 do provide some interesting metrics, the most important part of the RFC is that it requires stateful TCP, instead of “fake TCP” where UDP is sent using a TCP protocol header. For QoS-enabled connections there is currently no RFC that specifies how to test, but with Netrounds you can do this effectively.

Netrounds supports tests of TCP performance according to RFC 6349 and TCP bandwidth distribution between queues for up to six QoS classes.

Contact us for more information, we look forward to hear from you.