Network Assurance Made Easy - Site Activation Testing in the Cloud

|
August 6, 2020
|
|

For the original article visit Medium here.

I have been in the telco industry for some time, and it is a fact that technologies supporting our industry are constantly evolving. Most of us have experienced first hand the leap from searching information by shuffling through encyclopedias and text books to having the world’s information at our fingertips within seconds on the internet. Likewise, the economy has been disrupted several times by several technology shifts and most companies do operate quite differently today than 20 years ago. 

In the present day, the worldwide economy is dependent on reliable and high-performing network connections. Setting up a resilient and manageable network that can expand along the way is a big challenge. Predicting how it will behave when millions of users start using your services is almost impossible. Internal research shows that a majority of business service providers are unable to successfully turn up services and onboard their customers fast enough. One ability that has been missing for many years is for service delivery teams to quickly confirm that SD-WAN and wholesale transport services have been configured properly before customers start using the service. Most business service providers skip the turn-up testing phase, or use inadequate methods, which results in failed deliveries and expensive repair. The solution is to automate the activation testing process to guarantee that service levels are delivered, so that customers can be successfully onboarded.

As highlighted by our CEO, Mats Nordlund, our mission is to provide the most intuitive and versatile software for automated active testing and monitoring for the most demanding network environments. We believe that a solution to a complex problem does not itself need to be complex.

Let’s dive into an example right away.

The Netrounds solution consists of two main components: Netrounds Control Center (NCC) and Test Agents (TAs). 

  • NCC is the core component of the solution which orchestrates the activities of TAs: initializing tests and monitoring, collecting statistics, and more.
  • TAs are the worker nodes. They execute the tests created from templates, make all the measurements, and report data back to the NCC. The TAs do not store test results, statistics, or test configurations. TAs can be deployed basically on any x86 or x86-64 bit platform, and required system resources are minimal. 

Below is an example of a site activation test. This is to demonstrate how a task can be accomplished in a few simple steps with Netrounds Control Center and the Test Agents. 

Scenario: You have multiple network sites as shown below. They are on AWS but located in different regions due to business needs. Now suppose that the need arises for a new site in Singapore on AWS. Therefore, you would like to ensure that all services running at the Singapore site can be used optimally before you roll out and announce those services. 

Example of a Site Activation Test with Netrounds Control Center and Test AgentsImage 1: Example of a site activation test with Netrounds Control Center and Test Agents

Let’s go through, step-by-step, how we can conduct site activation testing via Netrounds Control Center.

What we’re going to do is to execute a test template with several tests. 

  • First we will do a maximum transmission unit (MTU) check to confirm that the MTU configuration is correct end-to-end.
  • Then, we do some transmission control protocol (TCP) testing to check the connectivity of certain ports for our new site.
  • The next step is to generate five simultaneous voice over Internet Protocol (VoIP) calls between two sites. This VoIP user datagram protocol (UDP) test tells us how our new network influences the quality of VoIP traffic. An objective quality score on the mean opinion score (MOS) scale is calculated for VoIP based mainly on network jitter and packet loss. 
  • We proceed with a TCP throughput test in order to see the link capacity between the two sites.
  • We finalize our testing with a domain name system (DNS) lookup to see whether host names can be resolved, and then execute HTTP tests in parallel to see the responses for certain web pages. 

Note: A test template allows engineers to easily combine one or more tests. Templates are completely configurable and allows users to select what tests to perform and in what order.

Summary: 

  • Step 1 – MTU check between two sites
  • Step 2 – TCP testing on certain ports (80, 443, 8080) to confirm accessibility at our new site
  • Step 3 – VoIP: 5 simultaneous calls between two sites 
  • Step 4 – TCP throughput testing to measure the link capacity
  • Step 5 – DNS lookup on certain host names, HTTP testing. These will be executed at our new site only.

In order to start doing the site activation testing, we log in to our NCC. 

Below is the Netrounds Control Center Web UI. 

Netrounds Control Center Web User Interface (UI)Image 2: Netrounds Control Center web user interface (UI)

Now we create a test sequence in NCC. Each test is a part of a jigsaw puzzle. We select pieces and combine them to create a sequence. 

Creation of a Test Sequence in Netrounds Control Center

Image 3: Creation of a test sequence in Netrounds Control Center

Clicking on these jigsaw pieces reveals the configurable settings for each task. Let’s take a look at each step of the test in detail. 

Path MTU Discovery

Path MTU between two Netrounds Test Agents (TAs)Image 4: Path MTU between two Netrounds Test Agents (TAs)

This step determines the path MTU between two sites, or more precisely between two TAs. Our Test Agent at the already-operational site in Japan acts as server, and the TA in Singapore is the client. The minimum acceptable MTU is 1400 bytes, and if it turns out that the configured MTU is smaller, the test will fail.

TCP Test for Port Connectivity Verification

Destination Port Numbers

TCP TestsImage 5 and 6: Destination port numbers and TCP tests

TCP packets will be sent from the Japan region to our new site in Singapore with a specified destination port number as shown above (8080, 443, 80). This is to check whether the ports are accessible from our Singapore site. We will be running three parallel TCP tests, one for each port, and generate 1 Mbit/s of traffic.

VoIP UDP

VoIP Media Streams over UDP - Calculate MOS ScoreImage 7: VoIP media streams over UDP setup of MOS score threshold

In this step we generate five simultaneous VoIP media streams over UDP, using the G.711 codec.

MOS scores are calculated for VoIP based mainly on network jitter and packet loss.

MOS scale: 5 = Excellent, 4 = Good, 3 = Fair, 2 = Poor, 1 = Bad.

A MOS score of 4 or worse will trigger an errored second and appear in the final report.

TCP Throughput Testing

TCP Throughput TestImage 8: TCP throughput test

One way of measuring the link capacity between these two regions is to trigger a TCP test. The TA can generate/support up to 10,000 Mbit/s. If you are expecting throughput higher than a certain value, there is a possibility to enter that value as a minimum threshold. An errored second will then be triggered whenever the throughput dips below that threshold. 

DNS and HTTP Testing

DNS Test

HTTP TestImage 9 and 10: DNS and HTTP testing

These two tests can be executed by using one TA. The Singapore TA is going to be used to measure the DNS response time towards 1.1.1.1 for google.com, and the same TA will perform an HTTP request towards the same URL. Anything different than the expected HTTP response code (200 OK) will fail the HTTP test. 

Running the Service Activation Test 

Service Activation TestImage 11: Service activation test

The test has five steps in total, and each one will be executed after the completion of the previous step. During the test execution, we can view detailed result data in real time for any task we are interested in by clicking on that task. 

TCP throughput test results

TCP throughput test resultsImage 12: TCP throughput test results

The line graph charts the TCP throughput as a function of time, while the main user interface displays the average TCP throughput of 127.7 Mbit/s between the Japan and Singapore sites. (This aligns with my settings on AWS, given the fact that I was using the smallest EC2 instance.) The line graph has a resolution of 1 second, allowing us to view results in fine detail. 

TCP Throughput Test ResultsImage 13: TCP throughput test results

The average DNS response time for google.com towards 1.1.1.1 is 2.17 ms. I personally tested this a couple of times and compared the results with the well-known address 8.8.8.8. The response from 1.1.1.1 seems much faster, most likely due to the fact that 8.8.8.8 is more popular and thus has higher resource usage. 

Netrounds Control Center also provides a nice overall report for an executed test template. The site activation test in our example had five steps, and having a PDF file with results in detail would make things much easier. A typical service provider has many departments which might be interested in seeing the results and see what they can do within their expertise to fix the issues if need be. 

Below is the VoIP test result seen in the final report.

VoIP Test ResultsImage 14: VoIP test results

Our test indicated an average MOS of 4.37, which correlates to near-excellent call quality. 

Summary

In this article, we have gone through the main components of Netrounds: Control Center and Test Agents. We have also created and executed a test template for site activation; further, we have explored the details of the test configuration and got a sneak peek of the results report.

I hope that this blog has demonstrated how easy it is to define and automate test templates with Netrounds, and that network engineers will find it reassuring that there is cost-efficient and easy-to-use technology to manage the complex networks of today.

For reference, the chart below shows the test types a TA can execute:

Test Agent Execution Test Types

Image 15: Test types executable by Test Agents

For more information on how service delivery teams (engineering) can quickly confirm that SD-WAN and wholesale transport services have been configured properly, watch our latest webinar “Zero-Touch Testing for SD-WAN and 5G Unified Transport” on-demand. 

If you are curious about our solution or have questions, please reach out to anyone on our team at any time.