Making the Most of Public WAN for Private Cloud Applications

Sometimes clouds mean storms. The enterprise cloud is no different.

Cloud-hosted applications create a number of challenges for network administrators, especially when they run over a public wide area network, as most do. Today’s delivery chain for applications is somewhat decoupled, with the WAN carrying data from enterprises to apps housed in offsite data centers outside of the enterprises’ purview. The public WAN’s performance is absolutely crucial to application delivery, but the public WAN doesn’t provide much visibility into those applications.

Further compounding the challenges, even small changes in the network can magnify application performance problems. And poor network performance is more than just a nuisance for business-critical applications like video, VoIP, unified communications and virtual desktops, all of which are creeping ever more into the cloud.

How do you detect and resolve issues in the network if it’s not actually your network?

The WAN Plays a Critical Role in Application Delivery

Packet loss within a WAN can occur for many reasons, including network congestion or protection events such as route reconvergence and network misconfigurations. Furthermore, most WANs are oversubscribed and leverage network statistics to maximize available capacity. Data traffic is unpredictable, so wide area networking equipment, such as switches, routers and gateways, can react to network congestion by selectively discarding certain packets.

And don’t be fooled by low packet loss numbers. Even in the 1 to 2 percent range, packet loss’s effects on application performance are often disproportionately large, causing effects ranging from jitter on a phone call to the crash of a virtual application. Packet loss often equals application throughput decrease or outright failure. It’s further compounded by a significant increase in latency.

Getting ahead of the curve is critical. Administrators must be able to pinpoint the location and source of latency and determine beforehand what the implications to specific applications could be.

The chart below shows the effect of packet loss and application latency for applications using TCP/IP.

The Situation Today

Full network visibility and understanding of the innards of an application require more than simply keeping an eye on bandwidth consumption per protocol. HTTP traffic has very diverse levels of business priority. Without sufficient application-level visibility, it’s difficult for enterprises to make sure their available network resources are, for example, bolstering ROI sufficiently.

But today there exists no simple way to measure user-application response times end-to-end throughout the entire service delivery network, since the network involves both proprietary and external networks. In order to understand and manage user experiences, isolate the sources of performance impairments, and make the necessary adjustments to network settings and policies in real time, enterprises need to employ new strategies.

Distributed applications require distributed data-capture strategies in order to be able to isolate issues. Although an increasing number of network devices provide embedded traffic classification and monitoring capabilities, the implementations often result in performance-impacting consequences, cost more, and often lack the ability to look beyond packet headers.

An effective solution must look deep into the content within the applications. The growing mobile-user presence and the increased complexity of network environments means that users typically pick up IP addresses dynamically. For this reason, a user could have several IP addresses during a single session. This evolution of the network makes it nearly impossible to monitor, secure, and manage solely by IP address.

Requirements for New WAN Monitoring Solutions

A truly network-wide monitoring solution, extending to remote branch offices, must provide access to key performance information, including

• Measurement and analysis of application performance for all transactions
• Comparisons of response times against intelligent baselines and thresholds
• Identification of abnormal latencies in the network
• Isolation of a problem to a specific link or application server, or the application itself
• Delivery of alerts on any performance deterioration

An effective monitoring solution must also provide a scalable approach to pervasive monitoring, management, and troubleshooting, with an architecture that decouples data collection from management, aggregation, and analysis. This approach allows a highly scalable, distributed, cost-effective method to add visibility throughout the network, reducing the complexity of capturing rich intelligence from the network, the content, and the end-user experience.

Outsourcing WAN Monitoring and Management

Unfortunately, many enterprises lack the funds to deploy a sophisticated, ubiquitous WAN monitoring strategy to each remote branch office. Others lack the in-house expertise. That’s where outsourcing comes in. Providing enhanced WAN traffic visibility creates opportunities for service providers and options for enterprise IT administrators.

It’s a win-win for service providers and enterprises. The enterprise gets better network visibility and response to network issues. The service provider becomes a more trusted and valuable resource for the enterprise, rather than a simple middleman between application and end user.

Outsourcing eliminates hardware maintenance and support staff costs associated with on-premises technology. The managed service provider can also take advantage of economies of scale to use the most sophisticated WAN monitoring and management products on the market and employ people with the expertise to more effectively assess the data collected. To deliver these types of services, network operators need to have visibility into their network’s capacity use, application performance, and bottleneck locations.

Service providers must be able to quickly isolate the sources of issues, indemnify their services should problems arise, and provide the information enterprise IT administrators require about network performance. Some questions enterprises should consider when assessing managed network services include:

• Does the service provider possess the necessary technical and business expertise to adapt to the enterprise’s evolving needs?
• Are the provider’s services customizable to the enterprise’s specific needs?
• Does the service provider offer a real-time network view?
• Is application and/or network performance backed with specific service level agreements?

The Value of Effective Monitoring

Industry studies show that just a half-second delay in generating search results worsens the user experience and, in effect, sheds a significant portion of a company website’s traffic. Poor business-critical application performance can also hinder productivity and damage reputations and relationships. Clearly, proactive and real-time monitoring to improve the quality of the end-user experience offers huge benefits.

With effective monitoring, those dark stormy enterprise clouds will begin to lighten.

Thanks to Enterprise Networking Planet for the article.

Your Guide to Troubleshooting VoIP

VoIP’s extreme sensitivity to delay and packet loss compared to other network applications such as web and e-mail services, presents a real challenge. A basic understanding of VoIP traffic and of the quality metrics provided by VoIP monitoring tools will help keep your network running smoothly

Your Guide to VoIP

This white paper guides you through the essentials of VoIP troubleshooting, including common problems and the metrics you should employ to fix and prevent them. You’ll learn how these metrics relate to one another and to overall VoIP health. You’ll also learn the best practices for keeping your VoIP network flowing smoothly, your time-to-resolution low, and end users happy.

Network Jitter and Delay

Real-time voice communications are sensitive to delay and variation in packet arrival times. Codecs require a steady, dependable stream of packets to provide reasonable playback quality. Packets arriving too early, too late, or out of sequence result in jerky, jumbled playback. This phenomenon is called jitter.

Increasing jitter buffer size can help, but only to a point.

Because no network can guarantee a perfectly steady stream of packets under real-world conditions, VoIP phones use jitter buffers to smooth out the kinks. A jitter buffer is simply a First-In, First Out (FIFO) memory cache that collects the packets as they arrive, forwarding them to the codec evenly spaced and in proper sequence for accurate playback.

While a jitter buffer can successfully mask mild delay and jitter problems, severe jitter can overwhelm the jitter buffer, which results in packet loss (see below). Increasing the size of the jitter buffer can help, but only to a point: A jitter buffer that increases overall round-trip delay to 300 ms will make normal conversation difficult.

Packet Loss

As mentioned above, packet loss can be the result of the jitter buffer being overwhelmed. Other reasons include landline media failure and poor wireless signal quality. The latter can be a big problem with VoFi (Voice over WiFi) service. Regardless of the source, VoIP phones and gateways attempt to conceal this type of signal degradation by duplicating packets to fill in the missing data. As with jitter, these techniques can maintain voice quality only to a point.

Packet loss on data networks has long been characterized as a “bursty” phenomenon, which is another way of saying “it never rains, it pours.” Networks tend to either sporadically drop single packets (these periods are called “gaps” in packet loss), or large numbers of contiguous packets in a “burst.” Packet loss concealment techniques typically have no problem handling packet loss during gap periods; it is the sustained bursts you must watch out for.

Call Management Problems

If the VoIP call manager (sometimes called the VoIP server) is overwhelmed with requests, or its connection to the network is impaired, call setup delays can reach the point where users abandon calls before they are able to connect to the other party. If IP phones are misconfigured, or their IP connection to the server is impaired, calls remain open in the call queue long after the parties have disconnected.

Managing VoIP Quality

You can manage only what you can measure. Managing a VoIP deployment therefore requires some hard numbers beyond subjective user assessments of quality (although these are obviously important as well). Beyond monitoring the network parameters discussed in this paper, having an overall quality score such as a Mean Opinion Score (MOS) or R-factor score can also be a useful VoIP network health index.

VoIP monitoring tools calculate the MOS and R-factor scores using a formula known as the E-model. Using the statistics it has collected from the network, the analyzer calculates how much the various impairment factors (such as codec compression, jitter, delay, and packet loss) would affect the typical user’s perception of call quality.

VoIP-Specific vs. All-Purpose Monitoring Tools

There are a number of different options on the market for managing VoIP quality, mainly falling into three categories:

• Dedicated VoIP tools originally developed for the telecom industry. These tools are great for testing IP phone and gateway designs, but not as good at solving deployment problems on a live network.
• Network protocol analyzers that have added “VoIP Support” by licensing technology developed for the telecom industry and integrating it into their product line.
• Network monitoring tools that approach VoIP quality management from an IT administrator’s point of view rather than from that of telecom engineer.

To the IT administrator, managing VoIP quality is just another network task. This makes the third approach (the “all-purpose” network monitoring tool) often the most practical choice. But note that “VoIP support” means more than just decoding the packets of various VoIP protocols; it also means being able to track and display network delay, jitter, and packet loss, and to distill this information into overall quality scores, both per-call and in aggregate.

“VoIP Support” means more than just decoding the packets of various VoIP protocols.

And to be really useful to the enterprise, the tool should also track, store, and analyze long-term trends. This is so that you can understand what is “normal” VoIP performance, and maintain a database of Call Detail Records (CDRs) from which you can generate reports for management or service providers. The VoIP monitoring tool should also be capable of automatically notifying you when selected statistics indicate a developing problem. On all of these counts, Network Instruments® Observer® meets the requirements.

VoIP Points of Visibility

In switched environments, where to deploy an analyzer or probe for maximum visibility isn’t necessarily obvious. Complicating matters for VoIP is the fact that each call includes both client-server communications (between IP phones and the call manager during setup and tear-down), and peer-to-peer (the streams of voice data passed between the parties). For example, consider the following VoIP network deployment:

Where to place probes on such a network depends on what you want or need to see. If you need access to all local conversations on either coast, including both call setup and actual voice data, use a SPAN session on the access layer switch to mirror VoIP traffic to the analyzer. Assigning all VoIP traffic to a dedicated VLAN makes this fairly straightforward.

Capturing Local IP Phone Traffic Shows:

• Any phone’s communications with its local call manager
• Both sides of the full-duplex connection between local phones talking to each other
• Both sides of the full-duplex connection between phones located on opposite coasts

What you will not be able to see from this probe is any communications between the East Coast and the call manager located on the West Coast.

If you are more interested in a coherent view of calls between the West Coast and East Coast, including all call manager communications, use a SPAN session to mirror both the uplink traffic between the core and MPLS mesh, and all traffic flowing to and from the call manager. This will give you a coherent view of inter-office calls, along with all call manager communications, both local and remote.

With a probe deployed in this manner, you will not be able to see the peer-to-peer voice traffic between local phones. For complete coverage, connect probes to both the core and access layers at each site. Another alternative is to deploy probes at the core 24/7/365, monitoring the access layer with a portable analyzer or software probe only to troubleshoot local call problems as needed.

VoIP Network Analysis

How can VoIP analysis help manage quality? By closely monitoring the network conditions that affect VoIP, you can begin to address developing infrastructure problems before they result in user complaints or downtime.

Tracking Network Performance

Consider the East Coast/West Coast example described in the previous section. Ken, the administrator responsible for ensuring VoIP quality, has set up a Network Instruments 10/100/1000 Probe Appliance on the core switch to monitor all call manager activity and any VoIP traffic traversing the link. He has configured Observer to send him an e-mail whenever any of the following conditions arise:

• MOS falls to 3.5 or less
• Jitter levels crossing the MPLS mesh exceed 20 ms
• Delay levels crossing the MPLS mesh exceed 80 ms

Any of these conditions are indications that VoIP quality is threatened. Given the topology involved, the most likely source of problems is the MPLS mesh routers, which are under the service provider’s control. By digging deeper into the statistics the analyzer provides, you can determine why the MOS is falling, and what is causing jitter, delay, or packet loss.

If jitter is the problem, a good place to start is by comparing jitter levels against bandwidth utilization to see if there is any correlation. The analysis shown below (taken from Network Instruments’ Observer Expert) shows just such a correlation.

As this scenario shows, such a link between spikes in bandwidth and jitter could mean it is time to invest in more bandwidth, or time to put more controls on employee Internet usage for applications such as streaming media and peer-to-peer file sharing unrelated to business.

If there isn’t an obvious correlation between jitter and bandwidth utilization, the depth of data provided by an all-purpose network analysis and monitoring tool can help you dig deeper for the correct diagnosis. For example, if VoIP traffic across an MPLS mesh is subject to excessive jitter, it could be the result of “route flapping” on the service provider’s routers. An analyzer can confirm and document this. Armed with the hard data provided by analysis, you could then contact the service provider so they can address the problem. If delay across the mesh exceeds the contractual obligations of the Service Level Agreement (SLA), the provider may owe your organization some refunds for service failure, in addition to being responsible for fixing the problem.

Troubleshooting Connection Problems

When a user can’t get a dial tone, or if there are excessive delays in ringing the other party’s phone, examining a graphical display of how the call is progressing between the parties and the call manager can indicate what is going wrong.

Network Instruments’ VoIP Expert displays just such a diagram: simply right-click on any call or connection stream. Because differing protocols dictate differing phone/call manager interactions, some knowledge of the protocol is necessary for detailed troubleshooting. But even if you lack a detailed knowledge of the protocol, the Connection Dynamics display highlights which party isn’t responding, or which party is responding slowly.

Better manage and troubleshoot VoIP across your network using Observer’s VoIP Expert, available in Observer Expert and Observer Suite. Whether you’re interested in the big picture or a specific conversation, Observer offers real-time statistics, Expert VoIP analysis, and reports to help you address all levels of VoIP traffic.

Over 70 VoIP-specific metrics.

Understanding VoIP performance is the key to accurate troubleshooting. Observer provides visibility into the network, application, and VoIP traffic to quickly resolve issues. VoIP Expert summaries provide easy understanding of overall VoIP network health. Over 70 VoIP-specific metrics like call quality can be tracked on an aggregate or per-call basis. Take advantage of over 50 VoIP Expert events for immediate problem identification. Once you’ve identified the issue, drill-down on specific conversations for an in-depth view.

VoIP Expert Analysis Features

Convenient, at-a-glance summaries

• VoIP Traffic Summary
• Call Summary
• Voice Quality Scoring
• Precedence (QoS)

Detailed views with in-depth Call Detail Records

• Addresses
• Status (Open, closed, fail)
• Number of packets, packet bytes, packet loss
• Start time, initial setup duration, duration
• Current jitter, maximum jitter
• MOS, R-factor
• QoS for each call
• Number of packets that arrive out of order
• Detailed analysis for packet loss and delay
• Gap and burst measurements
• Long-term Call Detail Records trending

Quickly Solve Problems with over 50 VoIP Experts  

Observer offers over 50 event-based and threshold-based VoIP Experts to immediately flag problems for faster problem resolution. Examples include:

• Alarms for unacceptable jitter level
• Lost packets
• Alterations in the QoS stream

Manage VoIP Audio Quality

Observer displays individual Mean Opinion Score (MOS) and R-factor for individual calls and as an average. Place alarms on this score to proactively manage overall VoIP quality.

Monitor Quality of Service (QoS)

Observer’s VoIP Expert reports QoS levels by call, packet, and protocol. This analysis also shows percentage of VoIP utilization compared to other network traffic, allowing you to plan network upgrades.

Evaluate Jitter

Observer’s VoIP Expert displays the statistical variance of packet arrival times, known as jitter, measured in timestamp units or RTP time units.

Measure Bursts and Gaps

Observer’s VoIP Expert provides comprehensive metrics for monitoring bursts, or periods of high packet loss, and gaps, or periods of little packet loss. Burst and gap percentages quickly indicate the quality of a call.

Track and Decode VoIP and Video

Observer offers complete VoIP and video decodes, including H.323, Session Initiation Protocol (SIP), MGCP, and SCCP (Cisco “skinny”). Observer also supports Avaya CCMS, Nortel UNIStim, and Mitel® systems.

Reconstruct and Review VoIP Calls

Capture and reconstruct calls for playback, so you can hear the same call quality issues experienced by users.

Compare VoIP to Network Performance

Use Observer to identify whether jitter or delay is being caused by other applications on the network. Observer will track VoIP issues along with overall network performance.

Monitor VoWLAN

Observer’s VoIP enhancements are automatically available across multiple topologies, thanks to the Network Instruments Distributed Network Analysis (NI-DNA™) architecture. Observer’s VoIP Expert will monitor VoIP traffic even over wireless networks.

Top 10 VoIP Best Practices

Our sales engineers have put together this valuable list of the best practices to use when implementing or troubleshooting VoIP. Keeping these steps in mind will help you get the most from your VoIP network.

1. Understand and measure call quality components

There are a variety of metrics you can use to assess VoIP call quality, including jitter, MOS, R-Factor, gap density, burst density, Quality of Service prioritization, and compression techniques. Ensure you are accurately analyzing VoIP communication by learning how to measure these attributes.

2. Implement Quality of Service prioritization

Incorrectly set QoS precedence for VoIP traffic leads to delays in packet delivery and reduced call quality.

3. Conduct site surveys

The more you know about your network, the better prepared you are to properly integrate VoIP. Conduct a site survey to review current WAN bandwidth levels, traffic flows, and existing switches for bottlenecks and choke points. Then, identify or determine specific needs through testing and modeling.

4. Deploy analysis tools strategically for maximum visibility

Placing network analysis consoles and probes on your network requires a clear understanding of VoIP traffic patterns. Are you concerned with monitoring VoIP traffic locally, over WAN links, both? Depending on your objectives, place your analysis tools to ensure optimal visibility of VoIP communications.

5. Implement VLANs to isolate and monitor VoIP issues

Organize your VoIP traffic by VLAN user groups. This practice will greatly simplify problem resolution.

6. Monitor rollouts to ensure a positive user experience

Determine whether users are receiving a positive experience by reviewing cumulative VoIP metrics, codecs, and other network performance variables during VoIP deployment. By evaluating VLAN setups and overall link utilization, you can judge overall network performance and quickly make adjustments during implementation.

7. Compare jitter to overall network bandwidth utilization to understand response time

When jitter becomes a problem, look at the big picture. A correlation between jitter and bandwidth usage means the problem is overall network usage. If there is no direct correlation, excessive jitter might be caused by isolated network factors that require further investigation.

8. Set up your analyzer to proactively monitor VoIP activity

Utilize monitoring and notification tools to speed problem resolution. Determine “normal” or “acceptable” levels of activity for your network and its users. Then set up thresholds within your analyzer to alert you when thresholds are broken or in danger.

9. Automate problem resolution

Expert Analysis functionality eliminates unnecessary trial and error when troubleshooting VoIP issues by automating problem resolution. Utilize Expert Analysis on VoIP communication to quickly pinpoint the source of common VoIP problems.

10. Baseline network traffic

For comprehensive understanding of VoIP traffic, capture and store long-term network data. Only with critical trending data can you accurately perform baselining activities. Baselining validates VoIP performance, helps future capacity planning efforts, and provides long-term understanding of VoIP health.

Summary of VoIP statistics and quality metrics

The following table summarizes the statistics and quality measurements discussed in this paper, both defining what is measured, and describing its relevance.

Thanks to Network Instruments for the article.

StableNet® – Managing End-to-End VoIP Networks

StableNet® is a unified End-to-End Service Quality Management platform and therefore, takes a customer-centric approach to the service assurance monitoring infrastructure, performance and fault management in a single solution.This White Paper provides details on how to engineer a complete End-to-End VoIP service monitoring and reporting solution with StableNet®.

There are any number of VoIP management solutions available in today’s market place. However, when you start to drill-­down into the capabilities of these tools they tend to focus on the performance elements of your network infrastructure and associated VoIP metrics (e.g. RTT, RTD\Latency, Packet Loss, Jitter, Moss, R-­Factor etc.) , assumptions are made on infrastructure and fault management being in place, so it is vitally important to assess the complete picture of your solution requirement before selecting the choice of tool to be deployed. VoIP monitoring lies central to this, as VoIP downtime and poor VoIP performance directly impacts such things as business performance, profitability and revenue. Achieving a consistent level of quality on VoIP calls requires multiple dependent components working properly, thus the importance of a monitoring system that correlates the infrastructure, performance, and fault management into an integrated End-­to-­End view is vital…….read more

DOWNLOAD THE WHITE PAPER HERE

Thanks to Infosim for the article.