Zap Provides Open Source Wireless Testing

By Charlie Schluting | Jan 27, 2010 | Print this Page
http://www.enterprisenetworkingplanet.com/netos/article.php/3861171/Zap-Provides-Open-Source-Wireless-Testing.htm

"Zap" is a wireless performance tool, previously used for internal development and testing by Ruckus Wireless. Ruckus has released the Zap source code under a modified BSD license to provide the tool to the world, and hopefully spur development of this and other related analysis tools. Zap measures performance, statistically, to provide insight into the true nature of how a network can perform.

Before Zap, a wireless network administrator might have done some manual calculation to determine a best-guess baseline of how a network would perform. Check to make sure there aren't too many overlapping channels that may cause interference, carefully plan thelayout, and hope. When testing time comes, we might run a large file copy with rsync or even use TTCP, which will tell us, on average, how fast the data transferred. This is fine for bulk transfers, and it may highlight an obvious problem in the network, but network-intensive applications today require more certainty.

How It Works

Testing wireless is difficult because the tester has no ability to control the environmental factors. We might run a crude test, like a file copy, multiple times and get multiple answers. Those answers can vary by MegaBits per second, which isn't exactly precise.

Zap lets network designers determine the sustained worst-case performance a network can deliver 99.5 percent of the time. Zap does this by sending controlled bursts of data, measuring loss, latency, and other factors that can cause fluctuations in performance. Learning the worst-case sustained rate provides network engineers with the tools to evaluate whether streaming video or other demanding applications will run effectively on the existing network.

Getting Zap

At first, it seemed Ruckus was hopping on the marketing bandwagon that is open source. Its choice of license, however, indicates otherwise. Ruckus released Zap under a great license, the BSD, but simplified a bit. Basically, you're free to use or even sell it, but keep the copyright notice intact and don't imply Ruckus is endorsing your product or service.

To get Zap, visit its Google Code page for subversion checkout information:

http://code.google.com/p/zapwireless/source/checkout

Part of the reason for that awkward first impression is that Zap lacks documentation. There is no README file, or other trace of documentation in the checked out code. There are good comments in the source code, but you shouldn't have to look there. On the Google Code page there is a Word document in the Downloads section, but it is mainly instructions for running Zap on Windows. Windows users, however, don't get easy access to the binaries.

Anyway, Linux users. We get the source and run 'make' to compile Zap. There are a few warnings, but it seems to compile and run fine. Being the Curious George most of us are, we also find that turning on all warnings in gcc (-Wall) yields 78 compile warnings. Regardless, it runs and does its job, and none of the warnings seem to indicate that a grave mathematical error has been made.

In addition to the missing README file, 'make install' does not install any man pages. In fact, there are no man pages to speak of. This clearly isn't Linux software, but it's useful nonetheless, so let's move on.

Output

The Windows documentation indicates that Zap has two components: zap and zapd. Running Zap on one node as the server requires simply invoking 'zapd' to start the service. Zap, then requires you to specify the source and destination IP address. Running 'zap -h' or any other option it doesn't understand will coincidentally spit out syntax help information.

We first tried to run zapd on one machine, and then zap on the same. This produced a segmentation fault, so maybe testing the loopback interface performance isn't implemented.

Running zap on two machines led us to discover it isn't a server/client type of scenario. In reality, you need to run zapd on both machines, and then run zap to instruct them to communicate. This is weird, at best, but keep in mind that this software is generally running on their embedded devices, so it makes some sense.

With zapd running on both nodes, we ran: 

'zap -s10.1.0.69 -d10.1.0.70 -X30'

The -X30 option instructs Zap to terminate after thirty seconds.

Every burst outputs a line, but the very last line of output is what you're after:

  274: 10.1.0.69->10.1.0.70 1071333=rx   0=dr   0=oo   0=rp  3880=rx in    50.1ms   912.8mbps   916.8 | 944.3 926.3 883.6 871.4 863.1 856.7

Left to right, here is what this all means:

274: this was the 274th burst test

10.1.0.69: the source (sending) station

10.1.0.70: the destination (receiving) station

1071333=rx: number of received packets

0=dr: number of dropped packets

0=oo: out of order packets

0=rp: retried packets

3880=rx in: received packets in this batch/sample

50.1ms: batch sample time

912.8mbps: throughput

916.8: cumulative average throughput

944.3: peak throughput observed

926.3: median throughput (50%)

883.6: throughput at 90 percentile, i.e. it was better than this 90% of the time

871.4: 95 percentile

863.1: 99 percentile

856.7: 99.5 percentile

As you can see, Zap is fun to use on wired networks, too.

The 99.5 percent number basically tells you that you can depend on the network to deliver that speed almost always. This is especially useful for IPTV traffic, where it is important to be able to sustain a minimum rate to keep the video flowing. 

In summary, Zap provides insights into the true nature of how a wireless network will perform. The unique method of sending bursts really puts wireless (and wired) networks through its paces and demonstrates what real-world throughput performance you can expect. It's great to have this tool, we just wish it were a little cleaner so that it might one day end up in various Linux distributions.


When he's not writing for Enterprise Networking Planet or riding his motorcycle, Charlie Schluting works as the VP of Strategic Alliances at the US Division of LINBIT (of DRBD fame). He also operates Longitude Technologies, which offers world-wide Linux & Network support and consulting services. Charlie also wrote Network Ninja, a must-read for every network engineer.