WiFi Direct Data Performance Tests

In this section

• WiFi Direct Data Performance Tests NoAP SingleChannel

• WiFi Direct Data Performance Tests PreAP24 SingleBand SingleChannel

• WiFi Direct Data Performance Tests PreAP5 MultiBand MultiChannel

• WiFi Direct Data Performance Tests PreAP5 SingleBand MultiChannel

• Troubleshooting WiFi Direct Data Performance Tests

In this article:

• Test suite overview

• Performance requirements

• Performance metrics

Test suite overview

This test suite validates that a Wi-Fi device meets the performance requirements for throughput, jitter and packet loss as defined by the Windows Certification Program for WLAN Devices over a Wi-Fi Direct Link, when the device under test (DUT) becomes either a Wi-Fi Direct Group Owner (GO) or a group client (GC) by joining an existing GO.

After successful Wi-Fi Direct pairing is achieved, network performance measurements are collected over the Wi-Fi Direct link. When an infrastructure connection exists to an access point (AP), throughput is simultaneously measured on the station (STA) connection. These metrics are then validated against Windows Hardware Certification Kit (Windows HCK) performance requirements to determine whether to pass or fail the tests.

Windows HCK Wi-Fi Direct Data Performance tests use the NTtTcp tool to collect network performance measurements over both infrastructure wireless and Wi-Fi Direct links for the following scenarios:

• single-band/single-channel

• single-band/multiple-channel

• multiple-band/multiple-channel

• Wi-Fi Direct GO

• Wi-Fi Direct GC

• Wi-Fi Direct Multi-Port (GO + GC)

• No AP connection

• .4Ghz N, 2, and 5Ghz (N and AC)

• Streaming mode

• AC_VO Quality of Service (QoS) tagging set on the WFD port(s)

• Connection to an infrastructure network AP

Note  

Whether 5 Ghz 802.11n or 5 Ghz 802.11ac are used when you connect from the DUT to the 802.11ac AP depends upon the capabilities of the driver and the wireless network interface adapter. Connection to 2.4 Ghz is always 802.11n. The DUT always connects to the 802.11ac AP. The support device connects to the second AP. Performance requirements are validated on a particular link only if QoS tagging is set on that link.

Performance requirements

Throughput requirements

The 802.11 devices must have no more than 15% aggregate throughput degradation when data flow is divided between two ports (with and without Multi-Channel/Band operation). These are ExtSTA, Wi-Fi, and either Direct Role Port (GO) or Wi-Fi Direct Role Port (Client), compared with aggregate throughput when only one Wi-Fi port is connected. This throughput requirement individually applies to all types of ports (ExtSTA, Wi-Fi Direct Role Port [GO] and Wi-Fi Direct Role Port [Client]).

Windows HCK requirements state that an 802.11 device must be able to maintain between 50% and 65% of the maximum theoretical throughput over a TCP connection, depending on the Phy type, spatial stream and channel width configuration. Details are provided in the following tables.

Note  

All speeds are in Mbps. Combinations are not required for Secure Digital (SD) IO (SDIO) bus type devices. Channel width is the number of spatial streams.

The following tables describe the throughput expectations for WLAN devices if a WLAN device supports 802.11ac Phy Type (802.11ac Phy Type support is optional for WLAN devices).

Table 1. WLAN 802.11ac Phy Type Throughput Expectation at 20 MHz

Table 2. WLAN 802.11ac Phy Type Throughput Expectation at 40 MHz

Table 3. WLAN 802.11ac Phy Type Throughput Expectation at 80 MHz

The following tables describe the throughput expectations for WLAN 802.11n Phy Type transmissions (802.11n Phy Type support is mandatory for WLAN devices).

Table 4. WLAN 802.11n Throughput Expectation at 20 MHz

Table 5. WLAN 802.11n Throughput Expectation at 40 MHz

The following table defines the requirements for jitter (one-way latency) and maximum consecutive packet loss. The maximum one-way latency and packet loss requirements are at UDP for AC_VI/AC_VO traffic. The maximum allowable consecutive packet loss in all cases is 0.5%, or <= 3 consecutive packets.

Table 6. Jitter requirements

Performance metrics

Throughput is the average rate of successful message delivery over a communication channel. The throughput that the Windows HCK validates comes directly from the NTtTcp tool.

NTtTcp output contains a measurement of latency. This measurement is used in validation results and should not be confused with the jitter measurement. Jitter is calculated as an approximation of one-way latency, and it is a measurement of how the real packet spacing differs from ideal fixed packet spacing.

To measure jitter, Windows HCK sends a stream of UDP packets of a specific size at a specific rate. The real packet spacing is calculated at the receiving end of the network performance measurement and is compared against the ideal packet spacing that is expected from a zero-delay network. The jitter is calculated by finding the absolute packet-to-packet spacing of every packet after the traffic has stabilized, and then analyzing its deviation from an ideal period of time T.

After all packet-to-packet spacing is calculated, Windows HCK finds the 99 percentile largest packet to packet Jitter (ji). The test case fails if this value is larger than 30, 40 or 50 milliseconds (depending on QoS tagging), and whether the measurement is for a single-channel or multiple-channel environment. Using 99 percentile means that if only 10 out of a 1000 packets are larger than the expected value, the IHV 802.11 driver passes the test. However if more than 1% of the packet to packet jitters are larger than the requirements, the test fails.

The formulas come from the Microsoft Voice over Wi-Fi Personal Certification Test Plan. The formulas that the algorithms implement to calculate Jitter are as follows:

Let the period of the sent packets be T(ms). The packet to packet spacing (with adjustment for dropped packets) is:

Note  

The denominator adjusts the packet spacing in the case where there is a packet gap; this prevents gaps from causing spikes in jitter.

The Packet to packet jitter, Ji, (the difference in packet spacing) is:

Packet jitter (the difference between the packet spacing and the ideal spacing, T) is the sum of Ji:

This reduces to:

Where:

defines that the first packet that is used in this computation arrived on time.

Jitter is defined as the absolute value of j₁(|j₁|). Because of instrumentation variations, we recommend that a percentile of jitter is used to characterize the system capabilities (for example, 99|95|90% of all jitter values falls below n ms), instead of just using max j₁.

The Windows HCK performance requirements and test tolerate fluctuations on conditions to reduce the possibility of false negative issue investigations. This test suite uses NttTcp to get network performance metrics. In some cases, performance measurements are calculated based on the tool’s output and then validated to make sure that they match performance requirements for throughput, jitter and packet loss.

For jitter calculations, NttTcp outputs a comma-separated value (CSV) file. The CSV contains the following headers:

<packet_num, send_count, send_freq, recv_count, recv_freq>

Send_count is the value that QueryPerformanceCounter() returns immediately before it calls send().Send_freq is from QueryPerformanceFrequency(), in units of counts/seconds. Therefore,

send_time[i] (ms) = send_count[i]/send_freq[i] * 1000.0; //this is only a relative time.  Absolute value is effectively meaningless.

Similarly, recv_count is the result of QueryPerformanceCounter() immediately after a packet is received:

recv_time[i] (ms) = recv_count[i]/recv_freq[i] * 1000.0

This send_time and recv_time is what is used to calculate the packet to packet spacing:

Packet loss is a much simpler calculation. When data is being transferred, each packet is tracked with a serial number (packet_num) in the CSV file header. If, during the transfer, the packet serial number of the current packet differs by more than 4 units to that of the previous packet, it means more than 3 consecutive packets were lost. In this case, the test fails.

Related topics

Wireless LAN (802.11) Tests

Send comments about this topic to Microsoft