Hyper-V performance tests (SharePoint Foundation 2010)

Applies to: SharePoint Foundation 2010

This article describes the results of Microsoft SharePoint Foundation 2010 performance testing in a virtualized computing environment that uses Windows Server 2008 Hyper-V technology.

In this article:

• Test goals and description

• Hardware platform

• Software platform

• Test farm topology

• Test methodology

• Results of tests

• Conclusions and recommendations

Test goals and description

In addition to the broad goal of determining how Microsoft SharePoint Foundation performs when virtualized in a Hyper-V environment, we wanted to identify the following issues:

• Performance bottlenecks.

• Performance gains when servers that are running SharePoint Foundation are virtualized on host servers that support new technologies such as hyper-threading (HT) and Second Level Address Translation (SLAT).

• Issues or limitations that have to be considered when virtualizing SharePoint Foundation 2010.

The following four groups of tests, which are documented in this article, are designed to meet the test goals:

• Comparison of virtual machine and physical computer performance

  These tests show the performance results when the same load is applied to a physical computer and a virtual machine configuration.

• Virtual machine memory scale up

  These tests show how performance is affected by increasing the memory allocation on a virtual machine (VM).

• Single host scale out

  These tests show how performance is affected when the number of VMs that are deployed on a single host is increased.

• Virtual machine scale out

  These tests show how performance is affected when the number of virtual machines that are deployed on one or more hosts is increased.

Hardware platform

Most groups of tests used the following two sets of Hewlett Packard (HP) hardware to host our virtual machines:

• HP BL680c-G5 (blade server)

• HP ProLiant SE326M1 (Mach 1 server)

Blade server tests were run for the benefit of customers who are considering repurposing existing server technology for hosting VMs instead of buying new servers.

The Mach 1 server was used to show the performance characteristics of an Intel chipset that supports hyper-threading (Intel Hyper-Threading (HT)). The Mach 1 chipset also supports Intel VT Extended Page Tables (NPT) technology, which Windows Server 2008 R2 Hyper-V can use to implement Second Level Address Translation (SLAT). For more information, see What's New in Hyper-V in Windows Server 2008 R2 (https://go.microsoft.com/fwlink/p/?LinkId=155234).

HP blade servers

The following table provides a summary of the hardware which tested SharePoint Foundation on HP blade servers.

HP Mach 1 servers

The following table provides a summary of the hardware which tested SharePoint Foundation on HP Mach 1 servers.

Software platform

The 64-bit versions of the following products were used for the performance tests:

• Windows Server 2008 R2 Enterprise

• SQL Server 2008 R2 Enterprise

• Microsoft SharePoint Foundation 2010

• Microsoft Visual Studio 2008 Team System (VSTS)

• Microsoft Visual Studio Team System 2008 Test Edition

Test farm topology

The same three-tier farm topology was used on both sets of HP server hardware. This topology was selected because it supports the following general Microsoft SharePoint Foundation roles: front-end Web servers, an application server, and a database server.

Test methodology

The test mix, which is derived from the usage pattern identified on Microsoft internal SharePoint Foundation servers, consisted of the following tasks:

• Create, use, update, and delete

• Page views such as list views and document libraries

• Browser-based views of Word, Excel, and PowerPoint by using Microsoft Office Web Apps

• Simulated traffic between the server and rich clients such as Microsoft Word 2010, Microsoft Excel 2010, and Microsoft PowerPoint 2010 as an application.

The previous tasks were tested by using 84 separate actions that ranged from .02% of the test mix to 80% of the test mix. Loads were stepped (1 to 10) for different groups of tests and the number of simulated users ranged from 1 to 1000 within a connection-per-user model.

After a warm-up time of 10 minutes, the tests ran for 10 minutes and collected data every 10 seconds.

Two key performance indicators were used for each of the four groups of tests: maximum passed request per second (Max. passed RPS) and the time taken for each request, expressed as milliseconds.

Results of tests

The results of our testing are specific to the hardware and workload that was described previously, and results are not indicative of product performance in all cases. The test results show the potential tradeoffs when you are deciding to virtualize a SharePoint Foundation 2010 configuration with a particular usage pattern. It is very important to interpret our test results in the context of your infrastructure and your implementation of SharePoint Foundation.

Comparison of virtual machine and physical computer performance

These tests show the performance results when the same load is applied to a physical computer and a range of virtual machines.

This group of tests had the following goals:

• To determine the decrease in performance or gain when virtualizing a physical server

• To determine the virtual configuration that is required to achieve the same performance level as a physical computer

• To determine the amount of resource overhead or degree of decreased performance when the workload that is applied to a physical server is tested on a virtual machine.

HP blade server

The physical server is used as a front-end Web server that has 24 cores and 24,576 MB of memory. For the virtual environment, six VMs are used as Web servers on two hosts. Each VM has four virtual processors and 15,000 MB of memory.

Notice that the following results on the previous graph:

• The Max. passed RPS is approximately 20% higher on the physical computer.

• The Avg. response time (ms) is approximately 15% lower on the physical computer.

• The Web server CPU usage (%) for the virtual machines is approximately 9% lower than on the physical server.

HP Mach 1 server

The physical server is used as a front-end Web server that has eight cores and 49,152 MB of memory. For the virtual environment, four VMs are used as Web servers on two hosts. Each VM has the configuration shown in the following list.

• Four virtual processors

• 8,192 MB of memory

• Two volumes using fixed size VHD files

The following graph shows performance results for VMs with hyper-threading enabled (HT on), VMs with hyper-threading disabled (HT off), and a physical computer.

Notice that the following results on the previous graph:

• When hyper-threading is enabled (HT on):

  • The Max. passed RPS is approximately 18% higher on the physical computer

  • The Avg. response time (ms) is approximately 15% lower on the physical computer.

  • The Web server CPU usage (%) is approximately 46% lower than a VM with hyper-threading disabled, and approximately 36% lower than the physical computer.

• When hyper-threading is disabled (HT off):

  • The Max. passed RPS is approximately 29% higher on the physical computer.

  • The Avg. response time (ms) is approximately 20% lower on the physical computer.

Virtual machine memory scale up

In this group of tests, the front-end Web servers are configured to use different amounts of memory to determine how scaling up memory on the VMs affects performance. The test uses four front-end Web servers on two host servers (HP blade server). Each VM is configured to use four virtual processors and two volumes using pass-through disks.

The following graph shows the average response time and requests per second (RPS) for VMs that have a starting configuration of 2,048 megabytes (MB) of RAM and are scaled up to 15,000 MB of RAM.

The previous graph shows that there is very little variation in CPU usage for the hosts, SQL Server, and the VSTS agent.

From a scale-up perspective, there is a minimal performance gain when scaling from 2 GB of memory to 4 GB of memory. Beyond the 4 GB memory allocation on the virtual machines there is no significant performance gain for the test workload.

Single host scale out

These tests show how performance is affected when the number of VMs is increased on a single host. The tests also show the effect on performance when the host cores are oversubscribed.

HP blade server

This test uses up to eight VMs and each VM has the configuration shown in the following list.

• Four virtual processors

• 15,000 MB of memory

• Two volumes using pass-through disks

The following graph shows that optimal throughput is achieved by using six front-end Web servers, which is a 1:1 mapping of logical processors to virtual processors. When more VMs are added, performance decreases. The only observed bottleneck is CPU contention when the CPU is oversubscribed.

The test mix used on the HP blade servers was then run two times on the Mach 1 servers. On the first run hyper-threading was enabled, and on the second run hyper-threading was disabled.

HP Mach 1 server

The test uses up to four VMs and each VM has the configuration shown in the following list.

• Four virtual processors

• 8,192 MB of memory

• Two volumes using fixed size VHD files

The following graph shows performance results when hyper-threading is enabled on the host.

The following graph shows performance results when hyper-threading is disabled.

The two previous graphs indicate that there is no need to oversubscribe the host CPU because there is no significant performance gain or loss.

Another key outcome of these tests is the increase in compute headroom, which is indicated by "Web server host logical CPU usage". By comparing this value in both graphs you can see that the increase in compute capacity ranges from 10% to 25% depending on the level of CPU oversubscription.

Virtual machine scale out

This group of tests shows the effect on performance when the number of virtual machines is scaled out across hosts and bottlenecks are identified. In addition, virtual machine roles are mixed to determine the effect on performance.

The test uses a range of host servers (2-3) and a range of virtual machines (6-12) deployed across the hosts. Each VM has the configuration shown in the following list.

• Two virtual processors

• 15,000 MB of memory

• Two volumes using pass-through disks

The following graph shows performance when virtual machines are scaled out.

Notice that the following results on the previous graphs:

• With dedicated roles for the virtual machines, the performance gain is noticeable when the number of hosts and the number of VMs is scaled up to three hosts with 12 VMs. Using Max. passed RPS as a benchmark, performance is approximately 2.5 times better. However, the average response time per request degrades as indicated by the approximately 26% increase in response time.

• With mixed roles for the virtual machines and using the number of hosts and the same number of VMs, Max. passed RPS improves by approximately 31%. Additionally, the Avg. response time (ms) is improved, as indicated by the drop of approximately 25%. Typically there is a lower duty cycle on an application server, which enables Hyper-V to use the additional resources for other VMs that are running on the host.

An additional performance observation is that there was a significant increase in SQL Server CPU usage when the tests were run with mixed virtual machine roles. This increase was approximately 35%.

HP Mach 1 server

The test uses a range of host servers (2-4) and a range of virtual machines (3-9) deployed across the hosts. Each VM has the configuration shown in the following list.

• Four virtual processors

• 8,192 MB of memory

• Two volumes using fixed size VHD files

The following graph shows performance with hyper-threading enabled on the host.

Notice the following results on the previous graph:

• With dedicated virtual machine roles, the performance gain is noticeable when the number of hosts and the number of VMs are scaled up to three hosts with six VMs. Using Max. passed RPS as a benchmark, performance is approximately 42% better and the average response time per request decreases by approximately 21%.

• When the number of hosts and number of virtual machines is scaled up again to four hosts and nine virtual machines, Max. passed RPS improves by approximately 57% compared to the three host and six VM configuration. Compared to the HP blade servers the Avg. response time (ms) only increases by approximately 4%.

• SQL Server CPU usage (%) follows the same pattern as on the HP blade servers, increasing linearly to 41%.

Conclusions and recommendations

The following table summarizes the conclusions drawn from our tests which are based on a specific workload, the hardware that was used, and the virtual machines used.

In our test environment we observed a decline in performance when SharePoint Foundation roles are virtualized. This is the result of Hyper-V overhead as well as operating system overhead on each virtual machine.

The amount of virtualization overhead and reduced performance (relative to a similarly configured physical computer) depends on the size (for example, the amount of memory, and the number of cores) as well as the features of the host hardware chipset (for example, support for Second Level Address Translation and hyper-threading.)

We recommend that you clearly understand what you hope to gain by virtualizing all or some of a SharePoint farm. After the decision to virtualize is made, thorough planning and testing is required to fully understand all performance options and implications.

You can use the results of tests in this article as a guide. However, results are not definitive and the results are for a specific workload in a specific physical environment.