Create a detailed design and system specifications for a virtual farm (SharePoint Server 2010)
This article provides information about capacity management and high availability for a virtual environment hosting Microsoft SharePoint Server 2010. We combine these two concepts in this article because capacity and sizing are very important parts of developing a virtualization plan and the architecture for a virtual environment, and because capacity management is not isolated from high availability in a virtual environment. In the case of virtualization hosts, insufficient capacity can block high availability at the farm level and at the host level.
As is the case with other aspects of a virtual environment, such as backup and recovery, capacity management and high availability have to accommodate the two layers of a virtual environment, which are the virtual machines used for SharePoint Server 2010 and the physical servers that are used to host the virtual machines. In the case of a hybrid environment, you also have to deal with physical Microsoft SharePoint Server farm servers.
In this article:
Virtualization overview
Capacity management
Virtualization server capacity and sizing
Creating and refining the architectures
Additional options for improving the architecture
High availability
Virtualization overview
Server virtualization, as implemented by Windows Server 2008 Hyper-V technology or Microsoft Hyper-V Server 2008, is hardware-based and is also referred to as hardware-assisted virtualization, as opposed to software-based virtualization. The Hyper-V hypervisor has a more direct communication path to, and interaction with, physical server hardware components than software-based virtualization technologies. The net result is better performance than a software-based virtualization technology. For more information about the Hyper-V architecture, see An Introduction to Hyper-V in Windows Server 2008 (https://go.microsoft.com/fwlink/p/?LinkId=188006) and Monitoring Hyper-V performance (https://go.microsoft.com/fwlink/p/?LinkID=187746&clcid=0x409).
Although a physical server may meet Hyper-V requirements, each physical server is unique. Every manufacturer uses its own implementation of processors, multi-core technology, memory, the data bus, hard disks, and network adapters. Additionally, hardware design and implementation varies from model to model, even if the models are produced by the same manufacturer. This highlights the need for rigorous testing when you deploy SharePoint Server 2010 in a virtual environment.
Software programs and applications exhibit the same variations in performance as hardware. Some programs are CPU intensive, other programs have high memory demands, and other programs are hard disk intensive. SharePoint Server has its own capacity needs, as does Internet Information Server (IIS) and SQL Server 2008. Once again, rigorous testing is required.
Capacity management requires you to consider the virtualization server, the storage solution, the network infrastructure, the technologies running in a SharePoint Server environment, and the features that are enabled to implement your SharePoint Server solution.
Capacity management
Capacity management extends the concept of capacity planning to express a cyclical approach in which the capacity of a SharePoint Server 2010 deployment is continually monitored and optimized to accommodate changing conditions and requirements. You can apply this approach to all SharePoint Server farms, including those that are fully virtualized, and those that are partly virtualized. For an overview of capacity management, see Capacity management and sizing for SharePoint Server 2010. Additional capacity management resources are located at the Capacity Management for SharePoint Server 2010 (https://go.microsoft.com/fwlink/p/?LinkId=194748) Resource Center.
Virtualization server capacity and sizing
After you have a SharePoint Server farm design and sizing recommendations for the farm servers, you design the physical virtualization host architecture required to support the virtual farm. For more information about virtual architectures, see Plan virtual architectures (SharePoint Server 2010).
We recommend that you use applicable principles from SharePoint Server 2010 capacity management and use them as guides for a virtual environment. The following activities illustrate the iterative nature of designing, sizing, and adjusting a virtual and physical architecture from initial planning to deployment in a production environment.
Note
If you do thorough planning and testing, changes to the architecture and server configurations should only be required if there is a significant and unanticipated increase in farm use or because new features are added to your SharePoint Server solution.
Before you start the farm deployment, create a virtual and physical architecture with virtual machine and virtualization server sizing. In the case of multiple virtualization hosts, this architecture must include virtual machine distribution.
During the pilot phase of deployment, collect health and performance data that you can use to establish benchmarks for the farm virtual machines and the virtualization hosts.
During the user acceptance test phase of deployment, adjust the virtualization host and virtual machine configurations based on the benchmark data. If necessary, change the physical architecture by redistributing virtual machines on the virtualization hosts.
After deployment, continue to collect health and performance benchmarks and refine virtual machine and, if applicable, physical machine configurations. If necessary, adjust both architectures.
It is essential that you can analyze virtualization host and virtual machine performance data and understand how it reflects capacity needs and application effect on capacity. Additionally, you have to understand performance and capacity limits. Given the interrelationship between the virtual layer and the physical layer, anything that affects virtual machine capacity and performance either has a direct effect on the host or has to be accommodated for by making changes to the virtualization host configuration in order to sustain acceptable performance across the farm.
In some cases it may be necessary to change the physical architecture by adding additional virtualization hosts and then changing the distribution of virtual machines on the physical architecture.
Important
In benchmark tests between a physical computer and a virtual machine, the throughput of the virtual machine typically cannot match that of a physical computer. Virtual machine performance is, with rare exceptions, always lower than that of a physical computer. The degree of performance difference depends on virtualization host capabilities, the applications that are running, and the benchmarks that you choose to use as primary performance indicators.
We recommend that you read the Hyper-V Performance FAQ R2 (https://go.microsoft.com/fwlink/p/?LinkID=187745), which is updated to reflect capacity and performance information for Windows Server 2008 R2 and Windows Server 2008 with Service Pack 2 (SP2). This FAQ contains answers to common Hyper-V questions, provides guidance, and includes links to detailed articles that you can use to develop benchmarks for the virtualization host, virtual machines, and Windows networking.
We also suggest you read the following posts about Hyper-V performance counters:
Hyper-V Performance Counters - Part one of many - The overview (https://go.microsoft.com/fwlink/p/?LinkID=125651)
Hyper-V Performance Counters – Part two of many – "Hyper-V Hypervisor" counter set (https://go.microsoft.com/fwlink/p/?LinkID=125652)
Hyper-V Performance Counters – Part three of many – "Hyper-V Hypervisor Logical Processors" counter set (https://go.microsoft.com/fwlink/p/?LinkID=125653)
Hyper-V Performance Counters – Part four of many – "Hyper-V Hypervisor Virtual Processor" and "Hyper-V Hypervisor Root Virtual Processor" counter set (https://go.microsoft.com/fwlink/p/?LinkID=125655)
Creating and refining the architectures
A complete architecture consists of the virtualization hosts, virtual machines, and physical machines that make up the SharePoint Server environment that you plan to deploy. For more information about virtualization architectures, see Plan virtual architectures (SharePoint Server 2010).
Developing and implementing a virtual architecture consists of the following steps:
Create the virtual and physical architecture. Create an architecture that will support the goals of your SharePoint Server 2010 farm.
Analyze the architectures. Identify and obtain any information that is missing or that will improve the design of the environment that you plan to deploy.
Refine the architectures. Use the information from step 2 to refine the architecture.
Continue to refine the architectures and server configurations as you move through the various deployment stages. For more information about the deployment stages, see the SharePoint 2010 Products Deployment and SharePoint 2010 Products: Virtualization Process models, available in the Technical diagrams (SharePoint Server 2010) article.
Create the architecture
Create a model of the architecture that you can use as a tool for evaluating and adjusting virtual machine and virtualization host configurations. Use the following criteria as a guide for developing your model:
Identify the number of virtual machines that are needed and the role of each in the SharePoint Server farm.
Specify individual virtual machine configuration requirements (disk space, memory, and number of processors). These are based on SharePoint Server capacity requirements.
Specify virtualization host requirements (disk space, memory, and number of logical processors). These are based on virtual machine requirements.
Identify virtual machine distribution on virtualization hosts. These are based on farm high-availability requirements and are constrained by virtualization host quantity and capacity.
Identify general networking and storage requirements.
Allow for growth on virtualization hosts and on the virtual machines (scale up or scale out).
After you create an architecture model, you have to analyze both architectures to validate the design as well as the virtualization host and virtual machine configurations.
Analyze the architectures
The fundamental purpose of analyzing the architecture is to determine whether it can successfully support the SharePoint Server 2010 solution that you want to deploy. However, it is reasonable to assume that design and server configurations will change as you move through the deployment process.
The following illustration shows a sample virtual architecture for a farm that consists of front-end Web servers, application servers, and database servers. This architecture is representative of the small to medium farms described in Example virtual architectures for small to medium size farms, and we can use it show the key elements that have to be considered when you analyze the capacity and availability requirements for a virtual farm.
Important
Virtualization server and virtual machine sizing in the following illustration is not prescriptive.
Figure 1. Preliminary architecture
.gif "Virtual SharePoint Server 2010 farm topology")
Use the criteria provided for creating a virtual architecture to analyze the sample architecture shown in the previous illustration. The architecture in the illustration assumes that all the Web servers and application servers are virtual machines. It has not been determined whether the farm database servers are physical machines or virtual machines.
Virtualization host analysis
The following tables (HOST-1 and HOST-2) provide an analysis of each virtualization host and use memory, processors, and scalability as criteria. The host analysis is followed by a design analysis.
HOST-1
| Criteria | Analysis |
|---|---|
Memory |
After factoring in 2 GB of RAM for the host operating system and using the projected RAM requirements, there is an estimated 2 GB of RAM available for future use. |
Processors |
The logical to virtual processor mapping is 8:10 (1:1.25), which means that the CPU is slightly oversubscribed, which would not be an issue in a test environment. Important Oversubscribing the CPU on a virtualization server will reduce overall performance. The extent of this effect is determined by the load put on the virtual machines. As a best practice, do not oversubscribe the virtualization server CPU if it can be avoided. |
Scalability |
This is not an option because there is insufficient memory. Additionally, the degree of CPU oversubscription (even by adding a virtual machine with two processors) would have a noticeable effect on performance. |
HOST-2
| Criteria | Analysis |
|---|---|
Memory |
After factoring in 2 GB of RAM for the host operating system and using the projected RAM requirements, there is an estimated 6 GB of RAM available for future use. |
Processors |
The logical to virtual processor mapping is 8:8 (1:1), which meets the best practice guidance. |
Scalability |
There is sufficient memory to increase the memory allocation to the virtual machines. There is enough capacity to add a new virtual machine with two processors and 4 GB of RAM. This means that the virtualization host CPU would be slightly oversubscribed (8:10), but like HOST-1, would not be an issue in a test environment. |
Design analysis
The sample architecture generally shows a degree of high availability for the farm servers. For example, there are three front-end Web servers distributed across HOST-1 and HOST-2, and the database servers (clustered or mirrored) also reside on separate virtualization hosts or separate physical servers. High availability at the virtualization host level is not part of the architecture and pertinent information is missing. The following information is required before the design can be revised:
Database size
The size of the content database determines how you configure and distribute all the farm servers.
Storage subsystem
For example, in the sample architecture, no information is provided about the number of disks required for each virtual machine, nor is there any indication of disk distribution and capacity. This information is very important for determining and configuring the storage system. The architecture sample uses local storage. You have to determine whether this is suitable for your environment, or if you want to use pass-through disk configuration to a LUN on a SAN.
Networking requirements
The number of network adapters and minimum throughput has to be identified.
Virtual hard disk configurations
You also have to determine which of the Hyper-V hard disk configurations you want to use (for example, fixed size, pass-through). For more information, see Planning for Disks and Storage (https://go.microsoft.com/fwlink/p/?LinkId=188007) and Virtual Hard Disk Performance: Windows Server 2008 / Windows Server 2008 R2 / Windows 7 (https://go.microsoft.com/fwlink/p/?LinkId=186519).
After you complete a design review, the next step is to refine the architecture.
Refine the architecture
The scope of refining the architecture depends on your initial architecture, the results of your analysis, and your implementation plan. Using the sample that is provided, there are scenarios where you might decide not to make any changes. For example:
The preliminary architecture is suitable for early testing, proof-of-concept, and a limited pilot deployment.
The virtualization hosts are for testing only and will be replaced by higher capacity hosts during the user acceptance test phase.
The virtual farm is for testing purposes only and will be shut down after testing is finished. In some cases the environment may be retained and used at a later date to test software updates.
The following illustration shows a revised architecture that is better suited for a production farm.
Figure 2. Revised architecture
.gif "Revising a virtual architecture")
In the revised architecture, the primary assumption is that you want to stay with eight core commodity virtualization servers. The changes in the preceding illustration reflect that assumption and include the following considerations:
The estimated size of the content database is 1 terabyte (TB).
The objective is to provide high availability for all the farm servers and to maximize performance across infrastructure.
The farm database servers are physical servers that can be clustered or mirrored to support high availability. Each server has 8 cores, 16 GB of RAM and uses local drives to reduce latency.
Virtualization host analysis
The following tables (HOST-1 revised and HOST-2 revised) provide an analysis of each virtualization host using memory, processors, and scalability as criteria. The host analysis is followed by a design analysis.
HOST-1 revised
| Criteria | Analysis |
|---|---|
Memory |
After factoring in 2 GB of RAM for the host operating system and using the projected RAM requirements, there is an estimated 2 GB of RAM available for future use. |
Processors |
The logical to virtual processor mapping is 8:10 (1:1.25), which is slightly oversubscribed. |
Scalability |
There is a marginal amount of memory available to increase the memory allocation to the virtual machines. Based on the amount of memory and the processor ratio, there is not enough host capacity to add an additional virtual machine. |
HOST-2 revised
| Criteria | Analysis |
|---|---|
Memory |
After factoring in 2 GB of RAM for the host operating system and using the projected RAM requirements, there is an estimated 4 GB of RAM available for future use. |
Processors |
The logical to virtual processor mapping is 8:12 (1:1.50), which is oversubscribed by 50 percent. |
Scalability |
There is a marginal amount of memory available to increase the memory allocation to the virtual machines. Based on the amount of memory and the processor ratio, there is not enough host capacity to add an additional virtual machine. |
Design analysis
Each virtual machine uses a three-drive configuration, sized according to SharePoint Server best practice guidance. These drives are typically configured as follows:
Drive C (50 GB) for Windows installation
Drive D (50 GB) for SharePoint Server 2010 files
Drive E (300 GB) for Web content and log files
Each front-end Web server is configured with four virtual processors (4xVP) and 8 GB of RAM. This is the minimum recommended configuration for a production environment.
The number of front-end Web servers is increased to four to support effective clustering and high availability. This four-server configuration is particularly well-suited for installing software updates because there will always be two servers available when you install updates.
The two application servers (App-1, App-2) provide high availability. App-1 hosts Central Administration, the Search crawl component, and the passive index for the Search query component. The number of processors and amount of memory is based on the estimated size of the content database.
App-2 is a dedicated Search query server. It also contains a copy of Central Administration. The number of processors and amount of memory is based on the estimated size of content database.
For high availability, Central Administration is also installed on a front-end Web server on another host.
The database servers are physical servers that are clustered or mirrored to ensure high availability. This move to physical servers has the benefits of increasing virtualization host capacity for the virtual farm servers and improving overall database performance.
Note
As indicated earlier in this article, the decision to virtualize or not virtualize database servers is a complex decision that requires extensive planning and testing.
From a networking perspective, both virtualization hosts are configured with two separate 1-gigabit physical network adapters. This is a recommended practice to ensure that virtualization host and virtual machine data traffic is separated to improve performance and provide some adapter redundancy.
Each virtualization host employs a virtual LAN (VLAN), which can provide the following benefits: network segregation, improved security, and performance.
The revised virtual and physical architecture is significantly improved and could be deployed into a production environment. However, it is important to note that, as configured, available virtualization host resources do not support farm scaling. Additionally, they cannot support the migration of a farm server from one host to another if the need arises.
Realistically, if you want to deploy the example farm into production, we recommend that you consider the following upgrades:
Increase virtualization host capacity by using a 16-core computer with 48 or 64 gigabytes of RAM.
Add one or more virtualization hosts.
To achieve the optimum level of high availability, consider the additional options in the following section.
Additional options for improving the architecture
The previous section provided options for revising the model. There are, of course, other options to achieve better performance and high availability. Scaling out the virtualization host environment or scaling up the virtualization hosts are good alternatives, although cost is always an issue. Your organization's virtualization strategy will help define the best approach.
Tip
In terms of cost, it is usually less expensive to purchase a server that has more capacity than you need in the short term than it is to upgrade a server to gain more capacity. This is especially true in the case of memory upgrades, where typically you have to throw away the existing memory modules and buy a full set of new memory in order to upgrade the memory.
Performance gains can be achieved with the following options:
Deploy or purchase servers that have Second-Level Address Translation (SLAT) enabled processors. In Intel processors, this feature is referred to as Nested Page Tables and is available in Nehalem 55xx series processors. For AMD, this feature is referred to as Enhanced Page Tables (EPT).
Deploy or purchase servers that provide CPU Core-Parking, a feature that allows the running Hyper-V to use the lowest number of processor cores to meet the workload demand.
Investigate TCP chimney offload, Virtual Machine Queues (VMQ) and jumbo frames. These features improve network performance and decreases CPU utilization, thereby increasing the overall system capacity.
Investigate jumbo frame support to speed up network performance when transferring large amounts of data. However, you must test this thoroughly because jumbo frames do not work in all environments.
Investigate adapter teaming. This feature can improve network performance and provide failover capability to the physical network adapters.
Important
Adapter teaming is a third-party solution and is only supported by the vendor. For more information, see Microsoft Support Policy for NIC Teaming with Hyper-V (https://go.microsoft.com/fwlink/p/?LinkId=194749).
High availability
To ensure high availability for a virtual environment, consider implementing Windows Server 2008 R2 failover clustering and Hyper-V live migration, as follows:
The scope of failover clustering can include virtualization hosts and the virtual machines on each host. If a virtualization host fails unexpectedly, the virtual machines automatically fail over to another virtualization host.
Live migration is a solution for planned downtime. You can migrate running virtual machines to another server (without downtime), shut down the physical server, and perform the maintenance. When you finish maintenance on the server, use live migration to move the virtual machines back to the original physical server. Microsoft supports the use of Live Migration with virtual machines running SharePoint Server 2010.
For more information, see Hyper-V: Using Hyper-V and Failover Clustering (https://go.microsoft.com/fwlink/p/?LinkID=187967) and Hyper-V: Using Live Migration with Cluster Shared Volumes in Windows Server 2008 R2 (https://go.microsoft.com/fwlink/p/?LinkId=188009).