Install-on-Demand for Games
This technical article discusses two techniques, install-on-demand and background install, using Windows Installer. Games can utilize these installation techniques to provide a better and more pleasant gaming experience for players by reducing the installation time.
- Overview
- Patching Support
- The InstallOnDemand SDK Sample
- The Sample's Features and Components
- Installation
- Limitation
Overview
Installation has been an element of computer-based applications for a long time. Most applications today require that they be installed on the user's local hard drive first before they can be used. Computer games are no exception; when a consumer buys a Microsoft Windows game and tries to run it, he must first go through an installation process that copies necessary files from the game disc to the hard drive. This installation process is usually lengthy and can take as much as an hour to complete. The installation time is a factor that makes console games more desirable than computer-based games for some players, as they are able to immediately play a console game after inserting the game disc. The technology discussed in this article will attempt to remedy this by drastically reducing the installation time.
Traditionally, games require that all or most of the files be installed prior to launching. In order to achieve install-on-demand, the game resources need to be modular; that is, the developer must divide the application's resources (graphics, audio, and so on) into components. Each component is a set of resources that can be installed or removed as a unit. Once this is done, the game developer defines one or more features, typically one or more per level or zone. Each feature of an application specifies a set of components necessary for running that particular feature. When an application is installed, its features can be marked as "install" (components copied to the local hard drive at installation time) or "advertised" (components copied to the local hard drive after the initial install when the application uses that feature). A game developer can reduce the installation time by designing the game to launch and run with a minimal set of features installed. The rest of its features can be marked as advertised and installed on demand when the application actually needs to use functionalities that those features provide.
Games can call Windows Installer to install a particular feature that may not have been installed. To make the installation appear in the background, a worker thread can be used to make the calls to the installer while the primary thread continues to handle game logic and to render the screen. This minimizes the disruption of gameplay caused by the installation. The game can initiate the installation at any time. However, because the installation consumes processor cycles, it is generally not a good idea to perform the installation when the primary thread is in critical need of processing power, such as when the user is in the middle of the action. For example, a good time to perform the installation may be during the time the user is in the game menu, when the game is paused or minimized, or when the user is watching the introduction movie or cutscenes.
Patching Support
Most games today need to be updated even after they ship as bugs get fixed and new features get added. Updating often requires patching, which is traditionally a straightforward procedure for games. Because all of the necessary files are installed onto the user's hard drive, patching a game involves copying revised files onto the hard drive, overwriting existing files. When install-on-demand is employed, not all files are installed and copied at the time of patching. Therefore, the patcher cannot simply write updated files onto the game folder.
Windows Installer has features for patching applications that use install-on-demand. When applying a patch, the installer caches the patch onto the system. This feature works well for patches with small deltas. The originally released files no longer need to be on the disk at the time of patching, so those files can be advertised. Later, when the application runs and needs to access the files, the installer installs the most recent version of those files by copying the originally released version from the media (for example, CDs) and applying the patch after reading the saved patch data.
The InstallOnDemand SDK Sample
The Install-on-Demand for Games sample demonstrates the install-on-demand techniques discussed in this article. Unlike other samples, Install-on-Demand for Games cannot be run directly from the sample browser. Because the sample uses the Windows Installer to manage its installation, it needs to be included in the installer's database of installed applications.
To launch the sample
- Use the Install Project link in the sample browser to copy the sample's files to a folder.
- Double click InstallOnDemand.msi to install the sample.
- Select Typical Install.
- Start the sample by launching the InstallOnDemand.exe in the installed folder (typically Program Files\InstallOnDemand) or by launching from Start Menu\Programs.
InstallOnDemand.msi is a database recognized by the installer. It defines the entire installation process: the directory structure, what will and will not be copied, what resources will be copied together, what registry values to write, what shortcuts to create, and so forth.
When launched, the sample plays an introduction sequence. The player can end it and enter the main menu by pressing the ESC key. Following the introduction, the player can start a new game by entering a character name and scrolling through the statistics. Before the sample begins to play the introduction sequence, the sample calls an installer function to check if the feature for level 1 is installed. If the level 1 feature has not been installed, the sample uses a background thread to ask the installer to install the game, while the primary thread is doing something else (such as playing the introduction sequence, rendering the menu, or interacting with the player on character creation). The experience is different from traditional game installation in that the user is occupied in the game (watching the introduction or creating a new character) while installation is in progress. After the player has finished creating a character, the sample loads the resources for level 1.
On the right side of the sample screen are five buttons marked "Play Level 1" through "Play Level 5." These buttons simulate the player's completion of the current level and advancement to the next. When one of these buttons is clicked, a statistics screen appears showing information about the level that he has just finished. The sample also takes this time to ask the installer to check and install the next level, if it is not already installed. The installation occurs while the player is reading the statistics screen, so that when the user clicks OK to enter the next level, the level's resources are all installed and ready to be loaded.
The Sample's Features and Components
Traditionally, games require that all or most of the files be installed prior to launching. In order to achieve install-on-demand, the game resources need to be modular; that is, the developer must divide the application's resources (graphics, audio, and so on) into components. Each component is a set of resources that can be installed or removed as a unit. Once this is done, the game developer defines one or more features, typically one or more per level or zone. Each feature of an application specifies a set of components necessary for running that particular feature. When an application is installed, its features can be marked as "install" (components copied to the local hard drive at installation time) or "advertised" (components copied to the local hard drive when the application later uses that feature). A game developer can reduce the installation time by designing the game to launch and start running with a minimal set of features installed. The rest of its features can be marked as advertised and installed on-demand when the application actually needs to use functionalities that those features provide.
The following table lists the six top-level features that the sample defines.
| Feature Name | Functionality | Components | Files |
|---|---|---|---|
| Core | Includes resources required at all times, regardless of level. These resources are: sample executable, media required by introduction sequence and loading screen, and .fx file that handles all rendering in the sample. | Core | InstallOnDemand.exe, InstallOnDemand.fx, Loading.bmp, Level.x |
| Core | (same as above) | CoreUI | Media\UI\dxutcontrols.dds, Media\UI\DXUTShared.fx, Media\UI\arrow.x |
| Core | (same as above) | CoreMisc | Media\Misc\seafloor.x, Media\Misc\seafloor.bmp |
| Core | (same as above) | CoreSpeeder | Media\PRT Demo\LandShark.x, Media\PRT Demo\speeder_diff.jpg |
| Core | (same as above) | CoreReg | N/A (registry value) |
| Level1 | Provides resources used by level 1. | Level1 | Level1.jpg |
| Level1 | (same as previous) | L1Skybox | Media\Light Probes\galileo_cross.dds |
| Level2 | Provides resources used by level 2. | Level2 | Level2.jpg |
| Level2 | (same as previous) | L2Skybox | Media\Light Probes\grace_cross.dds |
| Level3 | Provides resources used by level 3. | Level3 | Level3.jpg |
| Level3 | (same as previous) | L3Skybox | Media\Light Probes\rnl_cross.dds |
| Level4 | Provides resources used by level 4. | Level4 | Level4.jpg |
| Level4 | (same as previous) | L4Skybox | Media\Light Probes\stpeters_cross.dds |
| Level5 | Provides resources used by level 5. | Level5 | Level5.jpg |
| Level5 | (same as previous) | L5Skybox | Media\Light Probes\uffizi_cross.dds |
The Level1 through Level5 features have additional sub-features that contain files not directly used by the sample. These sub-features files have been added to make the installation take longer. This is done to illustrate the ongoing installation operation that is running in the background while the sample is running.
The following table lists the sub-features.
| Feature | Sub-features | Files |
|---|---|---|
| Level1 | L1PH1, L1PH2, L1PH3, L1PH4, L1PH5 | Level1 Placeholder Data\L1PH1.dat Level1 Placeholder Data\L1PH2.dat Level1 Placeholder Data\L1PH3.dat Level1 Placeholder Data\L1PH4.dat Level1 Placeholder Data\L1PH5.dat |
| Level2 | L2PH1, L2PH2, L2PH3, L2PH4, L2PH5 | Level2 Placeholder Data\L2PH1.dat Level2 Placeholder Data\L2PH2.dat Level2 Placeholder Data\L2PH3.dat Level2 Placeholder Data\L2PH4.dat Level2 Placeholder Data\L2PH5.dat |
| Level3 | L3PH1, L3PH2, L3PH3, L3PH4, L3PH5 | Level3 Placeholder Data\L3PH1.dat Level3 Placeholder Data\L3PH2.dat Level3 Placeholder Data\L3PH3.dat Level3 Placeholder Data\L3PH4.dat Level3 Placeholder Data\L3PH5.dat |
| Level4 | L4PH1, L4PH2, L4PH3, L4PH4, L4PH5 | Level4 Placeholder Data\L4PH1.dat Level4 Placeholder Data\L4PH2.dat Level4 Placeholder Data\L4PH3.dat Level4 Placeholder Data\L4PH4.dat Level4 Placeholder Data\L4PH5.dat |
| Level5 | L5PH1, L5PH2, L5PH3, L5PH4, L5PH5 | Level5 Placeholder Data\L5PH1.dat Level5 Placeholder Data\L5PH2.dat Level5 Placeholder Data\L5PH3.dat Level5 Placeholder Data\L5PH4.dat Level5 Placeholder Data\L5PH5.dat |
During installation, the core feature should be marked "install" and all other features should be marked "advertised." By only installing one feature instead of six, the time that the player must wait until the game launches is significantly reduced.
Installation
The Windows Installer provides a mechanism for an application to request that an advertised feature be installed. However, the mechanism is a synchronous application programming interface (API) call, which means that the application has to wait inside the call until the installation is complete. To achieve background installation, a worker thread is required so that the main application thread is free to perform other important tasks, such as rendering on the screen to continue giving the player visual feedback.
In the sample, there are three states of installation that happen during the sample execution: active installation, passive installation, and no installation.
- Active installation is a request initiated by the sample when it needs to access or load resources provided by one or more features. The sample does this when it cannot continue until the resource is installed.
- Passive installation is initiated when the sample is not performing a critical task such as when the player is at a menu or watching a cut scene. When this is the case, the worker thread checks if any feature of the sample is still advertised. If it can find one, it calls the installer to install that feature. This process repeats until every feature of the sample is installed. Essentially, passive installation utilizes extra processor cycles to perform installation in the background when it is least intrusive to the main sample.
- No installation happens when the player is actively engaged in the game; this prevents a framerate drop which would disrupt the user experience.
In the sample, a class CMsiUtil is defined to handle all installation-related tasks. Essentially, CMsiUtil uses a worker thread that invokes the installer to install the sample's features in a loop. The class has two queues that store installation requests: one high priority queue for active installation and one low priority queue for passive installation. During initialization, the class enumerates all features of the product and adds them to the passive installation queue. Because the entire product is queued this way, the entire product will eventually be installed if the sample has enough free processor cycles.
When the sample needs to request an active installation, the sample can call CMsiUtil::UseFeatureSet() and pass the name of the top-level feature. UseFeatureSet() will queue the requested feature and all of its sub-features in the active installation queue so that the worker thread can install them.
While executing an install request, the worker thread will check the active installation queue as well as the passive installation queue to see if either queue has any additional requests. Each time the thread finds a request, it will call the installer API to perform the actual installation. Once both queues are empty, the worker thread will go to sleep with a call to WaitForSingleObject. Because the entire product is placed in the passive installation queue during initialization, an empty queue implies that the entire product has been installed.
The sample calls CMsiUtil::EnablePassiveInstall() to enable or disable passive installation. EnablePassiveInstall(true) increments the enable count for passive installation, and EnablePassiveInstall(false) decrements it. If the enable count is greater than 0, the class will process the passive installation queue. The sample allows passive installation when any of the following is true:
- The user is viewing the initial introduction sequence.
- The user is navigating in the sample menu.
- The user is viewing the statistics at the end of a level.
- The sample application loses its focus and goes to the background.
The methods of CMsiUtil are listed below:
| Method | Description |
|---|---|
| AbortAllRequests | Causes the current installation to abort and empties the active installation request queue. |
| AbortCurrentRequest | Causes the installation in progress to abort. The worker thread then processes the next request in the queue, if one exists. |
| EnablePassiveInstall | Increments or decrements passive installation enable count. The sample uses this call to control when passive installation can and cannot happen. |
| GetCurrentFeatureName | Returns the name of the feature that is actively being installed. |
| GetFeatureProgress | Returns the current tick position for the feature that is being installed. |
| GetFeatureProgressMax | Returns the maximum number of progress tick for the feature that is being installed. |
| GetLastError | Use this method to retrieve the return code from the previous installation request. |
| GetPassiveProgress | Returns the progress bar tick position for passive installation. |
| GetPassiveProgressMax | Returns the current tick position and the maximum number of ticks for passive installation. Together, the sample can use them to show the overall progress of the passive installation. |
| GetProgress | Returns the progress bar tick position for the active feature set installation. This is used when the sample renders the installation progress bar. Because the Windows Installer only provides progress information for the one feature being installed, the method divides the progress bar among the requested features so that the user still sees the entire installation as one task. |
| GetProgressMax | Returns the maximum progress bar tick count for the active feature set installation. This is used when the sample renders the installation progress bar. |
| Initialize | Initializes the class with the product globally unique identifier (GUID). This method also enumerates every feature of the application which has been advertised but not yet installed, and places it in the passive installation queue to set up the passive installation. |
| IsInstallInProgress | Use this method to find out whether an active installation is being processed. |
| UseFeature | Private method called by UseFeatureSet. Checks if a feature is installed. If the requested feature is installed the method returns. If the feature is not yet installed (advertised), the method queues a new active installation request for the worker thread and then returns. The optional event handle that will be signaled when the requested installation is complete. |
| UseFeatureSet | Called by the sample when it needs to access functionalities provided by a particular feature or any of its sub-features. The method enumerates all features of the sample and calls UseFeature() for sub-features of the specified root feature. The sample can pass in an event handle that will be signaled when the entire feature set is installed. Because the features all installed as a set, The handle is specified for the last feature queued by UseFeature() instead of every feature, so that the sample gets notified once after all requested features are installed. |
| UseProduct | Queues an active installation request for the worker thread to call the installer to perform a full product installation. The optional event handle that will be signaled when the requested installation is complete. |
Limitation
The current version of installer is not designed for simultaneous access by multiple threads. Therefore, when the worker thread is calling into the installer, the main thread should not call the installer. An example of this limitation occurs in the sample when the main thread requests a feature, then requests the same feature again before the worker thread completes the installation. The second request calls MsiQueryFeatureState() to find out whether the requested feature is already installed, since the installer may sometimes indicate that the feature is completely installed when the worker thread is still copying the files.
Fortunately, there is an easy work-around for this. CMsiUtil will check whether a feature is being installed by the worker thread before it calls functions such as MsiQueryFeatureState() or MsiUseFeature() to ask for the install state of the feature in question. Be aware that this limitation can also become an issue elsewhere.
Patching can affect how well install-on-demand works on an end-user's computer. Applying a patch that only contains data that has changed from the previous version may require that the previous version of the updated file be installed in order to apply the delta. In this situation, the patch must request that the installer install affected advertised features before applying the patch to the game.
The sample is installed by launching InstallOnDemand.msi because it assumes that the Windows Installer is present on the computer. If the installer is absent, .msi files are not recognized and launching them will not work. To overcome this problem, an application should use an installation program to perform the task. This program should first check to see if the installer is present, and if so, its version. If the version does not meet the application's requirement, the installation program should install the Windows Installer, then launch the .msi file. This process is known as bootstrapping. Applications typically name their bootstrapping installation program Setup.exe. The sample does not handle bootstrapping. However, complete details on bootstrapping can be found in the Windows Installer.
Developers should also pay attention to the size of each feature in their games. When canceling an installation in progress, the installer restores the computer to the state before the installation. This means that the installation of the feature is completely undone, and there is no partial feature installation. A large feature requires longer time to install, which either increases the likelihood that the install will be interrupted and cancelled or the install will interfere with the main application. An example would be enabling passive installation when the user brings up the game menu in the middle of game play. If a feature is installing and the user goes back to playing, the game can do one of two things: it can let the passive installation complete, or it can cancel the passive installation. A large feature would fit poorly with either model. If the game lets the large installation complete, the installation may hinder the game's rendering performance for a long time. Conversely, if the game cancels the installation, then the user must stay in the menu for a long time before returning to the game. Developers should find a balanced feature size that works best for their individual games.