Share via

Graphics and Rendering Process in Silverlight for Windows Embedded (Compact 7)

  • Article

3/12/2014

This topic provides an overview of the Microsoft Silverlight for Windows Embedded rendering process. You can use this information to improve graphics performance in Silverlight for Windows Embedded applications.

Phases of the Rendering Process

The Silverlight for Windows Embedded rendering process consists of the following two phases:

  • Rasterization   Rasterization is the process of converting vector-based graphics (such as lines, text and rectangles) into the pixel-based representations that can be directly displayed on a screen. The renderer allocates a memory buffer that serves as a bitmap image and begins to rasterize the vector-based graphics into that bitmap image, pixel-by-pixel, taking opacity masks and gradients into account. Rasterization is handled purely within the CPU.
  • Composition   After each UI element is rasterized into a bitmap image, the renderer composes the final image by layering the individual memory buffers on top of one another, according to the z-order of the original elements. As it composes the individual bitmap images together, the renderer takes element opacity and any transformations (such as scale or translation) into consideration, ultimately producing the image that is displayed on the device screen. Composition can be handled by either the CPU or the graphics processing unit (GPU).

The following image shows the rasterization and composition phases as they are applied to two elements in a UI, one of which has an opacity mask.

Phases of the rendering process

The rendering process is complicated because the image on the screen is assembled from a set of UI elements, many of which are composed of other UI elements. Internally, Silverlight for Windows Embedded uses a visual tree, which is a hierarchical structure of all the elements that are visible and must be rendered. To render a typical screen, Silverlight for Windows Embedded steps through the elements in the visual tree, and interleaves rasterization and composition to reduce the number of memory buffers that must be allocated. The process will overlay an element into an existing buffer whenever possible. The designer or developer can use cached composition to specify how the particular UI elements are to be grouped and to be composed by the GPU. Appropriate caching can dramatically increase the UI frame rate.

When UI elements are animated, it wastes processor time to re-rasterize elements that are not moving. In addition, only the portions of the display that are affected by element movement need to be re-composed. To optimize the rendering process, Silverlight for Windows Embedded automatically defers composition of certain elements, so that the composition steps can be more rapidly completed by using a GPU. Using a GPU to speed the rendering process is commonly referred to as hardware acceleration. If your device does not have a GPU or is otherwise incapable of hardware acceleration, Silverlight for Windows Embedded uses Graphics Device Interface (GDI) to draw UI objects pixel by pixel onto the primary display surface in z-order, with a corresponding lower rendering speed.

Threads in the Rendering Process

There are two main threads in Silverlight for Windows Embedded: the UI thread and the compositor thread. To create a responsive UI, it is important to understand their roles.

The UI Thread

The UI thread handles the following tasks:

  • Processes user input
  • Parses and creates objects from XAML
  • Draws all visuals on the screen for the first time
  • Executes per-frame callbacks and other user code

To make your application as responsive as possible, you must keep the UI thread as lightweight and free as possible.

The Compositor Thread

The compositor is an ultra-lightweight thread that simply combines textures for the GPU; it drives the types of animation that are defined as storyboards and drawn by manipulating base textures on the GPU. These animation types include:

  • Translation. Changes the location of the object
  • Scaling. Zooms in and out and creates the illusion of depth
  • Rotation. Turns the object about a point or an axis
  • Deformation. Changes the skew or aspect ratio of the object
  • Plane Projection. Represents the object by mapping it to a two-dimensional plane

Note that if any of these animations use an opacity mask, a non-rectangular clip, or a texture size for an animated object greater than 2,048 x 2,048 pixels, they will be rasterized on the UI thread.

Example of the Rendering Process

To illustrate the preceding concepts, see the example UI pictured in the screen shown in the following figure. The globe image on the screen is animated and moves around the screen, remaining completely inside the screen boundaries. Although this screen shows a very simple UI, it includes all of the key building blocks listed above to illustrate the rendering process

Example UI

Code for the Example Screen

In the preceding figure, the screen is created by using XAML to define a user control. The MainPage.xaml file, which defines the Hello World user control shown in the figure, contains the following code block:

<UserControl
   xmlns="https://schemas.microsoft.com/winfx/2006/xaml/presentation"
   xmlns:x="https://schemas.microsoft.com/winfx/2006/xaml"
   x:Class="HelloWorld.MainPage"
   Width="300" Height="200">

   <Grid x:Name="LayoutRoot">
      <Grid x:Name="_Background" Background="{StaticResource BlueWashBrush}">
         <Image x:Name="_LightHexes" Source="Assets/Hexes.png" Stretch="Fill"/>
         <Image x:Name="_DarkHexes" Source="Assets/BlackHexes.png" Stretch="Fill">
            <Image.OpacityMask>
               <LinearGradientBrush EndPoint="0.5,1" StartPoint="0.5,0">
                  <GradientStop Offset="0"/>
                  <GradientStop Color="#C0FFFFFF" Offset="0.6"/>
               </LinearGradientBrush>
            </Image.OpacityMask>
         </Image>
      </Grid>
      <Rectangle x:Name="_Border" Fill="#FF969696" Stroke="White" Margin="0,10,90,10" RadiusX="10" RadiusY="10" 
         HorizontalAlignment="Center" Width="180" Opacity="0.5"/>
      <TextBlock x:Name="_Label" HorizontalAlignment="Center" Margin="0,0,90,15" VerticalAlignment="Bottom" 
         FontFamily="Verdana" FontSize="21.333" Text="Hello World" TextWrapping="Wrap" FontWeight="Bold"/>
      <Button x:Name="_Button" HorizontalAlignment="Right" Margin="0,0,8,8" VerticalAlignment="Bottom" Width="75" 
         Content="Close" FontFamily="Verdana" FontSize="16"/>
      <Image x:Name="_Globe" Height="138" HorizontalAlignment="Center" Margin="0,20,90,0" VerticalAlignment="Top" 
         Width="138" Source="Assets/Globe138x138.png" Stretch="Fill" RenderTransformOrigin="0.5,0.5" 
         CacheMode="BitmapCache">
         <Image.RenderTransform>
            <TransformGroup> 
                 <ScaleTransform/>
                 <SkewTransform/>
                 <RotateTransform/>
                 <TranslateTransform/> 
            </TransformGroup>
         </Image.RenderTransform>
      </Image>
   </Grid>
</UserControl>

Visual Tree Defined by the Code

The Microsoft Silverlight for Windows Embedded visual tree for the Hello World user control specified by this code is shown in the following figure.

Visual tree for the XAML code sample

Rendering Process for the Example Screen

The Silverlight for Windows Embedded rendering process carries out the following steps to render the visual tree illustrated in the preceding figure:

  1. Allocate main buffer for LayoutRoot.
  2. Rasterize the gradient brush for the _Background grid into the main buffer.
  3. Rasterize _LightHexes into a temporary buffer and compose with the main buffer.
  4. Rasterize _DarkHexes into a temporary buffer and compose with the main buffer.
  5. Rasterize the _Border rectangle into a temporary buffer and compose with the main buffer.
  6. Rasterize the _Label text into a temporary buffer and compose with the main buffer.
  7. Allocate a temporary buffer for _Button and then rasterize and compose all of the Button elements (in the template for the button) into this temporary buffer.
  8. Compose the _Button buffer with the main buffer.
  9. Compose _Globe with the main buffer.

Note that as the renderer executes the preceding steps, it adheres to the following rules:

  • It allocates buffers only as necessary. If it is possible to overlay an element into an existing buffer, it will generally do so.

  • It uses the CPU to perform all of the rasterizing and composing operations.

    Note

    If the device has a GPU and hardware acceleration is enabled, it composes on the GPU. For a description of a rendering process that uses a GPU, see OpenGL Rendering Process.

During animation, as the _Globe element moves around the screen, none of the other elements change. It is not necessary for Silverlight for Windows Embedded to perform all of the rasterization and composition of steps 1 through 8 for each frame of the animation. The optimal behavior is to save the results of steps 1 through 8 and then, for each frame of the animation, compose _Globe with those saved results. The CacheMode attribute, shown at the end of line 22 in the preceding code block, explicitly tells the renderer to treat the _Globe element individually, producing the desired results.

In Silverlight for Windows Embedded, caching is automatic, and it is not necessary to specify the CacheMode. Note that ordering the XAML elements to group background items and separate cached items is important for the success of the caching algorithm and, ultimately, the animation speed. For more information, see Reordering XAML Elements and Setting Z-Order for Optimal Display Buffer Usage.

See Also

Tasks

Test Graphics Performance of Silverlight for Windows Embedded
Optimize Hardware, BSP, or Drivers, for Graphics Performance
Optimize UI Elements and XAML Files for Graphics Performance

Concepts

Graphics and Performance in Silverlight for Windows Embedded
Hardware Acceleration in Silverlight for Windows Embedded
Optimize OS Components for Graphics Performance
Optimize Images for Graphics Performance