Video Capture
A version of this page is also available for
4/8/2010
The term video capture, as used in this documentation, describes any application where video is received from a hardware device. Video capture devices are typically cameras but can also be other video sources. The captured video can be saved to disk or previewed live.
This section describes some of the concepts you must understand to use DirectShow for video capture:
Building Graphs with the Capture Graph Builder
Interfaces for Capture Applications
Architecture
DirectShow uses an object called the Capture Graph Builder to help create filter graphs for video capture. For more information, see Capture Graph Builder. The structure of the filter graph it creates depends on the number of output pins provided by the camera driver.
A two-pin camera driver has one pin for video capture and another pin for still image capture. For two-pin drivers, the Capture Graph Builder inserts the Smart Tee filter into the capture graph. This filter splits the incoming video stream into two separate output streams, one for the actual data capture and the other for a live preview of the data, typically for a camera's LCD preview display.
The following diagram shows the architecture of a typical capture graph for a two-pin camera driver.
.gif "Aa924811.00ec17a6-633e-48e6-905e-50d4e240e3c5(en-us,MSDN.10).gif")
A three-pin camera driver has an output pin for video capture, an output pin for still images, and a preview pin to directly view what the camera is aimed at. Because the preview pin provides direct driver-level support for a device's preview pipeline, the Capture Graph Builder does not include the Smart Tee filter in the capture graph.
The following diagram shows the architecture of a typical capture graph for a three-pin camera driver.
.gif "Aa924811.0d3d5b0b-4cc9-4b73-9b85-285400836b86(en-us,MSDN.10).gif")
The major parts of both the two-pin and three-pin architectures are described below:
- Camera Driver
The camera driver provides the low level interface between the OS and the actual camera hardware.**
- Smart Tee
The Smart Tee Filter takes a single stream of incoming image data and splits it into two output streams. One stream is used for the actual video capture and the other is used to drive the camera's viewfinder or preview window.
- Buffer filter
The buffer filter is only added to the filter graph if the encoder DMO it is connecting to exposes the major media a filter that This filter is inserted into the filter graph to allow camera devices to support captures that do not take place in real time. This allows camera applications and devices to be developed on lower cost hardware.
Color Space Converter Filter (CC)
This filter transforms captured video data from one RGB or YUV color space to another. The Capture Graph Builder automatically inserts this filter into the capture and still image pipelines if the video capture filter's output pin does not have the same media type as the input pin on the encoder or image sink. The practical implication of this is that you can improve the performance of your capture graph by making sure that these output and input pins have matching media types, thereby eliminating the need for a color conversion step.The video renderer for the preview display must be able to fall back to GDI-based rendering. Because GDI uses RGB color formats, the Capture Graph Builder always inserts the color space converter filter into the preview pipeline if the video capture filter's output pin uses a YUV color format. The color space converter filter is omitted from the graph if the preview output from the video capture filter is in an RGB format and bit depth that GDI supports.
Using the Color Space Converter Filter will impact the performance of the graph. To omit the filter, make sure that your video capture filter outputs media types supported by the downstream filters. You cannot disable color conversion, but you can tune the system so it’s not necessary.
For a table showing the specific color conversions supported by this filter, see Color Space Converter Filter.
- File Writer
The file writer is a multiplexer that combines audio and video data and writes it out to an Advanced Systems Format (ASF) file.
- DMO Wrapper Filter
This filter provides an interface between DirectShow and a DirectX Media Objects (DMO). The Windows Media Video encoder is provided as a DMO.
Video Renderer
The video renderer supplies image data to a camera's viewfinder or preview window.The Video Renderer relies on GDI when it initiates its connections with the filter graph. To support this, the filter upstream from the video renderer must support the RGB color space. Depending on the video capture filter's output color format, the Capture Graph Builder may need to insert the color space converter filters before the video renderer.
At allocation time, the Video Renderer enumerates all media types supported by the upstream filter. During this enumeration, the Video Renderer attempts to create a DirectDraw surface for each media it encounters. If it can successfully create the surface then it will stop enumerating and use that DirectDraw surface as its rendering target.
There are two reasons that the video renderer has to support a GDI mode. First, the display hardware may not have native support for certain stretching and scaling operations. In these cases, the video render cannot use DirectDraw calls for those graphics operations and must instead rely on software-based routines through GDI. Second, in cases where there are complex clipping regions, the video renderer will fall back from DirectDraw-based rendering to GDI-based rendering.
When initiating the connection, the Video Renderer needs to use GDI, and as such, the upstream filter needs to support RGB. As RGB is not always available from the video capture driver, the Color Converter needs to be inserted. Before going into Run mode, at allocation time, the Video Renderer will enumerate all media types supported by the upstream filter, and attempt to create DDraw surfaces with those types. If there is a match, then the DDraw surface will be used.
- Image Sink
The image sink is the interface to the Imaging API, which is used to write still images.
You can see from the descriptions above that the Capture Graph Builder can configure the capture graph with a number of different filters to accommodate the hardware and encoding parameters of your device. You can improve the performance of your capture graph by carefully planning and coordinating the hardware capabilities and requirements on your device. The best capture graph performance comes occurs when the video renderer can render its output using DirectDraw, and when the color space converter filter is either not required in the graph or is simply acting as a pass-through filter.