Camera Performance Certification Test (Compact 7)
3/12/2014
This test suite collects the performance data of a camera on the Windows Embedded Compact device. The following information will be collected when the test suite is run on the Windows Embedded Compact device with a camera.
1. Camera Graph Build
2. Camera Preview Startup
3. Camera Preview Framerate
4. Camera Still image e2e latency
5. Camera Still Latency
6. Camera Vid Cap Encode Framerate
7. Camera Vid Cap Framerate
Test Prerequisites
Your device must meet the following requirements before you run this test.
*A DirectShow-based camera driver is required
Subtests
The table below lists the subtests included in this test.
| SubTest ID | Description |
|---|---|
100 |
Outputs graph layout and allocator information for reference. |
200 |
Enforces the minimum preview startup latency. |
201 |
Enforces the minimum preview frame rate. |
300 |
Enforces the minimum still capture latency. |
400 |
Outputs information about the video capture encode frame rate. |
Setting Up the Test
A DirectShow-based camera driver is required
Running the Test
If multiple camera drivers are in the system, the test's command line can be modified to select which camera driver to test with. If a third-party encoder is used, the command line can be modified to test with the third-party encoder.
Test command line options:
/? outputs this message
/CommandLineFile <test command line stored in a file> - reads in test command line options from a file. The file is just a standard text file containing the command line on a single line within the file.
Camera driver selection
/UseMonoCam - sets the framework to use the monolithic camera driver (nullcam.dll), tests fail if unavailable. WARNING: This can overwrite PIN registry key settings used by other monolithic camera drivers causing them to fail.
/UseTestCam - sets the framework to use the test camera driver, tests fail if unavailable.
/CamDriver <legacy name (CAM1:)> - sets the framework to use the camera driver with the legacy device name specified. If no driver is specified and the test camera driver is available it will be used. If no driver is specified and the test camera driver is unavailable the first enumerated driver will be used.
/TestCamXML <xml path and name> - sets the registry keys for the test camera driver to use the xml file specified.
/TestCamProfile <profile id from the file> - sets the registry keys for the test camera driver to use the profile specified.
Pluggable component modification
/VideoEncoderCLSID {xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx} - sets the framework to use the encoder dmo with the CLSID specified.
/AudioEncoderCLSID {xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx} - sets the framework to use the encoder dmo with the CLSID specified.
/MuxCLSID {xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx} - sets the framework to use the MUX with the CLSID specified.
/UseGSM - sets the framework to use the GSM encoder through the audio capture filter.
/ForceColorConversion - Forces the color conversion filter into the graph in front of the renderer, still sink, and encoder.
/VideoCaptureExtension <extension> - changes the default extension (asf) to the user requested extension for video files captured.
/StillCaptureExtension <extension> - changes the default extension (jpg) to the user requested extension for still images captured.
/FileStorageLocation <path to storage location for files created by the test> - overrides the default location to store captured files (the device root).
Captured video validation
/VideoFileConformanceLibrary <path> - this takes a path to a library (DLL) which exports a function called VerifyVideoFile which takes a single WCHAR * parameter which indicates a file name to verify. This function should return an HRESULT indicating whether or not the video file is conformant and playable. If no path is given and the file sink used is the default ASF file sink, asfverify.dll is used.
Failure diagnostics options
/OutputKeyEvents - enables the test to output debugging information for missing events, skip outputting high volume events.
/OutputAllEvents - enables the test to output debugging information for missing events, outputs every event.
/OutputFilterInformation - enables the test to output information about the graph topology.
/OutputPreviewDiagnostics - output preview diagnostics information.
/OutputStillDiagnostics - output still capture diagnostics information.
/OutputCaptureDiagnostics - output video capture diagnostics information.
/OutputAllDiagnostics - output video capture diagnostics information.
/SaveFailures <path> - saves video and still failures to the specified path.
Verifying the Test
The graph layout test is for informational purposes only. The presence of the color converter indicates a non-optimal color format negotiation. Allocator changes across the input and output pins of a filter indicate that the filter is using a different buffer for it's output than input, generally indicating some sort of memcopy of the image buffer. Allocators at filters other than source filters and encoders indicate a non-optimal allocator negotiation.
Preview startup latency test failures indicate that from the time the camera sensor is told to run and it delivers the first image data is larger than the maximum permitted latency. Check that the camera driver is doing as much work as possible when put into the pause state to prepare for running.
Preview framerate failures indicate that the preview frame rate is below the permitted level. Increase the preview frame rate delivered by the driver.
Still capture latency failures indicate that the time from when the driver is told to capture an image and it sends a still capture conformation notification was too long. This time does not include still image encoding, just the time from when the request is made and the image data is locked into memory and the user can move the camera away from the target.
The video capture encode framerate detection test is for informational purposes only. The test will output the encode frame rate for the capture set at the closest resolution to the device's full screen resolution. To increase the encode frame rate look into reducing CPU utilization by the preview window by optimizing color formats to remove color conversions, reducing the preview frame rate, remove other processes/threads using CPU, replacing the encoder with a hardware assisted encoder, or adding encoder optimized color formats to the camera driver output.
Troubleshooting the Test
Test failures are limited to blocking failures due to functional problems, and insufficient performance. Blocking failures can be investigated further by running the other camera certification tests; performance failures need to be fixed by improving driver performance for the scenario being tested.
For additional platform specific issues, consult the CTK articles on the TechNet wiki.