High dynamic range (HDR) and low-light photo capture

This article shows you how to use the AdvancedPhotoCapture class to capture high dynamic range (HDR) photos. This API also allows you to obtain a reference frame from the HDR capture before the processing of the final image is complete.

Other articles related to HDR capture include:

• You can use the SceneAnalysisEffect to allow the system to evaluate the content of the media capture preview stream to determine if HDR processing would improve the capture result. For more information, see Scene analysis for MediaCapture.

• Use the HdrVideoControl to capture video using the Windows built-in HDR processing algorithm. For more information, see Capture device controls for video capture.

• You can use the VariablePhotoSequenceCapture to capture a sequence of photos, each with different capture settings, and implement your own HDR or other processing algorithm. For more information, see Variable photo sequence.

There is a full sample demonstrating the use of the AdvancedPhotoCapture class that you can use to see the API used in context or as a starting point for your own app. For more information see, Camera Advanced Capture sample.

HDR photo capture

Determine if HDR photo capture is supported on the current device

The HDR capture technique described in this article is performed using the AdvancedPhotoCapture object. Not all devices support HDR capture with AdvancedPhotoCapture. Determine if the device on which your app is currently running supports the technique by getting the MediaCapture object's VideoDeviceController and then getting the AdvancedPhotoControl property. Check the video device controller's SupportedModes collection to see if it includes AdvancedPhotoMode.Hdr. If it does, HDR capture using AdvancedPhotoCapture is supported.

m_hdrSupported = m_mediaCapture.VideoDeviceController.AdvancedPhotoControl.SupportedModes.Contains(Windows.Media.Devices.AdvancedPhotoMode.Hdr);

Configure and prepare the AdvancedPhotoCapture object

Because you will need to access the AdvancedPhotoCapture instance from multiple places within your code, you should declare a member variable to hold the object.

private AdvancedPhotoCapture m_advancedCapture;

In your app, after you have initialized the MediaCapture object, create an AdvancedPhotoCaptureSettings object and set the mode to AdvancedPhotoMode.Hdr. Call the AdvancedPhotoControl object's Configure method, passing in the AdvancedPhotoCaptureSettings object you created.

Call the MediaCapture object's PrepareAdvancedPhotoCaptureAsync, passing in an ImageEncodingProperties object specifying the type of encoding the capture should use. The ImageEncodingProperties class provides static methods for creating the image encodings that are supported by MediaCapture.

PrepareAdvancedPhotoCaptureAsync returns the AdvancedPhotoCapture object you will use to initiate photo capture. You can use this object to register handlers for the OptionalReferencePhotoCaptured and AllPhotosCaptured which are discussed later in this article.

if (m_hdrSupported == false) return;

// Choose HDR mode
var settings = new AdvancedPhotoCaptureSettings { Mode = AdvancedPhotoMode.Hdr };

// Configure the mode
m_mediaCapture.VideoDeviceController.AdvancedPhotoControl.Configure(settings);

// Prepare for an advanced capture
m_advancedCapture =
    await m_mediaCapture.PrepareAdvancedPhotoCaptureAsync(ImageEncodingProperties.CreateUncompressed(MediaPixelFormat.Nv12));

// Register for events published by the AdvancedCapture
m_advancedCapture.AllPhotosCaptured += M_advancedCapture_AllPhotosCaptured;
m_advancedCapture.OptionalReferencePhotoCaptured += M_advancedCapture_OptionalReferencePhotoCaptured;

Capture an HDR photo

Capture an HDR photo by calling the AdvancedPhotoCapture object's CaptureAsync method. This method returns an AdvancedCapturedPhoto object that provides the captured photo in its Frame property. Next, the photo is saved to disk.

try
{

    // Start capture, and pass the context object
    AdvancedCapturedPhoto advancedCapturedPhoto = await m_advancedCapture.CaptureAsync();

    using (var frame = advancedCapturedPhoto.Frame)
    {
        var fileName = String.Format("SimplePhoto_{0}_HDR.jpg", DateTime.Now.ToString("HHmmss"));
        StorageFile photoFile = await KnownFolders.PicturesLibrary.CreateFileAsync(fileName);
        IRandomAccessStream stream = await photoFile.OpenAsync(FileAccessMode.ReadWrite);
        await RandomAccessStream.CopyAndCloseAsync(advancedCapturedPhoto.Frame, stream);
    }
}
catch (Exception ex)
{
    Debug.WriteLine("Exception when taking an HDR photo: {0}", ex.ToString());
}

Get optional reference frame

The HDR process captures multiple frames and then composites them into a single image after all of the frames have been captured. You can get access to a frame after it is captured but before the entire HDR process is complete by handling the OptionalReferencePhotoCaptured event. You don't need to do this if you are only interested in the final HDR photo result.

Because the reference frame arrives out of context of the call to CaptureAsync, a mechanism is provided to pass context information to the OptionalReferencePhotoCaptured handler. First you should call an object that will contain your context information. The name and contents of this object is up to you. This example defines an object that has members to track the file name and camera orientation of the capture.

public class MyAdvancedCaptureContextObject
{
    public string CaptureFileName;
    public PhotoOrientation CaptureOrientation;
}

Create a new instance of your context object, populate its members, and then pass it into the overload of CaptureAsync that accepts an object as a parameter.

// Add the the capture time to the file name
var fileName = String.Format("SimplePhoto_{0}_HDR.jpg", DateTime.Now.ToString("HHmmss"));

// Create a context object, to identify the capture in the OptionalReferencePhotoCaptured event
var context = new MyAdvancedCaptureContextObject()
{
    CaptureFileName = fileName,
};

// Start capture, and pass the context object
AdvancedCapturedPhoto advancedCapturedPhoto = await m_advancedCapture.CaptureAsync(context);

In the OptionalReferencePhotoCaptured event handler, cast the Context property of the OptionalReferencePhotoCapturedEventArgs object to your context object class. This example modifies the file name to distinguish the reference frame image from the final HDR image and then saves the image to disk.

private async void M_advancedCapture_OptionalReferencePhotoCaptured(AdvancedPhotoCapture sender, OptionalReferencePhotoCapturedEventArgs args)
{
    // Retrieve the context (i.e. what capture does this belong to?)
    var context = args.Context as MyAdvancedCaptureContextObject;

    // Remove "_HDR" from the name of the capture to create the name of the reference
    var referenceName = context.CaptureFileName.Replace("_HDR", "");

    using (var frame = args.Frame)
    {
        await SaveCapturedFrameAsync(frame, referenceName, context.CaptureOrientation);
    }
}

Receive a notification when all frames have been captured

The HDR photo capture has two steps. First, multiple frames are captured, and then the frames are processed into the final HDR image. You can't initiate another capture while the source HDR frames are still being captured, but you can initiate a capture after all of the frames have been captured but before the HDR post-processing is complete. The AllPhotosCaptured event is raised when the HDR captures are complete, letting you know that you can initiate another capture. A typical scenario is to disable your UI's capture button when HDR capture begins and then reenable it when AllPhotosCaptured is raised.

private void M_advancedCapture_AllPhotosCaptured(AdvancedPhotoCapture sender, object args)
{
    // Update UI to enable capture button
}

Clean up the AdvancedPhotoCapture object

When your app is done capturing, before disposing of the MediaCapture object, you should shut down the AdvancedPhotoCapture object by calling FinishAsync and setting your member variable to null.

await m_advancedCapture.FinishAsync();
m_advancedCapture = null;

Low-light photo capture

Starting with Windows 10, version 1607, AdvancedPhotoCapture can be used to capture photos using a built-in algorithm that enhances the quality of photos captured in low-light settings. When you use the low-light feature of the AdvancedPhotoCapture class, the system will evaluate the current scene and, if needed, apply an algorithm to compensate for low-light conditions. If the system determines that the algorithm is not needed, a regular capture is performed instead.

Before using low-light photo capture, determine if the device on which your app is currently running supports the technique by getting the MediaCapture object's VideoDeviceController and then getting the AdvancedPhotoControl property. Check the video device controller's SupportedModes collection to see if it includes AdvancedPhotoMode.LowLight. If it does, low-light capture using AdvancedPhotoCapture is supported.

m_lowLightSupported = m_mediaCapture.VideoDeviceController.AdvancedPhotoControl.SupportedModes.Contains(Windows.Media.Devices.AdvancedPhotoMode.LowLight);

In your app, after you have initialized the MediaCapture object, create an AdvancedPhotoCaptureSettings object and set the mode to AdvancedPhotoMode.LowLight. Call the AdvancedPhotoControl object's Configure method, passing in the AdvancedPhotoCaptureSettings object you created.

Call the MediaCapture object's PrepareAdvancedPhotoCaptureAsync, passing in an ImageEncodingProperties object specifying the type of encoding the capture should use.

if (m_lowLightSupported == false) return;

// Choose LowLight mode
var settings = new AdvancedPhotoCaptureSettings { Mode = AdvancedPhotoMode.LowLight };
m_mediaCapture.VideoDeviceController.AdvancedPhotoControl.Configure(settings);

// Prepare for an advanced capture
m_advancedCapture =
    await m_mediaCapture.PrepareAdvancedPhotoCaptureAsync(ImageEncodingProperties.CreateJpeg());

To capture a photo, call CaptureAsync.

AdvancedCapturedPhoto advancedCapturedPhoto = await m_advancedCapture.CaptureAsync();
StorageFile photoFile = await KnownFolders.PicturesLibrary.CreateFileAsync($"Photo_{advancedCapturedPhoto.Mode}.jpg", CreationCollisionOption.GenerateUniqueName);
IRandomAccessStream stream = await photoFile.OpenAsync(FileAccessMode.ReadWrite);
await RandomAccessStream.CopyAndCloseAsync(advancedCapturedPhoto.Frame, stream);

You can capture multiple low-light photos without reconfiguring the AdvancedPhotoCapture object, but when you are done capturing, you should call FinishAsync to clean up the object and associated resources.

await m_advancedCapture.FinishAsync();
m_advancedCapture = null;

Working with AdvancedCapturedPhoto objects

AdvancedPhotoCapture.CaptureAsync returns an AdvancedCapturedPhoto object representing the captured photo. This object exposes the Frame property which returns a CapturedFrame object representing the image. The OptionalReferencePhotoCaptured event also provides a CapturedFrame object in its event args. After you get an object of this type, there are a number of things you can do with it, including creating a SoftwareBitmap or saving the image to a file.

Get a SoftwareBitmap from a CapturedFrame

Get a SoftwareBitmap from a CapturedFrame object by simply accessing the SoftwareBitmap property of the object. However, most encoding formats do not support SoftwareBitmap with AdvancedPhotoCapture, so you should check and make sure the property is not null before using it.

SoftwareBitmap bitmap;
if (advancedCapturedPhoto.Frame.SoftwareBitmap != null)
{
    bitmap = advancedCapturedPhoto.Frame.SoftwareBitmap;
}

In the current release, the only encoding format that supports SoftwareBitmap for AdvancedPhotoCapture is uncompressed NV12. So, if you want to use this feature, you must specify that encoding when you call PrepareAdvancedPhotoCaptureAsync.

m_advancedCapture =
    await m_mediaCapture.PrepareAdvancedPhotoCaptureAsync(ImageEncodingProperties.CreateUncompressed(MediaPixelFormat.Nv12));

Of course, you can always save the image to a file and then load the file into a SoftwareBitmap in a separate step. For more information about working with SoftwareBitmap, see Create, edit, and save bitmap images.