Share via

Joint Filtering

  • Article

Kinect for Windows 1.5, 1.6, 1.7, 1.8

The skeletal tracking joint information can be adjusted (smoothed) across different frames to minimize jittering and stabilize the joint positions over time. For example, an application could drive a cursor icon on the screen using the position of the hand joint.

The Kinect for Windows SDK provides a mechanism to smooth the joint positions in a frame. Applications can enable smoothing and Kinect will adjust the position of the joints calculated in that frame according to the desired smoothing behavior.

The smoothing filter used is based on the Holt Double Exponential Smoothing method used for statistical analysis of economic data. This provides smoothing with less latency than other smoothing filter algorithms. The filter can be controlled via five smoothing parameters:

Smoothing

  • Smoothing parameter. Increasing the smoothing parameter value leads to more highly-smoothed skeleton position values being returned.
  • It is the nature of smoothing that, as the smoothing value is increased, responsiveness to the raw data decreases.
  • Thus, increased smoothing leads to increased latency in the returned skeleton values.
  • Values must be in the range 0 through 1.0. Passing 0 causes the raw data to be returned.

Correction

  • Correction parameter. Lower values are slower to correct towards the raw data and appear smoother, while higher values will correct toward the raw data more quickly.
  • Values must be in the range 0 through 1.0.

Prediction

  • The number of frames to predict into the future.
  • Values must be greater than or equal to zero.
  • Values greater than 0.5 will likely lead to overshooting when moving quickly. This effect can be damped by using small values of fMaxDeviationRadius.

JitterRadius

  • The radius in meters for jitter reduction.
  • Any jitter beyond this radius is clamped to the radius.

MaxDeviationRadius

  • The maximum radius in meters that filtered positions are allowed to deviate from raw data.
  • Filtered values that would be more than this radius from the raw data are clamped at this distance, in the direction of the filtered value.

For more information about smoothing, see the Skeletal Joint Smoothing White Paper.

Here are examples of the smoothing values in both managed and native code.

C# Examples

  // Some smoothing with little latency (defaults).
  // Only filters out small jitters.
  // Good for gesture recognition in games.
  TransformSmoothParameters smoothingParam = new TransformSmoothParameters();
  {
    smoothingParam.Smoothing = 0.5f;
    smoothingParam.Correction = 0.5f;
    smoothingParam.Prediction = 0.5f;
    smoothingParam.JitterRadius = 0.05f;
    smoothingParam.MaxDeviationRadius = 0.04f;
  };


  // Smoothed with some latency.
  // Filters out medium jitters.
  // Good for a menu system that needs to be smooth but
  // doesn't need the reduced latency as much as gesture recognition does.
  TransformSmoothParameters smoothingParam = new TransformSmoothParameters();
  {
    smoothingParam.Smoothing = 0.5f;
    smoothingParam.Correction = 0.1f;
    smoothingParam.Prediction = 0.5f;
    smoothingParam.JitterRadius = 0.1f;
    smoothingParam.MaxDeviationRadius = 0.1f;
  };
  
  // Very smooth, but with a lot of latency.
  // Filters out large jitters.
  // Good for situations where smooth data is absolutely required
  // and latency is not an issue.
  NUI_TRANSFORM_SMOOTH_PARAMETERS VerySmoothParams =
  {0.7f, 0.3f, 1.0f, 1.0f, 1.0f};
  TransformSmoothParameters smoothingParam = new TransformSmoothParameters();
  {
    smoothingParam.Smoothing = 0.7f;
    smoothingParam.Correction = 0.3f;
    smoothingParam.Prediction = 1.0f;
    smoothingParam.JitterRadius = 1.0f;
    smoothingParam.MaxDeviationRadius = 1.0f;
  };

For additional information, see TransformSmoothParameters Properties.

C++ Examples

  C++ // Some smoothing with little latency (defaults).
  // Only filters out small jitters.
  // Good for gesture recognition in games.
  const NUI_TRANSFORM_SMOOTH_PARAMETERS defaultParams =
  {0.5f, 0.5f, 0.5f, 0.05f, 0.04f};

  // Smoothed with some latency.
  // Filters out medium jitters.
  // Good for a menu system that needs to be smooth but
  // doesn't need the reduced latency as much as gesture recognition does.
  const NUI_TRANSFORM_SMOOTH_PARAMETERS somewhatLatentParams =
  {0.5f, 0.1f, 0.5f, 0.1f, 0.1f};

  // Very smooth, but with a lot of latency.
  // Filters out large jitters.
  // Good for situations where smooth data is absolutely required
  // and latency is not an issue.
  const NUI_TRANSFORM_SMOOTH_PARAMETERS verySmoothParams =
  {0.7f, 0.3f, 1.0f, 1.0f, 1.0f};

For additional information, see NUI_TRANSFORM_SMOOTH_PARAMETERS Structure.

How to Enable Smoothing in C#

In managed code, applications must enable smoothing when calling the SkeletonStream.Enable Method passing in the TransformSmoothParameters Structure to set the smoothing parameters. The joint positions returned by the skeletal tracking system will be smoothed based on the value of the smoothing parameters.

Note that, unlike native code applications, managed code applications do not have access to the raw joint position, but only to the smoothed ones.

    
  void StartKinectSTWithSmoothing()
  {
      kinect = KinectSensor.KinectSensors.FirstOrDefault(s => s.Status == KinectStatus.Connected); // Get first Kinect Sensor
      kinect.SkeletonStream.TrackingMode = SkeletonTrackingMode.Seated; // Use Seated Mode

      TransformSmoothParameters smoothingParam = new TransformSmoothParameters();
      {
          smoothingParam.Smoothing = 0.5f;,
          smoothingParam.Correction = 0.5f;,
          smoothingParam.Prediction = 0.5f;,
          smoothingParam.JitterRadius = 0.05f;,
          smoothingParam.MaxDeviationRadius = 0.04f;
      };

      kinect.SkeletonStream.Enable(smoothingParam); // Enable skeletal tracking

      kinect.SkeletonFrameReady += new EventHandler<SkeletonFrameReadyEventArgs>(kinect_SkeletonFrameReady); // Get Ready for Skeleton Ready Events

      kinect.Start(); // Start Kinect sensor
  }

How to Enable Smoothing in C++

In native code, no special initialization is needed. An application passes each skeleton frame to the INuiSensor::NuiTransformSmooth Method, which replaces the unsmoothed joint positions with their smoothed values.

  void MyApplication::UpdateKinectSTWithSmoothing()
  {
    // Wait for 0ms, just quickly test if it is time to process a skeleton
    if ( WAIT_OBJECT_0 == WaitForSingleObject(m_hNextSkeletonEvent, 0) )
    {
      NUI_TRANSFORM_SMOOTH_PARAMETERS defaultParams =
        {0.5f, 0.5f, 0.5f, 0.05f, 0.04f};
      NUI_SKELETON_FRAME skeletonFrame = {0};

      // Get the skeleton frame that is ready
      if (SUCCEEDED(m_pNuiSensor->NuiSkeletonGetNextFrame(0, &skeletonFrame)))
      {
        // Smooth the skeleton joint positions
        m_pNuiSensor->NuiTransformSmooth(&skeletonFrame, &defaultParams);

        // Process the skeleton frame
        SkeletonFrameReady(&skeletonFrame);
      }
    }
  }