Share via

Data processing and user-defined functions

  • Article

Important

A new version of the Azure Digital Twins service has been released. In light of the new service's expanded capabilities, the original Azure Digital Twins service (described in this documentation set) has been retired.

To view the documentation for the new service, visit the active Azure Digital Twins Documentation.

Azure Digital Twins offers advanced compute capabilities. Developers can define and run custom functions against incoming telemetry messages to send events to predefined endpoints.

Data processing flow

After devices send telemetry data to Azure Digital Twins, developers can process data in four phases: validate, match, compute, and dispatch.

Azure Digital Twins data processing flow

  1. The validate phase transforms the incoming telemetry message to a commonly understood data transfer object format. This phase also executes device and sensor validation.
  2. The match phase finds the appropriate user-defined functions to run. Predefined matchers find the user-defined functions based on the device, sensor, and space information from the incoming telemetry message.
  3. The compute phase runs the user-defined functions matched in the previous phase. These functions might read and update computed values on spatial graph nodes and can emit custom notifications.
  4. The dispatch phase routes any custom notifications from the compute phase to endpoints defined in the graph.

Data processing objects

Data processing in Azure Digital Twins consists of defining three objects: matchers, user-defined functions, and role assignments.

Azure Digital Twins data processing objects

Matchers

Matchers define a set of conditions that evaluate what actions take place based on incoming sensor telemetry. Conditions to determine the match might include properties from the sensor, the sensor's parent device, and the sensor's parent space. The conditions are expressed as comparisons against a JSON path as outlined in this example:

  • All sensors of datatype Temperature represented by the escaped String value \"Temperature\"
  • Having 01 in their port
  • Which belong to devices with the extended property key Manufacturer set to the escaped String value \"Contoso\"
  • Which belong to spaces of the type specified by the escaped String \"Venue\"
  • Which are descendants of parent SpaceId DE8F06CA-1138-4AD7-89F4-F782CC6F69FD
{
  "id": "23535afafd-f39b-46c0-9b0c-0dd3892a1c30",
  "name": "My custom matcher",
  "spaceId": "DE8F06CA-1138-4AD7-89F4-F782CC6F69FD",
  "description": "All sensors of datatype Temperature with 01 in their port that belong to devices with the extended property key Manufacturer set to the value Contoso and that belong to spaces of type Venue that are somewhere below space Id DE8F06CA-1138-4AD7-89F4-F782CC6F69FD",
  "conditions": [
    {
      "id": "43898sg43-e15a-4e9c-abb8-2gw464364",
      "target": "Sensor",
      "path": "$.dataType",
      "value": "\"Temperature\"",
      "comparison": "Equals"
    },
    {
      "id": "wt3th44-e15a-35sg-seg3-235wf3ga463",
      "target": "Sensor",
      "path": "$.port",
      "value": "01",
      "comparison": "Contains"
    },
    {
      "id": "735hs33-e15a-37jj-23532-db901d550af5",
      "target": "SensorDevice",
      "path": "$.properties[?(@.name == 'Manufacturer')].value",
      "value": "\"Contoso\"",
      "comparison": "Equals"
    },
    {
      "id": "222325-e15a-49fg-5744-463643644",
      "target": "SensorSpace",
      "path": "$.type",
      "value": "\"Venue\"",
      "comparison": "Equals"
    }
  ]
}

Important

  • JSON paths are case sensitive.
  • The JSON payload is the same as the payload that's returned by:
    • /sensors/{id}?includes=properties,types for the sensor.
    • /devices/{id}?includes=properties,types,sensors,sensorsproperties,sensorstypes for the sensor's parent device.
    • /spaces/{id}?includes=properties,types,location,timezone for the sensor's parent space.
  • The comparisons are case insensitive.

User-defined functions

A user-defined function is a custom function executed within an isolated Azure Digital Twins environment. User-defined functions have access to raw sensor telemetry message as it gets received. User-defined functions also have access to the spatial graph and dispatcher service. After the user-defined function is registered within a graph, a matcher (detailed above) must be created to specify when the function is executed. For example, when Azure Digital Twins receives new telemetry from a given sensor, the matched user-defined function can calculate a moving average of the last few sensor readings.

User-defined functions can be written in JavaScript. Helper methods interact with the graph in the user-defined execution environment. Developers can execute custom snippets of code against sensor telemetry messages. Examples include:

  • Set the sensor reading directly onto the sensor object within the graph.
  • Perform an action based on different sensor readings within a space in the graph.
  • Create a notification when certain conditions are met for an incoming sensor reading.
  • Attach graph metadata to the sensor reading before sending out a notification.

For more information, read How to use user-defined functions.

Examples

The GitHub repo for the Digital Twins C# sample contains a few examples of the user-defined functions:

  • This function looks for carbon dioxide, motion, and temperature values to determine whether a room is available with these values in range. The tutorials for Digital Twins explore this function in more details.
  • This function looks for data from multiple motion sensors, and determines that the space is available if none of them detect any motion. You can easily replace the user-defined function used in either the quickstart, or the tutorials, by making the changes mentioned in the comments section of the file.

Role assignment

A user-defined function's actions are subject to Azure Digital Twins role-based access control to secure data within the service. Role assignments define which user-defined functions have the proper permissions to interact with the spatial graph and its entities. For example, a user-defined function might have the ability and permission to CREATE, READ, UPDATE, or DELETE graph data under a given space. A user-defined function's level of access is checked when the user-defined function asks the graph for data or attempts an action. For more information, read Role-based access control.

It's possible for a matcher to trigger a user-defined function that has no role assignments. In this case, the user-defined function fails to read any data from the graph.

Next steps