Initial sync
The initial synchronization before you start dual-write on a table will handle existing data on both sides of the integration between finance and operations apps and Microsoft Dataverse. It will copy existing data from one app to another bidirectionally as API calls that are using CSV Unicode as the file format. An initial sync between finance and operations apps and Dataverse environments has a corresponding export and import project.
To begin troubleshooting this process, that export and import project is the place to start. Ensure that a data import or export project can be completed, and that the implementation is successful. If a data import or export project implementation fails, you should look for known issues or event logs that are associated with the service.
Restarting the service from Microsoft Dynamics Lifecycle Services for a finance and operations apps environment is a common step to resolve issues with the import and export framework, in addition to confirming the implementation details of the jobs.
If the data import and export jobs and packages can import successfully, and you’ve fixed issues that were present, you can re-run the implementation of the data packages by finding them in the Data management workspace > Dual-write screen, selecting the tab for the initial sync details from the list, and then right-clicking to view the re-run options.
The two re-run options are:
- Re-run execution, which will re-run the entire initial sync for the package that you’ve selected for all implementations.
- Re-run execution without errors, which will run the initial sync for the records that weren’t in the errored state or missing data.
Additionally, you can change the ability to do a full or incremental push for an initial sync based on the change tracking and whether it’s disabled or enabled. If you want to have incremental pushes of data that have changed since the last time when an initial sync was done, you’ll need to enable change tracking. Otherwise, if you don’t want to complete that action, you can disable the change tracking and the system will do a full initial sync the next time that it’s requested.
Self-referencing field issues
The best way to explain self-referencing fields is to give an example.
| Field A | Field B |
|---|---|
| Value 1 | Value 3 |
| Value 2 | Null |
| Value 3 | Value 4 |
| Value 4 | Value 1 |
In this example, the first value of Field B references the third value of Field A. If you perform an initial sync with this data, the first pass will fail because the third value in Field A hasn’t been synced; therefore, Field B can’t populate the third value either. A few standard maps complete this process, where you have a table that has self-referencing records.
To fix this issue, follow these steps:
- Remove Field B from the mapping.
- Save the new map.
- Disable change tracking.
- Sync the map with only Field A data.
When Field A is synced, then you can add Field B back into the mapping and then you can complete another initial sync. For the sync that includes Field B, you’ll need to ensure that change tracking is enabled to sync the data based on Field A, which has already been mapped and synced to Dataverse successfully.
The full process would resemble the following steps:
- Remove Field B from the table mapping.
- Save the new table map as the newest version.
- Remove change tracking.
- Start the initial sync.
- After initial sync has completed, stop the mapping.
- Restore the map to the previous version to add Field B back to the mapping.
- Turn on change tracking.
- Sync the table mapping again.
Filter issues
Occasionally, when you’ve added a filter to an entity or table, you’d expect that the information will be filtered when it’s moved to the other table. Unfortunately, issues might occur where the filter won’t appear in one system or the other after the initial sync. Typically, this issue occurs if the filter syntax is wrong during the initial sync, and it will be ignored. To ensure that the filters work, you’ll need to verify that the syntax for the filter is correct. You should ensure that filters are only applied on the fields that are present on the dual-write maps.
An extra runnable class is available for checking the filter expression in finance and operations apps, as follows:
var entityName = "PROJECTENTITY";
var filterExpression = '(ParentProject == "")';
Query query = new Query();
query.literals(NoYes::Yes);
QueryBuildDataSource qbd = query.addDataSource(tablename2id(entityName));
qbd.addRange(fieldname2id(qbd.table(),identifierStr(RecVersion))).
value(filterExpression);
qbd.addSelectionField(fieldname2id(qbd.table(),identifierStr(RecId)));
QueryRun qRun = new QueryRun(query);
// This provides the actual sql statement to execute
var actualSqlStatement = query.getSQLStatement();
while(qRun.next())
{
var rec = qRun.get(tableName2Id(entityName));
}
If the runnable class returns with expected results, then the filter is working with finance and operations apps and will be able to transfer over through the dual-write mapping and initial sync process. On the Dataverse side, if a filter isn’t working or is ignored in finance and operations apps after the initial sync, you can make the following OData call to ensure that the filter is working correctly.
https://<EnvironmentURL>/api/data/v9.0/<Table>?$filter=<fieldname> eq <value>
If that call returns expected results, it will work during the initial sync between the systems. Out-of-the-box standard maps do have filters, so we recommend that you compare custom filters with these standard maps to ensure that the syntax is similar.
Database refresh
Database refresh is the most common scenario where errors occur from Dataverse and finance and operations apps. It’s often caused by orphaned or trash records from one system or the other. After a database refresh has been performed, we recommended that you stop all maps and unlink the environment through the Dual-write workspace.
After you’ve completed that process, delete the data from the DualWriteProjectConfiguration, DualWriteProjectFieldConfiguration, and BusinessEventsDefinition key tables in finance and operations apps. You can use the DualWriteProjectConfigurationEntity table to clear these records from Common > Common > Office integration > Excel workbook designer.
On the Dataverse side, you can go to Advanced Settings > Advanced Find. Select the Dual-write Runtime Configuration option, select the column that you want to view, and then select Results to view the configurations. Select the records and then delete them.
When you’re finished, both environments will show that the data has been removed from those areas. Then, it’s time for you to re-sync and re-link the environments and enable the maps again. This approach will remove orphaned or lost records in the environments to ensure that the sync is working correctly.