Calculated columns + Azure Synapse Link != true

Calculated columns + Azure Synapse Link != true

I was recently helping my colleague Ebba Linnea Nilsson with a support ticket with data not being propagated correctly from dataverse to a datalake via Azure Synapse Link. It turned out that this was all by design. A design that might not be what normal users would expect.

Calculated columns and now recently the formula columns are both very useful way of being able to calculate data in a field that is based on other fields. Common scenarios are calculations like “Weighted revenue” which is the probability multiplied by the estimated revenue for an opportunity. However, there are scenarios where you need to be aware of how these fields actually work or you might get an unwanted or unexpected behaviour.

The first thing that needs to be understood is that these column types are calculated “on-the-fly” everytime dataverse attemts to access these columns. It might seem like the data is “in the columns” but it really isn’t, it is calculated. This is a big difference from for instance rollup-columns is that those columns are calculated on a regular interval by the system, and the result is stored in the record.

What does this mean for Azure Synapse Link? Well, let’s say we have a simple calculation, that sets the value “A” into all records for this calculated column. We then enable the Azure Synapse Link which will make an initial sync and set the column in the datalake to “A”. Now we change the calculation of the rule to output “B” instead. As no records are actually changed, this will not cause any records in the datalake to be updated, hence they will all still have the value “A”. From a user perspective comparing Dynamics 365 to the datalake without any underlying understanding of how this functions, it will look like an error. Same column has different values comparing what is in dataverse with what is in the datalake.

As soon as a record is actually changed, all columns for that record will then be sent to the datalake, and hence the calculated column will be set to “B” at that time. It is hence possible, to manually or semimanually force a resync, but it would require some bulk like for instance SSIS with Kingswaysoft especially for implementations with large amounts of records.

An important question to ask, is why would you want to calculate the data in dataverse and then use it in in the datalake. If you have a propper datalake architecture it should be easier to make calculated columns/fields in the datalake/datalakehouse. If the data is calculated only for use in the datalake, I would suggest moving the calculation to the datalake.

There are, of course, scenarios when it is preferrable to have calculations in one place and reuse the output in many places. However, this understanding of what can reasonably be expected is then essential.

As for product improvements, I have added an idea on the subject, if you agree with me, please vote! Microsoft Idea (

A final note is that this type of unexpected behaviour is not limited to just Azure Synapse Link but really to any integrations based on either “modified on” or change tracking without doing periodic synchronizations. Hence I would also like to give a general warning about this.

Our first connector released!

Our first connector released!

Previously integrations between systems have been rather specific. For instance, someone has build an integration between Dynamics 365 and Mail Chimp or something similar. The introduction of Logic Apps and Power Automate has, however, at least from a Microsoft ecosystem perspective, changed this quite a lot. Now, when someone asks for an integration it is usually preferable to build and use custom or standard connectors in Power Automate / Logic Apps instead. This allows for more flexibility in the integration with the downside that the actual logical flow of the integration still has to be built, which can be a daunting task in itself in some cases.

Hence, when we at CRM-Konsulterna started a conversation with one of the larger digital signature companies in Sweden, Verified, about helping them create an integration to Dynamics/Power Platform, we suggested to them that we acutally not build an integration at all but rather a connector to Logic Apps/Power Automate. This connector has now been released and we are very proud of this and hope that it can be of great use to many companies that want to build integrations to Verified.

And as LogicApps/Power Automate are more generic solutions, this actually enables integrations to any of the other 600+ systems to which there are connectors nowdays. Hence someone who wants to build an integration to Saleforce or even SurveyMonkey can now do this using Power Automate or Logic Apps.

One of the things that need to be mentioned with regards to Power Automate is, however, that there sometimes are issues with the execution of Flows. This is not particuarly common but it does happen. It has also been improved over time by Microsoft but we do feel that it is important to consider this fact when building integrations with Power Automate. So, for instance making sure the logic both triggers on a particular event and has a “clean up” job which runs periodically and catches any records which for some reason wasn’t processed correctly.

I don’t think this should discourage the use of Power Automate or Logic Apps but that you need to make the logic a bit more robust.

Another interesting issue which can arise due to the fact that Power Automate is so easy to work with is that people with no programming experience start making Flows. This might not be a problem if you make a simple flow but with more complex logic you typically have to consider some fringe cases, what we call exceptions in programming, for instance if there is no data in a field where data is expected. This can cause a Flow excution to break or do unforseen things. Hence, having a good process for monitoring Flows and having experts have a look at them to notice any potiential problems, especially when they grow in complexity, compliance or use.

With this, I hope you have fun with Power Automate and create some amazing Flows to help you in your everyday life! And thank you all amazing colleagues who made the launch of this connector possible.

Read more about it on Microsoft Docs:

Read more about Verified: