Export to Data Lake Alternatives: Fabric Link Vs. Azure Synapse Link

We previously explored Microsoft's deprecation of the "Export to Data Lake" feature for Dynamics 365 Finance and Operations (F&O), what it means, and your options for ensuring a smooth transition.

In this blog, we explain and compare the two options organizations should consider for replacing Export to Data Lake to help you make the best decision for your organization.

Why Did Microsoft Deprecate Export to Data Lake?

Microsoft decided to sunset the "Export to Data Lake" feature for 2 main reasons.

1. To streamline its data integration offerings and encourage the adoption of more advanced and flexible solutions.
2. To move away from legacy replication technology called change data capture (CDC).

Export to Data lake also delivers data in CSV format, which can result in poor performance, and read/write errors when used directly for reporting, and requires multiple transformations to optimize.

Microsoft has made significant advancements in their technology stack since the launch of Export to Data Lake, introducing technologies that cut down on storage and compute costs, as well as reduce the steps necessary to move and transform data. With the newer approaches, customers can land their data directly in the OneLake as delta tables, unlocking a wave of new functionality when paired with Power BI and other tools.

When Will Export to Data Lake Get Shut off?

Export to Data Lake is deprecated as of November 1, 2024. It will continue to function with limited support.  Microsoft is allowing a one-time extension through February 15, 2025.  You can submit the request via the Azure Synapse Link option in Power Apps. You must submit a request for each environment. Microsoft plans to officially shut off Export to Data Lake beginning March 25, 2025.

We are advising organizations using this legacy method to not delay their migration and fully understand the implications of each option and start your transition to alternative solutions as soon as possible.

Organizations have 2 main options to consider: Fabric Link or Azure Synapse Link. Synapse link requires a whole lot more manual configuration than Fabric link, but it does give you the benefit of not having to pay in perpetuity for data storage. More on each option below.

Option 1: Fabric Link

Microsoft Fabric Link is a feature that enables organizations to connect their data from Microsoft Dataverse directly to Microsoft Fabric. This integration allows users to access and analyze their Dynamics 365 and Power Apps data seamlessly within Microsoft Fabric’s environment, specifically utilizing OneLake, the built-in data lake.

If you are already utilizing Power BI for data consumption, employing a data warehouse, or using dataflows and notebooks for data transformation, the link to Fabric feature offers considerable benefits. This feature allows you to streamline your data integration architecture by eliminating the need for a separate storage account or Synapse services for Dataverse data. Instead of incurring costs for Azure resources such as storage and compute, you will pay for the additional Dataverse storage required. Compute charges, including near real-time data updates and management overhead, are integrated into the Dataverse storage costs.

Fabric link is the primary Export to Data Lake alternative that everybody's talking about. But Fabric link has some serious cost considerations that people are not always aware of.

The biggest consideration is when you use Fabric link, is that it stores the data in an intermediate form in the Dataverse environment at a very costly rate of $40/gig/mo. or $40,000 / TB / Mo. Yikes!  This cost is only incurred when you exceed the storage capacity allotment provided for with your licensing agreement.  Additionally, data storage is compressed so you cannot directly compare it to current storage requirements.

Data latency for Fabric link is about 1 hour, which may be an issue for some use cases.  Microsoft is working to reduce that latency, but timeframes and new latency metrics are not yet available.

There is also a 1000 table limit with Fabric Link.  All Dataverse tables are automatically added to Fabric link, so if you have Dynamics CE or CRM, or have a large Dataverse presence, you may hit the limit.  Microsoft is also working to remove this restriction.

Option 2: Azure Synapse Link

Microsoft Azure Synapse Link is the non-Fabric based direct replacement for Export to Data Lake.  It makes Dynamics 365 F&O and/or Dataverse data available within an Azure Synapse environment or storage container. This approach has 2 options:

1. CSV
   1. This option has the lowest data latency and only has storage costs but requires the greatest transformation effort to read and apply to an analytics solution.
2. Delta
   1. This option requires a Spark Pool and runs Microsoft managed notebooks to create delta tables in Synapse from the source system. This simplifies the logic and effort to make the source data ready for use in an analytics solution.

Both Synapse Link approaches eliminate the high cost of Dataverse storage, using Azure Storage instead. However, you will incur compute costs to transform the data for downstream use.

OmniData has developed processes to simplify and quickly implement both options.  If you need low data latency and have complex transformation processes, we have a framework for ingesting the CSV files efficiently.

If you choose the Delta approach, we can help to implement and integrate with existing solutions or even with Fabric using data virtualization shortcuts.

Resource: Microsoft’s Azure Synapse Transition Guide

What is the best Option to replace Export to Data Lake Feature?

Both provide enhanced performance, greater flexibility, and new features such as Delta Table functionality. The comparison table below helps illustrate the differences between the two options:

Fabric Link Vs. Azure Synapse Link – A Comparison