Tech Talk Thursday: Build a Synapse Pipeline Part 1: Handle Full and Incremental Refreshes Using Delta Lake Control Tables
What exactly is Delta Lake and how can you use it to build Azure Synapse Pipelines for full and incremental loads? Find out in Part One of a three-video series describing the step-by-step process.
In today’s Tech Talk, OmniData™’s Principal Solutions Architect, Ray Marquina, will discuss an architecture using Azure Data Lake as the storage layer to house your inserts and updates, while leveraging Delta Lake as the control table for your Synapse Pipelines.
In this video, Ray will show you how to work seamlessly in Synapse Studio using Delta Lake as the control table, just like you would in a traditional database setting.
Be prepared for the next video in this three-part series by watching Part One today.
OmniData is a leading Microsoft partner and professional services firm specializing in data and analytics modernization and AI solutions. We provide actionable insights and scalable solutions powered by AI, seamlessly integrating with existing systems to maximize efficiency. Our team of veteran specialists excels in solution architecture, data engineering, and business intelligence. We thrive on partnering with companies across various industries and global markets, helping them quickly conceptualize and address complex business data challenges.