“ForEach” ACTIVITY IN AZURE DATA FACTORY (ADF)

ForEach activity is used as a loop in the data factory. To understand ForEach activity consider a problem statement and try to solve it.

Problem Statement:- We have 3 tables in Azure SQL DB and we need to move these tables from Azure SQL DB to Azure Data Lake. Based on the conditions: 1) must have one pipeline for the whole process 2) Each table copied should have an individual folder name.

Solution:-

Step 1: Get ready with the Azure account and set up an Azure subscription, resource group, and storage account with a container and data factory account.

Step 2: Create a connection using linked services to link Azure SQL DB and Data Lake.

Two Linked services

1. Connection to the Azure SQL DB(Source)
2. Connection to the Data Lake (Destination)

Step 3: Create a dataset using Datasets to link Azure SQL DB dataset to the Azure SQL DB Linked service and the Data Lake storage dataset to the Data Lake storage linked service.

At Azure SQL DB dataset: Dynamics Parameters at Azure SQL DB