Flow lets in Mapping Data Flow within ADF(Azure Data Factory)

Flowlets are very similar to functions in programming languages. We create a function in a scenario where we need to create repetitive tasks with some return values.

Flow lets is also a reusable activity that can be created from existing mapping data flow or started from scratch.

Advantages

1. Prevent duplicate logic data flow.
2. Reusable the mapping data flow.

Problem Statement: Consider a scenario where we need to remove duplicate records from the sample file stored in Data Lake.

Solution:

Consider the mentioned employee table with duplicate records.

1. Create a data flow with the source connection and load the dataset.

2. Group by records based on the required columns.

3. Aggregate by count() to identify the duplicate records.