Unlocking Insights: Visualizing Data on GCP with Tableau Hey readers! I’m excited to share a new article with you. In this article, we’ll explore the seamless integration of GCP Big Query with Tableau, allowing you to effortlessly visualize your data on the Google Cloud Platform using the powerful features of Tableau. Let’s dive in and discover how you…

Azure Data Engineering Interview Questions Data engineering is a critical role in the world of data science and analytics. In this article, I will be sharing questions that I have faced during the interview process i.e., asked in product and service-based companies. Write an SQL query to find a day on which the weather temperature…

Joins In Pyspark In this article we will explore the reason behind joining tables/data frame and types of joins used in Pyspark. With the requirement of bringing more clear and meaningful way to store data in table led to coming up of data modelling concept in the world of data warehouse. Datamodelling…

Demystifying Azure Integration Runtime(IR) How to Seamlessly Connect and Manage Data Across Your Ecosystem. In simple terms, IR describes “how to take the data “.Let’s understand the detailed definition of IR. “Azure Integration Runtime Bridges the Gap Between Cloud Data and Between On-Premises Data and Cloud Data.” Integration Runtime provides computation power to the…

Working with Dates and Time in PySpark Recently working on my current project faced a scenario where we need to convert a string datatype column with the date value to a date or datetime datatype with the same value. In this article, we will learn how to convert different formats of date/datetime values using Pyspark. Functions to…

Unleashing the Power of Azure Synapse with External Table In this article, we will learn how to create an external table in Dedicated SQL Pool and Serverless SQL Pool. How do create an External Table in Serverless SQL Pool? # create database DROP DATABASE appdb; CREATE DATABASE appdb; # Change to connect to: built-in and db name: appdb CREATE…

Azure Triggers This article is for sure going to help all the new learners who are new to Azure or want to switch to the Azure data engineering domain. Azure Triggers is one of the favorite questions asked in all the data engineering interviews. The task of the trigger is just to…

Azure Databricks Company founded by the creators of Apache Spark. Databricks make use of Apache Spark to provide a Unified Analytics platform. Why do we need Azure Databricks? In order to make use of apache spark we need to provision the machines and install the spark, and the necessary libraries and maintain…

Copy Data from Rest API to ADL using dynamic parameters at Dataset and Linked Service. In this article, we will see how to copy data from an API with specific files to an Azure data lake. Step 1: Create a folder raw where we will copy data in the data lake. Create a file with .json format and add the configuration-related information.