Nidhi Gupta
4 min readSep 23, 2023

--

Unlocking Data Strategy-Which is Right for Your Business?

With the increase in demand of ingesting huge variety, volume and velocity of data on the day to day basis, led to the upcoming of new data warehouse solution which can resolve major problems of processing of big data.

Hold 🛑In the journey of this article we will explore pros and cons of Database, DataLake, Datawarehouse and how easily DataLakehouse can solves this using existing capabilities.

Database:

Introduction

  1. Structured Storage: Database are well suited for structuted data, where the data’s format is predefined and organized into tables and schema.
  2. ACID Properties: Database often follow the ACID(Atomicity, Consistency, Isolation, Durability) properties to ensure the reliability in the transactions).
  3. Schema Rigidity: Changes to the schema can be complex and time-consuming, making adaptability challenging.
  4. Querying: Databases can be query using SQL language to perform operation on the Databases.
  5. Example: MySQL, SQL Server, Oracle and PostgreSQL.

Shortcomings of database in handling big data in terms of format and storage led to the upcoming of DataLake.

DataLake:

Introduction

  1. Storage Flexibility: Data Lakes can store data in its raw and native format, making them suitable for a wide range of data types, including documents, images, logs, and more.
  2. Scalability: Data Lakes can scale horizontally to handle massive volumes of data, making them ideal for big data and IoT applications.
  3. Schema-on-Read: Unlike databases, which use a schema-on-write, DataLake uses schema-on-read.
  4. Example: AWS Glue and Azure Data Lake Storage

Shortcomings of DataLake for implementing update and delete were not possible.No ACID properties for the stored data or lack of data consistency and data integrity.

Data Warehouse:

Introduction

  1. Structured Storage: Data warehouses rely on structured, schema-on-write storage, requiring data to conform to predefined schemas before ingestion.
  2. Schema Rigidity: Changes to the schema can be complex and time-consuming, making adaptability challenging.
  3. Structured Data: They…

--

--

Nidhi Gupta

Azure Data Engineer 👨‍💻.Heading towards cloud technologies expertise✌️.