Data lake

A data lake stores large amounts of data in its raw form, structured or not, in cheap, scalable object storage. Instead of designing a schema up front, teams land the data first and decide how to interpret it when they read it.

Data lakes became popular because data science and machine learning need raw, detailed and varied data that traditional warehouses weren’t built to hold.

Diagram

flowchart LR
    apps[App databases] --> ingest[Ingestion]
    logs[Logs and clickstream] --> ingest
    files[Files and documents] --> ingest
    ingest --> raw[(Raw zone)]
    raw --> clean[(Cleaned zone)]
    clean --> curated[(Curated zone)]
    curated --> ds[Data science and ML]
    curated --> sql[SQL query engines]
    catalog[Data catalog] -.describes.- raw & clean & curated

How it works

Schema on read. Data is stored as-is; structure is applied by whoever queries it.
Object storage. Cheap, durable cloud storage holds files in open formats such as Parquet, JSON or CSV.
Zones. Data typically moves from a raw zone, through a cleaned zone, to a curated zone ready for use.
Separate compute. Query engines and processing frameworks read the files directly, so storage and compute scale independently.
A catalog records what datasets exist, where they live and what they contain.

Pros

Cheap storage at scale: keep everything, including detail you don’t yet need.
Any type of data: structured tables, logs, documents, images and audio.
Flexible: no schema design required before data arrives.
Good for data science: analysts and models get the full, raw history.

Cons

Data swamps: without cataloging and governance, nobody can find or trust anything.
Weak guarantees: plain files lack transactions, so concurrent writes and partial failures can leave data inconsistent.
Slower for BI: ad-hoc SQL over raw files is usually slower and harder than querying a warehouse.
Quality checks come late: problems are found when data is read, not when it arrives.

When to use it

Machine learning and data science on large or varied datasets.
Retaining raw history cheaply for future, not-yet-known uses.
Ingesting high-volume logs, events or IoT data.

When to avoid it

Business reporting that needs fast, consistent, governed numbers. A warehouse or lakehouse fits better.
Small data volumes, where a database is simpler.

Common pitfalls

No catalog or ownership. The lake fills up with unlabeled files and becomes a swamp.
Dumping without retention rules. Storage is cheap, but sensitive data kept forever is a compliance risk.
Too many small files, which slow down every query engine.