Sending Data to Kafka using Pyspark Permalink
“Kafka is a distributed event-streaming platform designed to handle real-time data feeds with high throughput and low latency.”
Recent Posts
Transforming Bank’s Data using PySpark Permalink
“Combining similar types of information hierarchically not only improves data transfer efficiency but also simplifies decoding for downstream teams.”
An ETL pipeline using Pyspark for a Bank Permalink
“While notebooks are great for exploring and gaining an initial understanding of the data, they are not versatile enough for building end-to-end data pipe...
Data Wrangling and Exploration using Pyspark Permalink
“Data wrangling is not just about cleaning; it’s about structuring data to make it usable and insightful for analysis.”
Designing a Data Lake House for a Big Bank Permalink
“A data lakehouse blends the scalability of a data lake with the reliability of a data warehouse, creating a unified platform for all types of data.”