Sorting in Spark Dataframe
In this blog, we will learn how to sort rows in spark dataframe based on…
Removing White Spaces From Data in Spark
White spaces can be a headache if not removed before processing data. We will learn…
Adding White Spaces to Data in Spark Dataframe
In this blog, we will learn how to use rpad and lpad functions to add…
String Functions in Spark
This blog is intended to be a quick reference for the most commonly used string…
Converting Strings to Dates in Spark
In this blog, we are going to learn how to format dates in spark along…
Date Difference functions in Spark
we need to find a difference between dates or find a date after or before…
Working With Timestamps in Spark
Working with timestamps while processing data can be a headache sometimes. Luckily Spark has some…
Date & Timestamp Functions in Spark
Spark provides multiple Date and Timestamp functions to make processing dates easier. In this blog,…
Select Expr in Spark Dataframe
In this blog, we will learn how to use select and expr in the Spark…
Add, Rename, Drop Columns in Spark Dataframe
We will go through common column operations like add, rename, list, select, and dropping a…
MSCK Repair – Fixing Partitions in Hive Table
We will learn how to add multiple partitions to hive table using msck repair table…
Inserting Data In Hive Partitioned tables
In this blog, we will learn how to insert data in partitions in hive table….