Date Difference functions in Spark

Updated On February 11, 2021 | By Mahesh Mogal

We may have a use case where we need to find a difference between dates or find a date after or before "x" days from a given date. For such use cases, the spark has provided a rich list of functions to easily solve these problems. we are going over such functions in this blog.

date_add - Date After x Days

Say we ant to get date after 7 days from today, we can get using spark function date_add

date_sub - Date Before x Days

similarly, we can use date_sub function to find a date before x days from the given date.

add_months - Adding Months to Date

We can add months to date using add_months function.

We can see that, this function has added 3 months to our date and showing us final result.

date_diff - Finding Difference Between Dates in Days

We may need to find a difference between two days. For such a use case, we can use date_diff function, which accepts 2 arguments and return as difference between first date and second date.

date_diff_function
Difference between two dates using date_diff

months_between - Months between Dates

Though in most cases we need to find the difference between dates in days, Spark has also provided us function to find the difference of months in two dates.

Conclusion

We have gone through many useful functions to find the difference between dates and finding date after certain days. I hope you found this useful.

Date Difference functions in Spark

Mahesh Mogal

I am passionate about Cloud, Data Analytics, Machine Learning, and Artificial Intelligence. I like to learn and try out new things. I have started blogging about my experience while learning these exciting technologies.

Stay Updated with Latest Blogs

Get latest blogs delivered to your mail directly.

Recent Posts

Where and Filter in Spark Dataframes

In this blog, we will learn how to filter rows from spark dataframe using Where and Filter functions.

Where and Filter in Spark Dataframes
Read More
Distinct Rows and Distinct Count from Spark Dataframe

Getting distinct values from columns or rows is one of most used operations. We will learn how to get distinct values as well as count of distinct values.

Distinct Rows and Distinct Count from Spark Dataframe
Read More
Sorting in Spark Dataframe

In this blog, we will learn how to sort rows in spark dataframe based on some column values.

Sorting in Spark Dataframe
Read More

Leave a Reply

Your email address will not be published. Required fields are marked *

linkedin facebook pinterest youtube rss twitter instagram facebook-blank rss-blank linkedin-blank pinterest youtube twitter instagram
Share via
Copy link
Powered by Social Snap