Working With Timestamps in Spark

Updated On February 12, 2021 | By Mahesh Mogal

Working with timestamps while processing data can be a headache sometimes. Luckily Spark has some in-built functions to make our life easier when working with timestamps. Let us go over these functions.

current_timestamp - Getting Current Timestamp

We can get current timestamp using current_timestamp function.

date_trunc - Truncated Date From Timestamp

We can get truncated date from timestamp using code below.

timestamp_trunc

Converting Timestamp to Unix Timestamp

Spark has provided function to convert timestamp to Unix Timestamp.

Get Timestamp from Unix Timestamp

Get Timestamp from Date

We can also get from timestamp using to_timestamp function

I hope you found this helpful. See you in in next blog.

Working With Timestamps in Spark

Mahesh Mogal

I am passionate about Cloud, Data Analytics, Machine Learning, and Artificial Intelligence. I like to learn and try out new things. I have started blogging about my experience while learning these exciting technologies.

Stay Updated with Latest Blogs

Get latest blogs delivered to your mail directly.

Recent Posts

Where and Filter in Spark Dataframes

In this blog, we will learn how to filter rows from spark dataframe using Where and Filter functions.

Where and Filter in Spark Dataframes
Read More
Distinct Rows and Distinct Count from Spark Dataframe

Getting distinct values from columns or rows is one of most used operations. We will learn how to get distinct values as well as count of distinct values.

Distinct Rows and Distinct Count from Spark Dataframe
Read More
Sorting in Spark Dataframe

In this blog, we will learn how to sort rows in spark dataframe based on some column values.

Sorting in Spark Dataframe
Read More

Leave a Reply

Your email address will not be published. Required fields are marked *

linkedin facebook pinterest youtube rss twitter instagram facebook-blank rss-blank linkedin-blank pinterest youtube twitter instagram
Share via
Copy link
Powered by Social Snap