Working With Timestamps in Spark

Updated On February 12, 2021 | By Mahesh Mogal

Working with timestamps while processing data can be a headache sometimes. Luckily Spark has some in-built functions to make our life easier when working with timestamps. Let us go over these functions.

current_timestamp - Getting Current Timestamp

We can get current timestamp using current_timestamp function.

date_trunc - Truncated Date From Timestamp

We can get truncated date from timestamp using code below.


Converting Timestamp to Unix Timestamp

Spark has provided function to convert timestamp to Unix Timestamp.

Get Timestamp from Unix Timestamp

Get Timestamp from Date

We can also get from timestamp using to_timestamp function

I hope you found this helpful. See you in in next blog.

Mahesh Mogal

I am passionate about Cloud, Data Analytics, Machine Learning, and Artificial Intelligence. I like to learn and try out new things. I have started blogging about my experience while learning these exciting technologies.

Stay Updated with Latest Blogs

Get latest blogs delivered to your mail directly.

Recent Posts

Spark Join Types With Examples

In this blog, we are going to learn different spark join types. We will also write code and validate data output for each join type to better understand them.

Read More
Integrate Spark with Jupyter Notebook and Visual Studio Code

In this blog, we are going to integrate spark with jupyter notebook and visual studio code to create easy-to-use development environment.

Read More
Reading Data From SQL Tables in Spark

In this blog, we are going to learn about reading data from SQL tables in Spark. We will create Spark data frames from tables and query results as well.

Read More

Leave a Reply

Your email address will not be published. Required fields are marked *

linkedin facebook pinterest youtube rss twitter instagram facebook-blank rss-blank linkedin-blank pinterest youtube twitter instagram