In this blog, we will learn how to filter rows from spark dataframe using Where and Filter functions.
White spaces can be a headache if not removed before processing data. We will learn how to remove spaces from data in spark using inbuilt functions.
In this blog, we will learn how to use rpad and lpad functions to add padding to data in spark dataframe.
This blog is intended to be a quick reference for the most commonly used string functions in Spark. It will cover all of the core string processing operations that are supported by Spark. In addition, it should serve as a useful guide for users who wish to easily integrate these into their own applications.
In this blog, we will learn how to use select and expr in the Spark data frame. We will learn multiple use cases along with selectExpr.
In this blog, we will learn how to insert data in partitions in hive table. We will write queries to insert data in static as well as dynamic partitions.
We will learn how to specify our custom schema with column names and data types for Spark data frames.
We will learn how to load data from JSON, CSV, TSV, Pipe Delimited or any other type for delimited file to spark Dataframe.
We will learn how to load and populate data to hive table. We will also learn how to copy data to hive tables from local system.