lakehouse.databricks.common.read_data_lake_query

lakehouse.databricks.common.read_data_lake_query(spark, query, using=None)

A function to read data from the data lake using a hive sql query, the data must be available as a hive table

Parameters

spark: spark context

spark context passed from the calling spark instance

query: string

the hive sql query used to get the data

using: string, default=None

the hive database/schema where the table(s) are stored creates a using statement so query doed not have to provide two part naming of tables Defaults to None.

Returns

dataframe:

spark dataframe containing data read by the function, if Exception nothing is returned