lakehouse.databricks.common.read_datalake_file

lakehouse.databricks.common.read_datalake_file(spark, mounting_point, file_path, file_type, options=None, lake_file_struct=None, complete_file_path=None, json_schema_path=None, **kwargs)

A function to read a set of files or file in the data lake

Parameters

sparkspark context

spark context passed from the calling spark instance

mounting_point: string

mounting point where data is located

file_path: string

file path to be appended to mount point

file_type: string

type of file to be read, delta, parquet, json, csv, text file types supported supported

options: string, default=None

options to be used while reading csv filed. Defaults to None.

lake_file_struct: string, default=None

lake file structure to be appeneded to file path when querying nested data. Defaults to None.

complete_file_path: string, default=None

complete file path used when querying a specific file. Defaults to None.

json_schema_path string, default=None:

location of the schema for a json dataset. Defaults to None.

kwargsobject

looking for file_store element

Returns

dataframe:

spark dataframe containing data read by the function, if Exception nothing is returned