lakehouse.databricks.common.generate_delta_load_stats

lakehouse.databricks.common.generate_delta_load_stats(spark, schema, table, load_type, source_row_count, quarantined_row_count=0)

a function to return the statistics from a delta table insert/update/delete operation, will always return the most recent history expluding optimize operations

Parameters

sparkspark context

spark context passed from the calling spark instance

schemastring

hive database/schema where table exists

tablestring

delta table name

load_typestring

type of data load being performed, eg type-1

source_row_countint

number of rows from the source table

Returns

dataframe:

a spark data frame containing the delta loads statistics from the load operation