lakehouse.databricks.common.generate_expectation_meta_behaviour

lakehouse.databricks.common.generate_expectation_meta_behaviour(spark, ge_df)

This function generates expectation meta data to control how the calling pipeline behaves on expectation failure

Parameters

sparkspark context

Spark context object passed from the calling Spark instance.

ge_dfdataframe

the great expecations dataframe containing the expectation results

Returns

object :

provides an object with the attributes defining the status of the expecation and the pipeline behaviour on expectation failure :

{'success': "False", 'fail_pipeline': "True"}