Common Data Module Documentation

The cdm module provides functionality to support the integration of Common Data Model Artifacts into a Delta Lakehouse implementation

Common Data Model (CDM)

get_cdm_tables_to_process(spark, file_mount)

This function returns a JSON list of tables from the CDM to be processed

get_cdm_entities_to_integrate(spark, ...[, ...])

This function returns the information required to integrate CDM Entities from the Bronze Zone of the Data Lakehouse to the Silver Zone

get_parent_entities_to_integrate(spark, ...)

This function returns the information required to integrate Parent CDM Entities from the Bronze Zone of the Data Lakehouse to the Silver Zone

get_dependent_entities_to_integrate(spark, ...)

This function returns the information required to integrate Dependent CDM Entities from the Bronze Zone of the Data Lakehouse to the Silver Zone

get_cdm_trickle_tables_to_process(spark, ...)

This function returns a list of all of the cdm tables that are configured for a trickle feed into the cdm Data Lake

stream_cdm_table_silver(spark, src_mount, ...)

This function streams Bronze CDM tables to the Silver Zone of the Data Lakehouse

process_cdm_table_bronze(spark, cdm_path, ...)

This fucntion moves CDM Data from the Common Data Model Storage account to the Data Lakehouse Bronze Zone in the form of append only delta tables

process_cdm_entity(spark, cdm_path, ...[, ...])

This function Processes a Common Data Model Entity and converts to a Data Lakehouse table