A cross tenant metadata driven processing framework for Azure Data Factory and Azure Synapse Analytics achieved by coupling orchestration pipelines with a SQL database and a set of Azure Functions.

- Overview
- Contents

View the Project on GitHub mrpaulandrew/procfwk


¬ę¬†Contents / Data Factory

Dataset IconThe processing framework requires only one dataset to run. In the development code repository this is called GetSetMetadata. This dataset offers all Lookup and Stored Procedure activities within the framework pipelines the ability to call and query database objects.

As the scope of the framework is to operate at the control flow level within a given solution, datasets used for data transformation are not within the remit of the framework. However, if you are using a single Data Factory instance for both the framework pipelines and the worker pipeline processing tasks other datasets can be created as required.