A cross tenant metadata driven processing framework for Azure Data Factory and Azure Synapse Analytics achieved by coupling orchestration pipelines with a SQL database and a set of Azure Functions.

- Overview
- Contents

View the Project on GitHub mrpaulandrew/procfwk

Worker Pipeline Decoupling


By design the worker pipelines called by the processing framework can be anywhere;

Or, simply contained within the same orchestrator instance as the framework pipelines (parent/child/infant).

The Azure Functions used to interact with the worker pipelines assume nothing about the target location and require a complete set of environment metadata provided from the database at runtime. This includes:

It is this level of pipeline decoupling that allows the processing framework to be used to bootstrap any existing solution.