A cross tenant metadata driven processing framework for Azure Data Factory and Azure Synapse Analytics achieved by coupling orchestration pipelines with a SQL database and a set of Azure Functions.

- Overview
- Contents

View the Project on GitHub mrpaulandrew/procfwk

Functions (Azure)

« Contents

Azure Functions Icon Azure Functions provide the middle ware in the processing framework allowing the orchestration pipelines to interact with the worker pipelines or access external resources.

The highly scalable, cheap, serverless compute provides the required customisations to extend the framework and acheive a level of decoupling between orchestration resources as well as supporting its own authenticate to Key Vault when needed.

All the Functions within the processing framework are created using:

Code Icons


To support the interchangeable use of orchestrators in the processing framework (Azure Data Factor or Azure Synapse Analytics) the Function App’s internal classes are setup using a services architecture.

At runtime the abstract class PipelineService is called which inspects the request for the orchestrator type. Given the enumerable value the required orchestrator service class is then instantiated.

See services for more details.


Alongside the primary function methods a set of helper classes are used to define return types and support the function requests.

See helpers for more details.

Managed Identity (MSI)

MSI Enabling the Azure Function App managed identity is optional if you want to store worker pipeline SPN details in Azure Key Vault.

The MSI could also be used to authenticate against the worker pipelines directly removing the need for the SPN metadata. Although this option is understood, it is not implemented or supported by the framework as it would underpin the cross Azure tenant execution support.