Azure Functions provide the middle ware in the processing framework allowing the orchestration pipelines to interact with the worker pipelines or access external resources.
The highly scalable, cheap, serverless compute provides the required customisations to extend the framework and acheive a level of decoupling between orchestration resources as well as supporting its own authenticate to Key Vault when needed.
All the Functions within the processing framework are created using:
To support the interchangeable use of orchestrators in the processing framework (Azure Data Factor or Azure Synapse Analytics) the Function App’s internal classes are setup using a services architecture.
See services for more details.
Alongside the primary function methods a set of helper classes are used to define return types and support the function requests.
See helpers for more details.
Enabling the Azure Function App managed identity is optional if you want to store worker pipeline SPN details in Azure Key Vault.
The MSI could also be used to authenticate against the worker pipelines directly removing the need for the SPN metadata. Although this option is understood, it is not implemented or supported by the framework as it would underpin the cross Azure tenant execution support.