A cross tenant metadata driven processing framework for Azure Data Factory and Azure Synapse Analytics achieved by coupling orchestration pipelines with a SQL database and a set of Azure Functions.

- Overview
- Contents

View the Project on GitHub mrpaulandrew/procfwk

Deploying ProcFwk

« Contents

ProcFwk Deployment Steps

Below are the basic steps you’ll need to take to deploy the processing framework to your environment. Currently these steps assume a new deployment is being done, rather than an upgrade from a previous version of the framework. In addition, a reasonable working knowledge of using the Microsoft Azure platform is assumed when completing these action points.

Please see Service Tiers for details on the recommended minimum levels of compute to deploy.

Note; in the case of most deployment steps, things can be tailored to your specific environment requirements. For example:

These custom setups are valid and given your knowledge of Azure can easily be applied. Otherwise the below steps offer a minimum vial product set of configurations.

Azure Resources

Key Vault Secrets

If using Azure Key Vault, complete the following steps, otherwise jump to the next section.

Deploying Code

Service Principals

If Using Azure Data Factory

If Using Azure Synapse Analytics

Checking Your Orchestrator

Adding Metadata

INSERT INTO [Recipients]
	'Alerting User 1',

EXEC [procfwkHelpers].[AddRecipientPipelineAlerts]
	@RecipientName = N'Alerting User 1',
	@PipelineName = 'Worker Pipeline 1',
	@AlertForStatus = 'Success, Failed';	
EXEC [procfwk].[CheckMetadataIntegrity]
	@DebugMode = 1;


For more support deploying the processing framework see the video guide playlist on YouTube:

YouTube Demo Video