Orchestrate
.procfwk

Logo

A cross tenant metadata driven processing framework for Azure Data Factory and Azure Synapse Analytics achieved by coupling orchestration pipelines with a SQL database and a set of Azure Functions.


- Overview
- Contents


View the Project on GitHub mrpaulandrew/procfwk

Helpers


¬ę¬†Contents / Functions


The Azure Functions App used by the framework contains the following internal classes that support the public functions with reuseable methods and clients to authenticate against external Azure resources in a common way.

This internal classes all live within the namespace ADFprocfwk.Helpers.


Data Factory Client Class

Methods:

CreateDataFactoryClient

Returns: Microsoft.Azure.Management.DataFactory.DataFactoryManagementClient

Role: this helper is used by the execute pipeline, check pipeline status and get error details function to authenticate against the target worker data factory at runtime before invoking pipeline operation requests.

Exmaple Use:

using (var client = DataFactoryClient.CreateDataFactoryClient(tenantId, applicationId, 
    authenticationKey, subscriptionId))
{
    PipelineRun pipelineRun;

    pipelineRun = client.PipelineRuns.Get(resourceGroup, factoryName, runResponse.RunId);
}

Synapse Client Class (coming soon)

Methods:

CreateSynapseClient

Returns: Microsoft.Azure.Management.Synapse.SynapseManagementClient

Role: Coming Soon

Example Use:

using var client = SynapseClient.CreateSynapseClient(tenantId, applicationId, authenticationKey, subscriptionId);

Key Vault Client Class

Methods:

GetSecretFromUri

Returns: value

Role: if the functions app needs to get a secret value from Azure Key Vault this methods wraps up the call to the respective key vault resource using the URL provided and authenticating against key vault using the Function App MSI.

Example Use:

string authenticationKey = 
    KeyVaultClient.GetSecretFromUri(authenticationKey);

SMTP Client Class

Methods:

CreateSMTPClient

Returns: System.Net.Mail.SmtpClient

Role: creates a means of sending emails to an SMTP mailing service from the framework send email function. The client wraps up the authentication details and host information so the public function can focus on constructing the email/content to be sent.

Example Use:

using (var client = SMTPClient.CreateSMTPClient())
{
    MailAddress from = new MailAddress(SMTPClient.FromEmail);
    MailMessage mail = new MailMessage
    {
        From = from,
        IsBodyHtml = true,
        Subject = subject,
        Body = message
    };

    mail.To.Add(toAddress);
    mail.CC.Add(ccAddress);
    mail.Bcc.Add(bccAddress);
    
    client.Send(mail);
}

Request Helper Class

Methods:

CheckUri

Returns: bool

Role: provides a simple validation check for values passed to the public functions as part of the request body.

Example Use:

if (RequestHelper.CheckUri(authenticationKey))
{
    log.LogInformation("Valid URL Provided");
}

CheckGuid

Returns: bool

Role: provides a simple validation check for values passed to the public functions as part of the request body.

Example Use:

if (!RequestHelper.CheckGuid(applicationId))
{
    log.Error("Invalid GUID Provided.");
}