ARM template deployment for Logic Apps with Datalake API Connection - azure-logic-apps

Error - The API connection 'azuredatalake' is not configured to support managed identity."
I tried deployment of Azure logic App along with API connection to access Azure DataLake Gen1 using Managed Identity. This failed due to above error.
Also deployed API connection separately which was successful but with status "Unauthenticated", so deploying of corresponding logic apps failed with missing api connection.
Part Template for API connection reference:
{
"type": "Microsoft.Web/connections",
"apiVersion": "2016-06-01",
"name": "[parameters('azuredatalake_1_Connection_Name')]",
"location": "[parameters('location')]",
"kind": "V1",
"properties": {
"displayName": "azuredatalakemsi",
"customParameterValues": {},
"api": {
"id": "[variables('managedadlsApi')]"
}
}
}

If we want to access the data stored in data lake gen1, we must configure right ACLs for the user or sp used to do auth. Otherwise, we cannot have permission to access data. For more details, please refer to the official document. Meanwhile, we cannot do that via arm template. We can do that by PowerShell or Portal.
Besides, regarding how to use MSI to access Azure data lake gen1 in azure logic app, please refer to the following steps
Enable MSI in Azure logic app
{
"apiVersion": "2016-06-01",
"type": "Microsoft.logic/workflows",
"name": "[variables('logicappName')]",
"location": "[resourceGroup().location]",
"identity": {
"type": "SystemAssigned"
},
"properties": {
"definition": {
"$schema": "https://schema.management.azure.com/providers/Microsoft.Logic/schemas/2016-06-01/workflowdefinition.json#",
"actions": {},
"parameters": {},
"triggers": {},
"contentVersion": "1.0.0.0",
"outputs": {}
},
"parameters": {},
"dependsOn": []
}
Configure ACLs
Create connection
{
"type": "Microsoft.Web/connections",
"apiVersion": "2016-06-01",
"name": "[parameters('azuredatalake_1_Connection_Name')]",
"location": "[parameters('location')]",
"tags": {
"CreatedTime": "2021-05-24T03:11:28.9371899Z"
},
"kind": "V1",
"properties": {
"displayName": "test",
"customParameterValues": {},
"api": {
"id": "[concat(subscription().id,'/providers/Microsoft.Web/locations/', parameters('location'), '/managedApis/azuredatalake')]"
}
}
},
{
"type": "Microsoft.Logic/workflows",
"apiVersion": "2017-07-01",
"name": "[parameters('workflows_testlogic05_name')]",
"location": "[parameters('location')]",
"dependsOn": [
"[resourceId('Microsoft.Web/connections', parameters('azuredatalake_1_Connection_Name'))]"
],
"properties": {
"state": "Enabled",
"definition": {
"$schema": "https://schema.management.azure.com/providers/Microsoft.Logic/schemas/2016-06-01/workflowdefinition.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"$connections": {
"defaultValue": {},
"type": "Object"
}
},
"triggers": {
},
"actions": {
},
"outputs": {}
},
"parameters": {
"$connections": {
"value": {
"azuredatalake": {
"connectionId": "[resourceId('Microsoft.Web/connections', parameters('azuredatalake_1_Connection_Name'))]",
"connectionName": "azuredatalake",
"connectionProperties": {
"authentication": {
"type": "ManagedServiceIdentity"
}
},
"id": "[concat(subscription().id,'/providers/Microsoft.Web/locations/', parameters('location'), '/managedApis/azuredatalake')]"
}
}
}
}
}
}
For more details, please refer to
https://learn.microsoft.com/en-us/azure/logic-apps/create-managed-service-identity

I was able to solve this by modifying ARM Template for this API connection, that is by adding "parameterValueType" as "Alternative".
Deployment of Azure logic App along with API connection to access Azure DataLake Gen1 using Managed Identity is successful.
Part Template for API connection reference:
{
"type": "Microsoft.Web/connections",
"apiVersion": "2016-06-01",
"name": "[parameters('azuredatalake_1_Connection_Name')]",
"location": "[parameters('location')]",
"kind": "V1",
"properties": {
"displayName": "azuredatalakemsi",
"parameterValueType": "Alternative",
"customParameterValues": {},
"api": {
"id": "[variables('managedadlsApi')]"
}
}
}

Related

Azure Logic App Standard - Some of the connections are not authorized yet

I'm creating a logic apps standard workflow hosted in an ase v3. I have the built-in file system connector to collect from an on-prem file share. I was able to create the workflow in the azure portal. However, when I return to it and click save I get the following message:
Some of the connections are not authorized yet. If you just created a
workflow from a template, please add the authorized connections to
your workflow before saving
The code for the workflow is shown below:
{
"definition": {
"$schema": "https://schema.management.azure.com/providers/Microsoft.Logic/schemas/2016-06-01/workflowdefinition.json#",
"actions": {
"Copy_file": {
"inputs": {
"parameters": {
"destination": "destination_path_here",
"source": "#triggerOutputs()?['body']?['path']"
},
"serviceProviderConfiguration": {
"connectionName": "FileSystem",
"operationId": "copyFile",
"serviceProviderId": "/serviceProviders/FileSystem"
}
},
"runAfter": {},
"type": "ServiceProvider"
}
},
"contentVersion": "1.0.0.0",
"outputs": {},
"triggers": {
"When_a_file_is_added": {
"conditions": [
{
"expression": "#not(equals(triggerBody().Size,0))"
}
],
"inputs": {
"parameters": {
"folderPath": "source_path_here"
},
"serviceProviderConfiguration": {
"connectionName": "FileSystem",
"operationId": "whenFilesAreAdded",
"serviceProviderId": "/serviceProviders/FileSystem"
}
},
"kind": "Polling",
"recurrence": {
"frequency": "Second",
"interval": 15
},
"splitOn": "#triggerOutputs()?['body']",
"type": "ServiceProvider"
}
}
},
"kind": "Stateful"
}

Need help to understand maintaining azure key vault in different environment and also to be able to use that in logic app

I have a logic app which makes HTTP call to Key Vault URI to get the secret needed to connect to external system. I have developed this in the dev resource group. I want to know how to setup the key vault from dev resource groups to other resource groups (test/prod). Also, how to migrate the logic app and get the secret per environment.
:) The solution is to use ARM templates and ADO/any other pipeline. You can create ARM templates with different parameters' values for different environments and use them to deploy your Logic App and Key vault to different environments.
Logic App Template sample:
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
// Template parameters
"parameters": {
"<template-parameter-name>": {
"type": "<parameter-type>",
"defaultValue": "<parameter-default-value>",
"metadata": {
"description": "<parameter-description>"
}
}
},
"variables": {},
"functions": [],
"resources": [
{
// Start logic app resource definition
"properties": {
<other-logic-app-resource-properties>,
"definition": {
"$schema": "https://schema.management.azure.com/providers/Microsoft.Logic/schemas/2016-06-01/workflowdefinition.json#",
"actions": {<action-definitions>},
// Workflow definition parameters
"parameters": {
"<workflow-definition-parameter-name>": {
"type": "<parameter-type>",
"defaultValue": "<parameter-default-value>",
"metadata": {
"description": "<parameter-description>"
}
}
},
"triggers": {
"<trigger-name>": {
"type": "<trigger-type>",
"inputs": {
// Workflow definition parameter reference
"<attribute-name>": "#parameters('<workflow-definition-parameter-name')"
}
}
},
<...>
},
// Workflow definition parameter value
"parameters": {
"<workflow-definition-parameter-name>": {
"value": "[parameters('<template-parameter-name>')]"
}
},
"accessControl": {}
},
<other-logic-app-resource-definition-attributes>
}
// End logic app resource definition
],
"outputs": {}
}
Key Vault template:
{
"name": "string",
"type": "Microsoft.KeyVault/vaults",
"apiVersion": "2018-02-14",
"location": "string",
"tags": {},
"properties": {
"tenantId": "string",
"sku": {
"family": "A",
"name": "string"
},
"accessPolicies": [
{
"tenantId": "string",
"objectId": "string",
"applicationId": "string",
"permissions": {
"keys": [
"string"
],
"secrets": [
"string"
],
"certificates": [
"string"
],
"storage": [
"string"
]
}
}
],
"vaultUri": "string",
"enabledForDeployment": "boolean",
"enabledForDiskEncryption": "boolean",
"enabledForTemplateDeployment": "boolean",
"enableSoftDelete": "boolean",
"createMode": "string",
"enablePurgeProtection": "boolean",
"networkAcls": {
"bypass": "string",
"defaultAction": "string",
"ipRules": [
{
"value": "string"
}
],
"virtualNetworkRules": [
{
"id": "string"
}
]
}
},
"resources": []
}
Moreover, you can read this article to understand more about setting up your ADO pipelines: Integrate ARM templates with Azure Pipelines

How to add Blob storage connection in logic app template file

I want to deploy the logic app to the portal using the Azure cloud shell, I have modified the all the files , and able to deploy the logic apps successfully. But when I see the changes in the portal all the actions are showing but the blob storage part it is showing like invalid connection.
Please help us how to provide the connections and all related to the blob storage in the logic app template file.
Thanks in advance.
Regards, Manikanta P.
Let's assume i already have a container called azureBlobContainer under MyStorageAccount Using the below template you should be able to deploy a LogicApp with a blob connection created.
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"LogicAppName": {
"defaultValue": "Test",
"type": "string"
},
"storageAccountName": {
"defaultValue": "MyStorageAccount",
"type": "string"
}
},
"variables": {},
"resources": [
{
"type": "Microsoft.Web/connections",
"apiVersion": "2016-06-01",
"name": "azureblobContainer",
"location": "[resourceGroup().location]",
"properties": {
"displayName": "BlobConnection",
"api": {
"id": "[concat(subscription().id, '/providers/Microsoft.Web/locations/', resourceGroup().location, '/managedApis/', 'azureblob')]"
},
"parameterValues": {
"accessKey": "[listKeys(resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName')), providers('Microsoft.Storage', 'storageAccounts').apiVersions[0]).keys[0].value]",
"accountName": "[parameters('storageAccountName')]"
}
}
},
{
"type": "Microsoft.Logic/workflows",
"apiVersion": "2017-07-01",
"name": "[parameters('LogicAppName')]",
"location": "[resourceGroup().location]",
"dependsOn": [
"[resourceId('Microsoft.Web/connections', 'azureblobContainer')]"
],
"properties": {
"state": "Enabled",
"definition": {
"$schema": "https://schema.management.azure.com/providers/Microsoft.Logic/schemas/2016-06-01/workflowdefinition.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"$connections": {
"defaultValue": {},
"type": "Object"
}
},
"triggers": {
"request": {
"type": "Request",
"kind": "Http",
"inputs": {
"schema": {}
}
}
},
"actions": {
"Create_blob": {
"runAfter": {},
"type": "ApiConnection",
"inputs": {
"body": "#triggerBody()",
"host": {
"connection": {
"name": "#parameters('$connections')['azureblob']['connectionId']"
}
},
"method": "post",
"path": "/datasets/default/files",
"queries": {
"folderPath": "/azureblobContainer",
"name": "Test",
"queryParametersSingleEncoded": true
}
},
"runtimeConfiguration": {
"contentTransfer": {
"transferMode": "Chunked"
}
}
},
"Response": {
"runAfter": {
"Create_blob": [
"Succeeded"
]
},
"type": "Response",
"inputs": {
"statusCode": 200
}
}
},
"outputs": {}
},
"parameters": {
"$connections": {
"value": {
"azureblob": {
"connectionId": "[resourceId('Microsoft.Web/connections', 'azureblobContainer')]",
"connectionName": "azureblobContainer",
"id": "[concat(subscription().id, '/providers/Microsoft.Web/locations/', resourceGroup().location, '/managedApis/', 'azureblob')]"
}
}
}
}
}
}
]
}

How to stop Azure Logic App SMS flooding?

I am trying to use this Azure Serverless IOT Button Project to send an SMS alert: https://azure.microsoft.com/en-in/resources/samples/azure-serverless-iot-button/
When I click the button once, I see that the Logic App starts running multiple times and I get flooded with hundreds of SMS messages in less than a minute. I only need one SMS alert per button click. What am I missing?
"$connections": {
"value": {
"plivo": {
"connectionId": "/subscriptions/---0-fb3c-406e-98--2-7fa0e00ef0d0/resourceGroups/--/providers/Microsoft.Web/connections/plivo",
"connectionName": "plivo",
"id": "/subscriptions/f--0-fb3c-406e-9-72-7f--0/providers/Microsoft.Web/locations/westus/managedApis/plivo"
}
}
},
"definition": {
"$schema": "https://schema.management.azure.com/providers/Microsoft.Logic/schemas/2016-06-01/workflowdefinition.json#",
"actions": {
"Send_SMS": {
"inputs": {
"body": {
"dst": "1 801-999-9999",
"src": "1 801-999-9982",
"text": "Button Pressed - #{triggerBody()?['location']}"
},
"host": {
"connection": {
"name": "#parameters('$connections')['plivo']['connectionId']"
}
},
"method": "post",
"path": "/v1/Account/auth_id_value/Message/"
},
"runAfter": {},
"type": "ApiConnection"
}
},
"contentVersion": "1.0.0.0",
"outputs": {},
"parameters": {
"$connections": {
"defaultValue": {},
"type": "Object"
}
},
"triggers": {
"manual": {
"inputs": {
"schema": {
"properties": {
"location": {
"type": "string"
}
},
"type": "object"
}
},
"kind": "Http",
"type": "Request"
}
}
}
}
You have tagged this question with azure-function-app, are you sure the azure function only calls the logic app one time? Could you please post the code of the azure function
I apparently wasnt filtering the messages that were coming through, and so it was triggering on other telemetry that had nothing to do with the buttons.

Create SQL Server via Azure Resource Manager (ARM) template

I am trying to create a new Azure instance of SQL Server in which I would like to then create a few new databases.
I know from the Azure Portal that some sort of admin users could be:
an SA user (I think this means "Server Admin" and it looks like some sort of old way of managing a SQL Server instance, but at the same time very "basic" and proved to work)
an Active Directory user (not sure about Azure terminology here, but it looks like this could be some "broad user" for the whole Azure platform, like e.g. my own login user for the Azure Portal, this is not specific to databases world).
I would like to create a SQL Server with a SA user to administer the server. From the Azure portal I can not find a way to generate an ARM template for a SA user for the SQL Server instance.
I am copy pasting from a 10000 lines ARM template for a very long list of SQL servers and databases but I am not able to isolate the basic steps to have a hopefully clean and short ARM template to start with.
This is the ARM template I am trying to deploy on Azure:
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"location": {
"type": "string",
"defaultValue": "westeurope"
},
"foo_sql_server_name": {
"defaultValue": "foo-sql-server",
"type": "String"
}
},
"resources": [
{
"type": "Microsoft.Sql/servers",
"kind": "v12.0",
"name": "[parameters('foo_sql_server_name')]",
"apiVersion": "2015-05-01-preview",
"location": "[parameters('location')]",
"scale": null,
"properties": {
"administratorLogin": "<MY_SA_USER_THAT_I_CAN_NOT_CREATE>",
"version": "12.0"
},
"dependsOn": []
}
]
}
When running the above with:
az group deployment create \
--name "deployDBs" \
--resource-group "MyCustomResourceGroup" \
--template-file ./templates/db.json # --verbose --debug
Then I get the following error message:
Deployment failed. Correlation ID: <A_CUSTOM_GUID>. {
"status": "Failed",
"error": {
"code": "ResourceDeploymentFailure",
"message": "The resource operation completed with terminal provisioning state 'Failed'.",
"details": [
{
"code": "InvalidParameterValue",
"message": "Invalid value given for parameter Password. Specify a valid parameter value."
}
]
}
}
When removing the JSON field administratorLogin (because hopefully I could create the SA user somehow somewhere else that I yet have to figure out), then I get the following error message:
Deployment failed. Correlation ID: <ANOTHER_CUSTOM_GUID>. {
"status": "Failed",
"error": {
"code": "ResourceDeploymentFailure",
"message": "The resource operation completed with terminal provisioning state 'Failed'.",
"details": [
{
"code": "InvalidParameterValue",
"message": "Invalid value given for parameter Login. Specify a valid parameter value."
}
]
}
}
I am not able to find the definition for the pair "username password" for the SA user (Server Admin) from the 10000 lines auto-generated ARM template.
How could I create/inject a SA user for the SQL Server while deploying a new instance of a SQL Server?
The sa login you use on an on-premises SQL Server instance is known on Azure SQL as the Admin login. You can provide the name of the admin login and its password as parameter as shown on below sample template:
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"yourservernameName": {
"type": "string",
"defaultValue": "yourservername2"
},
"yourservernameAdminLogin": {
"type": "string",
"defaultValue": "VeryWiseAdmin",
"minLength": 1
},
"yourservernameAdminLoginPassword": {
"type": "securestring",
"defaultValue": "ReplaceWithTheMostSecurePasswordThatEverExisted&NeverShareLikeThisWithAnyone!"
},
"dbnameName": {
"type": "string",
"defaultValue": "dbname",
"minLength": 1
},
"dbnameCollation": {
"type": "string",
"minLength": 1,
"defaultValue": "SQL_Latin1_General_CP1_CI_AS"
},
"dbnameEdition": {
"type": "string",
"defaultValue": "Basic"
},
"dbnameRequestedServiceObjectiveName": {
"type": "string",
"defaultValue": "Basic"
}
},
"variables": {
},
"resources": [
{
"name": "[parameters('yourservernameName')]",
"type": "Microsoft.Sql/servers",
"location": "West Europe",
"apiVersion": "2014-04-01-preview",
"dependsOn": [],
"tags": {
"displayName": "yourservername"
},
"properties": {
"administratorLogin": "[parameters('yourservernameAdminLogin')]",
"administratorLoginPassword": "[parameters('yourservernameAdminLoginPassword')]",
"version": "12.0"
},
"resources": [
{
"name": "[concat(parameters('yourservernameName'),'/AllowAllWindowsAzureIps')]",
"type": "Microsoft.Sql/servers/firewallRules",
"location": "[resourceGroup().location]",
"apiVersion": "2014-04-01-preview",
"dependsOn": [
"[resourceId('Microsoft.Sql/servers', parameters('yourservernameName'))]"
],
"properties": {
"startIpAddress": "0.0.0.0",
"endIpAddress": "0.0.0.0"
}
},
{
"name": "[concat(parameters('yourservernameName'),'/',parameters('dbnameName'))]",
"type": "Microsoft.Sql/servers/databases",
"location": "West Europe",
"apiVersion": "2014-04-01-preview",
"dependsOn": [
"[resourceId('Microsoft.Sql/servers', parameters('yourservernameName'))]"
],
"tags": {
"displayName": "dbname"
},
"properties": {
"collation": "[parameters('dbnameCollation')]",
"edition": "[parameters('dbnameEdition')]",
"maxSizeBytes": "1073741824",
"requestedServiceObjectiveName": "[parameters('dbnameRequestedServiceObjectiveName')]"
}
}
]
}
],
"outputs": {
"SomeString": {
"type": "string",
"value": "What ever you want to put here"
},
"ServerNameParam": {
"type": "string",
"value": "[parameters('yourservernameName')]"
},
"ServerResourceID": {
"type": "string",
"value": "[resourceId('Microsoft.Sql/servers', parameters('yourservernameName'))]"
},
"ServerObject": {
"type": "object",
"value": "[reference(parameters('yourservernameName'))]"
},
"SqlServerURL": {
"type": "string",
"value": "[reference(parameters('yourservernameName')).fullyQualifiedDomainName]"
},
"DbResourceID": {
"type": "string",
"value": "[resourceId('Microsoft.Sql/servers/databases', parameters('yourservernameName'), parameters('dbnameName'))]"
},
"DbObject": {
"type": "object",
"value": "[reference(parameters('dbnameName'))]"
},
"DbAdoConnString": {
"type": "string",
"value": "[concat('Server=tcp:',reference(parameters('yourservernameName')).fullyQualifiedDomainName,',1433;Initial Catalog=',parameters('dbnameName'),';Persist Security Info=False;User ID=',reference(parameters('yourservernameName')).administratorLogin,';Password=',reference(parameters('yourservernameName')).administratorLoginPassword,';MultipleActiveResultSets=False;Encrypt=True;TrustServerCertificate=False;Connection Timeout=30;')]"
}
}
}
Working sample:
"name": "name",
"type": "Microsoft.Sql/servers",
"location": "[resourceGroup().location]",
"apiVersion": "2014-04-01",
"properties": {
"administratorLogin": "somelogin",
"administratorLoginPassword": "somepasswordD1!"
}
please note that SA might not be allowed as a username and password has complexity requirements
We wanted to create a temporary unique password per resource group and don't have to worry about passwords in template or parameters files since these are checked into git. Solved it like this:
template.json:
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"vulnerabilityAssessments_Default_storageContainerPath": {
"type": "SecureString"
},
"servers_dev_name": {
"defaultValue": "dev-app",
"type": "String"
}
},
"variables": {
"servers_dev_password": "[concat('P', uniqueString(resourceGroup().id, '224F5A8B-51DB-46A3-A7C8-59B0DD584A41'), 'x', '!')]",
},
"resources": [
{
"type": "Microsoft.Sql/servers",
"apiVersion": "2019-06-01-preview",
"name": "[parameters('servers_dev_name')]",
"location": "northeurope",
"kind": "v12.0",
"properties": {
"administratorLogin": "OurSaName",
"administratorLoginPassword": "[variables('servers_dev_password')]",
"version": "12.0",
"publicNetworkAccess": "Enabled"
}
},
"To make sure that we are compliant with the Azure SQL database policy "Your password must contain characters from three of the following categories – English uppercase letters, English lowercase letters, numbers (0-9), and non-alphanumeric characters (!, $, #, %, etc.)", we insert one character for each category before and after the unique string."
Sources:
https://learn.microsoft.com/en-us/azure/azure-resource-manager/templates/template-functions-string#uniquestring
https://vivien-chevallier.com/Articles/automatically-generate-a-password-for-an-azure-sql-database-with-arm-template
Warning:
If you add to your parameters.json:
"servers_dev_password": {
"value": "[uniqueString(resourceGroup().id)]"
}
and add the parameter to template.json the actual password will be:[uniqueString(resourceGroup().id)].
"servers_dev_password": {
"type": "SecureString"
}
A thing to note is that the definition for uniqueString is:
Creates a deterministic hash string based on the values provided as
parameters.
This means that if you want to create a unique password per deployment it would have to look something like this:
"parameters": {
"newGuid": {
"type": "string",
"defaultValue": "[newGuid()]"
}
}
"variables": {
"sqlserverAdminPassword": "[concat(uniqueString(guid(resourceGroup().id, deployment().name)), parameters('newGuid'), 'Tg2%')]"
}
Your password would then be updated on every deploy.
https://stackoverflow.com/a/70325944/3850405

Resources