EventGrid Trigger - How to set clienttrackingid from triggerbody? - azure-logic-apps

In a microservice environment where requests span multiple services including eventgrid i'd like to configure an end-to-end logging with correlationid.
Inspired by this blog https://toonvanhoutte.wordpress.com/2018/08/05/end-to-end-correlation-across-logic-apps/
How can i configure the EventGrid triggers clientTrackingId with my correlationnr from Events data payload?
Checkout my definition below which does not work.
If i substitute "#{coalesce(json(triggerBody().Data)?.CorrelationNr, guid())}" with a string value or even "#parameters('$connections')['azureeventgrid']['connectionId']" it works like a charm.
"triggers": {
"When_a_resource_event_occurs": {
"correlation": {
"clientTrackingId": "#{coalesce(json(triggerBody().Data)?.CorrelationNr, guid())}"
},
"inputs": {
"body": {
"properties": {
"destination": {
"endpointType": "webhook",
"properties": {
"endpointUrl": "#{listCallbackUrl()}"
}
},
"filter": {
"includedEventTypes": [
"webhook.sp.updated"
]
},
"topic": "/subscriptions/xxxx/resourceGroups/xxx/providers/Microsoft.EventGrid/topics/WebHookManager"
}
},
"host": {
"connection": {
"name": "#parameters('$connections')['azureeventgrid']['connectionId']"
}
},
"path": "/subscriptions/#{encodeURIComponent('xxx')}/providers/#{encodeURIComponent('Microsoft.EventGrid.Topics')}/resource/eventSubscriptions",
"queries": {
"x-ms-api-version": "2017-06-15-preview"
}
},
"splitOn": "#triggerBody()",
"type": "ApiConnectionWebhook"
}
}
Logic App does not trigger. No Error message.

Please check the description about clientTrackingId, and your logic app no runs history is because your triggerBody() doesn't have CorrelationNr with the definition you show.
Actually your Event Grid trigger has detected the event, it just couldn't run with the logic. You could go to the EVALUATION and check the trigger history. It's because the value is null, then it won't run.
If you use HTTP request trigger, you could set the x-my-custom-correlation-id header. or set any key-value in the json body, then set the clientTrackingId with like #{coalesce(json(triggerBody())['keyname'], guid())}.
And if you are using some trigger without header, you have to point the value with string or other parameter like you said the connectionid or the parameter value you custom like below.
So the point is the clientTrackingId must be set before it runs and value could be obatined.

Related

is it possible to add safe URLs in AWS WAF?

I know that AWS WAF is pretty dumb and non-configurable, but last time it becomes stricter.
We can't send even request to backend like:
POST https://our.url/page_id
{
"data": "<a></a>"
}
In this case awswaf:managed:aws:core-rule-set:CrossSiteScripting_Body_RC_COUNT rule will be triggered.
I tried to find any ways how core-rule-set may be customized, but looks like it is impossible. But I want to trust that is my lack of search and it may be customized in some way. If not, current WAF is simply unusable. I can't imagine case when it may be used with these strict non-editable standard rules.
So the question is:
Is it possible to set some safe domains (like https://our.url) that will be passed without blocking? Or maybe some ways to allow <a> tag for example?
You can customize the action on the AWS managed rule in this way:
Edit the AWS managed core set and change the rule action to: "Override to Count". Take note of the aws label for this rule (something like "awswaf:managed:aws:core-rule-set:CrossSiteScripting_Body")
Create a new rule that you add the end of all existing rules. This new rule should trigger on conditions (1) Statement "has a label", where you specify the above label (2) url matches the url you want to let through. Specify the action as "Allow"
Create a new rule that you add after the previous rule. This rule should trigger on same condition (1). Specify the action as "Block"
Based on #Chris answer:
Add AWSManagedRulesCommonRuleSet and override Action to Count for desired Rule (CrossSiteScripting_BODY in my case) (if you have to allow safe URLs inside query parameters, CrossSiteScripting_QUERYARGUMENTS should be changed same way, etc.)
Add custom rule allow-safe-URLs with Block action that will block all requests with CrossSiteScripting_BODY label and with JSON body that don't match <[ ]*a[\-_a-zA-Z0-9 ='"()]*href[ ]*=[ "']https:\/\/([a-zA-Z0-9\-]+\.)?example\.com[ "']*>*.<[ ]*\/[ ]*a[ ]*> RegEx. (only <a href=https://example.com>xxx</a> or <a href=https://anysubdomain.example.com>xxx</a> will be detected as safe)
Last default action should be Allow.
Final JSON of the WEB ACL will be looks like:
{
"Name": "test-waf",
"Id": "some-uuid-of-web-acl",
"ARN": "arn:aws:wafv2:us-east-1:1234567890:regional/webacl/test-waf/some-uuid-of-web-acl",
"DefaultAction": {
"Allow": {}
},
"Description": "Web ACL for URL whitelisting tests",
"Rules": [
{
"Name": "AWS-AWSManagedRulesCommonRuleSet",
"Priority": 0,
"Statement": {
"ManagedRuleGroupStatement": {
"VendorName": "AWS",
"Name": "AWSManagedRulesCommonRuleSet",
"Version": "Version_1.5",
"RuleActionOverrides": [
{
"Name": "CrossSiteScripting_BODY",
"ActionToUse": {
"Count": {}
}
}
]
}
},
"OverrideAction": {
"None": {}
},
"VisibilityConfig": {
"SampledRequestsEnabled": true,
"CloudWatchMetricsEnabled": true,
"MetricName": "AWS-AWSManagedRulesCommonRuleSet"
}
},
{
"Name": "allow-safe-URLs",
"Priority": 1,
"Statement": {
"AndStatement": {
"Statements": [
{
"NotStatement": {
"Statement": {
"RegexMatchStatement": {
"RegexString": "<[ ]*a[\\-_a-zA-Z0-9 ='\"()]*href[ ]*=[ \"']https:\\/\\/([a-zA-Z0-9\\-]+\\.)?example\\.com[ \"']*>*.<[ ]*\\/[ ]*a[ ]*>",
"FieldToMatch": {
"JsonBody": {
"MatchPattern": {
"All": {}
},
"MatchScope": "VALUE",
"InvalidFallbackBehavior": "EVALUATE_AS_STRING",
"OversizeHandling": "MATCH"
}
},
"TextTransformations": [
{
"Priority": 0,
"Type": "NONE"
}
]
}
}
}
},
{
"LabelMatchStatement": {
"Scope": "LABEL",
"Key": "awswaf:managed:aws:core-rule-set:CrossSiteScripting_Body"
}
}
]
}
},
"Action": {
"Block": {}
},
"VisibilityConfig": {
"SampledRequestsEnabled": true,
"CloudWatchMetricsEnabled": true,
"MetricName": "allow-safe-URLs"
}
}
],
"VisibilityConfig": {
"SampledRequestsEnabled": true,
"CloudWatchMetricsEnabled": true,
"MetricName": "test-waf"
},
"Capacity": 707,
"ManagedByFirewallManager": false,
"LabelNamespace": "awswaf:1234567890:webacl:test-waf:"
}
WARNING: this JSON is just a simple reproducible example (Proof of concept). And this configuration vulnerable for attacks with body like: <a href=https://example.com></a><a href=https://www.evil-url.com></a>. RegEx also may be vulnerable. You have to check if ONLY safe URLs included in body. Actual checking (URL-matching) logic should be more complicated. Don't use it as is (by copy/paste).

change the header Content-Type to application/cloudevents+json when publishing to event grid

I'm using the trigger "When a HTTP request is received" to then publish multiple events to the event grid using the "Publish Event" action. The For loop works fine to split up the JSON that gets in and create the event to publish but still that publish fails with
{
"error": {
"code": "UnsupportedMediaType",
"message": "The Content-Type header is either missing or it doesn't have a valid value. The content type header must either be application/cloudevents+json; charset=utf-8 or application/cloudevents-batch+json; charset=UTF-8. Report 'edf36bbd-9221-4882-8a29-2264ffb16d72:3:3/6/2020 2:18:20 PM (UTC)' to our forums for assistance or raise a support ticket.",
"details": [
{
"code": "InvalidContentType",
"message": "The Content-Type header is either missing or it doesn't have a valid value. The content type header must either be application/cloudevents+json; charset=utf-8 or application/cloudevents-batch+json; charset=UTF-8. Report 'edf36bbd-9221-4882-8a29-2264ffb16d72:3:3/6/2020 2:18:20 PM (UTC)' to our forums for assistance or raise a support ticket."
}
]
}
}
I assume that the header from the input is used when publishing so I tried to change the header when publishing by changing the header in the Publish_Event block as follows (directly in code view as it is not supported in the UI) so I get the following (headers part is added):
"Publish_Event": {
"inputs": {
"body": [
{
"data": "#items('For_each_2')",
"eventType": "company-location",
"id": "ID : #{items('For_each')['businessId']}",
"subject": "Company Location changed"
}
],
"headers": {
"Content-Type": "application/cloudevents+json; charset=utf-8"
},
"host": {
"connection": {
"name": "#parameters('$connections')['azureeventgridpublish']['connectionId']"
}
},
"method": "post",
"path": "/eventGrid/api/events"
},
"runAfter": {},
"type": "ApiConnection"
}
But this is not working neither. Didn't find an action to make the change.
My full flow looks like this :
and as test data I have the following JSON I use to send with postman (a bit simplified):
[
{
"id": 3603,
"businessId": "QQTADOSH",
"locations": [
{
"id": 5316,
"businessId": "A-yelr3g"
},
{
"id": 5127,
"businessId": "A-c7i8gd"
},
{
"id": 5403,
"businessId": "A-fjdd2y"
},
{
"id": 6064,
"businessId": "A-rqvhz8"
}
]
},
{
"id": 3118,
"businessId": "Cr11_Macan_111qa",
"locations": [
{
"id": 4563,
"businessId": "A-3bv860"
}
]
}
]
Looks like the Official Event Grid Publish Connector doesn't support Cloud Events Schema.
You can set the topic to accept Event Grid Schema but I believe this is possible only at the time of creation.
Its best to open a feature request on UserVoice to add support for this and in the meantime, a workaround would be to use an HTTP Action to Post to Custom Topic instead.
But do note that the workaround would involve building the event payload to send (and things like fetching the access key from Key Vault instead of storing it in your workflow directly).

JSON schema deeper object uniqueness

I'm trying to get into JSON schema definitions and wanted to find out, how to achieve a deeper object uniqueness in the schema definition. Please look at the following example definition, in this case a simple IO of a module.
{
"$schema": "http://json-schema.org/draft-06/schema#",
"type": "object",
"required": ["modulIOs"],
"properties": {
"modulIOs": {
"type": "array",
"uniqueItems": true,
"items": {
"allOf": [
{
"type": "object",
"required": ["ioPosition","ioType","ioFunction"],
"additionalProperties": false,
"properties": {
"ioPosition": {
"type": "integer"
},
"ioType": {
"type":"string",
"enum": ["in","out"]
},
"ioFunction": {
"type":"string"
}
}
}
]
}
}
}
}
When I validate the following with i.E. draft-06 I get a positive validation.
{"modulIOs":
[
{
"ioPosition":1,
"ioType":"in",
"ioFunction":"240 V AC in"
},
{
"ioPosition":1,
"ioType":"in",
"ioFunction":"24 V DC in"
}
]
}
I'm aware that the validation is successfull because the validator does what he's intended to - it checks the structure of a JSON-object, but is there a possibility to validate object value data in deeper objects or do i need to perform the check elsewhere?
This is not currently possible with JSON Schema (at draft-7).
There is an issue raised on the official spec repo github for this: https://github.com/json-schema-org/json-schema-spec/issues/538
If you (or anyone reading this) really wants this, please thumbsup the first issue comment.
It's currently unlikely to make it into the next draft, and even if it did, time to impleemntations picking it up may be slow.
You'll need to do this validation after your JSON Schema validation process.
You can validate data value of your object fields by using JSON schema validation.
For example, if you need to check if ioPosition is between 0 and 100 you can use:
"ioPosition": {
"type": "integer",
"minimum": 0,
"maximum": 100
}
If you need to validate ioFunction field you can use regualr expression such as:
"ioFunction": {
"type": "string",
"pattern": "^[0-9]+ V [A,D]C"
}
Take a look at json-schema-validation.

Logic Apps variable in replace expression

I have a Logic Apps instance with a variable AddedPoKey set out of parsed JSON
"Set_PO_Key_variable_": {
"inputs": {
"name": "AddedPoKey",
"value": "#{body('Parse_JSON')?['poKey']}"
},
"runAfter": {
"Parse_JSON": [
"Succeeded"
]
},
"type": "SetVariable"
},
I want to use the value to set the Id in a RESTful API collection resource. I have not been able to get it to work, though I am sure it should. I am possible not finding the best MS documentation on this and if any could be pointed out to me, I would appreciate it. I have tried:
"Set_Po_Lin_Add_url_": {
"inputs": {
"name": "CreatePoLineResourceUrl",
"value": "#{replace('https://api.plex.com/EDI/sales-orders/{PoKey}/lines','{PoKey}',triggerBody()['AddedPoKey'])}"
},
"runAfter": {
"Set_PO_Key_variable_": [
"Succeeded"
]
},
"type": "SetVariable"
}
Which results in InvalidTemplate. Unable to process template language expressions in action 'Set_Po_Lin_Add_url_' inputs at line '1' and column '2459': 'The template language expression 'replace('https://api.plex.com/EDI/sales-orders/{PoKey}/lines','{PoKey}',triggerBody()['AddedPoKey'])' cannot be evaluated because property 'AddedPoKey' doesn't exist, available properties are 'ContentData, ContentType, ContentTransferEncoding, Properties, MessageId, To, ReplyTo, ReplyToSessionId, Label, ScheduledEnqueueTimeUtc, SessionId, CorrelationId, SequenceNumber, LockToken, TimeToLive'. Please see https://aka.ms/logicexpressions for usage details.'.
and then
"#{replace('https://api.plex.com/EDI/sales-orders/{PoKey}/lines','{PoKey}',variables('AddedPoKey')}"
which works.
Note that https://aka.ms/logicexpressions had been my main references, but does not even have the word "variable" on the page at this time.

Angular schema form destroyStrategy works only on siblings that are array of objects. It does not work on other siblings

I have not been successful in getting the destroy strategy to work on any sibling properties or sibling objects. It only works on sibling array of objects. Please check this example:
$scope.schema = {
"type": "object",
"properties": {
"propertyOne": {
"type": "string",
"enum": ["option1", "option2"],
"title": "Property One Select"
},
"propertyTwo": {
"type": "string",
"enum": ["option3", "option4"],
"title": "Property Two Select"
},
"objectOne": {
"type": "object",
"properties": {
"objectOnePropertyThree": {
"type": "string",
"enum": ["option5","option6"],
"title": "Property Three Select"
}
}
},
"arrayOfObjects": {
"type": "array",
"items": {
"type": "object",
"properties": {
"arrayObjectPropertyFour": {
"type": "string",
"enum": ["option7","option8"],
"title": "Property Four Select"
}
}
}
}
},
"required": ["propertyOne"]
};
$scope.form = [{
"key": "propertyOne"
}, {
"key": "propertyTwo",
"condition": "model.propertyOne === \"option1\""
},{
"key": "objectOne.objectOnePropertyThree",
"condition": "model.propertyOne === \"option1\""
},{
"key": "arrayOfObjects",
"condition": "model.propertyOne === \"option1\""
},
{
"type": "submit",
"title": "Save"
}];
http://jsfiddle.net/mutharasus/dp18a70b/
In here if you select the first dropdown to "Option1" then select all the other dropdowns and save. Then go back and switch the first dropdown to "Option2" and save you can see that only the very last array of objects is removed with the destroy strategy.
Am I doing something wrong or is this a bug in angular-schema-form? I looked under the issues that are currently open in the github project and I do not see an open issue about this.
You're correct, it's not currently behaving the way that you expect when an individual field is removed from the view because of a condition.
Here's what's happening: under the "old" bundled decorators for ASF, each field-type decorator is rendered on the page within an outer tag. The contents of the appropriate field template are then processed and rendered. Condition logic applies to everything within the tag, but not to the tag itself. Normally, this would be fine, but the destroyStrategy logic was assigned to the $destroy event of the tag. The end result is that the $destroy event will never fire unless the entire tag would be removed from the DOM. This is why the model values in the array of objects are being cleaned up - the container is removed when the "model.propertyOne === 'option1'" condition fails, which cascades the $destroy event to each object in the array.
I think that this got overlooked with the creation and release of the new builder, because I raised the issue at the end of the PR for the feature (https://github.com/Textalk/angular-schema-form/pull/371).
On the bright side, the new builder approach (which you can use by adding the bootstrap-decorator file from https://github.com/Textalk/angular-schema-form-bootstrap) doesn't have this issue. Instead, the destroyStrategy logic is applied via directive to the form fields because the tag is no longer used. Unless you have a need to stay with the old decorators at this time, I suggest grabbing the new ones and giving them a try.
Let us know how it goes!

Resources