Delta query for group properties not accurate for classification changes - azure-active-directory

We enforce base controls to Office 365 groups that are dependent on the data classification value. We have two ways these controls get applied:
Through a Graph subscription which automatically applies a set of base controls to an Office 365 group
A full scan to ensure each Office 365 group with a data classification value is assigned the correct base controls
We are trying to make the 1st option execute more efficiently by only proceeding with checking/applying the base controls if the data classification property changes. The delta query for group appears to be bugged; if the classification property changed values, it does not correctly report this. Any property that has been changed has a "null" value. Also the visibility property is always there.
Repro steps:
1: Initialization. Get delta token.
HTTP GET https://graph.microsoft.com/beta/groups/delta&$deltatoken=latest
2: Make a classification change to an Office 365 group. Make a description change to another Office 365 group.
3: Delta Query.
HTTP GET https://graph.microsoft.com/beta/groups/delta?$deltatoken=DeltaToken
Prefer:return=minimal
Response is:
{
"#odata.context": "https://graph.microsoft.com/beta/$metadata#groups",
"#odata.deltaLink": "https://graph.microsoft.com/beta/groups/delta?$deltatoken=NextDeltaToken",
"value": [
{
"classification": null,
"description": null,
"visibility": null,
"id": "Group1ID"
},
{
"description": null,
"visibility": null,
"id": "Group2ID"
}
]
}
Edit: Forgot to add... Microsoft docs mention a null value means the value is empty or the property has not changed. This observation doesn't reflect the docs.

Related

What syntax should be used for reading the final row in an Array on the Mapping tab on the Copy Data activity in Azure Data Factory / Synapse?

I'm using the copy data activity in Azure Data Factory to copy data from an API to our data lake for alerting & reporting purposes. The API response is comprised of multiple complex nested JSON arrays with key-value pairs. The API is updated on a quarter-hourly basis and data is only held for 2 days before falling off the stack. The API adopts an oldest-to-newest record structure and so the newest addition to the array would be the final item in the array as opposed to the first.
My requirement is to copy only the most recent record from the API as opposed to the collection - so the 192th reading or item 191 of the array (with the array starting at 0.)
Due to the nature of the solution, there are times when the API isn't being updated as the sensors that collect and send over the data to the server may not be reachable.
The current solution is triggered every 15 minutes and tries a copy data activity of item 191, then 190, then 189 and so on. After 6 attempts it fails and so the record is missed.
current pipeline structure
I have used the mapping tab to specify the items in the array as follows (copy attempt 1 example):
$['meta']['params']['sensors'][*]['name']
$['meta']['sensorReadings'][*]['readings'][191]['dateTime']
$['meta']['sensorReadings'][*]['readings'][191]['value']
Instead of explicitly referencing the array number, I was wondering if it is possible to reference the last item of the array in the above code?
I understand we can use 0 for the first record however I don't understand how to reference the final item. I've tried the following using the 'last' function but am unsure of how to place it:
$['meta']['sensorReadings'][*]['readings'][last]['dateTime']
$['meta']['sensorReadings'][*]['readings']['last']['dateTime']
last['meta']['sensorReadings'][*]['readings']['dateTime']
$['meta']['sensorReadings'][*]['readings']last['dateTime']
Any help or advice on a better way to proceed would be greatly appreciated.
Can you call your API with a Web activity? If so, this pulls the API result into the data pipeline and then apply ADF functions like last to it.
A simple example calling the UK Gov Bank Holidays API:
This returns a resultset that looks like this:
{
"england-and-wales": {
"division": "england-and-wales",
"events": [
{
"title": "New Year’s Day",
"date": "2017-01-02",
"notes": "Substitute day",
"bunting": true
},
{
"title": "Good Friday",
"date": "2017-04-14",
"notes": "",
"bunting": false
},
{
"title": "Easter Monday",
"date": "2017-04-17",
"notes": "",
"bunting": true
},
... etc
You can now apply the last function to is, e.g. using a Set Variable activity:
#string(last(activity('Web1').output['england-and-wales'].events))
Which yields the last bank holiday of 2023:
{
"name": "varWorking",
"value": "{\"title\":\"Boxing Day\",\"date\":\"2023-12-26\",\"notes\":\"\",\"bunting\":true}"
}
Or
#string(last(activity('Web1').output['england-and-wales'].events).date)

Microsoft Graph returns 400 when querying default photo

I'm querying the user's photo metadata and then use the ID from the metadata to query the actual photo:
https://graph.microsoft.com/v1.0/users/<user id>/photo returns
{
"#odata.context": "https://graph.microsoft.com/v1.0/$metadata#users('<user id>')/photo/$entity",
"#odata.mediaContentType": "image/jpeg",
"#odata.mediaEtag": "W/\"...\"",
"id": "default",
"height": 64,
"width": 64
}
Note that its ID is default. When I then try to get the content using https://graph.microsoft.com/v1.0/users/<user id>/photos/default/$value, graph returns HTTP 400 with
{
"error": {
"code": "ErrorInvalidImageId",
"message": "The image ID is not valid",
"innerError": {
"date": "2021-12-02T14:59:49",
"request-id": "...",
"client-request-id": "..."
}
}
}
If I replace default to 64x64 (the photo's dimensions from the metadata) in the above URL, everything's fine, i.e. https://graph.microsoft.com/v1.0/users/<user id>/photos/64x64/$value returns the photo.
For many users the ID returned from https://graph.microsoft.com/v1.0/users/<user id>/photo looks like 64x64 and this sort of IDs always work.
Why does Graph return default for some users and how should I handle it? Is it safe to create the an ID from width and height whenever I get default?
We may refer to the official document to find the reason for this scenario.
If we call the url /users/<user id>/photo to get the metadata of the photo, we will get profilePhoto as the response object which contains id, height, width and the id is made up of height and width.
Then let's see this feature of the api, it is designed for querying the metadata with a specific photo size, so if the id stands for the photo size, we can use id as the query parameter, but if the id is some other value, it will not work here.
My idea on your case is that the api is designed for using photo size as the query parameter and we may always use the height and the width as the query parameter all the time so that we don't need to check if it returns a correct id for querying.

Azure AD show group name in id token instead of group id

My id token has group (as role) ids only
"roles": [
"729b24b5-c527-440e-9ef6-81a04415e7ba",
"8d4f9343-10c3-43a2-9efe-34cfd740d020",
"81715416-9be4-43d7-807a-d5ccc9420cf7",
"1b5e6d7b-0ee0-4212-a5b9-cd5c3ca07a4a"
],
Even set to sAMAccountName
Any idea to return the group names instead?
If you are expecting group names in the claims of ID/Access/SAML token, unfortunately currently that is not supported due to some limitations. You would only have the object ids (guid) of the groups in the claim for AAD managed groups.
If you absolutely need group names for your purpose, consider a separate Graph API call to list group memberships of a user.
Also feel free to upvote on the feature request of group names in claims here.
Please refer to this similar question
I know this question is rather old already, but I experienced the same issue in summer 2021 still when trying to create a new .NET API project. Probably people stumbling across it in 2021 could make use of my solution:
I decided to create a NuGet package that resolves the group names using Microsoft Graph for applications using .NETs Microsoft.Identity.Web package (mostly ASP.NET Core applications).
Feel free to take a look into https://github.com/peterwurzinger/AuthOida if that's applicable to your use case.
This is relatively old, but there's answer to that.
You can translate your groups to names by adjusting your application manifest.
Go to your application manifest via Applications and SSO options, you should see there "Manifest option" it should return a JSON which you can modify.
The important bit is in the optionalClaims, you need to add to your groups.additionalProperties section cloud_displayname option like this:
...
"optionalClaims": {
"idToken": [
{
"name": "preferred_username",
"source": null,
"essential": false,
"additionalProperties": []
},
{
"name": "groups",
"source": null,
"essential": false,
"additionalProperties": [
"sam_account_name",
"emit_as_roles",
"cloud_displayname"
]
}
],
...

How to ensure all items in a collection match a filter in Azure Cognitive Search

I have Azure Cognitive Search running, and my index is working as expected.
We are trying to add a security filter into the search, based on the current users permissions.
The users permissions are coming to me in as IEnumerable, but I am currently selecting just a string[] and passing that into my filter, then do a string.join, which looks like this.
permission1, permission2, permission3, permission4
In our SQL database, we have a view that is where the index is getting it's data from. There is a column on the view called RequiredPermissions, it is a Collection(Edm.string) in the index, and the data looks like this.
[ 'permission1', 'permission2', 'permission3' ]
The requirement is that for a record to return in the results, a user's permissions must contain all of the RequiredPermissions for that record.
So if we have a user with the following permissions
permission1, permission3, permission5
And we have the following records
Id, SearchText, Type, Permissions
1, abc, User, [ 'permission1', 'permission2' ]
2, abc.pdf, Document, [ 'permission1' ]
3, abc, Thing, [ 'permission1', 'permission3' ]
4, abc, Stuff, [ 'permission3', 'permission4' ]
If the user searched for 'abc' and these four results would come back, I need to $filter results that do not have the proper permissions. So I would expect the following results
Id, Returned, Reason
1, no, the user does not have permission2
2, yes, the user has permission1 and nothing else is needed
3, yes, the user has both permission1 and permission3
4, no, the user does not have permission4
If I run the following filter, then I get back anything that has permission1 or permission3, which is not acceptable, since the user should not see items Id 1 or 4
RequiredPermissions/any(role: search.in(role, 'permission1, permission3', ','))
If I run this filter, then I get nothing back, everything is rejected, because no records have permission5, and the user has it
RequiredPermissions/all(role: not search.in(role, 'permission1, permission3', ','))
If I try to run the search using 'all' and without the 'not' I get the following error
RequiredPermissions/all(role: search.in(role, 'permission1, permission3', ','))
Invalid expression: Invalid lambda expression. Found a test for equality or inequality where the opposite was expected in a lambda expression that iterates over a field of type Collection(Edm.String). For 'any', please use expressions of the form 'x eq y' or 'search.in(...)'. For 'all', please use expressions of the form 'x ne y', 'not (x eq y)', or 'not search.in(...)'.\r\nParameter name: $filter
So it seems that I cannot use the 'not' with 'any', and I must use the 'not' with 'all'
What I wish for is a way to say that a user has all the permissions in their list that is in the RequiredPermissions column.
I am currently just working in Postman using the RestApi to solve this, but I will eventually move this into .Net.
Your scenario can't be implemented with Collection(Edm.String) due to the limitations on how all and any work on such collections (documented here).
Fortunately, there is an alternative. You can model permissions as a collection of complex types, which allows you to use all the way that you need to implement your permissions model. Here is a JSON example of how the field would be defined:
{
"name": "test",
"fields": [
{ "name": "Id", "type": "Edm.String", "key": true },
{ "name": "RequiredPermissions", "type": "Collection(Edm.ComplexType)", "fields": [{ "name": "Name", "type": "Edm.String" }] }
]
}
Here is a JSON example of what a document would look like with its permissions defined:
{ "#search.action": "upload", "Id": "1", "RequiredPermissions": [{"Name": "permission1"}, {"Name": "permission2"}] }
Here is how you could construct a filter that has the desired effect:
RequiredPermissions/all(perm: search.in(perm/Name, 'permission1,permission3,permission5'))
While this works, you are strongly advised to test the performance of this solution with a realistic set of data. Under the hood, all is executed as a negated any, and negated queries can sometimes perform poorly with the type of inverted indexes used by a search engine.
Also, please be aware that there is currently a limit on the number of elements in all complex collections across a document. This limit is currently 3000. So if RequiredPermissions were the only complex collection in your index, this means you could have at most 3000 permissions defined per document.

Simple request using googleapis to freebase

I would like to query freebase, to get some basic information about a celebrity.
For example, I would like to get the place of birth, and the gender of Madonna.
I achieved to get Madonna's friends by using:
https://www.googleapis.com/freebase/v1/mqlread?&query={"id":"/en/madonna","name":null,"type":"/base/popstra/celebrity","/base/popstra/celebrity/friendship":[{"participant":[]}]}
But, I'm understanding this request pretty badly. How can I change it to get the information I talked above ?
I was thinking about :
https://www.googleapis.com/freebase/v1/mqlread?&query={"id":"/en/madonna","name":null,"type":"/base/popstra/celebrity","/base/popstra/celebrity/gender":[{"gender":}], "/base/popstra/celebrity/place_of_birth":[{"place of birth":}]}
but it's not working ofc.
Here is how you build MQL queries for Freebase topics:
Find a sample topic that meets you query (ex. Madonna)
Go the the Query Editor and build a simple query for the ID of that topic:
[{
"id": "/en/madonna",
"name": null
}]​
Go to the sample topic page, click on "Edit this topic" and find the facts that you want to query for (ex. Date of birth)
You can see that Date of birth is part of the Person (/people/person) type.
Click on little wrench icon to the right of Person to go to the schema page for that type.
From there, you can see that the property that stores the date of birth is called /people/person/date_of_birth.
Add the properties that you want to your query like so:
[{
"id" "/en/madonna",
"name": null,
"/people/person/date_of_birth": null,
"/people/person/gender": null
}]​
Queries can also specify a default type (ex. /people/person) which makes the property paths shorter:
[{
"id" "/en/madonna",
"name": null,
"type": "/people/person",
"date_of_birth": null,
"gender": null
}]​
But each level of query can only have one default type so if you want to mix in properties from other types you'll need to use the full property IDs:
[{
"id": "/en/madonna",
"name": null,
"type": "/people/person",
"date_of_birth": null,
"gender": null,
"/base/popstra/celebrity/friendship": [{
"participant": []
}]
}]​
When you've got your query working in the query editor, just click the "Link" menu at the top right to get the MQLRead Link that will make that query to our API.

Resources