Struggling with optional claims in id/access token - azure-active-directory

Question

I am updating an internally developed single-page app (Typescript/React) that uses OAuth2 from AD-FS 2016 to Azure AD v2. Things are complicated slightly by the fact that I (the developer) don't have direct access to the Azure console and am working on this with a (non-developer) sysadmin who does.
I have implemented PKCE and got the flow working; I can now obtain JWT access, ID and refresh tokens from the server and authenticate them via JWKS. So far so good.
Now, my apps to know a couple more things:
whether or not the user should be treated as an administrator. This is inferred from group memberships
the preferred username and first name/surname of the user
The first of these we dealt with by setting up a "role" and mapping it out to groups in the Azure console. We then added the role claim to the tokens. I can find this as a string array in "id_token". No problem.
I was confused for a while because I was looking for it in "access_token", but it's not a problem for my app to use "id_token" instead.
The second is the thing that is really giving us problems. No matter what we put into the "optional claims" dialog - we've added all these fields and more, for the ID token, they do not appear in it. Nothing we are doing seems to affect the actual tokens that come out at all.
I am beginning to think that I have missed something out with regards to obtaining the information. I am using the https://graph.microsoft.com/profile, https://graph.microsoft.com/email and https://graph.microsoft.com/user.read scopes and the administrator has authorized these on behalf of the app. The user is synced from our in-house active directory, which the AD-FS is running from as well, so I know that this information is in there. I tried messing with the resource parameter but this is deprecated in Azure AD v2 apparently.
I've read and re-read https://learn.microsoft.com/en-us/azure/active-directory/develop/active-directory-optional-claims along with other online documentation, and the following passage confuses me and makes me think that the issue might be related to scopes:
Access tokens are always generated using the manifest of the resource, not the client. So in the request ...scope=https://graph.microsoft.com/user.read... the resource is the Microsoft Graph API. Thus, the access token is created using the Microsoft Graph API manifest, not the client's manifest. Changing the manifest for your application will never cause tokens for the Microsoft Graph API to look different. In order to validate that your accessToken changes are in effect, request a token for your application, not another app.
Or is that just the reason that I switched to using the id_token?
The optional_claims section of the configuration manifest looks like this:
"optionalClaims": {
"idToken": [
{
"name": "email",
"source": null,
"essential": false,
"additionalProperties": []
},
{
"name": "upn",
"source": null,
"essential": false,
"additionalProperties": []
},
{
"name": "groups",
"source": null,
"essential": false,
"additionalProperties": []
},
{
"name": "family_name",
"source": null,
"essential": false,
"additionalProperties": []
},
{
"name": "given_name",
"source": null,
"essential": false,
"additionalProperties": []
},
{
"name": "preferred_username",
"source": null,
"essential": false,
"additionalProperties": []
}
],
"accessToken": [
{
"name": "email",
"source": null,
"essential": false,
"additionalProperties": []
},
{
"name": "groups",
"source": null,
"essential": false,
"additionalProperties": []
},
{
"name": "preferred_username",
"source": null,
"essential": false,
"additionalProperties": []
}
],
"saml2Token": [
{
"name": "groups",
"source": null,
"essential": false,
"additionalProperties": []
}
]
},
But the resulting payload in the ID tag looks like this:
{
"aud": "redacted",
"iss": "https://login.microsoftonline.com/redacted/v2.0",
"iat": 1654770319,
"nbf": 1654770319,
"exp": 1654774219,
"email": "redacted",
"groups": [
"redacted",
"redacted",
"redacted",
"redacted"
],
"rh": "redacted",
"roles": [
"redacted"
],
"sub": "redacted",
"tid": "redacted",
"uti": "redacted",
"ver": "2.0"
}
Can anyone who has more experience of the platform help me understand what we are doing wrong here? Do we need to define custom scopes? Have we simply forgotten to turn an option on?
All help gratefully received! Thanks in advance...

Answer 1

I tried to reproduce the same in my environment and got below results:
I have implemented PKCE flow and got JWT access, ID and refresh tokens.
I added optional claims like below:
Go to Azure Portal -> Azure Active Directory -> App Registrations -> Your App -> Token Configuration
Please check the scopes you are using to get token.
When I gave only openid as scope, got response like below:
But when I gave scope as openid profile email user.read, got all optional claims successfully like below: