Identity Server configuration endpoint invalid json? - identityserver4

Question

We have an issue with identity server configuration endpoint generating invalid JSON, I cant show too much but the screenshot below shows the call to the .well-known/openid-configuration endpoint . The one with 7ee gives us this error, another environment that works shows valid JSON.
"#t": "2022-08-24T08:59:41.1177158Z",
"#mt": "{msg} {#dt}",
"#l": "Error",
"msg": "Exception caught while processing request",
"dt": {
"StackTrace": " at Newtonsoft.Json.JsonTextReader.ParseReadNumber(ReadType readType, Char firstChar, Int32 initialPosition)\r\n at Newtonsoft.Json.JsonTextReader.ParseValue()\r\n at Newtonsoft.Json.JsonReader.ReadForType(JsonContract contract, Boolean hasConverter)\r\n at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent)\r\n at Newtonsoft.Json.JsonSerializer.DeserializeInternal(JsonReader reader, Type objectType)\r\n at Newtonsoft.Json.JsonConvert.DeserializeObject(String value, Type type, JsonSerializerSettings settings)\r\n at Newtonsoft.Json.JsonConvert.DeserializeObject[T](String value, JsonSerializerSettings settings)\r\n at Microsoft.IdentityModel.Protocols.OpenIdConnect.OpenIdConnectConfigurationRetriever.GetAsync(String address, IDocumentRetriever retriever, CancellationToken cancel)\r\n at Microsoft.IdentityModel.Protocols.ConfigurationManager`1.GetConfigurationAsync(CancellationToken cancel)",
"Details": "Input string '7ee' is not a valid number. Path '', line 1, position 3.",
"CallingMethod": "Invoke",
"$type": "ErrorLogDetails"
},
Has anyone experienced this before and help point me in the right direction, many thanks.

Answer 1

This was a hard one, and really outside of our domain, we have a F5 load balancer that didn't have the right cors policy, unsure how this caused the extra characters, but it did. Adding this to the load balancer policy for that site corrected the issue.
when HTTP_REQUEST priority 200 {
unset -nocomplain cors_origin
if { ( [HTTP::header Origin] contains "example.com" ) } {
if { ( [HTTP::method] equals "OPTIONS" ) and ( [HTTP::header exists "Access-Control-Request-Method"] ) } {
# CORS preflight request - return response immediately
HTTP::respond 200 "Access-Control-Allow-Origin" [HTTP::header "Origin"] \
"Access-Control-Allow-Methods" [HTTP::header "Access-Control-Request-Method"] \
"Access-Control-Allow-Headers" [HTTP::header "Access-Control-Request-Headers"] \
"Access-Control-Max-Age" "86400" \
"Access-Control-Allow-Credentials" "true"
} else {
# CORS GET/POST requests - set cors_origin variable
set cors_origin [HTTP::header "Origin"]
}
}
}
when HTTP_RESPONSE {
set cors_origin [HTTP::header "example.com"]
# CORS GET/POST response - check cors_origin variable set in request
if { [info exists cors_origin] } {
HTTP::header remove Access-Control-Allow-Origin
HTTP::header remove Access-Control-Allow-Credentials
HTTP::header remove Vary
HTTP::header insert "Access-Control-Allow-Origin" example.com
HTTP::header insert "Access-Control-Allow-Credentials" "true"
HTTP::header insert "Vary" "Origin"
}
}