App Engine python runtime, latest SDK ... using urlfetch to request over https. No matter what value I use for verify_certificate (True, False, None), I get back the same response from an internal site with a fake certificate,
Invalid and/or missing SSL certificate for URL:
The documentation suggests that setting verify_certificate to False should bypass this check. But again, I get the same exception no matter what I use. I checked the bug database but did not find anything.
We are using a made up certificate because we want the data encrypted over the wire, but we can trust the endpoint as it is internal.
Thanks for any thoughts.
Verify that your self made certificate is valid and installed correctly by accessing the endpoint using other software such as curl, wget or a web browser. You might have already done that, but the question does not (yet) say so.
If other software can access the endpoint but the url fetch service still cannot, then please report the problem with a link to your question in the AppEngine Issue Tracker. Thank you.
Related
I'm new to IdentityServer4 (2.5) and certificate setup so please bear with me. I think that I've chased down everything I could. I am using it with ASP.Net Core 2.2.0 in a proof of concept app. I have OpenIdConnect with an authority app and a client using cookies with X509Certificate2. Works great on my local machine; however, when I deploy to IIS I get this error:
System.InvalidOperationException: IDX20803: Unable to obtain configuration from: 'https://my.com/mpauth/.well-known/openid-configuration'. ---> System.ArgumentException: IDX20108: The address specified 'http://my.com/mpauth/.well-known/openid-configuration/jwks' is not valid as per HTTPS scheme. Please specify an https address for security reasons. If you want to test with http address, set the RequireHttps property on IDocumentRetriever to false.
The problem is here - http://my.com/mpauth/.well-known/openid-configuration/jwks. If I put that in the browser I get an error; however, if I change http to https I get the data. What setting controls this?
TL;DR
In most cases IdentityServer defers the base hostname/URI from the incoming request but there might be deployment scenarios which require enforcing it via the IssuerUri and/or PublicOrigin options as documented here.
More Info
The URL you are getting in your exception is part of the discovery lookup. It is necessary for validating tokens (e.g. in an applications auth middleware).
There should be a first request to .../.well-known/openid-configuration (the main discovery document) that refers to several other URIs and one of them should be the jwks (signing key sets). In most cases the other URIs in openid-configuration will point to the same primary hostname and protocol scheme your identity server is using. In your case it looks like the scheme changes to HTTP which might be unwanted in this day and age.
Is it possible, that the deployed IdentityServer lives behind a load balancer/SSL termination appliance? This could cause behavior.
I am not sure about IIS details but there might also be some kind of default hostname/URI thing at play.
I'm trying to write a VSCode extension where users could log into Google AppEngine with a google account, and I need to get their SACSID cookie to make appengine requests.
So I'm opening a browser window at
https://accounts.google.com/ServiceLogin?service=ah&passive=true&continue=https://appengine.google.com/_ah/conflogin%3Fcontinue%3Dhttp://localhost:3000/
(generated by google.appengine.api.users.create_login_url)
The user logs in and is redirected to my local webserver at
localhost:3000/_ah/conflogin/?state={state}
Now I try to forward the request to my AppEngine app (since it knows how to decode the state parameter), so I do a request to
https://my-app.appspot.com/_ah/conflogin/?state={state}
basically just replacing localhost with the actual app.
but it doesn't work, presumably because the domain is different. I assume this is on purpose, for security.
Is there any way I can make this work ?
Not ideal, but the only solution I've found is to have an endpoint on my GAE instance that does the redirection. Then I can set that as the continue url, when I'm starting the authentication process
https://accounts.google.com/ServiceLogin?service=ah&passive=true&continue=https://appengine.google.com/_ah/conflogin%3Fcontinue%3Dhttps://my-app.appspot.com/redirect?to=http://localhost:3000
I think you should center the attention on the protocols you are using, since it’s known that the cookie name is based on the http protocol (HTTP : ACSID, HTTPS:SACSID), and that’s the security perspective till this point for me.
Having the error you are facing now would be helpful to understand the problem better. Also, how are you performing the call to the API and the code you are using would be helpful too.
UPDATE: Please, if anyone can help: Google is waiting for inputs and examples of this problem on their bug tracking tool. If you have reproducible steps for this issue, please share them on: https://code.google.com/p/googleappengine/issues/detail?id=10937
I'm trying to fetch data from the StackExchange API using a Google App Engine backend. As you may know, some of StackExchange's APIs are site-specific, requiring developers to run queries against every site the user is registered in.
So, here's my backend code for fetching timeline data from these sites. The feed_info_site variable holds the StackExchange site name (such as 'security', 'serverfault', etc.).
data = json.loads(urllib.urlopen("%sme/timeline?%s" %
(self.API_BASE_URL, urllib.urlencode({"pagesize": 100,
"fromdate": se_since_timestamp, "filter": "!9WWBR
(nmw", "site": feed_info_site, "access_token":
decrypt(self.API_ACCESS_TOKEN_SECRET, self.access_token), "key":
self.API_APP_KEY}))).read())
for item in data['items']:
... # code for parsing timeline items
When running this query on all sites except Stack Overflow, everything works OK. What's weird is, when the feed_info_site variable is set to 'stackoverflow', I get the following error from Google App Engine:
HTTPException: Invalid and/or missing SSL certificate for URL:
https://api.stackexchange.com/2.2/me/timeline?
filter=%219WWBR%28nmw&access_token=
<ACCESS_TOKEN_REMOVED>&fromdate=1&pagesize=100&key=
<API_KEY_REMOVED>&site=stackoverflow
Of course, if I run the same query in Safari, I get the JSON results I'm expecting from the API. So the problem really lies in Google's URLfetch service. I found several topics here on Stack Overflow related to similar HTTPS/SSL exceptions, but no accepted answer solved my problems. I tried removing cacerts.txt files. I also tried making the call with validate_certificate=False, with no success.
I think the problem is not strictly related to HTTPS/SSL. If so, how would you explain that changing a single API parameter would make the request to fail?
Wait for the next update to the app engine (scheduled one soon) then update.
Replace browserid.org/verify with another service (verifier.loogin.persona.org/verify is a good service hosted by Mozilla what could be used)
Make sure cacerts.txt doesnt exist (looks like you have sorted but just in-case :-) )
Attempt again
Good luck!
-Brendan
I was facing the same error, google has updated the app engine now, error resolved, please check the updated docs.
I'm doing https web requests in silverlight using "WebRequest"/"WebResponse" framework classes.
Problem is: I do a request to an url like: https://12.34.56.78
I receive back a versign signed certificate which has as subject a domain name like: www.mydomain.com.
Hence this results in a remote certificate mismatch error.
First question: Can I somehow accept the invalid certificate, and get the WebBresponse content ? (even if it involves using other libraries, I'm open to it)
Additional details: (for those interested on why I need this scenario)
I'm trying to give a client access to a silverlight app deployed on a test server.
Client accesses the silverlight app at: www.mydomain.com/app
Then I do some rest requests to: https://xx.mydomain.com
Problem is I don't want to do requests on https://xx.mydomain.com, since that is on our productive server. For this reason I use https://12.34.56.78 instead of https://xx.mydomain.com.
Client has some firewalls/proxies and if I simply change his hosts file and map https://xx.mydomain.com to 12.34.56.78, web requests don't resolve to the mapped IP.
I say this because on his network webrequests fail if I try that, on my network I can use the hosts changing without problems.
UPDATE: Fixed the problem by deploying test releases to an alternative: https://yy.domain.com and allowing the user to configure for test purposes, the base url to which I do requests to be: https://yy.domain.com.
Using an certificate that contained the IP in the subject or an alternative subject would've probably worked too, but would have cost some money to be issued by a certified provider and would not be so good because IP's might change.
After doing more research looks like Microsoft won't add this feature too soon, unless there's a scenario for non-testing/debugging uses.
See: http://connect.microsoft.com/VisualStudio/feedback/details/368047/add-system-net-servicepointmanager-servercertificatevalidationcallback-property
I'm working on a Windows Phone 7 application that makes a REST service call. The third party that hosts the web services has an invalid certificate in the current environment. When I hit the URL in Firefox, I get a warning about the cert and I am asked if I want to continue. I'm also using the Poster FF extension to test the call. It works with Poster if I first accept the invalid cert in Firefox. If I don't, then POSTER wont make the request.
In my WP7 Emulator, I can't make the request at all. I get a 404 at the EndGetResponse method. I making the same request as in Poster, so I know there is nothing wrong with the request. I have successfully hit another web service using the same code (no certs involved), so I don't think it's the code. The only thing I can think of is that WP7 doesn't allow requests to an invalid cert. Has anyone had experience with this situation? Is there any way around it?
Is there a way I can tell my app to accept all communication, even if there is an invalid cert?
There is sadly no way to do this on the phone. Ordinarily, i.e. on the desktop this simple line of code will disable certificate checking.
System.Net.ServicePointManager.ServerCertificateValidationCallback = (se, cert, chain, sslError) => { return true; };
If you look at the ServicePointManager on the phone, there's no callback to hook into. It's a massive pain in the arrrrse.
Have you considered writing to the service owner and asking why they're being bad internet citizens? (essentially, what you're seeing here is web security in action, for better or worse)
As Matt says, you might be able to code a simple relay on a web server. It doesn't have to be a special service, but maybe just a web page that does the call for you and spits out RAW text or XML. Your phone client just GETs this page and picks through the response manually.
Where there's a will there's a way.
Luke
You need to install the root CA cert of the issuing party on the phone.
You can do this by emailing the RootCA to the user of the phone. They click on the attachement and it will prompt them to ask if they want to install the certificate on the phone.
Once you have done that your requests should go through.
I dont believe there is a way to do this programatically in your app however.
I'm not aware of a way to install additional certificates on the phone.
In this situation I'd create a proxy service between your app and the 3rd party site and have your app call that. If you need to, you could put the proxy behind a valid cert.