It's not very well documented anywhere. Specifically, if I want to detect http vs https, how should I do that?
Here you go, with some names/IP addresses changed. You can use X-Forwarded-Proto to detect http vs https in App Engine Flex.
"X-Appengine-Region": "ca",
"X-Cloud-Trace-Context": "8b8613a90a389a6c1e125b330ab20b24/6635756377403605329",
"X-Appengine-Country": "US",
"X-Forwarded-Proto": "http",
"X-Appengine-Citylatlong": "23.774929,-120.419416",
"X-Appengine-City": "city name",
"Via": "1.1 google",
"X-Forwarded-For": "1.2.3.4, 5.6.7.8"
Do these public docs meet your needs? https://cloud.google.com/appengine/docs/flexible/python/how-requests-are-handled
Related
My goal is to limit access to my App Engine Service to my home office IP. I have configured the App Engine Firewall with allow rules for both my IPv4 and IPv6 addresses, and set the default rule to deny.
This works when browsing my application using the unique appspot.com address assigned to my app. But attempting to access my application using the custom domain I have configured for App Engine, is resulting in a 403.
I have further verified that the rules are working as intended on the appspot.com domain. Anything that isn't in my allow list is getting the 403 as intended.
This tells me that my rules are "working," but I am unable to find any reference as to why this would not influence access to my application through the configured custom domain.
Note: when the default rule is set to allow, my application does work using the custom domain, so I am certain that configuration is sound.
Are custom domains simply beyond the scope of App Engine's Firewall? I was hoping to avoid digging into the VPC configuration for now.
Firewall Rules
Custom Domain Config
Edit: Log shows my IPv6 IP address as the requesting IP when tailing the log:
{
"entries": [
{
"insertId": "dlpqxpfa090t8",
"jsonPayload": {
"appLatencySeconds": "0.011",
"trace": "b7f63eb3d2fb4c52480253c224821a23",
"latencySeconds": "0.011"
},
"httpRequest": {
"requestMethod": "GET",
"requestUrl": "/users/kind/add",
"status": 200,
"responseSize": "4810",
"userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.5112.81 Safari/537.36",
"remoteIp": "2600:****:****:****:****:****:****:9936",
"referer": "https://f******s.e******t.com/users",
"latency": "0.011s",
"protocol": "HTTP/1.1"
},
"resource": {
"type": "gae_app",
"labels": {
"zone": "",
"project_id": "f*******s",
"version_id": "20220801t212517",
"module_id": "default"
}
},
"timestamp": "2022-08-09T22:11:33.869Z",
"labels": {
"appengine.googleapis.com/trace_id": "b*****************a23",
"appengine.googleapis.com/instance_name": "aef-default-2*********7-770v",
"compute.googleapis.com/resource_name": "0**********3",
"compute.googleapis.com/resource_id": "21*********29",
"compute.googleapis.com/zone": "********"
},
"logName": "projects/f********s/logs/appengine.googleapis.com%2Fnginx.request",
"trace": "projects/f*********s/traces/b7f63eb3d2fb4c52480253c224821a23",
"receiveTimestamp": "2022-08-09T22:11:38.104875464Z"
}
]
}
Edit 2: As suggested in the comments, I tried hitting a URL w/ curl. Below is the result:
Microsoft Windows [Version 10.0.22000.856]
(c) Microsoft Corporation. All rights reserved.
C:\Users\shawn>curl
curl: try 'curl --help' for more information
C:\Users\shawn>curl https://f*****s.e*******t.com/index
<html><head>
<meta http-equiv="content-type" content="text/html;charset=utf-8">
<title>403 Forbidden</title>
</head>
<body text=#000000 bgcolor=#ffffff>
<h1>Error: Forbidden</h1>
<h2>Access is forbidden.</h2>
<h2></h2>
<script defer src="https://static.cloudflareinsights.com/beacon.min.js/v652eace1692a40cfa3763df669d7439c1639079717194" integrity="sha512-Gi7xpJR8tSkrpF7aordPZQlW2DLtzUlZcumS8dMQjwDHEnw9I7ZLyiOj/6tZStRBGtGgN6ceN6cMH8z7etPGlw==" data-cf-beacon='{"rayId":"738c818088a17d62","version":"2022.6.0","r":1,"token":"c070c2d4c5ad48d18815371af21d9e80","si":100}' crossorigin="anonymous"></script>
</body></html>
C:\Users\shawn>
NOTE: I thought I was on to something with IPv6 being the culprit, but I've since disabled IPv6 completely and https://whatismyipaddress.com/ is showing that I'm not broadcasting an IPv6 address any longer. Still no dice.
Cloudflare Proxied CNAME strikes again. Turning off this feature in Cloudflare for the CNAME pointing at ghs.googlehosted.com resolved the issue after about 5 minutes.
My react code:
http-commons.js
export default axios.create({
baseURL: "http://three-tier-backend-svc.default.svc.cluster.local:5000/",
headers: {
"Content-type": "application/json"
}
});
service.js
create(data) {
return http.post("/add", data);
}
When I tried the same command with curl inside the container by using kubectl exec
curl -XPOST -H "Content-type: application/json" -d '{"name": "Mat", "price": 5000, "breed": "Lab"}' http://three-tier-backend-svc.default.svc.cluster.local:5000/add
I get the response:
{"name": "Mat", "price": 5000, "breed": "Lab"}
But the same request doesn't work in frontend in kubernetes
Response in chrome
welcome on Stack Overflow
I think the main issue here is, that your Frontend javascript code (Axios + React) is executed at client-side.
Thus calling external service (three-tier-backend-svc.default.svc.cluster.local) from web browser, using DNS names reserved for Kubernetes in-cluster communication, is failing.
I would suggest you to take a look at three alternative options now:
Expose your backend outside Kubernetes cluster with Service (LoadBalancer/NodePort types).
Expose you backend with Ingress
Consider changing your web app code to be server side rendered.
Running an App Engine Java 8 app with Google Cloud Endpoints. I've generated the openapi.json, deployed it to my Endpoints Portal, and can see the API in my portal, with the various methods and resources listed correctly.
I'm running the dev app server locally in IntelliJ using the Cloud Code plugin. When I run it, it opens a browser tab that gives me an Error 403, with the following stack trace (abbreviated):
SEVERE: direct send of a check request service_name: "my-project-redacted.appspot.com"
operation {
operation_id: "11b8f9a6-c9cb-4895-95fb-8bb39176efb9"
operation_name: "1.my_project_dot_appspot_com.MyAPI"
consumer_id: "project:my-project"
start_time {
seconds: 1596075164
nanos: 812000000
}
end_time {
seconds: 1596075164
nanos: 812000000
}
labels {
key: "servicecontrol.googleapis.com/caller_ip"
value: "127.0.0.1"
}
labels {
key: "servicecontrol.googleapis.com/user_agent"
value: "ESP"
}
labels {
key: "servicecontrol.googleapis.com/service_agent"
value: "EF_JAVA/1.0.12"
}
}
failed
endpoints.repackaged.com.google.api.client.http.HttpResponseException: 403
{
"error": {
"code": 403,
"message": "The caller does not have permission",
"errors": [
{
"message": "The caller does not have permission",
"domain": "global",
"reason": "forbidden"
}
],
"status": "PERMISSION_DENIED"
}
}
at endpoints.repackaged.com.google.api.client.googleapis.services.AbstractGoogleClientRequest.newExceptionOnError(AbstractGoogleClientRequest.java:456)
at endpoints.repackaged.com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:321)
at endpoints.repackaged.com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1065)
at endpoints.repackaged.com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:419)
at endpoints.repackaged.com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at endpoints.repackaged.com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.api.control.Client.check(Client.java:205)
It's worth noting that the API seems to work -- I have the iOS simulator connecting to my local dev app server and making Endpoints calls. I'm just tired of seeing the 403 in my browser every time I start the dev app server, and fear it may portend some similar issue in production when I eventually push this new service.
This error indicates that there is something wrong with the permissions or that the Service Control API is disabled in your project, so in order to fix it you can:
Make sure your service has access to servicecontrol.googleapis.com enabled by running the following command on Cloud Shell:
gcloud services enable servicecontrol.googleapis.com
Double check ENDPOINTS_SERVICE_NAME parameter in your appengine-web.xml file, it should look like this:
<env-var name="ENDPOINTS_SERVICE_NAME" value="$PROJECT"/>
Check if OpenAPI specs are deployed to Cloud, you can check it by running this command on Cloud Shell:
gcloud endpoints configs list --service=$PROJECT
Double check if the Service account running your instance has the proper IAM roles.
We've got an application hosted on a VM in Azure, which is behind a WAF that we've got a lot of trouble with for some users.
Some users are plagued by the HTTP Error 400. The size of the request headers is too long. The application is protected by Azure AD login.
The full repsonse to the browser is:
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN""http://www.w3.org/TR/html4/strict.dtd">
{
"data": "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01//EN\"\"http://www.w3.org/TR/html4/strict.dtd\">\r\n<HTML><HEAD><TITLE>Bad Request</TITLE>\r\n<META HTTP-EQUIV=\"Content-Type\" Content=\"text/html; charset=us-ascii\"></HEAD>\r\n<BODY><h2>Bad Request - Request Too Long</h2>\r\n<hr><p>HTTP Error 400. The size of the request headers is too long.</p>\r\n</BODY></HTML>\r\n",
"status": 400,
"config": {
"method": "GET",
"transformRequest": [
null
],
"transformResponse": [
null
],
"url": "api/datacontext/workbooks/876dac86e00e42878d9e239a8efb00a3/session/start",
"headers": {
"Accept": "application/json, text/plain, */*",
"x-invision-app-language": "EN",
"Authorization": "Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsIng1dCI6ImFQY3R3X29kdlJPb0VOZzNWb09sSWgydGlFcyIsImtpZCI6ImFQY3R3X29kdlJPb0VOZzNWb09sSWgydGlFcyJ9.eyJhdWQiOiI2MGMyZDcwNi02M2JmLTRhYzItOGQyZi01M2QzMzM1ZTAwMDIiLCJpc3MiOiJodHRwczovL3N0cy53aW5kb3dzLm5ldC80M2I0MWVlMy01MDllLTRmNDYtOTFmZi1hYWMxNGY5MjAwYTAvIiwiaWF0IjoxNTY5NTg0NTIyLCJuYmYiOjE1Njk1ODQ1MjIsImV4cCI6MTU2OTU4ODQyMiwiYWNyIjoiMSIsImFpbyI6IkFTUUEyLzhNQUFBQWtlMDNycUJycmJoZlNBdDB4SnY4bkF4RzA1V1VPdnd4SjZDUmdVeFdkeGc9IiwiYW1yIjpbInB3ZCJdLCJhcHBpZCI6IjYwYzJkNzA2LTYzYmYtNGFjMi04ZDJmLTUzZDMzMzVlMDAwMiIsImFwcGlkYWNyIjoiMSIsImZhbWlseV9uYW1lIjoiUmV0dmVkdCIsImdpdmVuX25hbWUiOiJFcmlrIiwiaXBhZGRyIjoiMTk0LjI0OC4xNDcuMTMiLCJuYW1lIjoiRXJpayBSZXR2ZWR0Iiwib2lkIjoiMGJjMDUyMDgtZDQ1YS00MTk4LTk3MzItMzZiN2U2MDJiYmYwIiwib25wcmVtX3NpZCI6IlMtMS01LTIxLTEzNDI3OTMzNzAtNDI1NDg1MTE1LTU5NDUyMDk5NS0xODU0Iiwic2NwIjoiRGlyZWN0b3J5LlJlYWQuQWxsIFVzZXIuUmVhZCBVc2VyLlJlYWQuQWxsIFVzZXIuUmVhZEJhc2ljLkFsbCIsInN1YiI6ImJ0M3pIOFNQcjRtMWFqenZJa2ZiMjBnVUZkR3BxblpEYmg3QWJ4M3B5N28iLCJ0aWQiOiI0M2I0MWVlMy01MDllLTRmNDYtOTFmZi1hYWMxNGY5MjAwYTAiLCJ1bmlxdWVfbmFtZSI6ImVyaWsucmV0dmVkdEBjaG9pY2Uubm8iLCJ1cG4iOiJlcmlrLnJldHZlZHRAY2hvaWNlLm5vIiwidXRpIjoiUERYS0FfSl9CRWlmc0tDLVJJbHhBQSIsInZlciI6IjEuMCJ9.aghrUBArpEvvvXBs2MBPTCL2nUPZ3aMCJ-1r3EqB5a9UaqaX7Ego5mSw1gb_68y3KhsGfO7kAv49uCB7cy80kEXV4ES4htLefQmmp-Bx-1Et_w6vstoki9ojWuKP97NsaGlQBjPYCZcbCRptBIZJIr_H71dMuFhAPWYEImcGtOrF2RNQA4AFvlx6WL2dIONHVPar3sjgLWEvFxhPFZsml3Ht3M1OtLj5drAJrkUjgxfV3-00bqCwYCm5_t_BAtxWsd-LZEpjDLpN7nDBFIJF14oFrPB7yXCBM_q-Y4FCCwGE14NoRcUrJNJPYMt5b0LKHEAbIopdq_zmFQ6XnUmcjg"
},
"withCredentials": true
},
"statusText": "Bad Request"
}
The error show up on different paths in the application and seemingly at random. It might work fine for a while then the user gets the error message again. We've narrowed the problem down to beeing the WAF as when we've changed the traffic to flow directly to the IIS webserver the users are seeing none of theese errors. It seems that there's nowhere to change the size limit for the request header that I can find in the WAF. Anyone got any idea as for where to start looking for a solution?
The Azure WAF is configured as follows:
Configure
Tier: WAF
Firewall Status: Enabled
Firewall mode: Detection
There are no exclusions configured.
Global Parameters:
Inspect request body: Off
Rules
Rule set: OWASP 3.0
Advanced rule configuration: Disabled
I've defined a pull queue for my app engine application and want to fill it from outside via the REST api.
Unfortunately it doesn't work, I always get an error 500: backend error.
Even if I try to fill it with the api explorer from the documentation.
It responses:
500 Internal Server Error
- Show headers -
{
"error": {
"errors": [
{
"domain": "global",
"reason": "backendError",
"message": "Backend Error"
}
],
"code": 500,
"message": "Backend Error"
}
}
Any idea how to solve it?
Can you paste your code that is doing the leasing? One common mistake is forgetting to include the s~ in your App Engine Project. For example, if your application ID is my-awesome-app, then you are calling
tq.get(project='my-awesome-app', taskqueue=PULL_QUEUE_NAME...
when you should be calling
tq.get(project='s~my-awesome-app', taskqueue=PULL_QUEUE_NAME...
(thanks to bossylobster for this original answer: pull queues authorization from compute)
Also it turns out you must supply the queueName in the JSON body, in addition to payloadBase64, as discovered by Frank Ren in this thread