I have a GCP App Engine running that serves a bunch of GET endpoints with a Cache-Control: public, max-age=86400 header. I notice that often, the responses are served from Google Edge Network and logged with a 204 code.
My question is, do these responses still count towards the Out Bandwith quota, or are they treated differently?
The Google Frontend will cache content when Cache-Control is set to public.
Traffic delivered by Google services counts to your network egress quotas and costs. An HTTP 204 response is an egress network activity.
Related
I have an API running in Google App Engine and I want to introspect HTTP POST body to reject poorly formed calls before it hits my API running in app engine. Is there a way to front an App Engine application whereby I can introspect the URI, the POST body and return a 400 without it actually being processed by my code running in App Engine? Please let me know, thanks.
If you want to inspect HTTP, then you will need to write a proxy application to do so. Neither the Google Cloud Load Balancer nor App Engine support hooking HTTP requests.
Proxies have the MITM (Man In The Middle) problem in that HTTP traffic is often encrypted. You will not be able to inspect that traffic unless your proxy is the endpoint. If you implement your proxy with Apache or Nginx then you can use the ModSecurity project.
Basically, you want to implement a WAF. Google offers Cloud Armor which supports inspecting traffic via Cloud Armor rules.
I'm trying to configure my Google App Engine instance with Cloudflare for Saas, and more precisely Cloudflare's SSL for SaaS offering. The objective being that I can provide to my customer a "custom domain" (also known as "vanity domain"), such that they don't go to dashboard.mywebsite.com, but instead app.customerwebsite.com.
Configuration part
To make sure that my App Engine instance is correctly serving content on dashboard.mywebsite.com, I've made the following:
On Google Cloud side:
I've configured the custom domain dashboard.mywebsite.com.
I've let Google manage the SSL configuration (no custom key/certificate)
Here is my app.yaml configuration file:
runtime: nodejs14
env_variables:
NODE_ENV: 'production'
basic_scaling:
max_instances: 10
idle_timeout: 5m
On Cloudflare side:
I've updated the DNS records so that dashboard.mywebsite.com is perfectly working
I've configured the SSL on the Full mode (while I've tried with Flexible as well - both work)
I waited for a few hours and I confirm that dashboard.mywebsite.com resolves correctly and serves my content (from Google App Engine).
Next, custom domains
According to Cloudflare documentation, I had to register the fallback origin (i.e. dashboard.website.com) and then configure a custom hostname (e.g. app.customerwebsite.com). Which I did.
Now, according to Cloudflare documentation again, my customer has to create a CNAME record. Which I did with a domain of mine:
app.customerwebsite.com CNAME dashboard.mycompany.com
The issue
I waited a few hours again. Then, when I open app.customerwebsite.com in my browser, it shows a Google 404 error page instead of my dashboard. Which makes me think that Cloudflare successfully "redirects" the traffic to Google, but App Engine refuses to serve it. Probably because it doesn't know app.customerwebsite.com?
Any thoughts that would help?
As you noticed, the issue is not related to Cloudflare, but App Engine. The problem with your configuration is that, when App Engine receives a request, based on the Host header, it forwards the request to the right instance.
App Engine lets you map any custom domains that has been previously validated by Google. But in your situation, that would mean you have to register each custom domain of your customers on your App Engine instance. That's too cumbersome (if even possible).
What you need to do instead is the following:
enable a static IP address with Google Cloud
change your DNS record from dashboard CNAME ghs.googlehosted.com to dashboard A YOUR_IP_ADDRESS
configure a Google Cloud Load Balancer to map requests received on that IP address to your App Engine instance.
Google's documentation has a great guide on how to setup a load balancer with Cloud Run. By changing a few settings it works great with App Engine. As an extra help, below is the configuration details of our load balancer that allows us to provide vanity domains / custom domains to our customers through Google Cloud:
Again, the load balancer is here responsible to map all requests received by your IP address (no matter the Host header) straight to your App Engine instance.
As a best practice, it might be useful to push a dispatch.yaml file to your instance:
dispatch:
- url: '*/*'
service: default
Which tells App Engine to send all requests to the default service. It works a bit like a wildcard virtual hosts on an Apache server.
I'm working on an project on GAE which contains several high traffic endpoints and one endpoint that has low traffic but need authentication. My question is, can I enable Cloud Endpoints on the low traffic endpoint for authentication, but not on those high traffic endpoints, because Endpoints is expensive?
My understanding of how Cloud Endpoints work with GAE is that once the swagger file is deployed, and you specify a Endpoints version in app.yaml file, all traffic that goes through the Endpoints proxy server before reaching the application server. Also because Endpoints is charged based on # of requests, so in my case the entire application will take tens of millions request per day, but only a tiny fraction of that request will goes to a specific endpoint in the application that requires authentication.
In order to avoid spending thousands of dollars per month on Endpoint, It would be way cheaper if I can only let those privileged request go through Cloud Endpoints, and be charged only on those request.
You can have only one Endpoint (thus only one charged Endpoint pricing), if you separate it from the other URLs. You can do that by separating the Services or Projects
App Engine Standard supports automatic gzip compression of responses if the client has the correct Accept-Encoding and User-Agent headers set on the request. Info on this can be found here and here.
I am running a project in the App Engine Flexible Beta and it does not auto compress the responses. Is auto response compression present on the Flexible Beta? If not, what is the recommended approach to compress responses?
Presently, App Engine Flexible Beta will not compress responses. Most web servers can be configured to compress responses. Which web server are you using?
i'm using https on appengine. i checked not all request are using https spdy in firefox. Is there any thing that need to configure so that all requests are using spdy ?
?
As long as the file is served over https and by appengine servers, then it will be served over SPDY -- if the browser supports SPDY that is. Can't tell from your screenshot.. but if any of those files come from a different origin (ex, a third party widget), then those files may be served over plain HTTPS if they don't support SPDY.