Is it possible to make configurations in Google app engine for not recording IP addresses from clients to the Java application in stackdriver logs?
No, you cannot configure or opt-out from request logs, where the client IP addresses are recorded. From Using Stackdriver Logging in App Engine apps:
The App Engine standard environment produces the following logs:
Request logs, appengine.googleapis.com/request_log, called request_log in the Logs Viewer. This log records requests sent to
all App Engine apps. The request log is provided by default and you
cannot opt out of receiving it. For more details, see the RequestLog
type.
...
The App Engine flexible environment produces the following logs:
Request logs record requests sent to all App Engine apps. The request log is provided by default and you cannot opt out of receiving
it.
Related
I'm trying to configure my Google App Engine instance with Cloudflare for Saas, and more precisely Cloudflare's SSL for SaaS offering. The objective being that I can provide to my customer a "custom domain" (also known as "vanity domain"), such that they don't go to dashboard.mywebsite.com, but instead app.customerwebsite.com.
Configuration part
To make sure that my App Engine instance is correctly serving content on dashboard.mywebsite.com, I've made the following:
On Google Cloud side:
I've configured the custom domain dashboard.mywebsite.com.
I've let Google manage the SSL configuration (no custom key/certificate)
Here is my app.yaml configuration file:
runtime: nodejs14
env_variables:
NODE_ENV: 'production'
basic_scaling:
max_instances: 10
idle_timeout: 5m
On Cloudflare side:
I've updated the DNS records so that dashboard.mywebsite.com is perfectly working
I've configured the SSL on the Full mode (while I've tried with Flexible as well - both work)
I waited for a few hours and I confirm that dashboard.mywebsite.com resolves correctly and serves my content (from Google App Engine).
Next, custom domains
According to Cloudflare documentation, I had to register the fallback origin (i.e. dashboard.website.com) and then configure a custom hostname (e.g. app.customerwebsite.com). Which I did.
Now, according to Cloudflare documentation again, my customer has to create a CNAME record. Which I did with a domain of mine:
app.customerwebsite.com CNAME dashboard.mycompany.com
The issue
I waited a few hours again. Then, when I open app.customerwebsite.com in my browser, it shows a Google 404 error page instead of my dashboard. Which makes me think that Cloudflare successfully "redirects" the traffic to Google, but App Engine refuses to serve it. Probably because it doesn't know app.customerwebsite.com?
Any thoughts that would help?
As you noticed, the issue is not related to Cloudflare, but App Engine. The problem with your configuration is that, when App Engine receives a request, based on the Host header, it forwards the request to the right instance.
App Engine lets you map any custom domains that has been previously validated by Google. But in your situation, that would mean you have to register each custom domain of your customers on your App Engine instance. That's too cumbersome (if even possible).
What you need to do instead is the following:
enable a static IP address with Google Cloud
change your DNS record from dashboard CNAME ghs.googlehosted.com to dashboard A YOUR_IP_ADDRESS
configure a Google Cloud Load Balancer to map requests received on that IP address to your App Engine instance.
Google's documentation has a great guide on how to setup a load balancer with Cloud Run. By changing a few settings it works great with App Engine. As an extra help, below is the configuration details of our load balancer that allows us to provide vanity domains / custom domains to our customers through Google Cloud:
Again, the load balancer is here responsible to map all requests received by your IP address (no matter the Host header) straight to your App Engine instance.
As a best practice, it might be useful to push a dispatch.yaml file to your instance:
dispatch:
- url: '*/*'
service: default
Which tells App Engine to send all requests to the default service. It works a bit like a wildcard virtual hosts on an Apache server.
I created a Google Pub/Sub push subscription which points to a Google App Engine endpoint, but GAE is not receiving anything.
The messages are being published (I can see it on the Pub/Sub console), but never acknowledged.
The endpoint URL should be correct since I made my GAE endpoint public and tested using Postman.
Does Google Pub/Sub not support calling Google App Engine endpoints?
Or I'm I missing something here?
Appreciate it someone could help with this.
For a message to be acknowledged, it means it has to be received by the subscriber. How are you trying to do this? I ask because GAE (at least GAE standard) doesn't support streaming which means you can't have an indefinitely open connection listening for messages from pubsub.
Yes Google Pub/Sub supports calling to Google App Engine endpoints. I referred to this documentation.
As per the document, I used App Engine Flex Environment with Pub/Sub push subscription and I am able to fetch the messages that were published from Pub/Sub to the endpoint.
You can refer to the below mentioned steps :
Create one application using App Engine Flex Environment by referring to this document.
Deploy the application to GCP .
The files needed for application deployment are app.yaml, requirements.txt, main.py and index.html file inside a template folder.
Folder structure:
Demo→
app.yaml
requirements.txt
main.py
templates→
index.html
Provide the Pub/Sub topic name and token id ( should be the same as Pub/Sub push endpoint token id ) in the app.yaml file.
env_variables:
PUBSUB_TOPIC: your-topic
PUBSUB_VERIFICATION_TOKEN: 1234abcd
Run gcloud app deploy on the demo directory.
After the application is deployed, you will get an endpoint URL with the format : https://PROJECT_ID.REGION_ID.r.appspot.com
Example: https://mydemoproject.uc.r.appspot.com
Create a Pub/Sub topic and subscription and make the delivery type as push.
We need to provide a push endpoint with the format : https://mydemoproject.uc.r.appspot.com/pubsub/push?token=1234abcd
When an application is deployed in App Engine we can access that application at - https://PROJECT_ID.REGION_ID.r.appspot.com
The messages will be received by any of the instances of your application deployed in App Engine.
SSH into your instances and use the App Engine endpoint URL in the VM.
Output of instance 1:
Output of instance 2:
my website is hosted with google cloud app engine. I have an increase request amount from the same user agent. Is there a possibility to block specific incoming requests with app engine, e.g. through the app.yaml file? I don't find anything related to that in the documentation.
Thanks in advance
The Nginx load balancer of Google Cloud App Engine Flexible Environment (with custom runtime) logs remote IP addresses etc. to its access logs. What if I wanted to disable that behavior? Is it possible in any way? Docs don't say anything about configuring Nginx by yourself. Haven't find anything from the console either.
In order to disable any IP logging, you have to disable the Google Cloud Load Balancer logs and also any logging done by NGINX.
In order to disable the ones done by the Load Balancer of Google Cloud, this page of their official documentation explains how to do it. You would need select "Disable log source" from the cloud HTTP Load Balancer.
On the other hand, to configure NGINX, apparently the only way to do so is modifying the nginx.conf file.
Because this you are using App Engine Flex and it's not possible to access these virtual machines/containers once the App had been deployed, you would need make sure to change the configuration before executing the deployment.
I've found a Serverfault post which does explain how to disable NGINX logging, and another article which does explain the NGINX configurations with better depth in case they might be of use for you.
I believe this won't be possible. As per this Logging docs page:
The App Engine flexible environment produces the following logs:
Request logs record requests sent to all App Engine apps. The request
log is provided by default and you cannot opt out of receiving it.
App logs record activity by software within the App Engine app. The
log is provided by default and you cannot opt out of receiving it.
Runtime logs are provided from the flexible environment using a
preinstalled Logging agent.
I have created google app engine application. It works fine in my local system and also deployee successfully on server. but that url http://****.appspot.com/_ah/api/employeeendpoint/v1/employee/1 gives me below response.
{"error":{"errors":[{"domain":"global","reason":"sslRequired","message":"SSL is required to perform this operation."}],"code":403,"message":"SSL is required to perform this operation."}}
I have also try with https://****.appspot.com/_ah/api/employeeendpoint/v1/employee/1 but still not working.
Certain Google Cloud Platform products require SSL, you can simply enable SSL from your Cloud Console and it should work.
The documentation for App Engine, for Compute Engine, and for Cloud SQL.