I am trying to send email using client's on-prem SMTP server using app engine standard. For this we have created Serverless VPC access connector in default network and Cloud NAT with static ip address to send egress traffic. Client has whitelisted static ip address and port. Following is code snippet in app engine
msg.set_content('This is a HTML email')
msg.add_alternative(cleared_html_content, subtype='html')
try:
context = ssl._create_unverified_context()
print("starting conectn")
with smtplib.SMTP('xx.xxxx.edu', 2525) as server:
server.starttls(context=context)
server.send_message(msg)
print("sent almost")
except Exception as e:
print('Error: ', e)
Following is app.yaml
runtime: python37
entrypoint: gunicorn -t 120 -b :$PORT main:app
vpc_access_connector:
name: projects/xxxxxxxxx/locations/us-central1/connectors/yyyyyyyyy
When i run my app using app engine url, I am getting following error in logs viewer
Error: (554, b"xxx.xxxxx.edu\nYour access to this mail system has been rejected due to the sending MTA's poor reputation. If you believe that this failure is in error, please contact the intended recipient via alternate means."
Also i have created cloud function with same code as in app engine to test and surprisingly email was sent to intended recepient with out any issue. When i checked cloud NAT logs, it has all details when triggered via cloud function (in short it is using static ip address) but there are no logs related to app engine trigger. So i think my app engine traffic is not going via static ip address and not sure how to mention that in app.yaml
There might be code issue in email function as well but since it is working in cloud function, i really doubt about my app.yaml and not email python code. Any help is really appreciated
I understood that your SMTP IP was public. There is a caveat to know with serverless VPC connector.
With Cloud Function, and Cloud Run, you have the capacity to choose if only private IP or Public and Private IP are routed through the serverless VPC Connector
With app engine, I didn't find a clear description of the egress control, but I guess that only private IP (RFC1918) are routed through the VPC, and not the public one. And so, your Cloud Nat isn't used and thus you aren't authorised on the SMTP server of your school.
Edit 1:
You have 3 solutions to solve this
You can create a Cloud Functions (or a Cloud Run service) that your App Engine calls when you need to send an email.
You can switch from App Engine to Cloud Run (use the new beta command gcloud beta run deploy --source=. --region=<REGION> --platform=managed <Service Name>). Like this, you can deploy as with App Engine. The same Container engine builder as App Engine is used (Buildpack). You have to adapt the content of the app.yaml file (share it if you need help). However, up to now, IAP isn't compliant with Cloud Run. If you want to use it, wait!
Create a VPN between your VPC and your school network. Like this, you will call your SMTP server with a private IP. On the smtp server, grant only the serverless VPC connector range to access it. And you no longer need a Cloud NAT configuration.
Related
We have two separate projects, project A: has a Cloud SQL instance (has both public IP and private IP)
Now we have a new project B: It's a App Engine standard Java11 environment. For this project we have a requirement where we need to make a REST API call to a third party api, and it has to come from a specific GEO location. So we have setup on provisioning a static IP address explicitly in the geography where our app engine service is provisioned. We followed this documentation: Set up a static outbound IP address
When we deploy the App Engine after those changes, it is unable to reach the CLoudSQL anymore from project A, however when we remove from app.yaml the egress_setting: all-traffic and re-deploy, it connects to the Cloud Sql successfully but then we are unable to call our REST Api call, as third party api returns back 403 because of our IP address location.
Note that both Project A and Project B are in the same region.
Our app.yaml file looks like this:
`
runtime: java11
instance_class: F2
env_variables:
GAE_USE_SOCKETS_HTTPLIB : ''
vpc_access_connector:
name: projects/<projectB-id>/locations/northamerica-northeast1/connectors/cb-connector
egress_setting: all-traffic
This is the connection url used for Cloud SQL (MySql),
We have tried with the cloud SQL private IP as follow
spring:
datasource:
url: jdbc:mysql://<cloudSql-PrivateIp>:3306/_operations?user=<user>&password=<pass>&ipTypes=PRIVATE
Then we have also tried with cloud SQL public IP as follow:
spring:
datasource:
url: jdbc:mysql://_operations?cloudSqlInstance=<projectA-id>northamerica-northeast1:<sql-instanceName>&socketFactory=com.google.cloud.sql.mysql.SocketFactory&user=<user>&password=<pass>
We have followed these instructions here to connect from App-Engine to CloudSql: Connect from App Engine standard environment We tried with both option Public Ip and Private IP, but we kept on getting a Communication Link error. It seems to timeout after a while loading. That is when the egress_setting: all-traffic is set in app.yaml. If we remove that egress_setting: all-traffic, then connection works perfectly but we are unable to make our REST call to that third party api we need.
Not sure how we can set this up so we can still have connection to both the cloud Sql instance (project A) and still be able to reach our third party api from App engine (project B) using a static outbound IP. If someone can share some insight on what possible cause or solution that we can do?
EDIT
We have also tried to make a peering vpc between project A and project B, but still not working when egress_setting: all-traffic is set in app.yaml file.
Finally the issue was with the IP range that was configured in my VPC. The static egress IP range did not include the IP address of my Cloud SQL instance. After adding the appropriate IP range, now everything is working.
I have a python (flask) application running on Google App Engine (flex); the application is protected by the GAE firewall where:
Default rule is 'Deny' all ingress
There is a whitelist of IP addresses from which traffic is allowed.
I have some microservices deployed on Cloud Run (fully managed) which:
Receive requests from the GAE app (e.g. for heavy duty tasks)
Send the results of whatever they process as http requests back to handlers/endpoints in the GAE app
Thus the GAE app is the main point of interaction with clients and a dispatcher of heavy tasks, while the processing of those tasks is carried out by the microservices. I have set up a static outbound IP address of the Cloud Run hosted service which verfiedly works and traffic is routed through the NAT gateway as required in the documentation. The respective NAT IP address is on the firewall whitelist.
The problem is that the firewall still does not let in the Cloud Run >>> GAE app requests which bounce back with 403 statuses (of course, if I change the default firewall rule to 'Allow', traffic goes through). If I host the same microservice in a docker container on a GCE VM with a static IP address like this everything works flawlessly. This makes me hypothesize that albeit Cloud Run outbound traffic is indeed routed through the static IP address when traffic is towards addressees outside GCP, when I try to ping an internal (project-wise) asset it still goes though some dynamically selected IP (i.e. the static IP solution simply does not work). Unfortunately the logs don't show the 403-ed attempt so I can't see from what IP addresses those request seem to come (from a GAE standpoint).
I would be very grateful for ideas how this can be fixed as it greatly diminishes the value of the otherwise wonderful idea to have static outbound IP addresses for Cloud Run.
First, thank you both for your help and suggestions, they are very helpful. I found the solution with some kind help from Google:
When the Cloud Run microservice and the GAE app are hosted in the same project traffic is still routed through internal channels and appears to come from IP address 0.0.0.0 which can be whitelisted (so it would work) as long as one considers this address encompasses GCP assets which are parts of other projects too (to the best of my understanding)
A more robust solution seems to be setting up an externally facing load balancer as described here and putting it in front of the GAE app; in such a case, Cloud Run will indeed consistently use its static outbound IP address as described in the documentation
You are correct saying that the static IP is not honoured when packets are routed internally to GCP.
I think this is what you want. You have to allow in the firewall one of the IPs mentioned there (not sure which one right now).
Just as you and #Ema mentioned, this is an expected behavior having in mind that the traffic from Cloud Run to App Engine is intern.
When you use Cloud Nat to send all traffic there, it does happen. If you create a container and ping, let's say to www.github.com. You will find that the traffic goes through the IP you set. On the other hand, if you ping to www.google.com, given that the traffic is intern, and the site to reach out is in the same infrastructure, the request doesn't even goes through public internet.
Additionally, just to keep in mind Static outbound IP address is still in Beta and it is not recommended to use Beta features/products in production environments.
As you mentioned and as it is stated in Allowing requests from your services:
Creating a rule for IP 0.0.0.0 will apply to all Compute Engine instances with Private Google Access enabled, not only the ones you own. Similarly, allowing requests from 0.1.0.40 or 10.0.0.1 will allow any App Engine app to make URL Fetch requests to your app.
This questions might be of your interest:
What are the outbound IP ranges for GCP managed Cloud Run?
Possible to get static IP address for Google Cloud Functions?
I have 2 question related to GCP App Engine.
Q1. When App Engine instances are created, they are assigned with VM IP for SSH. Can we customize those IP values ?
Q2. I have created App Engine Firewall rules as below.
1. Priority: default , Action: deny , IP: *
2. Priority: 1000 , Action: allow , IP: 192.*.*.*
Where the IP 192.* is private network ethernet IP of my laptop.
But when I am running curl from local Cloud SDK, the request is failing with Error 403 (Forbidden).
How to configure App Engine FW to deny internet access and allow specific IP range ?
Q1: AppEngine is a serverless platform. You can find the scalability logic here, and you can't log in the VM, it's managed for you, there is no value to log in. You can't update things, your instance can be killed at any time,.... You deploy your code and let Google scaling your environment
Q2: Your local IP is in 192.xxx, but it's a local network, it belongs to the RFC 1918. When you go to internet, your local IP is NATed into the public IP of your internet connection. Go to this page (for example) to know your public IP
Then, all this public and only you (and all the computer belonging to your network (mobile phone, tablet, other computer, that use the same public IP)) can access to the App Engine service
We have 2 app engine app (flex and standard) running on separate projects and we want project A to request project B with https to xxx.appspot.com URL.
Our firewall on both projects Denies all IPs(*) and whitelisted App Engine internal addresss (10.1.0.41, 0.1.0.40, 10.0.0.1 and 0.1.0.30) as explained in the doc.
Yet we receive a "403 error forbidden access" (which disappears when disabling the firewall).
This post is similar to mine but the responses didn't help me.
Is there anything else I can do ?
Did anyone got this to work ?
Thank you in advance.
As you may already know, GCP Projects represent a trust boundary within an organization. Hence, inter-project communication between App Engine services would require Public IP communication or using Shared VPC, which allows connecting between networks from different projects. There should be no internal communication between App Engine Services over different projects. Hence, whitelisting App Engine internal IP addresses might not be useful in this situation.
About using Public App Engine IP addresses, as illustrated in this document. App Engine hosts services on a dynamic public IP address of a Google load balancer. Due to that, the IP address can be changed any time and any Static IP can not be provided. For outbound services, a large pool of IP addresses are used which you can obtain as outlined in this document
I have a GAE project (flexible) consisting of 1 default and 2 subservices:
foo.appspot.com
service1.foo.appspot.com
service2.foo.appspot.com
Now I want to use foo.appspot.com as API proxy & auth gateway to the internal services service1 and service2. The proxy itself I wrote and it is working fine.
I am struggling with adjusting the GAE Firewall to forbid incoming world traffic to service1 and service2 because I would like force an API user to send requests to foo.appspot.com. Traffic to the default service foo should be allowed.
It seems I can just enter IPs in the Firewall settings but not service names. The docs says that it should work but does not show how.
Thanks for the help!
App engine Flex environment is built on the Google Compute Engine and consequently, it supports the Virtual Private Cloud networking system.
With the VPC networks, you can configure firewall rules that would use Instance Tags to determine the target or source component in a firewall rule. Hence, you simply have to configure the app.yaml files of the target service/version to use the appropriate instance tags.