We have two separate projects, project A: has a Cloud SQL instance (has both public IP and private IP)
Now we have a new project B: It's a App Engine standard Java11 environment. For this project we have a requirement where we need to make a REST API call to a third party api, and it has to come from a specific GEO location. So we have setup on provisioning a static IP address explicitly in the geography where our app engine service is provisioned. We followed this documentation: Set up a static outbound IP address
When we deploy the App Engine after those changes, it is unable to reach the CLoudSQL anymore from project A, however when we remove from app.yaml the egress_setting: all-traffic and re-deploy, it connects to the Cloud Sql successfully but then we are unable to call our REST Api call, as third party api returns back 403 because of our IP address location.
Note that both Project A and Project B are in the same region.
Our app.yaml file looks like this:
`
runtime: java11
instance_class: F2
env_variables:
GAE_USE_SOCKETS_HTTPLIB : ''
vpc_access_connector:
name: projects/<projectB-id>/locations/northamerica-northeast1/connectors/cb-connector
egress_setting: all-traffic
This is the connection url used for Cloud SQL (MySql),
We have tried with the cloud SQL private IP as follow
spring:
datasource:
url: jdbc:mysql://<cloudSql-PrivateIp>:3306/_operations?user=<user>&password=<pass>&ipTypes=PRIVATE
Then we have also tried with cloud SQL public IP as follow:
spring:
datasource:
url: jdbc:mysql://_operations?cloudSqlInstance=<projectA-id>northamerica-northeast1:<sql-instanceName>&socketFactory=com.google.cloud.sql.mysql.SocketFactory&user=<user>&password=<pass>
We have followed these instructions here to connect from App-Engine to CloudSql: Connect from App Engine standard environment We tried with both option Public Ip and Private IP, but we kept on getting a Communication Link error. It seems to timeout after a while loading. That is when the egress_setting: all-traffic is set in app.yaml. If we remove that egress_setting: all-traffic, then connection works perfectly but we are unable to make our REST call to that third party api we need.
Not sure how we can set this up so we can still have connection to both the cloud Sql instance (project A) and still be able to reach our third party api from App engine (project B) using a static outbound IP. If someone can share some insight on what possible cause or solution that we can do?
EDIT
We have also tried to make a peering vpc between project A and project B, but still not working when egress_setting: all-traffic is set in app.yaml file.
Finally the issue was with the IP range that was configured in my VPC. The static egress IP range did not include the IP address of my Cloud SQL instance. After adding the appropriate IP range, now everything is working.
Related
I have a server hosted on app engine. I need to call an API from it with a static outbound IP. I followed step-by-step the following guide https://cloud.google.com/appengine/docs/standard/python3/outbound-ip-addresses#static-ip
Now, I do not quite get how to configure the cloud NAT in order to call the external API through my static outbound IP (external IP address). I guess, I should call a local VPC address from my server and configure somewhere the API IP it should redirect to.
Any help appreciated.
Thanks a lot,
Paul
If you configured everything per the doc you linked in your question, then you don't need to do anything special from within your AppEngine application. Setting the Serverless VPC Access egress_setting to all-traffic means that anytime your AppEngine app tries to call an external address, it will be routed this way and will use the static IP configured as part of the setup.
I am trying to send email using client's on-prem SMTP server using app engine standard. For this we have created Serverless VPC access connector in default network and Cloud NAT with static ip address to send egress traffic. Client has whitelisted static ip address and port. Following is code snippet in app engine
msg.set_content('This is a HTML email')
msg.add_alternative(cleared_html_content, subtype='html')
try:
context = ssl._create_unverified_context()
print("starting conectn")
with smtplib.SMTP('xx.xxxx.edu', 2525) as server:
server.starttls(context=context)
server.send_message(msg)
print("sent almost")
except Exception as e:
print('Error: ', e)
Following is app.yaml
runtime: python37
entrypoint: gunicorn -t 120 -b :$PORT main:app
vpc_access_connector:
name: projects/xxxxxxxxx/locations/us-central1/connectors/yyyyyyyyy
When i run my app using app engine url, I am getting following error in logs viewer
Error: (554, b"xxx.xxxxx.edu\nYour access to this mail system has been rejected due to the sending MTA's poor reputation. If you believe that this failure is in error, please contact the intended recipient via alternate means."
Also i have created cloud function with same code as in app engine to test and surprisingly email was sent to intended recepient with out any issue. When i checked cloud NAT logs, it has all details when triggered via cloud function (in short it is using static ip address) but there are no logs related to app engine trigger. So i think my app engine traffic is not going via static ip address and not sure how to mention that in app.yaml
There might be code issue in email function as well but since it is working in cloud function, i really doubt about my app.yaml and not email python code. Any help is really appreciated
I understood that your SMTP IP was public. There is a caveat to know with serverless VPC connector.
With Cloud Function, and Cloud Run, you have the capacity to choose if only private IP or Public and Private IP are routed through the serverless VPC Connector
With app engine, I didn't find a clear description of the egress control, but I guess that only private IP (RFC1918) are routed through the VPC, and not the public one. And so, your Cloud Nat isn't used and thus you aren't authorised on the SMTP server of your school.
Edit 1:
You have 3 solutions to solve this
You can create a Cloud Functions (or a Cloud Run service) that your App Engine calls when you need to send an email.
You can switch from App Engine to Cloud Run (use the new beta command gcloud beta run deploy --source=. --region=<REGION> --platform=managed <Service Name>). Like this, you can deploy as with App Engine. The same Container engine builder as App Engine is used (Buildpack). You have to adapt the content of the app.yaml file (share it if you need help). However, up to now, IAP isn't compliant with Cloud Run. If you want to use it, wait!
Create a VPN between your VPC and your school network. Like this, you will call your SMTP server with a private IP. On the smtp server, grant only the serverless VPC connector range to access it. And you no longer need a Cloud NAT configuration.
I have 2 question related to GCP App Engine.
Q1. When App Engine instances are created, they are assigned with VM IP for SSH. Can we customize those IP values ?
Q2. I have created App Engine Firewall rules as below.
1. Priority: default , Action: deny , IP: *
2. Priority: 1000 , Action: allow , IP: 192.*.*.*
Where the IP 192.* is private network ethernet IP of my laptop.
But when I am running curl from local Cloud SDK, the request is failing with Error 403 (Forbidden).
How to configure App Engine FW to deny internet access and allow specific IP range ?
Q1: AppEngine is a serverless platform. You can find the scalability logic here, and you can't log in the VM, it's managed for you, there is no value to log in. You can't update things, your instance can be killed at any time,.... You deploy your code and let Google scaling your environment
Q2: Your local IP is in 192.xxx, but it's a local network, it belongs to the RFC 1918. When you go to internet, your local IP is NATed into the public IP of your internet connection. Go to this page (for example) to know your public IP
Then, all this public and only you (and all the computer belonging to your network (mobile phone, tablet, other computer, that use the same public IP)) can access to the App Engine service
Our AppEngine app is connecting to a remote service which requires a VPN and also required me to add entries to the hosts file on my local machine in order to connect to their endpoints.
e.g.
10.200.30.150 foo.bar.com
This is working fine when running the app locally, but I can't figure out how to set this up on Google Cloud to work once deployed.
I can't use the IP addresses directly because it errors that the IP is not on the cert's list.
How do I map the host names to the IPs in Google Cloud so that AppEngine can use them?
From the error mentioned in the comment I suspect connecting directly through the IP fails because the certificate doesn't recognize the IP to DNS mapping as valid and therefore the secure connection setup breaks. Based on the requirements of connecting to the API by VPN and tweaking the hosts mapping there are few things you may try.
The simplest approach that may work would be using a Google Compute Engine VM instance, since there you would able to manipulate the etc/hosts file and replicate the local machine setup. This VM could be used either as the main app service or as a proxy from App Engine to the 3rd party API endpoint. To go that route I would suggest taking a look at these two posts which explain how to change the etc/hosts file on GCE (Changing the file once wouldn't work as the VM periodically overrides it, see the posts for cronjob like workaround).
Separately, as your app runs in App Engine flexible environment there is the chance to provide a docker container with the app packaged. It may be possible to set the workaround above in the docker file and have it working in App Engine too.
I am new to Google Cloud Platform and trying to understand it. I have launched a compute engine and have installed a web application. I can access that application using External IP address of the engine or with private IP address from within the VPC.
How can I refer to them with any kind of name?
Just like AWS has this concept of having a name for each instance as ec2-54-54-54-54.eu-west-1.compute.amazonaws.com. So, I can use this name in my browser and with some DNS, this name will be resolved to actual IP address.
Possible duplicate of Where can i find the name servers of Google Compute Engine.
But the answer is that Google Cloud Platform does not provide default public DNS for external IP addresses.
You will need to use Cloud DNS or another service or leverage an external service like xip-io which will map any IP to a DNS name.