My GAE (Standard) app is hosted in europe-west region.
I am looking into creating a Cloud SQL instance in support of this app and would like to place it closest to the GAE.
Current Cloud SQL instances are available at below locations:
europe-west1 Belgium
europe-west2 London
europe-west3 Frankfurt
Is there any way to find out additional location details of my GAE app in order to decide which Cloud SQL location to use?
You can use gcloud app describe from the cloud shell in order to find out your app's location info. Typing this command (gcloud app describe) in the shell returns something like this:
authDomain: gmail.com
codeBucket: staging.my-project-id.appspot.com
defaultBucket: my-project-id.appspot.com
defaultHostname: my-project-id.appspot.com
featureSettings:
splitHealthChecks: true
gcrDomain: eu.gcr.io
id: my-project-id
locationId: europe-west2
name: apps/my-project-id
servingStatus: SERVING
See the command description here.
Next you can create your SQL instance by typing gcloud sql instances create [your-instance-name] --region=[region-of-your-choice]. For example:
user-id#my-project-id:~$ gcloud sql instances create test-instance --region=europe-west2
Creating Cloud SQL instance...done.
Created [https://www.googleapis.com/sql/v1beta4/projects/my-project-id/instances/test-instance].
NAME DATABASE_VERSION REGION TIER ADDRESS STATUS
test-instance MYSQL_5_6 europe-west2 db-n1-standard-1 00.000.000.000 RUNNABLE
user-id#my-project-id:~$
All the available options are here.
Related
I'm facing a problem with gcloud and their support can't seem to help me.
So, to put my app in prod I need to use a redis instance to host some data. I'm using memorystore because I like to have everything on gcloud.
My app is in the standard environment on app engine so on their doc (https://cloud.google.com/memorystore/docs/redis/connect-redis-instance-standard) they ask me to configure a VPC connector. But I think that the CIDR that I put is always wrong, can someone help me finding the good CIDR.
connectMode: DIRECT_PEERING
createTime: '2020-03-13T17:20:51.590448560Z'
currentLocationId: europe-west1-d
displayName: APP-EPG
host: 10.240.224.179
locationId: europe-west1-d
memorySizeGb: 1
name: projects/*************/locations/europe-west1/instances/app-epg
persistenceIamIdentity: *************
port: 6379
redisVersion: REDIS_4_0
reservedIpRange: 10.240.224.176/29
state: READY
tier: BASIC
Thank you all !
First in order to VPC connector work yor App Engine instances have to be in the same VPC & region that your Redis instance is. If not there will not be connectivity between the two.
Also make sure you redis and app use one of the approved locations. By now it's a lot of them.
Your redis instance is in europe-west1 region so to create your VPC connector you have to set the name of the VPC network your redis instance is in
(for example "default").
IP range you were asking about is any range (not reserved by the network redis instance is in).
So - for example if your "default" network is 10.13.0.0/28 then you have to specify something else like 10.140.0.0/28 etc. It has to be /29 - otherwise you won't be able to create the connector.
Why 10.13.0.0 or any other addresses ? They are going to be assigned as the source network for you apps to connect to the Redis (or any
other VM's) in the specified network.
I've tested it using the command:
cloud compute networks vpc-access connectors create conn2 --network default /
--range 10.13.0.0/28 --region=europe-west1
Or you can do it using console in Serverless VPC Access and clicking "Add new connector";
You can also read documentation on how to create a connector.
Im writing a NodeJS app and trying to connect to GCPs Redis MemoryStore, but I'm getting the ETIMEDOUT 10.47.29.131:6379 error message. The 10.47.29.131 corresponds to the REDISHOST. I'm trying to reach the server by it's internal private IP.
While the app works locally with a local Redis installed, it does not when deployed to the GCP AppEngine.
My GCP-Setup
Redis instance running at location europe-west3-a
Created a connector under "Serverless VPC access" which is in europe-west3
Redis and the VPC-connector are on the same network "default".
App Engine running in europe-west
Redis isntance:
VPC-connector:
The app.yml
runtime: nodejs
env: flex
automatic_scaling:
// or this but without env: flex (standard)
vpc_access_connector:
name: "projects/project-ID/locations/europe-west/connectors/connector-name"
beta_settings:
cloud_sql_instances: project-ID:europe-west3:name
env_variables:
REDISHOST: '10.47.29.131'
REDISPORT: '6379'
// removed this when trying without env: flex (standard)
network:
name: default
session_affinity: true
I followed these instructions to set everything up: https://cloud.google.com/memorystore/docs/redis/connect-redis-instance-standard
Digging deeper, I found: https://cloud.google.com/vpc/docs/configure-serverless-vpc-access where they mention something about permissions and serverless-vpc-access-images, and while trying to follow the instructions: https://cloud.google.com/compute/docs/images/restricting-image-access#trusted_images I couldn't find "Define trusted image projects." anywhere
What am I missing here?
Well, turns out, the problem was the region I've selected for the Redis instance.
From Documentation:
Important: In order to connect to a Memorystore for Redis instance, the connecting client must be located within the same region as the instance.
A region is a specific geographical location where you can run your resources. Each region is subdivided into several zones.
For example, the us-central1 region in the central United States has zones us-central1-a, us-central1-b, us-central1-c, and us-central1-f.
Althouh the documentation clearly says, that AppEngine and Memorystore have to be in the same region, my assumption on what regions actually are, was false.
When I created the AppEngine, I created it in europe-west, which is the same as europe-west1. On the other hand, when I created the redis instance, I used europe-west3, with the assumption that west3 is the same region as west, which is not.
Since the AppEngines region cannot be changed, I created another redis-instance in europe-west1 and now everything works.
So, the redis region must be exactly the same as the AppEngine region. region1 is the same as region, but region2 or region3 are not the same as region.
With an Instance ID, can the details of the instance can be read from the google console?
To see the startTime of an instance in the Google Cloud Console click on the Navigation Menu > App Engine > Instances. Select the service you are interested on and below the graphic there will be displayed a table with the startTime of each instance.
Or you can run the following command:
gcloud app instances describe <INSTANCE_ID> -s=<SERVICE> -v=<VERSION> | grep startTime
EDIT:
In case the instances no longer exists you can use this filter in the Google Cloud Stackdriver Logging Console:
resource.type="gae_app"
resource.labels.module_id="SERVICE"
resource.labels.version_id="VERSION"
protoPayload.instanceId="INSTANCE_ID"
The timestamp of the first entry will roughly coincide with the StartTime of the instance.
To access a remote datastore locally using the original dev_appserver I would set --default_partition=s as mentioned here
In March 2013 Google made devappserver2 the default development server, and it does not support --default_partition resulting in the original, dreaded:
BadRequestError: app s~appname cannot access app dev~appname's data
It appears like the first few requests are served correctly with
os.environ["APPLICATION_ID"] == 's~appname'
Then a subsequent request results in a call to /_ah/warmup and then
os.environ["APPLICATION_ID"] == 'dev~appname'
The docs specifically mention related topics but appear geared to dev_appserver here
Warning! Do not get the App ID from the environment variable. The development server simulates the production App Engine service. One way in which it does this is to prepend a string (dev~) to the APPLICATION_ID environment variable, which is similar to the string prepended in production for applications using the High Replication Datastore. You can modify this behavior with the --default_partition flag, choosing a value of "" to match the master-slave option in production. Google recommends always getting the application ID using the get_application_id() method, and never using the APPLICATION_ID environment variable.
You can do the following dirty little trick:
from google.appengine.datastore.entity_pb import Reference
DEV = os.environ['SERVER_SOFTWARE'].startswith('Development')
def myApp(*args):
return os.environ['APPLICATION_ID'].replace("dev~", "s~")
if DEV:
Reference.app = myApp
I am using the Python 2.7 runtime with NDB from the 1.6.2 SDK on Google App Engine.
I get the following error:
BadRequestError: app s~myapphr cannot access app dev~myapphr's data
Originating from this code:
device = model.Key(urlsafe=device_id).get()
I am accessing my app from dev.myapp.appspot.com that is aliased to myapphr. device_id was created on the same dev.myapphr version.
What is going on?
the dev server has a default default_partition of 'dev' and on production, HRD apps get a partition of 's'. If you create a urlsafe key on the dev server and store it as a string it will not work on a the production server with a different partition. the way to make keys portable is to save them in a ReferenceProperty with db or KeyProperty on ndb.
The prefix you see there ("s~" and "dev~") is called a partition. You can use the --default_partition flag for dev_appserver.py to change the prefix from "dev~" to "s~".