How to connect Google Cloud SQL via Google App engine Flex to botnet core app? - google-app-engine

I have a .NET Core 2.2 app using a PostgreSQL DB.
Now I want to deploy it on Google Cloud App Engine Flex and Google Cloud SQL PostgreSQL.
I tried the official way and found this, where I found that you have to use $ gcloud beta app deploy instead of $ gcloud app deploy
My app.yaml configuration file:
env: flex
runtime: aspnetcore
beta_settings:
cloud_sql_instances: "<SQL-NAME>=tcp:<PORT>"
The issue is that I get this error:
Trying to connect to
Host=127.0.0.1;Port=XXX;Username=XXX;Password=XXX;Database=XXX;
Application startup exception: System.Net.Sockets.SocketException
(111): Connection refused
Do I have to include special libraries into .NET Core 2.2 to support Google App Engine?

You don't need to include any special libraries in .NET Core 2.2 rather than those that are already included in the Quckstart that you have shared above, unless of course you are using any additional libraries in your code.
I have tried the quickstart and deployed my .NET GAE app that connects to CloudSQL PostgreSQL database.
Taking a look at your app.yaml configuration file I can see that there is an issue with it. Also based on the error message you are getting I assume that you are having an issue in your appsettings.json file as well.
Based on the quickstart of GitHub that you shared, the configuration when deploying should be:
app.yaml
runtime: aspnetcore
env: flex
beta_settings:
cloud_sql_instances: "[PROJECT_ID]:[INSTANCE_REGION]:[INSTANCE_NAME]=tcp:[TCP_PORT_NUMPER]"
TCP Port number for PostgreSQL: 5432
TCP Port number for MySQL: 3306
To find the Instance connection name which is the part before tcp, you can go to Cloud Console > SQL > [YOU_INSTANCE_NAME] and you can find the entire string under Instance connection name
appsettings.json
{
"CloudSQL" : {
"Database" : "PostgreSQL", // Set to "PostgreSQL" when using a PostgreSQL database.
// [START gae_flex_mysql_settings]
"ConnectionString": "Uid=[USER_ID];Pwd=[PASSWORD];Host=cloudsql;Database=[DATABASE_NAME]"
// [END gae_flex_mysql_settings]
}
}
Database should be PostgreSQL as also specified by the comment
ConnectionString should be filled with your own data that you used to setup the database
The Host part should be cloudsql when you are deploying and 127.0.0.1 when running locally.
127.0.0.1 indicates that the database is running locally and the cloudsql string indicates that it should use the connection string name. Based on the error, I assume that you used the 127.0.0.1 to test locally but then forgot to change it back when you were deploying.

Related

Keycloak 19 docker can't find suitable driver for sqlserver

I deployed keycloak 19 in azure app service using the quay.io/keycloak/keycloak:19.0.1 image; It's working fine when using the dev file database; but I'm having trouble when I tried to connect on my SQL server database, I followed the instruction here but I'm getting the error below
ERROR [org.keycloak.quarkus.runtime.cli.ExecutionExceptionHandler] (main) ERROR: No suitable driver found for jdbc:sqlserver://MySDBServer:1433;databaseName=MyDatabaseName
Here's my Setup in app settings configurations
KC_DB:mssql
KC_DB_URL_HOST:[MyDatabaseServer]
KC_DB_PASSWORD:[MyDatabasePassword]
KC_DB_USERNAME:[MyDatatabaseUsername]
KC_DB_URL_DATABASE:[MydatabaseName]
KC_PROXY:edge
Passing the --optimized parameter to the start should fix it.
bin/kc.[sh|bat] start --optimized
In case of a Dockerfile, your ENTRYPOINT will look like:
ENTRYPOINT ["/opt/keycloak/bin/kc.sh", "start", "--optimized"]
You can read about the --optimized parameter at the links given below.
Optimize the Keycloak startup
Changes to the server configuration and startup

Gcloud cloud build local component failing with error "Error loading config file: unknown field "availableSecrets" in cloudbuild.Build"

Greetings stackoverflow community! First time asker, long time user.
I am testing out my cloudbuild.yaml file locally using Cloud Build Local component and Secret Manager and it is failing on "availableSecrets".
Error message: Error loading config file: unknown field "availableSecrets" in cloudbuild.Build
OS Platform: Windows 10/WSL2/Ubuntu 18.04
cloud-build-local: v0.5.2
Docker engine: v20.10.2
Nodejs version: v14.15.3
NPM version: 6.14.9
gcloud version: 326.0.0
Installed components: [BigQuery Command Line Tool, Cloud Datastore Emulator, Cloud SDK Core Libraries, Cloud Storage Command Line Tool, Google Cloud Build Local Builder, gcloud Beta Commands]
Documentation on Cloud Build build file: https://cloud.google.com/cloud-build/docs/build-config
Documentation to configure secrets with cloud build: https://cloud.google.com/cloud-build/docs/securing-builds/use-secrets
Documentation for cloud build local: https://cloud.google.com/cloud-build/docs/build-debug-locally
Steps performed:
Added secrets to Secret Manager
Enabled API between Cloud Build and Secrets Manager
Added cloudbuild service account as member of each secret password.
Added IAM permission Secret Manager Secrets Accessor to cloudbuild user. I don't know where I got this info from but it is residual at this point from other attempts to use Secret Manager with cloudbuild. I am not sure of the difference between applying access here vs applying to the Secret Manager secret.
Command: cloud-build-local --config=cloudbuild.staging.yaml --dryrun=false .
cloudbuild.staging.yaml:
- name: gcr.io/cloud-builders/npm
entrypoint: 'npm'
args: [ 'install' ]
- name: 'gcr.io/cloud-builders/gcloud'
args: ["app", "deploy"]
env:
- 'DAO_FACTORY=datastore'
- 'POLL_INTERVAL=15'
- 'PROMPT=staging>'
- 'ENVIRONMENT=staging'
- 'NAMESPACE=staging'
- 'RESET_DATASTORE=false'
secretEnv: ['ADMIN_USER', 'SUPER_ADMINS', 'BOT_TOKEN']
availableSecrets:
secretManager:
- versionName: projects/{project token}/secrets/SYSTEM_USER/versions/1
env: 'ADMIN_USER'
- versionName: projects/{project token}/secrets/SUPER_ADMINS/versions/1
env: 'SUPER_ADMINS'
- versionName: projects/{project token}/secrets/BOT_TOKEN/versions/2
env: 'BOT_TOKEN'```
Tag: cloud-build-local. I guess without reputation a meaningful tag cannot be created. Maybe an esteemed community member will create this as this may be specific to cloud-build-local only.
Support for Google Secret Manager in Google Cloud Build descriptor file is apparently very new and does not appear to be supported by cloud-build-local component at this time; please see comment from Guillaume about feature being a week old. When cloud build descriptor is ran in Cloud Build, it works fine.
I fixed a similar issue by upgrading the gcloud tool.

Error deploying java google app engine flexible application - Timed out waiting for the app infrastructure to become healthy

Writing this issue as I have no idea how to investigate it.
We're having problems in deploying an app engine flexible application.
The problem is, that the only error we get is the following:
GCLOUD: ERROR: (gcloud.app.deploy) Error Response: [4] Timed out waiting for the app infrastructure to become healthy.
I tried already the following:
Try a simple helloWorld app, to make sure it's not an application issue
Check quota settings -> All green
Check activity stream for warnings or errors
Check logs for warngings or errors
Grant owner role to service account which is deploying the app
App.yaml:
service: test-service # Id of the service
env: flex # Flex environment
runtime: java # Java runtime
runtime_config:
jdk: openjdk8 # use OpenJDK 8
resources:
cpu: 1
memory_gb: 2.8
gcloud version
Google Cloud SDK 214.0.0 alpha 2018.08.24
app-engine-java 1.9.64
app-engine-python 1.9.74 beta 2018.08.24 bq 2.0.34
cloud-datastore-emulator 2.0.2
core 2018.08.24
gsutil 4.33
kubectl 2018.08.24
pubsub-emulator 2018.08.24
After contacting the google technical support, we found out, that the default app engine service account didn't have the Editor role. After assigning the editor role the deployment worked again.
This error is often reported when your application has reached the quota limit for "In-use IP addresses". Similar error was reported on this Google Cloud Platform issue link. The default value for the in-use addresses is '8', and this quota value can be increased clicking the 'Edit' button in the Cloud Console — Ensure you are editing the value for In-use IP addresses.
The Google engineer confirmed that there is a planned improvement to the quota error details to be implemented in one of the next versions of gcloud SDK. You can track updates on the CloudSDK within this Google Group link

Elasticsearch deployment on google app engine flex

Is it possible to deploy Elasticsearch on App engine flex environment using a docker image.
I have tried the following
My files on the local machine
Folder : elasticsearch
app.yaml
Dockerfile
docker-entrypoint.sh
config folder(containing elasticsearch.yml)file
Contents of app.yaml
runtime: custom
env: flex
Dockerfile and docker-entrypoint.sh copied from https://github.com/GoogleCloudPlatform/elasticsearch-docker/tree/master/5/5.2.0
Modifications to the Dockerfile
replaced EXPOSE 9200 9300 to EXPOSE 8080
Modification to the elasticsearch.yml
cluster.name: "beaconinside-docker-cluster"
path.data: /usr/share/elasticsearch/data
http.host: 0.0.0.0
http.port: 8080
discovery.zen.minimum_master_nodes: 1
I build a container using the docker file on my local machine
docker build -t elasticdemo .
Then, I run the container
docker run -p 8080:8080 elasticdemo
I am able to access elasticsearch on 0.0.0.0:8080
Problem:
I am trying to deploy elasticsearch as an app to Google app engine flex environment
gcloud app deploy app.yaml --version elasticdocker --project myproject
The deployment fails with the following error
Updating service [default]...failed.
ERROR: (gcloud.app.deploy) Error Response: [9]
I was expected elasticsearch to deploy as an app and be available on the deployed url.
Could you please provide pointers/help/suggestions with this approach?
While you can deploy ES to App Engine Flexible environment it's not particularly useful. The VMs hosting GAE Flexible containers are restarted regularly as part of maintenance and whatever data is stored on the local disk will be lost on restart. If you want to use local disk for long term storage, I'd suggest to deploy the GCE VM's (or alternatively use a solution from the GCP Marketplace) or deploy to GKE which supports persistent disks
As for the actual question: you probably don't have a health check handler and therefore App Engine Flexible environment doesn't consider your app healthy after deploying it. The error message is useless, I agree.
From the GAE Flexible docs for building custom images:
"A health check is an HTTP request to the URL /_ah/health. A healthy application should respond with status code 200."
Alternatively you can turn off health checks by adding into app.yaml
enable_health_check: False

Google App Engine Bulkloader “Authentication Failed”

I'm trying to use the bulkloader to load my data to the App-Engine server. I run the following command using Python 2.5:-
appcfg.py upload_data --application=myappname --kind=mykind
--filename=data_archive.csv --url=http://myappname.appspot.com/remote_api
But its failing with this Authentication error:-
[INFO ] Connecting to myappname.appspot.com/remote_api
[ERROR ] Exception during authentication
URLError: <urlopen error [Errno 10061] No connection could
be made because the target machine actively refused it>
[INFO ] Authentication Failed
My idea is to do a bulk download from my development server and then use this dump to do a upload to the app-engine server. The bulk download worked fine. I used this format for this:-
appcfg.py download_data --application=myappname --kind=mykind
--url=http://localhost:8888/remote_api --filename=data_archive.csv
But the bulk upoad fails. A couple of things: the bulk download asked me for a userid and password, but the bulk upload does not. Also, I don't currently have a app.yaml file which I see mentioned a lot - do I need one to do this ?
Thanks in advance for any help.
M.
EDIT
For anyone else struggling with this, the problem was indeed being behind the proxy server, but there was another 'error' with what is above. The app-id needs the "s~" bit added to it.
appcfg.py upload_data --application=s~myappname --kind=mykind
--filename=data_archive.csv --url=http://myappname.appspot.com/remote_api
This isn't an authentication issue - that message is a red-herring - your machine is unable to contact the App Engine app at all. Do you have a proxy you need to transit through in order to make external connections?
You do not need --application=s~myappname when using bulkloader - Google have mentioned before:
Warning! Do not use the --application= flag to get the application ID
when using the bulk loader. Instead, use --url=.
For more detail take a look here:
https://developers.google.com/appengine/docs/python/tools/uploadingdata
app.yaml is how it finds your server. I am not sure how you can try and upload without one.
In addition to having an app.yaml that points to the production server, the production server also needs to have remote_api turned on (in it's app.yaml and in the version you are trying to reach):
builtins:
- remote_api: on

Resources