Grails and Google app engine deployment errors - google-app-engine

I have Grails 3.3.11 application, and I'm trying to deploy it on flexible Google App Engine, however, I have some errors mainly produced from connecting to Google cloud SQL, I made MySQL 5.7 instance on it and able to access it from my IDE and while I run the application locally,I searched for some solutions for theses errors but unfortunately, nothing worked for me, here are the exceptions
org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'methodValidationPostProcessor' defined in class path resource [org/springframework/boot/autoconfigure/validation/ValidationAutoConfiguration.class]: Unsatisfied dependency expressed through method 'methodValidationPostProcessor' parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'hibernateDatastoreServiceRegistry': Cannot resolve reference to bean 'hibernateDatastore' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'hibernateDatastore': Bean instantiation via constructor failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.grails.orm.hibernate.HibernateDatastore]: Constructor threw exception; nested exception is org.hibernate.exception.JDBCConnectionException: Unable to check JDBC Connection auto-commit in preparation for DDL execution
at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:749)
at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:467)
at
org.hibernate.resource.transaction.backend.jdbc.internal.DdlTransactionIsolatorNonJtaImpl.getIsolatedConnection(DdlTransactionIsolatorNonJtaImpl.java:46)
"
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
Caused by: java.net.ConnectException: Connection timed out (Connection timed out)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at com.mysql.jdbc.StandardSocketFactory.connect(StandardSocketFactory.java:211)
at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:299)
here are the production configurations in application.yml
dataSource:
dbCreate: create-drop
driverClassName: com.mysql.jdbc.Driver
dialect: org.hibernate.dialect.MySQL5Dialect
username: myUsername
password: myPassword
url: jdbc:mysql://DBIP:3306/CusOrderIns
properties:
jmxEnabled: true
initialSize: 5
maxActive: 50
minIdle: 5
maxIdle: 25
maxWait: 10000
maxAge: 600000
timeBetweenEvictionRunsMillis: 5000
minEvictableIdleTimeMillis: 60000
validationQuery: SELECT 1
validationQueryTimeout: 3
validationInterval: 15000
testOnBorrow: true
testWhileIdle: true
testOnReturn: false
jdbcInterceptors: ConnectionState
defaultTransactionIsolation: 2
build.gradle
buildscript {
repositories {
mavenLocal()
maven { url "https://repo.grails.org/grails/core" }
}
dependencies {
classpath "org.grails:grails-gradle-plugin:$grailsVersion"
classpath "com.bertramlabs.plugins:asset-pipeline-gradle:2.15.1"
classpath "org.grails.plugins:hibernate5:${gormVersion - ".RELEASE"}"
classpath 'com.google.cloud.tools:appengine-gradle-plugin:1.3.2'
}
}
provided "org.springframework.boot:spring-boot-starter-jetty"
// provided "org.springframework.boot:spring-boot-starter-tomcat" //commented
App engine config in src\main\appengine\app.yaml
runtime: java
env: flex
runtime_config:
jdk: openjdk8
server: jetty9
resources:
cpu: 2
memory_gb: 5
disk_size_gb: 10
manual_scaling:
instances: 1
App engine service account roles in Google IAM:
Cloud SQL Admin
Cloud SQL Client
Cloud SQL Editor
Cloud SQL Instance User
Editor
Cloud Run Admin
Anything missed or not configured probably?

I'm following this Google Cloud Community documentation about Deploy a Grails app to App Engine flexible environment. If you are still unable to connect using IP address of Cloud SQL, try using the Instance Connection Name of your Cloud SQL instance in url of dataSource and additional MYSQL dependencies in build.gradle:
application.yml
production:
dataSource:
dbCreate: update
dialect: org.hibernate.dialect.MySQL5InnoDBDialect
driverClassName: com.mysql.cj.jdbc.Driver
url: jdbc:mysql://google/[DATABASE_NAME]?socketFactory=com.google.cloud.sql.mysql.SocketFactory&cloudSqlInstance=[INSTANCE_NAME]&useSSL=true
username: root
password: password
For [DATABASE_NAME], use the database name you used when you created the database.
For [INSTANCE_NAME], use your instance name, which is visible in your Cloud SQL instance details.
Added in build.gradle:
runtime 'mysql:mysql-connector-java:8.0.16'
runtime 'com.google.cloud.sql:mysql-socket-factory-connector-j-8:1.0.14'

Related

Sql Server with spring boot HikariPool Failed to validate connection Possibly consider using a shorter maxLifetime value

I'm facing the following problem in my application that after a while running it starts to show the following warn.
HikariPool-1 - Failed to validate connection ConnectionID:19 ClientConnectionId: cea9b6e9-baac-495d-8f9b-08723893b6d9 (The connection is closed.). Possibly consider using a shorter maxLifetime value.
I already set the maxLifetime property to 600000 in my application.yml but the problem still persists.
Do I have other Hikari settings to configure?
I'm using spring boot 2.5.12 and data base Sql Server 2017
application.yml
spring:
main:
allow-circular-references: true
datasource:
hikari:
maxLifetime: 600000
url: jdbc-url
username: userName
password: passwaord
driver-class-name: com.microsoft.sqlserver.jdbc.SQLServerDriver
jpa:
open-in-view: false
properties:
hibernate:
dialect: org.hibernate.dialect.SQLServer2012Dialect
proc:
param_null_passing: true
show-sql: false
data:
web:
pageable:
default-page-size: 20

Unable to Deploy Application to App Engine Flexible Environment with a Shared VPC

I am unable to deploy a Dockerized application to App Engine Flexible Environment (AEF) in a Google Cloud Platform (GCP) project with a provisioned Shared Virtual Private Cloud (XPN).
In other words, my application with the following app.yaml:
automatic_scaling:
max_num_instances: 1
min_num_instances: 1
env: flex
network:
instance_tag: incorrect-target-tag
name: projects/$GCP_PROJECT_ID/global/networks/$XPN_NETWORK_NAME
service: $AEF_APPLICATION_NAME
and a confirmed Docker image name and tag in Google Container Registry (GCR):
gcloud container images list-tags \
us.gcr.io/$GCP_PROJECT_NAME/$AEF_APPLICATION_NAME \
--flatten=tags \
--format='value(format("us.gcr.io/$GCP_PROJECT_NAME/$AEF_APPLICATION_NAME:{0}", tags))' \
--project=$GCP_PROJECT_NAME
#=>
. . .
us.gcr.io/$GCP_PROJECT_NAME/$AEF_APPLICATION_NAME:$DOCKER_IMAGE_TAG
. . .
is unable to be deployed to AEF:
yes | gcloud app deploy \
--appyaml=./app.yaml \
--image-url=us.gcr.io/$GCP_PROJECT_NAME/$AEF_APPLICATION_NAME:$DOCKER_IMAGE_TAG
#=>
Services to deploy:
descriptor: [/. . ./app.yaml]
source: [/. . ./$AEF_APPLICATION_NAME]
target project: [$GCP_PROJECT_NAME]
target service: [$AEF_APPLICATION_NAME]
target version: [$AEF_APPLICATION_VERSION]
target url: [. . .]
target service account: [App Engine default service account]
Do you want to continue (Y/n)?
Beginning deployment of service [$AEF_APPLICATION_NAME]...
WARNING: Deployment of service [$AEF_APPLICATION_NAME] will ignore the skip_files field in the configuration file, because the image has already been built.
Updating service [$AEF_APPLICATION_NAME] (this may take several minutes)...
.............................................................failed.
ERROR: (gcloud.app.deploy) Error Response: [13] Flex operation projects/$GCP_PROJECT_NAME/regions/$AEF_APPLICATION_REGION/operations/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx error [INTERNAL]: An internal error occurred while processing task /app-engine-flex/insert_flex_deployment/flex_create_resources>1970-01-01T00:00:00.001Z000001.jc.2: <eye3 title='FAILED_PRECONDITION'/> generic::FAILED_PRECONDITION: Validation error: The App Engine flexible Environment Service Agent is unable to find a suitable Flex Firewall Rule in network '$XPN_NETWORK_NAME' in project '$GCP_PROJECT_ID'. Have the Shared VPC Admin create a Flex Firewall Rule as described in https://cloud.google.com/appengine/docs/flexible/python/using-shared-vpc
with the following Virtual Private Cloud (VPC) firewall rule supporting AEF communication through the XPN:
cloud compute firewall-rules list \
--filter="allowed[].ports=(8443) AND allowed[].ports=(10402)" \
--project=$GCP_PROJECT_NAME
#=>
NAME NETWORK DIRECTION PRIORITY ALLOW DENY DISABLED
aef-instance $XPN_NETWORK_NAME INGRESS 1000 tcp:8443,tcp:10402 False
To show all fields of the firewall, please show in JSON format: --format=json
To show all fields in table format, please see the examples in --help.
gcloud compute firewall-rules describe \
aef-instance \
--format=yaml \
--project=$GCP_PROJECT_NAME
#=>
allowed:
- IPProtocol: tcp
ports:
- '8443'
- '10402'
creationTimestamp: '1970-01-01T00:00:00.000-01:00'
description: allows traffic between aef and xpn
direction: INGRESS
disabled: false
id: 'xxxxxxxxxxxxxxxxxxx'
kind: compute#firewall
logConfig:
enable: false
name: aef-instance
network: https://www.googleapis.com/compute/v1/projects/$GCP_PROJECT_NAME/global/networks/$XPN_NETWORK_NAME
priority: 1000
selfLink: https://www.googleapis.com/compute/v1/projects/$GCP_PROJECT_NAME/global/firewalls/aef-instance
sourceRanges:
- 35.191.0.0/16
- 130.211.0.0/22
targetTags:
- incorrect-target-tag
Note: this rule is required for using any AEF application with the XPN, described here.
Following the guide to linking AEF and the XPN here, the target tag for VPC Firewall rule aef-instance MUST be aef-instance. Update VPC Firewall rule aef-instance with the correct target tag:
gcloud compute firewall-rules update \
aef-instance \
--project=$GCP_PROJECT_NAME \
--target-tags=aef-instance
#=>
Updated [https://www.googleapis.com/compute/v1/projects/$GCP_PROJECT_NAME/global/firewalls/aef-instance].
and you will be able to redeploy to AEF without that validation error.
Note: changing the target tag in the app.yaml isn't necessary: the AEF application will be able to communicate over a provisioned XPN as long as there is a firewall rule that meets this criteria exactly, regardless of tags specified in the app.yaml.

Connectivity issues from App Engine to CloudSQL using Private IP and Serverless VPC Access

I am doing a PoC to connect from Google App Engine to CloudSQL instance running with Private IP on a SharedVPC . The sample application for testing is from
https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/cloud-sql/mysql/sqlalchemy/README.md
My Serverless VPC Connector Range is as follows
$ gcloud compute networks vpc-access connectors list --region=us-central1
CONNECTOR_ID REGION NETWORK IP_CIDR_RANGE MIN_THROUGHPUT MAX_THROUGHPUT STATE
testserverlessvpc us-central1 kube-shared-vpc 192.168.60.0/28 200 300 READY
Private IP Range is as follows
$ gcloud compute addresses list --global --filter="purpose=VPC_PEERING"
NAME ADDRESS/RANGE TYPE PURPOSE NETWORK REGION SUBNET STATUS
cloudsqltestrangenew 10.0.100.0/20 INTERNAL VPC_PEERING kube-shared-vpc RESERVED
MySQL instance is as follows
$ gcloud sql instances list
NAME DATABASE_VERSION LOCATION TIER PRIMARY_ADDRESS PRIVATE_ADDRESS STATUS
mysql2 MYSQL_5_7 us-central1-b db-f1-micro - 10.0.100.5 RUNNABLE
app.standard.yaml is as follows
runtime: python37
service: appcentralpri
env_variables:
CLOUD_SQL_CONNECTION_NAME: projectname:us-central1:mysql2
DB_USER: guestdb
DB_PASS: password
DB_NAME: testdb
DB_HOST: 10.0.100.5:3306
vpc_access_connector:
name: projects/hostproject-26a2/locations/us-central1/connectors/testserverlessvpc
Deployment went through fine and no errors in log encountered
gcloud app deploy app.standard.yaml
However when I try to connect to the application endpoint , the page is not accessible with "Server Not Found"
I have given VPC Access and Network Compute User role to the below App Engine Default Service account on VPC Host Project
<hostproject>-26a2#appspot.gserviceaccount.com
UPDATES
Also added below in app.yaml but no luck
env_variables:
GAE_USE_SOCKETS_HTTPLIB : 'anyvalue'
https://cloud.google.com/appengine/docs/standard/python/sockets#making_httplib_use_sockets
Same setup for CloudRun and CloudSQL worked for me but struggling with GAE. Any suggestions to resolve this issue ?
Your app.yaml file is wrong formatted. The vpc_access_connector: need to be at the root level. Here it's taken as an environment variable value.
runtime: python37
service: appcentralpri
env_variables:
CLOUD_SQL_CONNECTION_NAME: projectname:us-central1:mysql2
DB_USER: guestdb
DB_PASS: password
DB_NAME: testdb
DB_HOST: 10.0.100.5:3306
vpc_access_connector:
name: projects/hostproject-26a2/locations/us-central1/connectors/testserverlessvpc

Driver:com.mysql.jdbc.Driver#75b3ef1a returned null for URL:jdbc for grails app on app engine

Followed the steps described here to connect my grails 3.2.9 app to google cloud-sql instance on google-app-engine flexible env
http://guides.grails.org/grails-google-cloud/guide/index.html#deployingTheApp
My grails version is as follows
==> grails -version
| Grails Version: 3.2.9
| Groovy Version: 2.4.10
| JVM Version: 1.8.0_131
My application.yml looks as follows
# tag::dataSourceConfiguration[]
dataSource:
pooled: true
jmxExport: true
dialect: org.hibernate.dialect.MySQL5InnoDBDialect
environments:
development:
dataSource:
dbCreate: create-drop
url: jdbc:h2:mem:devDb;MVCC=TRUE;LOCK_TIMEOUT=10000;DB_CLOSE_ON_EXIT=FALSE
test:
dataSource:
dbCreate: update
url: jdbc:h2:mem:testDb;MVCC=TRUE;LOCK_TIMEOUT=10000;DB_CLOSE_ON_EXIT=FALSE
production:
dataSource:
driverClassName: com.mysql.jdbc.Driver
dbCreate: create-drop
url: jdbc:cloudsql://google/{DATABASE_NAME}?cloudSqlInstance={INSTANCE_NAME}&socketFactory=com.google.cloud.sql.mysql.SocketFactory&user={USERNAME}&password={PASSWORD}&useSSL=false
properties:
When I run locally using
grails run-app
the app runs correctly
I run
./gradlew appengineDeploy to deploy and it deploys correctly
But when I try to open the scaffolded pages in the browser, I see the following error in the logs
==> gcloud app logs tail -s default
ERROR --- [ main] o.h.engine.jdbc.spi.SqlExceptionHelper
: Driver:com.mysql.jdbc.Driver#75b3ef1a returned null for
URL:jdbc:cloudsql://google/{DATABASE_NAME}?cloudSqlInstance=
{INSTANCE_NAME}&socketFactory=com.google.cloud.sql.mysql.SocketFactory&us
er={USERNAME}&password={PASSWORD}&useSSL=false
In addition the following error is also seen in the logs
ERROR --- [ Thread-16] .SchemaDropperImpl$DelayedDropActionImpl :
HHH000478: Unsuccessful: alter table property drop foreign key
FKgcduyfiunk1ewg7920pw4l3o9
Does the HH indicate that it is using the h2 database in production env?
Please help debug.
It seems the issue linked to hibernate. The same error for grails discussed here:
https://hibernate.atlassian.net/browse/HHH-11470
You are using MySQL because as you can see the error code say
Driver:com.mysql.jdbc.Driver#75b3ef1a returned null for
Your problem is that you need to configure the URL with your particular details changing {DATABASE_NAME} with the name of your database
you can see how to replace in the example at http://guides.grails.org/grails-google-cloud/guide/index.html#dataSourceGoogleCloudSQL

Unable to SSH to Google Cloud

I installed the Google Cloud SDK
Thought Web UI I created a new instance. I am not knowledgeable of SSH. I followed steps as described here: https://cloud.google.com/compute/docs/instances#sshkeys
I have Window 7 OS
I checked firewall rules as suggested here: https://cloud.google.com/compute/docs/troubleshooting#ssherrors
I checked these through Web UI and found rule
"default-allow-ssh 0.0.0.0/0 tcp:22 Apply to all targets"
STEPS I FOLLOWED:
1) > gcloud auth login
(default browser opens up and I authorize the Google Cloud SDK)
Google SDK Shell outputs:
"Saved Application Credentails. You are now logged as [someuser#gmail]
Your current project is [some-project-999].
2) > gcloud compute ssh my-instance --zone us-central1-a
Google SDK Shell outputs:
WARNING: You do not have an SSH key for Google Compute Engine.
WARNING: [C:\Program Files\Google\Cloud SDK\google-cloud-sdk\bin\..\bin\sdk\ssh-keygen.EXE] will be executed to generate
a key.
Generating public/private rsa key pair.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
The key fingerprint is:
ssh-rsa 2048 06:73:ac:e8:f2:31:c8:df:d4:b0:a2:3b:a2:53:6c:09
Your private key has been saved in C:\Users\First Last\.ssh\google_compute_engine.
Your public key has been saved in C:\Users\First Last\.ssh\google_compute_engine.pub.
Your putty key has been saved in C:\Users\First Last\.ssh\google_compute_engine.ppk.
Updated [https://www.googleapis.com/compute/v1/projects/arctic-depth-863].
Server refused our key
FATAL ERROR: Disconnected: No supported authentication methods available (server sent: publickey)
Server refused our key
FATAL ERROR: Disconnected: No supported authentication methods available (server sent: publickey)
Server refused our key
FATAL ERROR: Disconnected: No supported authentication methods available (server sent: publickey)
FATAL ERROR: Network error: Software caused connection abort
FATAL ERROR: Network error: Connection timed out
ERROR: (gcloud.compute.ssh) Could not SSH to the instance. It is possible that your SSH key has not propagated to the i
nstance yet. Try running this command again. If you still cannot connect, verify that the firewall and instance are set
to accept ssh traffic.
On the browser's Web UI, I open the Broser's SSH and I navigate to .ssh folder
someuser_gmail_com#my-instance:~$ cd .ssh
someuser_gmail_com#my-instance:~$ cat authorized_keys
# Added by Google
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4OxYxWvIlp...F7As google-ssh {"userName":"someuser#gmail.com","expireOn":"2015-02-21T23:29:06+0000"}
# Added by Google
ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzd...KRqcUZmvWr= google-ssh {"userName":"someuser#gmail.com","expireOn":"2015-02-21T23:28:55+0000"}
on Web UI, I navigate to Project's > Compute > Compute Engine > Metadata > SSH KEYS and I see three records
USERNAME KEY
someuser_gmail_com ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4...", "edpireOn":"2015-02-21T23:29:06+0000"}
someuser_gmail_com ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTIt...", "edpireOn":"2015-02-21T23:29:06+0000"}
First Last ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAi...ZkpSpRt6RQ== First Last#MYPC
In my local computer, I navigate to Users/First Last/.ssh/google_compute_engine.pub and I see
ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAit...mGhUKZRgFZkpSpRt6RQ== First Last#MYPC
QUESTIONS:
Does white space in the user's folder path causes problems (i.e. "First Last")?
When the key is created by the Google Cloud SDK, it sets the comment to First Last#MYPC. Is this the correct setting? (I have been reading and trying this and that and I suspect it should be something like someuser#my-instance-public-IP)
When I
Google Cloud SDK > gcloud compute instances describe my-instance --zone us-central1-a --format yaml
canIpForward: false
creationTimestamp: '2015-02-21T14:53:37.276-08:00'
disks:
- autoDelete: true
boot: true
deviceName: my-instance
index: 0
interface: SCSI
kind: compute#attachedDisk
licenses:
- https://www.googleapis.com/compute/v1/projects/ubuntu-os-cloud/global/licenses/ubuntu-1204-precise
mode: READ_WRITE
source: https://www.googleapis.com/compute/v1/projects/some-project-999/zones/us-central1-a/disks/my-instance
type: PERSISTENT
id: '111812933445597333'
kind: compute#instance
machineType: https://www.googleapis.com/compute/v1/projects/some-project-999/zones/us-central1-a/machineTypes/g1-small
metadata:
fingerprint: w3steEkuQUS=
kind: compute#metadata
name: my-instance
networkInterfaces:
- accessConfigs:
- kind: compute#accessConfig
name: External NAT
natIP: 112.134.99.170
type: ONE_TO_ONE_NAT
name: nic0
network: https://www.googleapis.com/compute/v1/projects/some-project-999/global/networks/default
networkIP: 10.356.252.66
scheduling:
automaticRestart: true
onHostMaintenance: MIGRATE
selfLink: https://www.googleapis.com/compute/v1/projects/some-project-999/zones/us-central1-a/instances/my-instance
serviceAccounts:
- email: 78111222333-compute#developer.gserviceaccount.com
scopes:
- https://www.googleapis.com/auth/devstorage.read_only
- https://www.googleapis.com/auth/logging.write
status: RUNNING
tags:
fingerprint: DLYFgkKTlB3=
items:
- http-server
zone: https://www.googleapis.com/compute/v1/projects/some-project-999/zones/us-central1-a
C:\Program Files\Google\Cloud SDK>
This is a known issue when using Cloud SDK from Windows.
Please download pageant.exe at [1] and use it to load your ppk key or use Putty (downloadable from the same link) to SSH to instance as documented at [2].
As a workaround you can even rename ssh.exe to ssh-bak.exe and ssh-term.exe to ssh.exe in C:\Program Files\Google\Cloud SDK\google-cloud-sdk\bin\sdk\
Link:
[1] - http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html
[2] - https://cloud.google.com/compute/docs/console#sshkeys

Resources