I'm trying to access my Cloud SQL instance from my App Engine flexible environment NodeJS application but kept getting timeout issues.
When I add the IP of the compute engine running the App engine to the Cloud SQL Access Control it works. The documentation states that access should be granted automatically if both are in the same project.
What am I missing?
I'm using the following connection settings for knex:
const config = {
host: 'myIP',
user: 'user',
password: 'password',
database: 'database',
port: 3306
};
The socket settings aren't working either. throwing a "Unhandled rejection Error: connect ENOENT /cloudsql" exception:
const config = {
socketPath: '/cloudsql/project:zone:instance',
user: 'user',
password: 'password',
database: 'database'
};
This error most likely indicates that the proxy process is not running on the GAE Flexible instance.
As per the documentation, you must have cloud_sql_instances set in your app.yaml file for the proxy to be automatically started.
Related
I want to use mongo atlas with strapi. SO I am reading the values from .env
these are the settings
connections: {
default: {
connector: 'mongoose',
settings: {
host: env('DB_MONGO_URL', '127.0.0.1'),
srv: env.bool('DATABASE_SRV', false),
port: env.int('DB_PORT', 27017),
database: env('DATABASE_NAME', 'mydb'),
username: env('DB_USER_NAME', null),
password: env('DB_PASSWORD', null)
},
options: {
authenticationDatabase: env('AUTHENTICATION_DATABASE', null),
ssl: env.bool('DB_SSL_ENABLE', false),
},
},
and here are the values
DB_MONGO_URL=myproject.random.mongodb.net/mydbt?ssl_cert_reqs=CERT_NONE
DB_USER_NAME=myuser
DB_PASSWORD=mypasswor
but this is showing
[2021-10-12T13:18:25.485Z] debug ⛔️ Server wasn't able to start properly.
[2021-10-12T13:18:25.490Z] error Error connecting to the Mongo database. Could not connect to any servers in your MongoDB Atlas cluster. One common reason is that you're trying to access the database from an IP that isn't whitelisted. Make sure your current IP address is on your Atlas cluster's IP whitelist: https://docs.atlas.mongodb.com/security-whitelist/
error Command failed with exit code 1.
In network i have allowed, 0.0.0.0 that includes any IP address.
If i pass this url in MongoDB Compass
mongodb+srv://myproject.random.mongodb.net/mydbt?ssl_cert_reqs=CERT_NONE
then this works
I tried the same as well
DB_MONGO_URL=mongodb+srv://myproject.random.mongodb.net/mydbt?ssl_cert_reqs=CERT_NONE
DB_USER_NAME=myuser
DB_PASSWORD=mypassword
but this shows
[2021-10-13T06:59:01.372Z] debug ⛔️ Server wasn't able to start properly.
[2021-10-13T06:59:01.374Z] error Error connecting to the Mongo database. getaddrinfo ENOTFOUND mongodb+srv
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
If i tried enable srv true
[2021-10-13T07:01:20.065Z] debug ⛔️ Server wasn't able to start properly.
[2021-10-13T07:01:20.067Z] error Error connecting to the Mongo database. URI does not have hostname, domain name and tld
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
how can i solve this?
I'm trying to proxy outbound API calls made from a Google App Engine application via a Google Compute Engine server VM instance running Squid proxy server.
The aim is that the REST api calls will all be made from a static ip address so that the 3rd party API will be able to identify and permit the calls via their firewall.
I have read and followed the instructions on this post:
connect Google App Engine and Google Compute Engine
I have managed to do the following so far:
Created a Google cloud compute VM and successfully assigned it a static external IP address.
Created a Serverless VPC access connector successfully (all resources are located in the same GAE region).
Added the vpc_access_connector name to my app.yaml in the Google App Engine project (which runs on Node.js).
Deployed the app using gcloud beta, with api calls being targeted towards the internal IP address of the proxy server, using the correct Squid default port (3128).
On issuing a request from the GAE app, I can see from the server logs that the correct IP address and port are being attempted but get the following error: "Error: tunneling socket could not be established, cause=connect ECONNREFUSED [my-internal-ip-address]:3128"
I've also tried running a curl command from the cloud shell interface, but the request times out every time.
If anyone could help solve this issue, I will be very grateful.
Here is a possible example of how to proxy outbound HTTP requests from an App Engine Standard application on NodeJS runtime via a Compute Engine VM running Squid, based on a slight modification of the available Google Cloud Platform documentation 1 2 and Quickstarts 3.
1. Create a Serverless VPC Access conector: Basically follow 2 to create the connector. After updating the gcloud components and enabling the Serverless VPC Access API on your project running the following command should suffice:
gcloud compute networks vpc-access connectors create [CONNECTOR_NAME] \
--network [VPC_NETWORK] \
--region [REGION] \
--range [IP_RANGE]
2. Create a Compute Engine VM to use as proxy: Basically follow 1 to set up a Squid proxy server:
a. Reserve a static external IP address and assign it to a Compute Engine VM.
b. Add a Firewall rule to allow traffic on Squid's default port: 3128. This command should work if you are using the default VPC network: gcloud compute firewall-rules create [FIREWALL_RULE_NAME] --network default --allow tcp:3128
c. Install Squid on the VM with the following command sudo apt-get install squid3.
d. Enable the acl localnet src entries in the Squid config files for the VPC Access connector:
sudo sed -i 's:#\(http_access allow localnet\):\1:' /etc/squid/squid.conf
sudo sed -i 's:#\(acl localnet src [IP_RANGE]/28.*\):\1:' /etc/squid/squid.conf
For example: if you used 10.8.0.0 as value for the [IP_RANGE] field for creating the connector, it should look something like sudo sed -i 's:#\(acl localnet src 10.8.0.0/28.*\):\1:' /etc/squid/squid.conf
e. Start the server with sudo service squid start
3. Modifications on App Engine application: Based on the Quickstart for Node.js modify the following files in order to create an application that crawls a webpage using the request-promise library and displays the HTML of the webpage. The request is send to the webpage using the VPC Access connector and the VM as a proxy with the modifications of the app.yaml and app.js files.
a. package.json
...
"test": "mocha --exit test/*.test.js"
},
"dependencies": {
"express": "^4.16.3",
"request": "^2.88.0",
"request-promise": "^4.2.5"
},
"devDependencies": {
"mocha": "^7.0.0",
...
b. app.js
'use strict';
// [START gae_node_request_example]
const express = require('express');
const app = express();
app.get('/', (req, res) => {
res
.status(200)
.send('Hello, world!')
.end();
});
//Add a handler to test the web crawler
app.get('/test', (req, res) => {
var request = require('request-promise');
request('http://www.input-your-awesome-website.com')
.then(function (htmlString) {
res.send(htmlString)
.end();
})
.catch(function (err) {
res.send("Crawling Failed...")
.end();
});
});
// Start the server
const PORT = process.env.PORT || 8080;
app.listen(PORT, () => {
console.log(`App listening on port ${PORT}`);
console.log('Press Ctrl+C to quit.');
});
// [END gae_node_request_example]
c. app.yaml
runtime: nodejs10
vpc_access_connector:
name: "projects/[PROJECT]/locations/[REGION]/connectors/[CONNECTOR_NAME]"
env_variables:
HTTP_PROXY: "http://[Compute-Engine-IP-Address]:3128"
HTTPS_PROXY: "http://[Compute-Engine-IP-Address]:3128"
Each time you go to the /test handler monitor that the requests go through the proxy by using sudo tail -f /var/log/squid/access.log command from the VM and checking the changes on the logs.
Notes: The connector, application and VM need to be on the same region to work and these are the supported regions for the connector.
I am new to both GKE and Django. I made an app in Django, made a docker container and push it to gcr and deploy it via GKE. The deployment works fine but when i try to login, I got the OperationalError. For database connection, I am using CloudSQL proxy.I have collected the static file and stored in google storage. Any help will be highly appreciated.
I have tried quite many opinions available already online but failed to succeed.
When i try to login as admin, I got the following output after input my username and password for login.
OperationalError at /admin/login
server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
Following are my database setting in Django.
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'polls',
'USER': os.getenv('DATABASE_USER'),
'PASSWORD': os.getenv('DATABASE_PASSWORD'),
'HOST': '127.0.0.1',
'PORT': '5432',
}
}
error while trying to login as admin
You should check in the docker logs , and check if there is any error connecting to the database
If it is a databse connection issue, then you can try the following in your docker-compose.yml . You can customize the rest of the variables mentioned, as needed for your polls application
you can try this
web:
build: ./app
image: {imagename}
depends_on:
- cloud-sql-proxy
environment:
- SQL_ENGINE=django.db.backends.postgresql_psycopg2
- SQL_DATABASE=test_db
- SQL_USER=postgres1
- SQL_PASSWORD=6728298
- SQL_HOST=cloud-sql-proxy
- SQL_PORT=5432
- DATABASE=postgres
cloud-sql-proxy:
image: gcr.io/cloudsql-docker/gce-proxy:1.11
command: /cloud_sql_proxy -instances=<INSTANCE_CONNECTION_NAME>=tcp:0.0.0.0:5432 -credential_file=/config
volumes:
- {service_account_creds_path.json}:/config
You could read this article https://adilsoncarvalho.com/how-to-use-cloud-sql-proxy-on-docker-compose-f7418c53eed9 for reference. The article is about mysql, but the concepts are the same . Good Luck!
I'm trying to connect to google sql cloud instance from custom runtime environment in App Engine.
When I follow the doc to connect using unix domain socket, it works. The problem is when I try to connect using a TCP connect. It shows:
Warning: mysqli_connect(): (HY000/2002): Connection refused in
/var/www/html/index.php on line 3
Connect error: Connection refused
This is my app.yaml file:
runtime: custom
env: flex
beta_settings:
cloud_sql_instances: testing-mvalcam:europe-west1:testdb=tcp:3306
resources:
cpu: 1
memory_gb: 0.5
disk_size_gb: 10
The Dockerfile:
FROM php:7.0-apache
ENV PORT 8080
CMD sed -i "s/80/$PORT/g" /etc/apache2/sites-available/000-default.conf /etc/apache2/ports.conf && docker-php-entrypoint apache2-foreground
RUN docker-php-ext-install mysqli
RUN a2enmod rewrite
COPY ./src /var/www/html
EXPOSE $PORT
And index.php:
<?php
$link = mysqli_connect('127.0.0.1', 'root', 'root', 'test');
if (!$link){
die('Connect error: '. mysqli_connect_error());
}
echo 'successfully connected';
mysqli_close($link);
?>
What am I doing Wrong?
The ip address ‘172.17.0.1’ is related with the docker container where the webserver is running, you can get more context on that in this documentation.
The documentation page you’re using might be lacking on adjusting the use case if you’re deploying with a presence of a Dockerfile. In the following documentation you can read more information about App Engine flexible runtimes.
As demonstrated by the documentation you’re using (remember to click on the TCP CONNECTION tab on this page), on the section of the app.yaml related to Cloud SQL instances information about the TCP port in use by the database server is needed.
you are my last chance here. So I've build a project with generator-sql-fullstack package and trying to deploy it to heroku.
Unfortunately, I can't connect to Heroku database as I get Error R10.
I've tried different ways to connect to postgres through sequelize - none of them worked. The main thing was to make it ssl compatible, but it still doesn't connect (or maybe I am using it wrong)
Project runs great locally, app is build without any errors also.
Please, take a look into Heroku logs and main files for connection. Hopefully, you will be able to help me here. Thank you.
$ heroku logs
2016-04-18T21:27:36.994697+00:00 app[web.1]: > node server/app.js
2016-04-18T21:27:36.994705+00:00 app[web.1]:
2016-04-18T21:27:37.784886+00:00 app[web.1]: Express server listening on 55648, in production mode
2016-04-18T21:28:34.891786+00:00 heroku[web.1]: Error R10 (Boot timeout) -> Web process failed to bind to $PORT within 60 seconds of launch
2016-04-18T21:28:34.891786+00:00 heroku[web.1]: Stopping process with SIGKILL
2016-04-18T21:28:35.750471+00:00 heroku[web.1]: Process exited with status 137
/server/config/environment/production.js
module.exports = {
database: 'DBNAME',
username: 'DBUSERNAME',
password: 'DBPASSWORD',
sql: {
host: 'DBHOST',
port: process.env.PORT || '5432',
dialect: 'postgres',
dialectOptions: {
ssl: true
}
}
};
/server/api/index.js
var sequelize = new Sequelize(config.database, config.username, config.password, config.sql);
Other than that, nothing was touched on server side.
My guess is that I am using Sequelize incorrectly, but I followed documentation (to adapt SSL for heroku).
Please, let me know if you want to see more configuration files.
Thank you for any input!
This:
var sequelize = new Sequelize(process.env.DATABASE_URL);
should be enough for connecting to Heroku's Postgres. You can check that DATABASE_URL in Heroku Dashboard -> YourApp -> Settings -> Config Variables. You can also take db credentials from that string, but you don't need to.
process.env.PORT is not the database port, but port on which you should start your server(so Heroku can do its internal routing things), like:
server.listen(process.env.PORT || 5432)
So you config file should look something like:
module.exports = {
port: process.env.PORT || 5432,
database: {
uri: process.env.DATABASE_URL
}
};
And connecting to database with:
var sequelize = new Sequelize(config.database.uri);