Google AppEngine Getting 403 forbidden trying to update cron.yaml - google-app-engine

I am following the docs on how to backup datastore using AppEngine.
I am performing a gcloud app deploy cron.yaml command on a GCE VM that is meant to update a cronjob in AppEngine. the GCE VM and AppEngine cron are in the same project, and I have granted AppEngine admin to the GCE VM via a default Service Account. When I run this on my local machine, it updates fine. However on the GCE instance, thats where issues arise
Here are the files
app.yaml
runtime: python27
api_version: 1
threadsafe: true
service: cloud-datastore-admin
libraries:
- name: webapp2
version: "latest"
handlers:
- url: /cloud-datastore-export
script: cloud_datastore_admin.app
login: admin
cron.yaml
cron:
- description: "Daily Cloud Datastore Export"
url: /cloud-datastore-export?namespace_id=&output_url_prefix=gs://<my-project-id>-bucket
target: cloud-datastore-admin
schedule: every 24 hours
cloud_datastore_export.yaml
import datetime
import httplib
import json
import logging
import webapp2
from google.appengine.api import app_identity
from google.appengine.api import urlfetch
class Export(webapp2.RequestHandler):
def get(self):
access_token, _ = app_identity.get_access_token(
'https://www.googleapis.com/auth/datastore')
app_id = app_identity.get_application_id()
timestamp = datetime.datetime.now().strftime('%Y%m%d-%H%M%S')
output_url_prefix = self.request.get('output_url_prefix')
assert output_url_prefix and output_url_prefix.startswith('gs://')
if '/' not in output_url_prefix[5:]:
# Only a bucket name has been provided - no prefix or trailing slash
output_url_prefix += '/' + timestamp
else:
output_url_prefix += timestamp
entity_filter = {
'kinds': self.request.get_all('kind'),
'namespace_ids': self.request.get_all('namespace_id')
}
request = {
'project_id': app_id,
'output_url_prefix': output_url_prefix,
'entity_filter': entity_filter
}
headers = {
'Content-Type': 'application/json',
'Authorization': 'Bearer ' + access_token
}
url = 'https://datastore.googleapis.com/v1/projects/%s:export' % app_id
try:
result = urlfetch.fetch(
url=url,
payload=json.dumps(request),
method=urlfetch.POST,
deadline=60,
headers=headers)
if result.status_code == httplib.OK:
logging.info(result.content)
elif result.status_code >= 500:
logging.error(result.content)
else:
logging.warning(result.content)
self.response.status_int = result.status_code
except urlfetch.Error:
logging.exception('Failed to initiate export.')
self.response.status_int = httplib.INTERNAL_SERVER_ERROR
app = webapp2.WSGIApplication(
[
('/cloud-datastore-export', Export),
], debug=True)
The Error I'm getting is
Configurations to update:
descriptor: [/usr/local/sbin/pluto/<my-project-id>/datastore/cron.yaml]
type: [cron jobs]
target project: [<my-project-id>]
Do you want to continue (Y/n)?
Updating config [cron]...
failed.
ERROR: (gcloud.app.deploy) Server responded with code [403]:
Forbidden Unexpected HTTP status 403.
You do not have permission to modify this app (app_id=u'e~<my-project-id>').
I have checked other posts related to this, however they seem to deal with an old version/deployment of appengine
Service Accounts!

From Deploying using IAM roles:
To grant a user account the ability to deploy to App Engine:
Click Add member to add the user account to the project and then select all of the roles for that account by using the dropdown menu:
Required roles to allow an account to deploy to App Engine:
a. Set the one of the following roles:
Use the App Engine > App Engine Deployer role to allow the account to deploy a version of an app.
To also allow the dos.yaml or dispatch.yaml files to be deployed with an app, use the App Engine > App Engine Admin role
instead.
The user account now has adequate permission to use the Admin API to deploy apps.
b. To allow use of App Engine tooling to deploy apps, you must also give the user account the Storage > Storage Admin role
so that the tooling has permission to upload to Cloud Storage.
Optional. Give the user account the following roles to grant permission for uploading additional configuration files:
Cloud Scheduler > Cloud Scheduler Admin role: Permissions for uploading cron.yaml files.
Potentially of interest:
Deployments with predefined roles
Predefined roles comparison matrix

Okay after some tinkering. I added the project editor role to the service account linked to the GCE instance running my server. I am not fully aware if this is the role with least priviledge to enable this to work.

Related

Connectivity issues from App Engine to CloudSQL using Private IP and Serverless VPC Access

I am doing a PoC to connect from Google App Engine to CloudSQL instance running with Private IP on a SharedVPC . The sample application for testing is from
https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/cloud-sql/mysql/sqlalchemy/README.md
My Serverless VPC Connector Range is as follows
$ gcloud compute networks vpc-access connectors list --region=us-central1
CONNECTOR_ID REGION NETWORK IP_CIDR_RANGE MIN_THROUGHPUT MAX_THROUGHPUT STATE
testserverlessvpc us-central1 kube-shared-vpc 192.168.60.0/28 200 300 READY
Private IP Range is as follows
$ gcloud compute addresses list --global --filter="purpose=VPC_PEERING"
NAME ADDRESS/RANGE TYPE PURPOSE NETWORK REGION SUBNET STATUS
cloudsqltestrangenew 10.0.100.0/20 INTERNAL VPC_PEERING kube-shared-vpc RESERVED
MySQL instance is as follows
$ gcloud sql instances list
NAME DATABASE_VERSION LOCATION TIER PRIMARY_ADDRESS PRIVATE_ADDRESS STATUS
mysql2 MYSQL_5_7 us-central1-b db-f1-micro - 10.0.100.5 RUNNABLE
app.standard.yaml is as follows
runtime: python37
service: appcentralpri
env_variables:
CLOUD_SQL_CONNECTION_NAME: projectname:us-central1:mysql2
DB_USER: guestdb
DB_PASS: password
DB_NAME: testdb
DB_HOST: 10.0.100.5:3306
vpc_access_connector:
name: projects/hostproject-26a2/locations/us-central1/connectors/testserverlessvpc
Deployment went through fine and no errors in log encountered
gcloud app deploy app.standard.yaml
However when I try to connect to the application endpoint , the page is not accessible with "Server Not Found"
I have given VPC Access and Network Compute User role to the below App Engine Default Service account on VPC Host Project
<hostproject>-26a2#appspot.gserviceaccount.com
UPDATES
Also added below in app.yaml but no luck
env_variables:
GAE_USE_SOCKETS_HTTPLIB : 'anyvalue'
https://cloud.google.com/appengine/docs/standard/python/sockets#making_httplib_use_sockets
Same setup for CloudRun and CloudSQL worked for me but struggling with GAE. Any suggestions to resolve this issue ?
Your app.yaml file is wrong formatted. The vpc_access_connector: need to be at the root level. Here it's taken as an environment variable value.
runtime: python37
service: appcentralpri
env_variables:
CLOUD_SQL_CONNECTION_NAME: projectname:us-central1:mysql2
DB_USER: guestdb
DB_PASS: password
DB_NAME: testdb
DB_HOST: 10.0.100.5:3306
vpc_access_connector:
name: projects/hostproject-26a2/locations/us-central1/connectors/testserverlessvpc

Google app engine removes session cookies when authorizing socket.io connection

I am trying to get the sessionID of a user from express-session when the user opens up a WebSocket connection. The code below shows how this is done. The problem is that when I deploy my server to google app engine, I can no longer see the session cookie when authorizing the socket connection. It works fine when running locally, so google app engine must be doing something to remove the cookie? I'm not too familiar with the google app engine so any help would be greatly appreciated.
I've tried many different solutions I found online such as passport.socketio from npm, many of which I believe implement the same solution that I ended up writing myself.
Edit 1:
To be more specific, data.headers.cookies contains an io, connect.sid and express.sid cookie when running from localhost. When running on the google app engine, it only contains the io cookie. (End of edit)
Edit 2:
I thought it could be an issue with having different instances of my server due to google's automatic scaling, so I've changed my setting in app.yaml to ensure that wont happen:
network:
session_affinity: true
manual_scaling:
instances: 1
(End of edit)
Code:
var session = require('express-session');
var MemoryStore = require('memorystore')(session);
var store = new MemoryStore({
checkPeriod: 86400000
}); // Likely to switch to RedisStore
app.use(session({
store: store,
secret: 'jpcs-0001080900DRXPXL',
saveUninitialized: false,
resave: true,
key: 'express.sid'
}));
// ...
io.set('authorization', (data, accept) => {
if (data && data.headers && data.headers.cookie) {
console.log('COOKIES:');
console.log(data.headers.cookie);
cookies_str = data.headers.cookie;
cookies_arr = cookies_str.split(';');
cookies = {};
for (index in cookies_arr) {
cookie = cookies_arr[index].split('=');
if (cookie.length != 2) continue;
key = cookie[0].replace(/ /g,'');
val = cookie[1];
cookies[key] = val;
}
if (!cookies['express.sid']) accept('User not signed in', null);
// HERE IS MY PROBLEM:
// When running from google app engine cookies['express.sid'] is null
sessionId = cookies['express.sid'].split('.')[0].substring(4);
data.sessionId = sessionId;
store.get(sessionId, (err, session) => {
if (!err && session) {
data.session = session;
accept(null, true);
}
else if (err) accept('Could not get session', false);
else if (!session) accept('Could not get session', false)
});
}
accept('Could not get session', false);
});
So I expect cookies['express.sid'] to contain the sessionID in it, but on google app engine it is null.
App.yaml
runtime: nodejs10 # For Node.js 8, use runtime: nodejs8
#instance_class: F2
#env_variables:
#BUCKET_NAME: "example-gcs-bucket"
handlers:
- url: .*
secure: always
redirect_http_response_code: 301
script: auto
network:
session_affinity: true
manual_scaling:
instances: 1
It's more likely that your application cannot retrieve the user from the session store when running on GAE due to a network or app configuration. Some more info on which session store you are using, the configuration for it in express, and the app.yaml would be more helpful in solving this. Its unlikely that GAE would be removing the cookie from the HTTP response headers.
I found that in order to use redis with express/express-session I needed to set the network name in app.yaml.
network:
name: default
Unfortunately it doesn't seem like you can use this for standard environments, so I also had to change:
runtime: nodejs
env: flex
And also needed to specify an engine in package.json when switching to a flexible environment.
"engines": {
"node": "10.x"
}

Permission denied during gcloud app deploy using Google Cloud SDK

It is insanely hard, to deploy an app to Google App Engine, using Google Cloud SDK.
I had tried the below 2 commands
C:\Users\yccheok\Desktop\jstock-android-appengine>gcloud config set project jstock-android
Updated property [core/project].
C:\Users\yccheok\Desktop\jstock-android-appengine>gcloud app deploy app.yaml --log-http --verbosity=debug
DEBUG: Running [gcloud.app.deploy] with arguments: [--log-http: "true", --verbosity: "debug", DEPLOYABLES:1: "['app.yaml']"]
DEBUG: No staging command found for runtime [python27] and environment [STANDARD].
DEBUG: API endpoint: [https://appengine.googleapis.com/], API version: [v1]
=======================
==== request start ====
uri: https://appengine.googleapis.com/v1/apps/jstock-android?alt=json
method: GET
== headers start ==
Authorization: Bearer ya29.GlxEBb1XVP1JK93-ARiaN_ZgiMbvZmw5KWfvJVfibDJ4FK_ZaMRoU1jVDTiWzsY606GSduJKJd9Nm8zA-_Iql5mGn4AMk4QVl8mPRycfekeZnOOHtbUvpkBMgOLOQA
accept: application/json
accept-encoding: gzip, deflate
content-length: 0
user-agent: google-cloud-sdk x_Tw5K8nnjoRAqULM9PFAC2b gcloud/184.0.0 command/gcloud.app.deploy invocation-id/c9ae232d33b346d787b95a36e28c38c0 environment/None environment-version/None interactive/True python/2.7.13 (Windows NT 10.0.16299)
== headers end ==
== body start ==
== body end ==
==== request end ====
---- response start ----
-- headers start --
-content-encoding: gzip
alt-svc: hq=":443"; ma=2592000; quic=51303431; quic=51303339; quic=51303338; quic=51303337; quic=51303335,quic=":443"; ma=2592000; v="41,39,38,37,35"
cache-control: private
content-length: 335
content-type: application/json; charset=UTF-8
date: Tue, 16 Jan 2018 19:16:21 GMT
server: ESF
status: 403
transfer-encoding: chunked
vary: Origin, X-Origin, Referer
x-content-type-options: nosniff
x-frame-options: SAMEORIGIN
x-xss-protection: 1; mode=block
-- headers end --
-- body start --
{
"error": {
"code": 403,
"message": "Operation not allowed",
"status": "PERMISSION_DENIED",
"details": [
{
"#type": "type.googleapis.com/google.rpc.ResourceInfo",
"resourceType": "gae.api",
"description": "The \"appengine.applications.get\" permission is required."
}
]
}
}
-- body end --
total round trip time (request+response): 1.796 secs
---- response end ----
----------------------
DEBUG: (gcloud.app.deploy) Permissions error fetching application [apps/jstock-android]. Please make sure you are using the correct project ID and that you have permission to view applications on the project.
Traceback (most recent call last):
File "C:\Program Files (x86)\Google\Cloud SDK\google-cloud-sdk\lib\googlecloudsdk\calliope\cli.py", line 797, in Execute
resources = calliope_command.Run(cli=self, args=args)
File "C:\Program Files (x86)\Google\Cloud SDK\google-cloud-sdk\lib\googlecloudsdk\calliope\backend.py", line 757, in Run
resources = command_instance.Run(args)
File "C:\Program Files (x86)\Google\Cloud SDK\google-cloud-sdk\lib\surface\app\deploy.py", line 65, in Run
parallel_build=False)
File "C:\Program Files (x86)\Google\Cloud SDK\google-cloud-sdk\lib\googlecloudsdk\command_lib\app\deploy_util.py", line 543, in RunDeploy
app = _PossiblyCreateApp(api_client, project)
File "C:\Program Files (x86)\Google\Cloud SDK\google-cloud-sdk\lib\googlecloudsdk\command_lib\app\deploy_util.py", line 703, in _PossiblyCreateApp
api_client._FormatApp()))) # pylint: disable=protected-access
HttpException: Permissions error fetching application [apps/jstock-android]. Please make sure you are using the correct project ID and that you have permission to view applications on the project.
ERROR: (gcloud.app.deploy) Permissions error fetching application [apps/jstock-android]. Please make sure you are using the correct project ID and that you have permission to view applications on the project.
C:\Users\yccheok\Desktop\jstock-android-appengine>
Then, I went through https://cloud.google.com/appengine/docs/admin-api/accessing-the-api , it mentioned I need to use Admin API. So, I do it step by step carefully.
Step 1
Step 2
It mentions Admin API is enabled. Now I need credential.
Step 3
Step 4
OK. Now they mention I don't need create new credential. I can use Application Default Credentials ?!
Step 5
So, I went to https://developers.google.com/identity/protocols/application-default-credentials?hl=en_GB . I learn that I need to run
C:\Users\yccheok\Desktop\jstock-android-appengine>gcloud auth application-default login
Your browser has been opened to visit:
https://accounts.google.com/o/oauth2/auth?redirect_uri=http%3A%2F%2Flocalhost%3A8085%2F&prompt=select_account&response_type=code&client_id=764086051850-6qr4p6gpi6hn506pt8ejuq83di341hur.apps.googleusercontent.com&scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fuserinfo.email+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fcloud-platform&access_type=offline
Credentials saved to file: [C:\Users\yccheok\AppData\Roaming\gcloud\application_default_credentials.json]
These credentials will be used by any library that requests
Application Default Credentials.
Step 6
Step 7
Still, after completing the above 7 steps, I still get the exact same error message, when trying to run
gcloud app deploy app.yaml --log-http --verbosity=debug
Can anyone let me know, what step I'm still require, in order to deploy my Python app to Google App Engine, using Google Cloud SDK?
The Admin API is for programmatically deploying the app, not for deploying using gcloud app deploy, for which you don't even need the Admin API enabled for your app.
From Deploying a Python App:
To programmatically deploy your apps, use the Admin API.
Before you begin
Before you can deploy your app:
The Owner of the GCP project must create the App Engine application.
Ensure that your user account includes the required privileges.
(but I can see how the above could be mis-intrepreted as an invitation to use the Admin API)
Most likely the account actually used by gcloud app deploy is missing or doesn't have the required permissions.
You can check the acount used with gcloud auth list. In my case the account is my email address, not a service account (I'm not sure if a service account can be used).
Use gcloud auth login (and maybe gcloud auth revoke) if you need a different account.
And you can check the account's privileges (if any) on the project/app on the IAM Page.
One additional note on this,
When you enable the App Engine API and the cloud builder api, make sure the Cloud Build Service Account also has access to the project.
I ran into that problem after enabling the correct apis.
This was using a build trigger. I could deploy locally from a command line because I was authenticated as myself. However, if you are using a build trigger, it will use the build service account, which needs access.
Hope this helps.
I had this issue. In my case it was solved by setting the project using the project ID, rather than the project name. See this answer
gcloud app deploy ERROR: Permissions error fetching application [apps/<PROJECT_NAME>]

gcloud app deploy give 400 / forbidden error / cannot push img to google container registry

I am trying to deploy .net core application from google compute vm to google app engine using gcloud app deploy. I get the following error
> WARNING: We couldn't validate that your project is ready to deploy to App Engine Flexible Environment. If deployment fails, please check the following mess
age and try again:
Server responded with code [400]:
Bad Request Unexpected HTTP status 400.
Failed Project Preparation (app_id='s~project-id'). Out of retries. Last error: Temporary error occurred while verifying project: TEMPORARY_ERROR: Unabl
e to check API status
Beginning deployment of service [default]...
WARNING: Deployment of App Engine Flexible Environment apps is currently in Beta
Building and pushing image for service [default]
Some files were skipped. Pass `--verbosity=info` to see which ones.
ERROR: (gcloud.app.deploy) Could not copy [/tmp/tmpLwvVOb/src.tgz] to [us.gcr.io/project-id/appengine/default.20170118t043919:latest]: HttpError accessing
<https://www.googleapis.com/resumable/upload/storage/v1/b/staging.project-id.appspot.com/o?uploadType=resumable&alt=json&name=us.gcr.io%2Fcasepro-v3%2Fappe
ngine%2Fdefault.20170118t043919%3Alatest>: response: <{'status': '403', 'content-length': '166', 'vary': 'Origin, X-Origin', 'server': 'UploadServer', 'x-g
uploader-uploadid': 'AEnB2UqprxH-2tIhsSZdGxDOtS8UnWSI29YTo4kaptNK67SWJpLVqR0zEtCAHgFyE64wj1HfCyUL5sy9z4AZkTRFYuxXfdw5TA', 'date': 'Wed, 18 Jan 2017 04:40:0
0 GMT', 'alt-svc': 'quic=":443"; ma=2592000; v="35,34"', 'content-type': 'application/json; charset=UTF-8'}>, content <{
"error": {
"errors": [
{
"domain": "global",
"reason": "forbidden",
"message": "Forbidden"
}
],
"code": 403,
"message": "Forbidden"
}
}
>. Please retry.
I have already enabled billing api, app engine admin api and storage api. Service a/c that is being used has editor rights. VM instance has been created using cloud launcher for Jenkins Bitnami package. I am trying to deploy app from command line from the vm before I configure Jenkins to do the same.
What to do to resolve this?
The problem is that gcloud app deploy is trying to deploy to the project id 'project-id', which cannot be your project id.
Try setting the project like this:
gcloud config set project MY-PROJECT-ID
Then, retry the gcloud app deploy command.
If this fails, please reply with your full gcloud command line, and the results of these two commands:
gcloud config list
gcloud version

Google App Engine Admin SDK Reports API returning 403 Insufficient Permission error

I started with some sample code for App Engine from Google.
My app needs to use the Directory API and the Reports API from the Google Admin SDK.
I have created a project in the API Console and turned on the Admin SDK in Services.
I added the scopes (the same ones as used in the code below) to the "Manage API client access" section of Advanced Tools in my domain's Google cpanel.
The call to the Directory API works.
After that, the call to the Reports API fails with the error message:
"HttpError: https://www.googleapis.com/admin/reports/v1/activity/users/all/applications/admin?alt=json returned "Insufficient Permission">"
Thanks much for the assistance.
import webapp2
import os
from apiclient.discovery import build
from oauth2client.appengine import OAuth2Decorator
from oauth2client.appengine import OAuth2DecoratorFromClientSecrets
from apiclient import errors
import logging
import json
decorator = OAuth2DecoratorFromClientSecrets(
os.path.join(os.path.dirname(__file__), 'client_secrets.json'),
'https://www.googleapis.com/auth/admin.directory.user.readonly')
directoryauthdecorator = OAuth2Decorator(
client_id='123.apps.googleusercontent.com',
client_secret='456-abc',
callback_path='/oauth2callback',
scope='https://www.googleapis.com/auth/admin.directory.user.readonly '
'https://www.googleapis.com/auth/admin.reports.audit.readonly '
'https://www.googleapis.com/auth/admin.reports.usage.readonly'
)
class MainHandler(webapp2.RequestHandler):
def get(self):
self.response.write('Hello world!')
class OAuthHandler(webapp2.RequestHandler):
#directoryauthdecorator.oauth_required
def get(self):
users = []
# Get the authorized Http object created by the decorator.
auth_http = directoryauthdecorator.http()
# Get the directory service
service = build("admin", "directory_v1", http=auth_http)
result = []
page_token = None
while True:
try:
param = {}
param['domain'] = 'mydomain.com'
if page_token:
param['pageToken'] = page_token
files = service.users().list(**param).execute()
result.extend(files['users'])
page_token = files.get('nextPageToken')
if not page_token:
break
except errors.HttpError, error:
print 'An error occurred: %s' % error
break
users = []
for user in result:
logging.info(user['primaryEmail'])
users.append(user['primaryEmail'])
param = {}
param['userKey'] = 'all'
param['applicationName'] = 'admin'
service = build('admin', 'reports_v1', http=auth_http)
# this call fails with the 403 Insufficient Permissions error
results = service.activities().list(**param).execute()
logging.info(results)
app = webapp2.WSGIApplication([
('/', MainHandler),
('/users', OAuthHandler),
(directoryauthdecorator.callback_path, directoryauthdecorator.callback_handler()),
], debug=True)
I read this post and cleared the credentials from the datastore.
Hitting the /users url again I got the redirect_uri error message.
I went back to the API project, fixed the Redirect URIs, and downloaded the client_secrets.json file.
Now both calls work (one to Directory API, the other to Reports API).

Resources