Gcloud deploy is not deploying a folder - google-app-engine

I am servicing a system.
An old App Engine tool was used to deploy, but it is now obsolete and I need to use the gcloud command line.
However, gcloud is ignoring one system folder and deploying all others.
One big reason this must be happening is because they developed this system based on node_modules and didn't build the code before uploading. In short, I need the node_modules folder for the code to execute.
I already disabled .gcloudignore, but there have been no changes.
Source folder structure:
Folder Structure in App Engine:
I ran the --verbosity = info code on my gcloud deploy and it appeared that they were ignoring it:
My app.yaml:
runtime: python27
api_version: 1
threadsafe: true
libraries:
- name: jinja2
version: 2.6
- name: lxml
version: 2.3
- name: pycrypto
version: 2.6
handlers:
- url: /static
static_dir: app/static
secure: always
- url: /.*
script: manage.app
secure: always
builtins:
- remote_api: on

By default, flask use a "templates" folder to contain all your template files(any plain-text file, but usually .html or some kind of template language such as jinja2 ) & a "static" folder to contain all your static files(i.e. .js .css and your images).
So ideally you should not use /static folder
you will need to update app.yaml to include handler static_dir: static
runtime: python27
api_version: 1
threadsafe: true
libraries:
- name: ssl
version: latest
# [START handlers]
handlers:
- url: /static
static_dir: static
- url: /.*
script: main.app
# [END handlers]
The best way to use is
from flask import Flask
app = Flask(__name__, static_folder='static', static_url_path='')

Related

Why does Google App Engine Standard Edition for Java create a __static__ folder and double the number of files deployed to a project?

I am deploying a web application to Google App Engine Standard edition using Java 8.
It is a documentation web site plus servlets that includes a huge amount of static web content (built with Jekyll).
I recently started getting this error trying to deploy the app:
ERROR: (gcloud.app.deploy) INVALID_ARGUMENT: This deployment has too many files.
New versions are limited to 10000 files for this app.
- '#type': type.googleapis.com/google.rpc.BadRequest
fieldViolations:
- description: This deployment has too many files.
New versions are limited to 10000 files for this app.
field: version.deployment.files[...]
My src/main/webapp folder has 5016 files.
But after running:
mvn clean install appengine:stage
my target/appengine-staging folder has 10113 files.
I see that the files are in 2 locations, both
target/appengine-staging
and
target/appengine-staging/__static__
Why do they appear twice?
Is this normal?
Or am I doing something wrong?
How can I avoid this duplication?
Or do I have to limit my files to under 5000?
Thanks for any advice.
Note: The generated app.yaml looks like this:
runtime: java8
instance_class: F1
inbound_services:
- warmup
derived_file_type:
- java_precompiled
threadsafe: True
auto_id_policy: default
beta_settings:
'source_reference': 'https://...'
api_version: 'user_defined'
handlers:
- url: (/.*/)
static_files: __static__\1index.html
upload: __NOT_USED__
require_matching_file: True
login: optional
secure: always
- url: (/)
static_files: __static__\1index.html
upload: __NOT_USED__
require_matching_file: True
login: optional
secure: always
- url: (/.*)
static_files: __static__\1
upload: __NOT_USED__
require_matching_file: True
login: optional
secure: always
- url: /.*
script: unused
login: optional
secure: always
skip_files: app.yaml

Access static json file in google app engine 2.7 standard environment - IOError(2, 'No such file or directory')

We are trying to access .json file inside our python code in Google App Engine.
Here is our app.yaml
service: worker
instance_class: F2
runtime: python27
api_version: 1
threadsafe: true
automatic_scaling:
max_instances: 100
min_pending_latency: 200ms # default value
max_pending_latency: automatic
handlers:
- url: /stylesheets
static_dir: stylesheets
- url: /images
static_dir: images
- url: /javascripts
static_dir: javascripts
- url: /(.*\.(gif|png|jpg|json))$
static_files: static/\1
upload: static/.*\.(gif|png|jpg|json)$
application_readable: true
- url: /static
static_dir: static
application_readable: true
- url: /favicon\.ico
static_files: favicon.ico
upload: favicon\.ico
- url: /.*
script: worker.app
login: admin
libraries:
- name: webapp2
version: "2.5.2"
- name: jinja2
version: latest
We are using it in our application like this.
path = os.path.join(os.path.dirname(__file__)[0], '/static/google_web_app_script_credentials.json')
flow = InstalledAppFlow.from_client_secrets_file(path,self.GOOGLESHEETS_SCOPES )
But we are receiving the error.
IOError(2, 'No such file or directory')
We have tried various options of changing path including path = os.path.join(os.path.dirname(file)[0], '../static/google_web_app_script_credentials.json')
When we go to the GCP console -> Debug and review the current code we are able to see the .json file under static folder. This means that the file is there, we are not able to get the application to read it. This service is as "worker".
From your App Engine app, you don't have access to your static files. Move the file to another location within your project, and you will then be able to access it.
Even more importantly, you don't want to store credentials in your static folder! Anyone on the web will be able to access them via the URL!

How to serve source folder on Google App Engine?

The below app.yaml pasted below gives error:
sre_constants.error: cannot refer to open group
application: villagegamedev
runtime: python27
api_version: 1
threadsafe: yes
handlers:
- url: /(.*)
static_files: ./\1
upload: ./\1
My express purpose for this test is to serve the source folder. When I Google this effort, the search only comes up with all the docs that say "App Engine does not serve files directly out of your application's source directory unless configured to do so." So how do I configure it to do so?
handlers:
- url: /
static_dir: static
Then just put all of the files you want to serve in a directory named static.
This link gives a full description.
Basically what you should do is;
Create a project on google cloud platform
Create a directory on your computer with the same name as the
project created above. Create your app.yaml file here.
Paste this code into the app.yaml file
runtime: python27
api_version: 1
threadsafe: true
handlers:
- url: /
static_files: www/index.html
upload: www/index.html
- url: /(.*)
static_files: www/\1
upload: www/(.*)
Install the google sdk into this folder (browse and select this
folder when prompted during installation).
Create a directory named www in your project folder. Save your static website files here.
Start up the google cloud sdk on your computer and run the following command
gcloud app deploy
7.run gcloud app browse to see your website in your browser.
Hope that helps!

How to properly setup modules for Google App Engine (PHP runtime)

I want to create very standard setup for GAE (php runtime): 2 modules with specific URLs (routings):
module-api for REST API
module-oli for backend static process ..
regarding to doc (https://cloud.google.com/appengine/docs/php/modules/ and https://cloud.google.com/appengine/docs/php/modules/routing) I've created 3 .yaml config files:
dispatch.yaml
application: ABC
dispatch:
- url: "*/oli/*"
module: module-oli
- url: "*/"
module: module-api
app.yaml
application: ABC
version: v1
module: module-api
runtime: php55
api_version: 1
handlers:
- url: /.*
script: public/api.php
module-oli.yaml
application: ABC
version: v1
module: module-oli
runtime: php55
api_version: 1
manual_scaling:
instances: 1
handlers:
- url: /.*
script: public/oli.php
I also tried many changes in URL handling, but the error I always get is "Duplicate module: module-api".
Can you help me please? Thank you in advance ..
I have no idea why, but (on my computer) it does not work just from googleAppEngineLauncher.app (OSX 10.11.3, googleAppEngineLauncher version 1.9.32), but it works if I use command line tools:
dev_appserver.py app.yaml module-oli.yaml
and only if I do not use param --skip_sdk_update_check (does not matter value)
more about naming modules (and using default module or not) .. it all works even if I name default module with specific name (of course, it causes error when not all request are dispatched to the module).
more about dispatching to modules; it works (as it's documented here). of course, it does not work at local development environment, as every single module runs at a different port (than the dispatch does not work, and if there is a module with manual_scaling, the _ah/start can not be handled and as err500 it stops the start)
here are all setup files, and how to run it:
app.yaml
application: <APPLICATION-ID>
module: default
version: v1
runtime: php55
api_version: 1
threadsafe: yes
handlers:
- url: /favicon\.ico
static_files: favicon.ico
upload: favicon\.ico
- url: .*
script: blablabla.php
module-oli.yaml
application: <APPLICATION-ID>
version: v1
module: module-oli
runtime: php55
api_version: 1
threadsafe: yes
instance_class: B1
manual_scaling:
instances: 1
handlers:
- url: /oli/.*
script: blebleble-oli.php
dispatch.yaml
dispatch:
- url: "*/oli/*"
module: module-oli
- url: "*/*"
module: default
to run on local development environment: dev_appserver.py app.yaml module-oli.yaml .. (no automatic routing, instances are running on different ports)
one everything is deployed to gcloud and dispatching is updated (appcfg.py -A wellfedcat-1221 update_dispatch .), it works like this:
APPLICATION-ID.appspot.com/* : served by default module ..
APPLICATION-ID.appspot.com/oli/* : served by module-oli ..
the dispatching is necessary when you want to use our own domain to map to gcloud ..
thank you #Tom for help!

Is xlrd supported in Google App Engine?

I have seen posts in Google Groups and here that mention using xlrd in Google App Engine:
How do you read excel files with xlrd on Appengine
https://groups.google.com/forum/?fromgroups=#!searchin/google-appengine-python/xlrd/google-appengine-python/lMix6vXhvtA/O_ExzkGhsKEJ
but when I try to add it to the app.yaml file, it says "the library xlrd is not supported". Also, in the list of 3rd party libraries I don't see xlrd.
https://developers.google.com/appengine/docs/python/tools/libraries27
But then other people seem to be using it on GAE, so does GAE support it or not? In the past I was able to import numpy successfully. Here is app.yaml. On main.py I have "import xlrd". Thank you very much.
application: uploadsample
version: 1
runtime: python27
api_version: 1
threadsafe: yes
handlers:
- url: /favicon\.ico
static_files: favicon.ico
upload: favicon\.ico
- url: .*
script: main.app
libraries:
- name: webapp2
version: "2.5.1"
- name: xlrd
version: "0.7.3"
You can include any pure python third-party libraries with your app that you want. You don't specify them in app.yaml, that's only for third-party libraries that are included by Google on the servers.
xlrd is not "supported" per se, but it should work.

Resources