How to properly setup modules for Google App Engine (PHP runtime) - google-app-engine

I want to create very standard setup for GAE (php runtime): 2 modules with specific URLs (routings):
module-api for REST API
module-oli for backend static process ..
regarding to doc (https://cloud.google.com/appengine/docs/php/modules/ and https://cloud.google.com/appengine/docs/php/modules/routing) I've created 3 .yaml config files:
dispatch.yaml
application: ABC
dispatch:
- url: "*/oli/*"
module: module-oli
- url: "*/"
module: module-api
app.yaml
application: ABC
version: v1
module: module-api
runtime: php55
api_version: 1
handlers:
- url: /.*
script: public/api.php
module-oli.yaml
application: ABC
version: v1
module: module-oli
runtime: php55
api_version: 1
manual_scaling:
instances: 1
handlers:
- url: /.*
script: public/oli.php
I also tried many changes in URL handling, but the error I always get is "Duplicate module: module-api".
Can you help me please? Thank you in advance ..

I have no idea why, but (on my computer) it does not work just from googleAppEngineLauncher.app (OSX 10.11.3, googleAppEngineLauncher version 1.9.32), but it works if I use command line tools:
dev_appserver.py app.yaml module-oli.yaml
and only if I do not use param --skip_sdk_update_check (does not matter value)
more about naming modules (and using default module or not) .. it all works even if I name default module with specific name (of course, it causes error when not all request are dispatched to the module).
more about dispatching to modules; it works (as it's documented here). of course, it does not work at local development environment, as every single module runs at a different port (than the dispatch does not work, and if there is a module with manual_scaling, the _ah/start can not be handled and as err500 it stops the start)
here are all setup files, and how to run it:
app.yaml
application: <APPLICATION-ID>
module: default
version: v1
runtime: php55
api_version: 1
threadsafe: yes
handlers:
- url: /favicon\.ico
static_files: favicon.ico
upload: favicon\.ico
- url: .*
script: blablabla.php
module-oli.yaml
application: <APPLICATION-ID>
version: v1
module: module-oli
runtime: php55
api_version: 1
threadsafe: yes
instance_class: B1
manual_scaling:
instances: 1
handlers:
- url: /oli/.*
script: blebleble-oli.php
dispatch.yaml
dispatch:
- url: "*/oli/*"
module: module-oli
- url: "*/*"
module: default
to run on local development environment: dev_appserver.py app.yaml module-oli.yaml .. (no automatic routing, instances are running on different ports)
one everything is deployed to gcloud and dispatching is updated (appcfg.py -A wellfedcat-1221 update_dispatch .), it works like this:
APPLICATION-ID.appspot.com/* : served by default module ..
APPLICATION-ID.appspot.com/oli/* : served by module-oli ..
the dispatching is necessary when you want to use our own domain to map to gcloud ..
thank you #Tom for help!

Related

Gcloud deploy is not deploying a folder

I am servicing a system.
An old App Engine tool was used to deploy, but it is now obsolete and I need to use the gcloud command line.
However, gcloud is ignoring one system folder and deploying all others.
One big reason this must be happening is because they developed this system based on node_modules and didn't build the code before uploading. In short, I need the node_modules folder for the code to execute.
I already disabled .gcloudignore, but there have been no changes.
Source folder structure:
Folder Structure in App Engine:
I ran the --verbosity = info code on my gcloud deploy and it appeared that they were ignoring it:
My app.yaml:
runtime: python27
api_version: 1
threadsafe: true
libraries:
- name: jinja2
version: 2.6
- name: lxml
version: 2.3
- name: pycrypto
version: 2.6
handlers:
- url: /static
static_dir: app/static
secure: always
- url: /.*
script: manage.app
secure: always
builtins:
- remote_api: on
By default, flask use a "templates" folder to contain all your template files(any plain-text file, but usually .html or some kind of template language such as jinja2 ) & a "static" folder to contain all your static files(i.e. .js .css and your images).
So ideally you should not use /static folder
you will need to update app.yaml to include handler static_dir: static
runtime: python27
api_version: 1
threadsafe: true
libraries:
- name: ssl
version: latest
# [START handlers]
handlers:
- url: /static
static_dir: static
- url: /.*
script: main.app
# [END handlers]
The best way to use is
from flask import Flask
app = Flask(__name__, static_folder='static', static_url_path='')

How to configure GCP App Engine app.yaml for PrivateBin?

I would like to host PrivateBin from a GCP App Engine, but I don't know how to configure the app.yaml file. If somebody has done this before, I would appreciate the help. :)
I have tried two options, but neither of them worked:
runtime: php55
api_version: 1
threadsafe: yes
handlers:
- url: /.*
script: index.php
&
runtime: php55
api_version: 1
handlers:
- url: /(.*.(ico$|jpg$|png$|gif$|htm$|html$|css$|js$|xml$))
static_files: \1
upload: (.*.(ico$|jpg$|png$|gif$|htm$|html$|css$|js$|xml$))
application_readable: true
- url: /(.+)
script: \1
- url: /
script: index.php
How do you intend to store your data?
https://github.com/PrivateBin/PrivateBin/blob/master/INSTALL.md#using-a-database-instead-of-flat-files
Looks like private bin uses the filesystem by default, which wont work on GAE Standard. You could do that in GAE flex, but the directory where you are reading and writing to would need to be a mounted Google Cloud Storage Bucket or something for the data to be persistent (as well as be available to multiple instances) https://cloud.google.com/storage/docs/gcs-fuse
The better route seems to be to configure privatebin to use a SQL db, which would be Google Cloud SQL:
https://cloud.google.com/sql/docs/
https://cloud.google.com/php/getting-started/using-cloud-sql-with-mysql

main domain + subdomain + CSS not found

I have one dispatch.yaml that splits 2 services; admin_main.py that controls the admin login, and main.py that controls the user landing. My problem is that admin_main.py does not see its own CSS that I directed it to it. However, it keeps matching with the main.py CSS.
my files are structured as
admin
|assets
|CSS
+styles.min.css
www
|assets
|CSS
+styles.min.css
dispatch.yaml:
dispatch:
# Default service serves simple hostname request.
- url: "example.net/"
service: default
# Default service serves simple hostname request.
- url: "app-example.appspot.com/"
service: default
# Default service serves simple hostname request.
- url: "admin.example.net/"
service: admin
- url: "admin-dot-app-example.appspot.com/"
service: admin
admin_main.py:
service: admin
runtime: python27
api_version: 1
threadsafe: yes
handlers:
- url: /assets/css/styles.min.css
static_files: admin/assets/css/styles.min.css
upload: admin/assets/css/styles.min.css
- url: /.*
script: subdomain.app
libraries:
- name: webapp2
version: latest
- name: jinja2
version: latest
main.py:
service: default
runtime: python27
api_version: 1
threadsafe: yes
default_expiration: "4d 5h"
handlers:
- url: /assets/css
static_dir: www/assets/css
- url: /assets/img
static_dir: www/assets/img
- url: /.*
script: main.app
libraries:
- name: webapp2
version: latest
- name: jinja2
version: latest
in the .html the links to the CSS are
<link rel="stylesheet" href="assets/css/styles.min.css">
for both landing sites. But that shouldn't be a problem since the dispatch.yaml separate the incoming calls, right?
I'm not really sure what is causing the problem. Also, I'm new to yaml and I been reading it's documentation.
One solution for this is to store every CSS in the same file, and give them different names. Then in app.yaml in the handlers section add this:
- url: "/assets/css/(.*\\.(css))$"
static_files: {CSS_DIR_IN_PROJECT}/\1
upload: {CSS_DIR_IN_PROJECT}/.*\.(css)$
application_readable: true
that was my only way of solving the problem

Endpoints: /_ah/api/ returns 404 after update

Problem fixed via Google Cloud support:
I removed the /_ah/spi/* handler from my endpoints yaml file, and the
- url: /.*
script: api.app
did not instantiate the endpoint.
It used to work before since the API was deployed for the previous version, but with the new version, there was nothing explicit to deploy the endopoints. So handlers need to be
handlers:
- url: /_ah/spi/.*
script: api.app
- url: /.*
script: api.app
Keeping the issue below for reference purposes
I've just deployed a new version of my application, and calls to http://app.appspot.com/_ah/api/app/version/method now return a 404. It worked perfectly before the update.
However, there's no trace at all in the logs, and no instance launched when I call/ping those URIs.
While if I call /_ah/whatever/app/version/method, I still have a 404, but it appears in my module logs, and it has the following message
{"state": "APPLICATION_ERROR", "error_message": "Not Found"}
The app is configured using modules, my app.yaml is defined with
application: appname
version: 2015-04-07
runtime: python27
api_version: 1
threadsafe: true
handlers:
- url: /_ah/spi/.*
script: api.api.app
- url: /.*
script: www.www.app
secure: always
libraries:
- name: pycrypto
version: latest
- name: endpoints
version: 1.0
- name: webapp2
version: latest
- name: jinja2
version: latest
And, in api/api.yaml
application: appname
module: default
version: 2015-04-07
runtime: python27
api_version: 1
threadsafe: true
inbound_services:
- warmup
handlers:
- url: /.*
script: api.app
libraries:
- name: pycrypto
version: latest
- name: endpoints
version: 1.0
- name: ssl
version: latest
I've updated the app to serve this new version in the admin console, and all other modules work perfectly.
Also, I can't see my API in the API explorer, https://appname.appspot.com/_ah/api/explorer returns an empty list (while I see it when running the dev server on localhost).
Update: I've just noticed, looking at #bossylobster reply in GAE cloud endpoints - Api not updating after deploy, that I do not have "Successfully updated API configuration" in my logs after the "Completed update of a new alternate version". However, I have "API deletion serving" at about the time everything started to be 404. Yet, I have no idea why there's this API deletion query in my logs. Any idea of what can be wrong?
That's an app in production and so the mobile version is down at the moment. I'm happy to send the app ID to a devrel in PM if that helps.

App Engine Python Modules and inbound mail service

I am using App Engine Modules in my python project. (https://developers.google.com/appengine/docs/python/modules/#Python_Background_threads)
I am also receiving email in m project:
https://developers.google.com/appengine/docs/python/mail/receivingmail
I want to direct the emails to my worker module and not the default module. To that end my worker.yaml has the following settings
worker.yaml
api_version: 1
application: integrate
module: worker
version: 1-0-0
runtime: python27
threadsafe: true
inbound_services:
- mail
builtins:
- deferred: on
handlers:
- url: /admin/.+
script: src.worker.main.app
login: admin
- url: /_ah/mail/.+
script: src.worker.main.app
login: admin
- url: /.*
script: src.worker.main.app
app.yaml
api_version: 1
application: integrate
version: 1-0-0
runtime: python27
threadsafe: true
builtins:
- deferred: on
handlers:
- url: /admin/.+
script: src.default.main.app
login: admin
- url: /.*
script: src.default.main.app
I even tried adding a dispatch.yaml
application: integrate
dispatch:
- url: "*/_ah/mail/.+"
module: worker
But no matter what I do the emails which reach my app are handled by the default module. Any idea what I am missing here? I see the emails coming in but no matter what I do they only go to the default module.
Inbound services could be used only within default module and that is expected behavior. The fact that it works for you locally in devserver is a bug, actually.
Just some additional info for the answer which may help folks in a similar situation.
I noticed in the DevServer log:
"Skipping dispatch.yaml rules because /_ah/mail/[EMAIL_ADDRESS_FOR_APP] is not a dispatchable path."
This is no doubt due to local config, however.
Regardless, the workaround I have now using Tasks is:
Dispatch or directly handle Inbound Mail in the default module
Provide a script handler that creates a Task, taking the relevant MailMessage data as the payload
Set the TaskQueue in queue.yaml to target the module you wish to process the payload data, e.g. a 'worker' module

Resources