I am still working to get my old style appengines to work under at least go111 (go112 would not work due to dependencies on memcache). I am now stumbling over app.yaml configuration issues with my static files, I used a completely static directory layout before and just specified a few dynamic handlers in the root like this:
runtime: go111
handlers:
- url: /_ah/.*
script: auto
login: admin
secure: always
- url: /dynamic
script: auto
secure: always
- url: /admin/.*
script: auto
login: admin
secure: always
- url: (.*)/
static_files: html\1/index.html
upload: html/index.html
secure: always
- url: /(.*\.map)
mime_type: application/json
static_files: html/\1
upload: html/(.*\.map)
secure: always
- url: /
static_dir: html
secure: always
dev_appserver.py will never call my dynamic entry point. In production this does work, but I am still working on the conversion and would like to test locally. Any hints how to convince dev_appserver.py to let me do this? By the way my gcloud tools are updated as of today.
Your question is why your application works well when you deploy it to production but doesn't work when you use dev_appserver.py to run it locally, and how could you run it with dev_appserver.py. The answer to that is:
You won't be able to run it locally properly using dev_appserver.py, since it does not support runtime Go 1.11. Look at the Local Development Server Options documentation, there's only a link for "Go 1.9".
(as you can see, the links for "Go 1.11" and "Go 1.12" are disabled, which translates to: not supported.)
Documentation for Go 1.11 on App Engine Standard states that in order to test your application locally you would have to use "go run" command (notice how it doesn't mention dev_appserver.py tool). The command would be something like this:
go run [build flags] [-exec xprog] package [arguments...]
For more information about the command go here.
I'm sure you might have already read this but, to know more about the migrations process from Go1.9 to Go1.11 read this documentation.
You have stated that "go run" command wouldn't work for your case. So, a workaround for that would be to test your application directly into App Engine but without migrating the traffic.
When deploying your test version use:
gcloud app deploy --no-promote
To access to it go to:
http://VERSION_ID.default.YOUR_PROJECT_ID.appspot.com
If everything turned out great, you can migrate the traffic on the Cloud Console UI selecting the version you just deployed and clicking "Migrate traffic".
Related
We've deployed a Vue SPA to Google App Engine and it's served completely by the static handlers.
The issue that we are facing is that if a user is active on our site mid-deploy, then their old Webpack chunk manifest becomes invalid (since some chunks' hashes are overwritten). If they now try to route to a new page and that page tries to fetch a chunk that got overwritten, we get the following error:
ChunkLoadError: Loading chunk Conversations failed.
(error: https://example.com/js/Conversations.71762189.js)
Ideally, we'd like to keep N (2-3?) previous versions of the app's static files.
Is our only option to push all the assets to a Cloud Storage Bucket? If so, how would we go about pruning older versions?
Here is my app.yaml for reference:
runtime: nodejs10
instance_class: F4
automatic_scaling:
min_instances: 2
max_instances: 10
default_expiration: "30d"
error_handlers:
- file: default_error.html
handlers:
- url: /api/*
secure: always
redirect_http_response_code: 301
script: auto
- url: /js/*
secure: always
redirect_http_response_code: 301
static_dir: dist/js
- url: /css/*
secure: always
redirect_http_response_code: 301
static_dir: dist/css
- url: /img/*
secure: always
redirect_http_response_code: 301
static_dir: dist/img
- url: /(.*\.(json|js|txt))$
secure: always
redirect_http_response_code: 301
static_files: dist/\1
upload: dist/.*\.(json|js|txt)$
expiration: "10m"
- url: /.*
secure: always
redirect_http_response_code: 301
static_files: dist/index.html
upload: dist/index.html
expiration: "2m"
The issue typically happens when the deployment overwrites an existing service version which receives traffic (i.e. the service version is not changed). From Deploying an app:
Note: If you deploy a version that specifies the same version ID as a version that already exists on App Engine, the files that you
deploy will overwrite the existing version. This can be problematic if
the version is serving traffic because traffic to your application
might be disrupted. You can avoid disrupting traffic if you deploy
your new version with a different version ID and then move traffic to
that version.
As long as a service version is deployed and not deleted or overwritten its respective static assets remain accessible.
To prevent the issue always deploy using a fresh service version, then (gradually) migrate traffic to the newly deployed version. Keeping the latest N service versions around will give you those N sets of static assets you desire.
In general, this deployment practice is good/recommended for a few other reasons:
avoids potential outages, see Continuous integration/deployment/delivery on Google App Engine, too risky?
avoids potential traffic loss while GAE spins up enough new version instances to handle the traffic load, see 2nd half of GAE shutdown or restart all the active instances of a service/app
Potentially of interest: Google Frontend Retention between deployments
Deploy using the --no-promote flag, and utilize the Traffic Migration feature in the Standard Environment to gradually migrate traffic over to the new version so that all users don't experience an immediate switchover the moment the new version goes live. App Engine will host both the old and new versions (or, "blue" and "green") for a period of time until all traffic points to the new version, and then the old version will be shut down.
See also:
Testing on App Engine
Migrating and Splitting Traffic with the Admin API
Blue-Green Deployment Pattern
I tried to use the documented way of restricting access to urls marked as static by way of login: required rules in the app.yaml file. My intention is to have access to script urls handled by the go programming language by xmlhttprequests, but the first step of authenticating the user before she can load the file dist/index.html fails.
Surprisingly for me the user is not prompted to login, instead receives the dist/index.html file and all other files it asks for from the static folder as if no restricting rule were present.
This is my app.yaml file:
application: helloworld
version: 1
runtime: go
api_version: go1
handlers:
- url: /
static_files: dist/index.html
upload: dist/index.html
secure: always
login: required - this is what fails as far as I'm concerned
- url: /(.*\.(txt|html|json|png|js|log|md|css|ico))
static_files: dist/\1
upload: dist/(.*\.(txt|html|json|png|js|log|md|css|ico))
secure: always
login: required
- url: /.*
script: _go_app
secure: always
login: required
The folder that I uploaded to appengine looks like this:
app.yaml
index.yaml
xhr_responses.go - this is the intended future non static AJAX part
dist/
index.html
loads of other stuff that is static
The 'login:' handler options in the .yaml config files rely on Google's authentication, which can be persisted using cookies and survive a browser restart.
To properly test the authentication you need to either use a fresh incognito browser session or go to one of the Google sites and ensure you're not logged in (explicitly log out from all Google accounts if needed) before testing.
Apparently I was signed in when trying stuff on the live google app engine, which I just forgot is the way it knows not to redirect access to a new login prompt.
I'm facing a problem where a static file is showing up correctly on the local machine via dev_appserver.py but once uploaded to the server it doesn't show and gives me a 404 error.
The partial directory file structre is:
- static/
articles.html
images/*.png
The partial app.yaml file is:
runtime: go
api_version: go1
- url: /(articles\.html)
static_files: static/\1
upload: static/*.html
I then to access this via the http url appname.appspot.com/articles.html.
I downloaded the code after I put it on the appengine server to ensure that articles.html has actually been uploaded in the said directory, and it has. So the exact same file structre locally and on the server, but it gives a 404 on the server and works locally. I have the latest versions of the appengine. Any help on what I could be doing wrong?
thanks in advance
Sathish
Just putting this up for completeness ... I went back and tried various reasonable combinations again and found that this config works:
- url: /(articles)\.html
static_files: static/\1.html
upload: static/.*.html
It is ridiculous of me to make the claim now, but I remember attempting a regex for the "upload" entry earlier and it hadn't worked. So, I don't clearly know the issue, however, my best guess is that dev_appserver.py is probably taking some shortcuts to serve static files and not behaving exactly like the appengine would. I shall raise an issue on that and see if there is any resolution or clarification.
In the answer to the question Error sending e-mail via SMTP server on App Engine development server there is a nice solution by Blixt:
"dev_appserver.py does not support TLS
which is required by Gmail. You can
enable it by adding a few lines in api/mail_stub.py:"
# After smtp.connect(self._smtp_host, self._smtp_port)
smtp.ehlo()
smtp.starttls()
smtp.ehlo()
But windows Vista would not let me change api/mail_stub.py
Is there another way to send mail from development server. Other options (Sendmail and ISP are not good for me). Thanks!
EDIT
I changed the api/mail_stub.py according to instructions here and I use the following command-line options:
dev_appserver.py
--smtp_host=smtp.gmail.com
--smtp_port=25
--smtp_user=xxxx#gmail.com
--smtp_password=gmail_pw
C:\Users\A\Desktop\repeater # path to root directory
But I get this error from Log Console:
***********************************************************
2010-11-18 10:24:37 Running command: "['C:\\Python26\\pythonw.exe',
'C:\\Program Files(x86)\\Google\\google_appengine\\dev_appserver.py',
'--admin_console_server=',
'--port=8080',
u'dev_appserver.py',
u'--smtp_host=smtp.gmail.com',
u'--smtp_port=25',
u'--smtp_user=xxxx#gmail.com',
u'--smtp_password=gmail_pw
C:\\Users\\A\\Desktop\\repeater',
'C:\\Users\\A\\Desktop\\repeater']"
Runs a development application server for an application.
dev_appserver.py [options] <application root>
Application root must be the path to the application to run in this server.
Must contain a valid app.yaml or app.yml file.
****************************************************
This is the app.yaml:
application: re-peater
version: 1
runtime: python
api_version: 1
handlers:
- url: /favicon.ico
static_files: static/images/favicon.ico
upload: static/images/favicon.ico
- url: /stylesheets
static_dir: stylesheets
- url: /.*
script: repeater.py
All this works without these command line options. Any suggestions why this is not working?
If you really, really need to send real email from the dev_appserver, you should set up your own mail relay on your machine, and point the SDK at that. I'm curious why it's so important to send real email, though - this is the development server, and you shouldn't be using it for anything other than development.
Change the file permissions so that you can change the api/mail_stub.py file.
This is no longer necessary
in /appengine/api/mail_stub.py
if self._allow_tls and smtp.has_extn ('STARTTLS'):
smtp.starttls ()
I am using appengine sdk version 1.9.15.
I am running my app on the production server for first time. I have a url with admin logon enabled in the app.yml. The script runs when I browse to its URL when running on the development server. However, after uploading to the production server when I go to the same URL I get the following error:
The requested URL /tasks/ was not found on this server.
Why would this occur? I tried updating again.
The URL and script is the third one in the my app.yml file:
##app.yml file
application: generic_app_name
version: 1
runtime: python
api_version: 1
handlers:
- url: /remote_api
script: $PYTHON_LIB/google/appengine/ext/remote_api/handler.py
login: admin
- url: /stats.*
script: $PYTHON_LIB/google/appengine/ext/appstats/ui.py
- url: /tasks/SR2pop
script: PopulateSR2.py
login: admin
- url: /
script: dbsample.py
Is the capitalization of your script exactly as specified in app.yaml? The production servers are case-sensitive, but if you're developing on Windows, the development server isn't. Check the capitalization of PopulateSR2.py matches the one in app.yaml.
Also, your file is called app.yaml, not app.yml, right?
Your 3rd handler is going to match only the exact string /tasks/SR2pop. None of your handlers will match /tasks/.