I'm trying to get Push-to-Deploy to work with GAE through GitHub set-up as the code repo source.
This is now Day 3 and I've run into several bugs like:
Creating a Release pipeline failing for no apparent reason
Getting Error 500 when I finally managed get the source pulled over to GAE
Changes in Github are propagated to GAE (they're visible under Source code -> Browse), but the Release routine isn't run and (understandably so) the app is not being served at the web address.
I tried creating several new projects and I exhibited various versions of the bugs above.
Here's where I'm at now:
I created a brand new project, setup Github as my repo, created a Release Pipeline and did a push afterwards (to Github) just to try and trigger the Deploy routine to no avail. The code tree is visible under Source code -> Browse, along with commit dates and so on, but the Release pipeline isn't run.
App ID is skillful-signer-695 and the platform of choice is php.
Any ideas?
Update:
Nearly 24 hours after I set-up the project and did a test commit to trigger the Release pipeline, it finally got executed! Its result is marked as 'SUCCESS.
Now if I try and hit the project's URL I get a:
Error 500 - The server encountered an error and could not complete your request.
Any help will be greatly appreciated!
Update 2:
I got some more info by looking at the logs:
A problem was encountered with the process that handled this request, causing it to exit. This is likely to cause a new process to be used for the next request to your application. (Error code 204)
Perhaps there's something wrong with my (quite simple) app.yaml?
application: skillful-signer-695
version: 1
runtime: php
api_version: 1
threadsafe: false
handlers:
# Serve images as static resources.
- url: /(.+\.(gif|png|jpg))$
static_files: \1
upload: .+\.(gif|png|jpg)$
application_readable: true
# Serve php scripts.
- url: /.*
script: index.php
It seems a lot of different people are experiencing the same thing and it has to do with app.yaml, but I can't figure out what I'm doing wrong... The above is pretty much pasted from the example in the docs.
we were finally able to get this working - turns out there were some issues with the codebase for the website, which were leading to the 500 error (that's Code Igniter for you, I guess).
In addition, we had to add some lines to the app.yaml for css and fonts to show up. Here's what we ended up with:
application: skillful-signer-695
version: 1
runtime: php
api_version: 1
threadsafe: false
handlers:
- url: /assets/(.*\.(css|js|ttf))$
static_files: assets/\1
upload: assets/.*\.(css|js|ttf)$
# Serve images as static resources.
- url: /(.+\.(gif|png|jpg))$
static_files: \1
upload: .+\.(gif|png|jpg)$
application_readable: true
# Serve php scripts.
- url: /.*
script: index.php
Aside from the issues on our end, I believe we ran into several GAE bugs (inconsistent pull times from GitHub, Dashboard issues, incomplete docs, etc.) which are normal for Beta software, but nasty nonetheless. Thanks for the help!
p.s. We ran something else, which might save somebody some time digging around - if you're trying to use sockets (for example to send an email via SMTP from your support form hosted on GAE since the standard PHP mail command doesn't work) you'll find that Sockets are enabled only after you enter your billing info. They don't really charge you anything for light use, but I guess Google considers this a Premium feature.
All the best!
I am having issues with Push 2 Not Deploy feature, because -at least for now- GAE does not resolve git submodules.
If you have submodules or ignoring essential files from git, your application will not work.
For example: I am sending e-mails from GAE and my login_data.py is ignored by git via .gitignore
If you use appcfg.py update --oauth . your login_data.py will be sent. Using push2deploy in a this situation will end up a big error like this:
Error: Server Error
The server encountered an error and could not complete your request.
Please try again in 30 seconds.
Related
I am following the instructions on https://cloud.google.com/appengine/docs/go/googlecloudstorageclient/download to begin migrating some code from the, now deprecated, Files API to the new Cloud Storage API without success.
The steps I'm following are ...
I'm running appengine v1.9.23 which is later than the required appengine v1.8.1.
My $GOPATH is set, so I skip step #1.
I proceed to step #2:
goapp get -u golang.org/x/oauth2
goapp get -u google.golang.org/cloud/storage
I am not developing on a managed VM, so I skip step #3.
Now when I run the application, I get:
go-app-builder: Failed parsing input: parser: bad import "syscall" in goapp/src/golang.org/x/net/internal/nettest/error_posix.go
What am I doing wrong?
Steps to reproduce
Download an install the Google Appengine runtime, version 1.9.23 from https://console.cloud.google.com/storage/browser/appengine-sdks/featured/ . Follow the installation instructions documented on https://cloud.google.com/appengine/downloads?hl=en
Create an appengine project directory:
% mkdir $HOME/myapp
Create a new app.yaml file as ~/myapp/app.yaml. Read the directions on the Google website for details: https://cloud.google.com/appengine/docs/go/config/appconfig
I use a version that does not have the static resources:
application: myapp
version: alpha-001
runtime: go
api_version: go1
handlers:
- url: /.*
script: _go_app
Create a location for the Go source files.
% mkdir $HOME/myapp/go
Set your GOPATH to the location of your sources
% export GOPATH=$HOME/myapp/go
Get the Go appengine example project: https://github.com/golang/example
% goapp get github.com/golang/example/appengine-hello
This command will download the example app to the first path entry in the GOPATH
Install the Google Cloud Storage client libraries as directed in https://cloud.google.com/appengine/docs/go/googlecloudstorageclient/download . Reference the steps at the top of this question for more details. Following the directions should result in you running 2 commands:
% go get -u golang.org/x/oauth2
% go get -u google.golang.org/cloud/storage
Attempt to run your go application
% goapp serve
You will see the following compilation error (no stack trace):
2015/12/23 10:37:07 go-app-builder: Failed parsing input: parser: bad import "syscall" in go/src/golang.org/x/net/ipv6/control_unix.go
This error is caused by either of two scenarios:
1) Implicitly importing syscall by importing another package that uses it, as referenced in this related question.
2) Having your package source files in your GOPATH located in a directory at or below the same level as your project's app.yaml (eg. app.yaml in ~/go, and packages sources in ~/go/gopath/src). If a package like x/net/internal/nettest exists in your GOPATH the syscall import will be parsed by goapp at compile time and throw the compilation error.
Avoiding these two scenarios should be sufficient to prevent any bad import "syscall" errors or related compilation errors.
Reproduced the initial steps above and got a similar error, even if not explicitly mentioning syscall. However, running “goapp serve” in the appengine-hello directory results in no error at all.
Adam’s explanation at point 2 applies here correctly: one needs to place the app.yaml file at the right level in the directory structure.
sirupsen/logrus references syscall.
They have an appengine tag specified, not to include syscall so it's usable in AppEngine, something like go build -tags appengine as per issue 310.
However I haven't yet succeeded including it in an AppEngine project so that this build param could be forwarded and specified somewhere so that it goes through. I'll come back to update if I manage.
I'm trying to create a managed vm for my node 4 application using google custom runtime.
I created the following Dockerfile:
FROM node:4.2.1
ENV PORT 8080
ADD package.json package.json
RUN npm install
ADD . .
CMD [ "npm", "start" ]
Along with this app.yaml:
# [START runtime]
runtime: custom
vm: true
api_version: 1
# [END runtime]
health_check:
enable_health_check: false
skip_files:
- ^(.*/)?#.*#$
- ^(.*/)?.*~$
- ^(.*/)?.*\.py[co]$
- ^(.*/)?.*/RCS/.*$
- ^(.*/)?\..*$
- ^(.*/)?.*/node_modules/.*$
- ^(.*/)?.*\.log$
I deploy the app using gcloud preview command:
gcloud preview app deploy app.yaml --promote
It seems like the docker is being built correctly but the at the end of the process I get this message:
Copying files to Google Cloud Storage...
Synchronizing files to [gs://staging.my-project-id.appspot.com/].
Updating module [default]...\Deleted [https://www.googleapis.com/compute/v1/projects/my-project-id/zones/us-central1-f/instances/gae-builder-vm-20151030t142257].
Updating module [default]...failed.
ERROR: (gcloud.preview.app.deploy) Error Response: [4] Timed out creating VMs.
I have my deployment working now. I have had to troubleshoot the same problem before, for another project, but I didn't have the code on hand, so I had to work through the problems again.
The deployment ran smoothly up until the last steps, where updating the module would timeout. This made me think it was something to do with the application starting up on VM and not responding appropriately, so the final hook would time out.
You'll find a lot of information here - https://cloud.google.com/appengine/docs/managed-vms/config . I checked the following things:
logging - ensure that you are writing to the correct log file. See https://cloud.google.com/appengine/docs/managed-vms/custom-runtimes#logging
ensure you have a .dockerignore file and are skipping files in app.yaml so you are not asking the process to copy across unneeded node_modules or log files
turn off health checking if you are not using it, or ensure you have the correct express.js routes configured for it
check that your environment variables are set and match what GAE can use. This was my final step - GAE will let you bind to a VM port on 8080. I had to pass through a NODE_ENV flag in my app.yaml which told the app to use 8080 and not 3000.
Lift the resources of the GAE instance in app.yaml. I specified two logical CPUs and made the ram 2 gig.
Good luck.
I have a basic appengine project with multiple modules and a dispatch.yaml:
my-project/boxes/app.yaml (default module)
my-project/users/app.yaml (users module)
my-project/dispatch.yaml
I'm trying to configure a single hourly cronjob with the following definition:
cron:
- description: hourly box purging
url: /api/boxes.purge
schedule: every 1 hours
target: default
I've tried adding it to the module it concerns, so put the above definition in file: 'my-project/boxes/cron.yaml' and running appcfg.py cron_info boxes/. My terminal seems to indicate all went well:
hourly box purging:
URL: /api/boxes.purge
Schedule: every 1 hours (UTC)
2015-04-30 10:08:00Z, 0:59:55 from now
2015-04-30 11:08:00Z, 1:59:55 from now
2015-04-30 12:08:00Z, 2:59:55 from now
2015-04-30 13:08:00Z, 3:59:55 from now
2015-04-30 14:08:00Z, 4:59:55 from now
Ye the Appengine Developer console fails to reflect this and cron jobs are not run. It does show on the local development panel.
Putting the definition in the root of the projects (besides dispatch.yaml) yields the same results. Other things i've tried (in vain): Redeploying all code, appcfg.py update_dispatch, waiting a while before refreshing the developer console.
Hopefull someone is able to help me find the obvious mistake, or confirm that their is some bug.
In the Configuration section of the doc it's stated:
Optional application-level configuration files (dispatch.yaml,
cron.yaml, index.yaml, and queue.yaml) are included in the top level
app directory.
I agree, the paragraph context appears to leave room for interpretation (typically...). But the quoted text also indicates that these files are considered app-level configs. So I'd keep them at the top.
About the update: I noticed, for example, that the index.yaml file was NOT uploaded with the rest of the multi-module app at my first deployment, I had to explicitly use appcfg.py update_indexes. This was not happening with a single module app. Maybe appcfg.py update_cron also needs to be explicit?
Im having problems deploying my GAE. I've got two running already so I should have all the right lirbaries and whatnot. But obviously Im missing something.
I registered my applet with googe appengine developers.
i enter the app_id in the app.yaml of the project as application name.
in cmd, i go to google_appengine and run the "appcfg.py update" on the root directory of my project. "C:/Development/GAE/projectname"
type in email and pw,
and then i get this error msg
12:47 AM Application: processing#######; version: 1
12:47 AM Host: appengine.google.com
12:47 AM
Starting update of app: processing######, version: 1
12:47 AM Getting current resource limits.
Password for ############gmail.com: Invalid username or password.
2014-09-19 00:47:36,404 ERROR appcfg.py:2416 An error occurred processing file '': HTTP Error 401: Unauthorized. Aborting.
Error 401: --- begin server output ---
Must authenticate first.
--- end server output ---
Any ideas what Im missing? Ive triple-checked all these steps cause thats all i can think of.
My App.yaml contains this
application: processing######
version: 1
runtime: python27
api_version: 1
threadsafe: yes
handlers:
- url: /statics
static_dir: statics
- url: /sketches
static_dir: sketches
- url: .*
script: main.app
libraries:
- name: webapp2
version: "2.5.2"
- name: jinja2
version: latest
Ive upgraded my GAE to the latest version too, still no change. I also tried uploading it to a different application, same problem.
Ive added 2 static folder I use for to run Processing embedded in my website.
PS my app runs fine locally and there's nothing that throws errors or doesnt compile. I think the fault lies somewehre else.
Cheers
Ok so as usual, more research eventually revealed the answer. And stackoverflow.
Guess I missed this first because it specified with php.Stack Overflow Question
Basically i wasnt doing anything wrong afterall, the permission problem was that it was being blocked by my google account.
I had to go to my Account Security permissions, and enable access from less-secure devices.
This must be a new, recent setting Google changed, becuase Ive already deployed 2 apps before.
To mention here for other people with the same problem:
First go through the steps I went through to make sure everything is set up correctly.
Then you should solving it the way I did.
Otherwise you can try setting it up so you can upload without password, this might avoid the problem if it lies somewehre else --oauth without pw
Finally I read that for some people the problem was in timezones. When the time on their computer was different to GAE or server I think that lead to miscommunication, so double-check that your time is set correctly and that it agrees with the time you see in the Launcher Log.
well for me the below 02 options worked
appcfg.py update . --oauth2 --noauth_local_webserver
The authentication flow was :
the command and options generated a url in stdout, which i copied and accessed from my browser, the browser generated a code, which I gave in my command line. I could see appcfg as one of the apps under connected apps and services of my google account.
It seems like an error with your user or password.
Are you sure you can access the project in the developer console, the email configured under permissions is the one which you are using to connect and the app id in yaml file is the same ID on the developers console? If so, the only thing I can imagine is a fail in your password.
Maybe a key pressed? Try typing it in a text processor.
A last idea, deploy the app via the launcher of the sdk. Add the existing app in the file menu (search the app.yaml file) and click deploy to see if there you can authenticate.
I'm new to GAE, and have big doutbts on how it works. Mainly the app.yaml configuration file. If i have this structure for example:
/
/index.php
/scripts/script.php
On app.yaml I should write something like this for example (the basic to work), right?
application: myapp
version: 1
runtime: php
api_version: 1
handlers:
- url: /
script: index.php
- url: /scripts/script.php
script: scripts/script.php
Is this correct? And if i have a lot of scripts on the scripts directory, should i put an entry for all of them?
I ask this (I think its a really stupid question but i want to be sure) because i have realised that if i put the scripts directory as a static_dir, like:
- url: /scripts
static_dir: scripts
... if i go to the URL of that file (http://mydomain.com/scripts.script.php) i can download the source code.
So I guess that static_dir is not for accessing a whole directory's content, and as it's name says... is just for static content like photos for example, right? so for each script i should write a line on the app.yaml to be accesible?
Sorry about my stupid question, my english is not so good and i'm new at this so i have this doutbt ;)
Thanks a lot in advance!
You can find some detailed examples here of PHP and app.yaml here:
https://developers.google.com/appengine/docs/php/config/appconfig
If you have a large number of *.php files that you want to make executable, you can use wildcards. For example:
- url: /(.+\.php)$
script: \1
Will map anything URL ending in .php to the equivalent filepath. So
http://myapp.appspot.com/some/directory/file.php
will map to
some/directory/file.php
in your app's directory.
Don't put anything in static directories that you don't want the world to see. static directories are intended as a fast shortcut for serving things like images, CSS, and JavaScript.
.php files (or .py files if you're using Python) need to be 'executed' (evaluated) on the server (App Engine) side. Putting them in a static_dir puts them out of reach of the execution environment, which isn't what you want.