I upgraded to GAE 1.7.7 today and found out that task queues stopped working on my development setup.
I'm using https on my development environment through an nginx set up to proxy the connections from fakedomain.local:80 and fakedomain.local:443 to localhost:8080 (where GAE listens).
With this setup, taskqueues end up being created to be executed at fakedomain.local:80. This used to work because the request would be picked up by nginx, but the version 1.7.7 of the development server seems to have a port registry which won't serve a request unless the port is known (if I understand google.appengine.tools.devappserver2.Dispatcher._resolve_target correctly). Of course, GAE listens on port 8080 and my tasks marked to run on fakedomain.local:80 never get executed (GAE logs this error: An error occured while sending the task "task1" (Url: "...") in queue...).
I tried patching dispatcher.py:577 so instead of raising a ServerDoesNotExistError when the port is not known it will just use the default server. With this modification I can get the taskqueues running again, but I'd rather use a solution which doesn't involve changing GAE's code.
How can I make GAE register the port 80 and 443 in version 1.7.7? Alternatively, is there a way I could specify the complete target URL for the task? (ie fakedomain.local:8080/my_task, instead of just /mytask).
taskqueue.add(target=taskqueue.DEFAULT_APP_VERSION, ...)
will run it on your default app, which should do exactly what you want.
taskqueue.DEFAULT_APP_VERSION =>
app_identity.get_default_version_hostname() =>
environ['DEFAULT_VERSION_HOSTNAME'] =>
'%s:%s' % (environ['SERVER_NAME'], server_port)
Related
I am new to the node world. I created a node js rest API. When I run npm start in my local machine or in the terminal for the first time, I can see console.log() in my terminal. Now, I am running the same application on an AWS Ec2 instance with Ubuntu as os. I run npm start and serve my app on port 80. I do this via ssh and after running my server I close the ssh connection. But when I reconnect via ssh, I want to see those console.log() messages in my terminal for some purpose.
I completely understand that logging messages in the terminal is not a good idea and there can be so many alternatives. Just want to know how to access the same terminal window/result as we see when I start my application.
if you are using pm2, you can try "pm2 logs"
So Nodemon won't work well in a production server or in any instance where you need to have the app going by itself.
Nodemon is a dev tool to enable you to restart your server during development. In a "real" vps you need to place the process in the background or it will be automatically killed when the connection times out.
Check out this YT series for a correct deploy architecture in an NGINX server with the use of pm2 and NGINX on a Red Hat server, I've personally used it more than once:
https://www.youtube.com/playlist?list=PLQlWzK5tU-gDyxC1JTpyC2avvJlt3hrIh
I followed this tutorial
to get a Bigtable client up and running in Google Managed VMs. But is there a way to run this locally? Reason is that deploying the code remotely in development is a pain.
Normally I can use dev_appserver.sh to run GAE app locally. But when I run it, I'm getting this error:
Caused by: java.lang.IllegalStateException: Jetty ALPN has not been
properly configured.
Which means we need to include ALPN library? Since our codebase is in Java 7, I used this ALPN version: 7.1.3.v20150130.
I then tried again with this:
dev_appserver.sh --jvm_flag=-Xbootclasspath/p:/Users/shouguoli/tmp/alpn-boot-7.1.3.v20150130.jar
still getting this error:
Caused by: com.google.apphosting.api.ApiProxy$CallNotFoundException:
The API package 'urlfetch' or call 'Fetch()' was not found.
How do you get it to work locally?
The sample was updated last week. It's based on the java 8 compat runtime, which means that you have access to most of the App Engine API's including Users, Task Queues, and Datastore.
There is a new Netty TCNative module that uses Boring SSL.
To use it with the pom.xml in the sample, do:
mvn clean -Pmac jetty:run -Dbigtable.projectID=<your-project> -Dbigtable.clusterID=<your-cluster> -Dbigtable.zone=<your-zone>
To use on Windows, use -Pwindows instead of -Pmac. For linux, omit the Profile -P as it's the default.
To deploy:
mvn clean gcloud:deploy -Dbigtable.projectID=<your-project> -Dbigtable.clusterID=<your-cluster> -Dbigtable.zone=<your-zone>
NOTE - it is advisable to do the clean between running locally and running remotely as the TCNative module is currently specific to the platform the code runs on.
We are in the process of updating all of our samples to use TCNative, we hope to have this by 3/10/16.
I can start and run my Angular app fine from my local computer. But I have never run it on a server before.
I have tried uploading it to the server. On the server I type the following to start it:
http-server
Then I visit http://serverip:8080/. Instead of seeing my app I see:
Index of /
(-rw-rw-r--) 847B app.js
(-rw-rw-r--) 748B index.htm
Node.js v0.10.25/ ecstatic server running # 195.189.182.160:8080
What is missing in my configuration?
ecstatic is a static file server. It's not set up to be a web server by default, even though it can work that way. When you do run it as a web server, certain standard options in most web servers - like treating a request for /path/ as a request for /path/index.htm - are disabled by default.
You need to enable opts.autoIndex. The package readme tells you how to do that. In your case, it means editing http-server or the .js file that it fires off; it's hard to say from what you have posted.
Currently the GAE Managed VMs feature is broken for any project with a name longer than 27 characters.
The underlying issue is that Docker restricts image namespace to between 4-30 chars. This has been fixed (https://github.com/docker/docker/issues/10392) but is still awaiting a release at time of writing.
It seems when deploying a Managed VM to GAE that the namespace is automatically generated from your project name plus _m_ prefix. This leads to error when attempting to deploy the vm:
DEBUG: "POST /v1.10/images/gcr.io/_m_<my project name>/<my project name>.default.20150330t140211/push HTTP/1.1" 500 111
INFO: Exception 500 Server Error: Internal Server Error ("Invalid namespace name (_m_<my project name>). Cannot be fewer than 4 or more than 30 characters.") thrown in ProgressHandler. Retrying.
The obvious solution would be for GAE gcloud tools to respect the underlying limit via some auto-truncation or hashing scheme.
Does anyone know a way around this? Or I have to wait for Google to fix or Docker to release a new version and Google to update?
We're aware of the issue and we're working on a long-term fix. For now, you can switch to an old version of gcloud. You can do this by setting this variable to point to an old version (0.9.51):
gcloud config set --scope=installation component_manager/fixed_sdk_version 0.9.51
then run "gcloud components update"
Then run "gcloud config set app/hosted_registry false"
and you should be able to deploy. I'll update this answer when we've fixed the naming issue.
UPDATE:
The naming issue has been fixed as of this week's release (0.9.57).
I'm working on a JS based project that runs off GAE and part of the code gets the user's avatar using OAuth from Facebook, Twitter or Google. I'm trying to write tests in Mocha in order to test this but I'm running into some problems.
The code works when I test it in the front end, and the way I envisaged it to work would be to use ZombieJS to run the app on GAE's dev_appserver.py, fire the OAuth functions, fill in the appropriate auth stuff and then complete the test by returning the image URL.
However the first hurdle I've got is that it appears that NodeJS's server is not allowing GAE's server to run on the same IP address. For example:
exec 'dev_appserver.py .', ->
console.log arguments
This returns the error 'Address already in use'. How can I get around this apart from running it on a different machine? Is it possible to tell NodeJS to not reserve the whole IP and just a port? I'm running GAE on 8080 and it works fine when it isn't invoked by NodeJS.
The second problem is ZombieJS. I'm trying to figure out a way I can listen to when new windows are opened and, essentially, tail the console of the browser. I've started two discussions on the Google group but no one has responded yet (https://groups.google.com/forum/?hl=en#!topic/zombie-js/cJklyMbwxRE and https://groups.google.com/forum/?hl=en#!topic/zombie-js/tOhk_lZv5eA)
While the latter isn't as important as I can find ways around it (I hope), the former is the main issue, so I'd greatly appreciate any direction on how to resolve this address conflict.
Here's my NodeJS script:
exec = ( require 'child_process' ).exec
fs = require 'fs'
should = require 'should'
yaml = require 'yaml'
Zombie = require 'zombie'
common = require '../../static/assets/js/common'
url = 'ahmeds.local'
browser = new Zombie()
config = null
consoleCb = 'function consoleSuccess(){console.log("success",arguments)}function consoleFailure(){console.log("failure",arguments)}'
browser.debug = true
browser.silent = false
fs.readFile '../../config.yaml', (error, data) ->
config = yaml.eval data.toString 'ascii'
exec 'cd ../../ && dev_appserver.py -a ' + url + ' .', ->
console.log arguments
# browser.visit config.local.url, ->
browser.visit 'http://' + url + ':8080', ->
browser.evaluate consoleCb
browser.evaluate 'profileImage("facebook",consoleSuccess,consoleFailure)'
console.log browser.window.console.output
I have only limited familiarity with NodeJS, but I just tested running a NodeJS server and App Engine local dev server on the same machine — it works just fine. Without seeing your NodeJS code, I'm guessing you're also trying to run NodeJS on port 8080, and so the App Engine server complains when it's started (8080 is the default, and you noted it's the port you are using).
Try passing --port=8081 (or some other port) to your invocation of dev_appserver.py and it should resolve the conflict.
Nothing in the code you've shown (other than the invocation of dev_appserver) should even be listening on any port (unless zombie implements a "server" for remote debugging or something like that). It looks like the port conflict is coming from somewhere else.
Note that zombie's own Mocha test framework does set up an express server, so if you're using it or code lifted from it, that might be doing it.
What does netstat have to say about who's binding to what port?