I have an appengine project using golang that I haven't touched in about a year. I cannot get it to build now on the machine where it was building before. I get the following error
go-app-builder: Failed parsing input: parser: bad import "syscall" in src/golang.org/x/net/internal/nettest/rlimit_unix.go
I am running on OS X if that helps at all. I am not a Golang expert, by any stretch, and my attempts to find anything even related to this have all turned up nothing. Any thoughts?
That usage is in net/internal. You are probably importing net in your app engine project someplace, and thus doing the illegal import.
If you constrain your usage to net/http on an app engine project, you should be just fine.
I think the problem is this:
You are not permitted to import the syscall package in a Go App Engine
app for security reasons. Whatever file is importing it needs to be
changed to not do that, or be excluded from the build.
as per this
So you might want to try this workaround. If you are importing the 'os' module. It fixes this
It would be helpful if you can post your code too.
Related
While I was deploying my google app engine project (Java) using gradle appengineDeploy my internet connection was interrupted.
I re-deployed the project. Although the console said BUILD SUCCESSFUL, the app engine instance no longer works. No matter how many times I re-deploy or update my application, the logs show nothing but errors:
java.lang.NoClassDefFoundError: com/google/appengine/api/ThreadManager
at
com.google.api.control.extensions.appengine.GoogleAppEngineControlFilter.createClient
(GoogleAppEngineControlFilter.java:61) at
com.google.api.control.ControlFilter.init (ControlFilter.java:141) at
org.eclipse.jetty.servlet.FilterHolder.initialize
(FilterHolder.java:139) at
org.eclipse.jetty.servlet.ServletHandler.initialize
(ServletHandler.java:873) at
org.eclipse.jetty.servlet.ServletContextHandler.startContext
(ServletContextHandler.java:349) at
org.eclipse.jetty.webapp.WebAppContext.startWebapp
(WebAppContext.java:1406) at
com.google.apphosting.runtime.jetty9.AppEngineWebAppContext.startWebapp
(AppEngineWebAppContext.java:175) at
org.eclipse.jetty.webapp.WebAppContext.startContext
(WebAppContext.java:1368) at
org.eclipse.jetty.server.handler.ContextHandler.doStart
(ContextHandler.java:778) at
org.eclipse.jetty.servlet.ServletContextHandler.doStart
(ServletContextHandler.java:262) at
org.eclipse.jetty.webapp.WebAppContext.doStart
(WebAppContext.java:522) at
com.google.apphosting.runtime.jetty9.AppEngineWebAppContext.doStart
(AppEngineWebAppContext.java:120) at
org.eclipse.jetty.util.component.AbstractLifeCycle.start
(AbstractLifeCycle.java:68) at
com.google.apphosting.runtime.jetty9.AppVersionHandlerMap.createHandler
(AppVersionHandlerMap.java:240) at
com.google.apphosting.runtime.jetty9.AppVersionHandlerMap.getHandler
(AppVersionHandlerMap.java:178) at
com.google.apphosting.runtime.jetty9.JettyServletEngineAdapter.serviceRequest
(JettyServletEngineAdapter.java:120) at
com.google.apphosting.runtime.JavaRuntime$RequestRunnable.dispatchServletRequest
(JavaRuntime.java:747) at
com.google.apphosting.runtime.JavaRuntime$RequestRunnable.dispatchRequest
(JavaRuntime.java:710) at
com.google.apphosting.runtime.JavaRuntime$RequestRunnable.run
(JavaRuntime.java:680) at
com.google.apphosting.runtime.JavaRuntime$NullSandboxRequestRunnable.run
(JavaRuntime.java:872) at
com.google.apphosting.runtime.ThreadGroupPool$PoolEntry.run
(ThreadGroupPool.java:270) at java.lang.Thread.run (Thread.java:748)
Caused by: java.lang.ClassNotFoundException:
com.google.appengine.api.ThreadManager at
java.net.URLClassLoader.findClass (URLClassLoader.java:381) at
com.google.apphosting.runtime.ApplicationClassLoader.findClass
(ApplicationClassLoader.java:135) at java.lang.ClassLoader.loadClass
(ClassLoader.java:424) at java.lang.ClassLoader.loadClass
(ClassLoader.java:357)
Is there some way for me to clear out the partial build (or whatever there may be) in google app engine?
I tried deleting all versions and instances (you cannot delete them all, it won't let you delete the serving one).
I tried increasing the version number in appengine-web.xml.
I tried gradle clean.
I tried disabling and enabling the application in the google cloud console.
No luck.
One idea I have is maybe if I can somehow force all the files for the app to be uploaded again? Because with gradle appengineDeploy only the changed files get uploaded.
EDIT:
I managed to fix one part of this by upgrading classpath 'com.google.cloud.tools:appengine-gradle-plugin:2.2.0' in my gradle to the latest version. (Felt kind of hackish to do that and not reliable in the future if this happens again.) Maybe this flushed whatever was in app engine or something by doing that. I still have one problem remaining:
I am using the firebase admin SDK to send firebase cloud messages (authenticated with a .json credentials file) like this:
FirebaseMessaging.getInstance().send(msg);
When I do I am getting this error:
com.google.firebase.messaging.FirebaseMessagingException: Unexpected
HTTP response with status: 401; body: null
Which is strange because other parts of the firebase admin SDK are working (like writing/reading from firestore).
So there is still something weird going on and I think it has to do with the fact that as I was uploading my google app engine project the connection was interrupted.
EDIT 2:
Here is something interesting: When I deploy my application using gcloud app deploy appengine-web.xml the application again will not work. Deploying with gradle appengineDeploy does though. I also noticed that the size of the application is shown as much smaller in the GCP console when deploying with gcloud app deploy appengine-web.xml. So something is messed up here. I tried looking up some sort of gcloud command to clear cache? Or something like that but no luck.
EDIT 3:
Additional Info:
My app was already deployed and working before the failed attempt. I changed one small piece of code in a function and upon uploading the app, the connection was interrupted because the internet went out.
I am on app engine standard environment.
I am deploying my application from the macOS terminal and using android studio to develop.
I have tried the stopPreviousVersion promote and version configs in gradle (actually the first two are true by default and version gets auto-generated if you do not set it).
Running gcloud app deploy appengine-web.xml --verbosity=debug shows a lot of sensitive information but one thing I am seeing is all the files in WEB-INF are being skipped:
DEBUG: Skipping upload of [WEB-INF/....
INFO: Incremental upload skipped 100.0% of data
DEBUG: Uploading 0 files to Google Cloud Storage
DEBUG: Using [16] threads
So perhaps files are not all being uploaded? This SO post raises a similar problem but has no solution: stackoverflow.com/q/42137452/3075340
It's weird but when I do gcloud app deploy, the app won't work at all. There are run-time errors all over the place. Doing gradle appengineDeploy fixes that but I still am having the firebase-admin issue.
To actually answer your question, GAE uses a staging bucket to cache files.
You can delete this bucket, and it will get recreated next time you try to deploy.
The bucket is always named staging.[PROJECT-ID].appspot.com. It should show on the buckets overview page
Just to be clear, I'm not anywhere near sure that this will actually resolve your issue, but this will most definitely clear whatever files GAE cached
Here there was a discussion on a, more or less, similar issue on App Engine from last year.
You can use it as a source of inspiration to fix your current issue, because I think they are really similar as a concept. From there I think it may worth trying to put the appengine-api-1.0-sdk-1.9.63.jar ( or whatever your version is ) file under WEB-INF/lib.
It the same post there were some guys who found out the there was a problem with gcloud v194 and switching to another version fixed the issue.
Here there is a similar issue which was resolved by upgrading the gcloud tool.
Anyhow, this behaviour is not normal and things should not work like this. You can try to find a workaround, but my recommendation is to report the issue and let an App Engine engineer taking a look over it. There are good chances to be something internal.
In the google release notes it says:
November 15, 2017
Go runtime notes
Updated Go SDK to 1.9.61
Add --go_debugging flag to dev_appserver.py to enable Delve debugging.
I'm using dev_appserver.py to fire up several services simultaneously, so that they all share the same datastore emulator, and it works great.
However, when I add that --go_debugging flag, I get lots of errors which I don't understand. If anyone has insight, I'd be grateful.
I get these kinds of errors for each service it tries to build:
can't load package: package -N: unknown import path "-N": cannot find module providing package -N
can't load package: package -l: unknown import path "-l": cannot find module providing package -l
Is there maybe a problem with launching several services at once?
Thanks!
Update Nov 2020:
With all the new changes to app engine being able to use standard go libraries at google APIs, now I can just run my go app locally without using dev_appserver, and let it attach to my local datastore emulator etc. Because of that, and thanks to vscode, delve debugging works great (with the go extension). So now I can step through my app engine standard go code.
By following this tutorial I created a go lang project and opened it by Visual Studio Code.
The code itself works fine I can run the server, but somehow VS Code shows
cannot find package "appengine" in any of:
/usr/local/Cellar/go/1.8.3/libexec/src/appengine (from $GOROOT)
/Users/ironsand/go/src/appengine (from $GOPATH)
I thought I must set GOROOT for the Google App Engine, but according to this stackoverflow question I shouldn't.
How to make VS Code recognize google app engine library properly?
More info
The appengine package exists in ~/dev/google-cloud-sdk/platform/google_appengine/goroot-1.8/src/appengine
I'm using macOS Sierra 10.12.6.
I did use that tutorial and it's working on my Mac via terminal and via VS Code.
You may follow this tutorial to be able use VS code to deploy GAE app with python:
Deploy GAE app with VS code
• Dont forget to update path of dev_appserver.py in tasks.json.
If that didn't work out, you would need to re-install/init google-cloud-sdk.
Note that it's necessary to provide root privileges during installing/initializing in order to allow installer script to add paths properly.
Last thing: avoid tilde expansion ~ in path. Use absolute path in config.( in general absolute path must be used always unless there is a reason to use relative address with tilde )
Good luck,'.
I am building a Google App Engine (GAE) project under Android Studio. I want to be able to have different "buildTypes" similar to how one can do for an android app project, however this is not supported by the appEngine Gradle plugin (see https://github.com/GoogleCloudPlatform/gradle-appengine-plugin/issues/177). As a workaround, I want to run some kind of script (e.g.: ant, bash, or similar), to do some arbitrary command prior to the build, so that I can copy or rename the right source files into place for the build. Unfortunately, being somewhat unfamiliar with Gradle, I am at a loss of how to do that. I've been reading the Gradle documentation but have been unable to discover how to add a hook before the :backend:compileJava task is executed. Can anyone explain how I might be able to do this?
If you haven't solved this yet
You can have any task depend on any other :
task myPrecompileTask {
doLast {
// execute some code here
}
}
compileJava.dependsOn myPrecompileTask
That will force any call to compileJava to call myPrecomileTask first.
I am trying to add Google Cloud Storage functionality to a Python GAE app that is already running with significant functionality. I work entirely within PyCharm on my development computer, which is a Mac running OS X 10.9.5.
I have created a new Python module that contains this statement:
import cloudstorage as gcs
as shown in the sample code at https://cloud.google.com/appengine/docs/python/googlecloudstorageclient/getstarted
When I first added that line, PyCharm said "No module named cloudstorage" in the editor.
I then followed both the "pip" and the "svn" instructions at https://cloud.google.com/appengine/docs/python/googlecloudstorageclient/download
to download the GCS Client Library.
In trying to follow those instructions, taking into account my prior experience with this programming environment, I actually tried using "pip" three times:
Once without the "-t" option, since I've never needed that option with "pip" before
Once using the "-t" option to specify my application directory's "lib" subdirectory
Once using:
pip install GoogleAppEngineCloudStorageClient -t /Applications/GoogleAppEngineLauncher.app//Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib, since I wasn't sure what the instructions meant by "<your_app_directory/lib>"
As mentioned, I also executed the "svn" command. Then, as mentioned in install python google cloud storage client on Ubuntu 14.04, I ran "cd gcs-client/src" and "sudo python setup.py install". I ran these commands in my user root directory.
After each of those successful but different installations of the GCS Client Library, I looked at the PyCharm editor window for my module, and it always had the same "No module named cloudstorage" error. But as an experiment, I would also try restarting PyCharm, and also try running my app.
At some point, the editor window stopped showing the error. It was not immediately after one of those steps above, but after I would go away to read various webpages and then come back to look at the error again. I don't know which of the installations was the one that got rid of the error message in the PyCharm editor.
In any case, whenever I try to run the app (again, inside PyCharm), I always get the runtime error "ImportError: No module named cloudstorage" on the same import statement.
The Run/Debug Configuration page for this app has both "Add content roots to PYTHONPATH" and "Add source roots to PYTHONPATH" checked.
Of course the main help I want is how to get past the "No module named cloudstorage" runtime error, even though the import statement no longer shows an error.
I think I also have as many as three spurious versions of the GCS Client Library. I'm much more concerned with getting past "Module not found", since it's a show-stopper, but if you have any idea how I can delete the spurious versions so that they're not just lying around, I'd be most grateful for that help as well.
If the "cloudstorage" directory is at <app>/lib/cloudstorage, then the import statement has to specify "lib":
import lib.cloudstorage
In my case, it's:
import lib.cloudstorage as gcs
By the way, the <app>/lib/GoogleAppEngineCloudStorageClient-1.9.5.0-py2.7.egg-info directory does not seem to be needed and can be deleted.
Actually, you also need to
touch __init__.py
in the lib directory. This will make the cloudstorage module visible to the "import lib.cloudstorage" command.
Dear Google: The distributions should include this file (or the procedure should account for it), and the demo script should be changed to reflect the expected usage. But more importantly why are you distributing/PROLIFERATING library code like this??!!! Why is this not distributed via gcloud? How am I ever going to pick up a patch for this library?
The accepted answer's solution
import lib.module_name
definitely can solve the problem. But I don't like add lib in front of every single module and happened to see how Google suggest import third party libs like this.
appengine will automatically run a file called appengine_config.py. So you can create such a file and put
from google.appengine.ext import vendor
vendor.add('lib')
inside that file. This will help you tell appengine to find dependencies in that folder, so you can simply write
import cloudstorage as gcs
I solve the missing module issue by adding the following to my main application file (main.py):
import os
import sys
sys.path.append(os.path.join(os.path.dirname(__file__), "lib"))
I think this is the way Guido intended. Now my code simply says import cloudstorage as gcs. None of the lib.cloudstorage or lib/__init__.py business.
From
https://stackoverflow.com/a/37645984/1740008