I develop some GO libraries using Google Cloud SQL and MySQL server. When I imported `appengine/cloudsql, an error below occured.
cloud.go:20:2: cannot find package "appengine/cloudsql" in any of:
/usr/local/Cellar/go/1.1.2/src/pkg/appengine/cloudsql (from $GOROOT)
/Users/lameduck/myGo/src/appengine/cloudsql (from $GOPATH)
I know this package, appengine/cloudsql, is only for Google App Engine and it doesn't exist on everywhere else.
I'm wondering how can I use it for GAE and standard sql library for other environments in a single library.
PS: I can setup Google App Engine SDK correctly. My question is not relevant to it. I hope my library runs on Google App Engine and standalone environment together. (I already made a code for GAE and a code for other dabatases.) It is Ok that users have to setup some configurations. But I don't want that users have to modify a library source code.
Thanks for any help.
I solved the problem. I used a build constraint to use the proper routine and avoid an error. There is a build constraint for App Engine, appengine.
The App Engine SDK introduces a new build constraint term:
"appengine". Files that specify
// +build appengine will be built by the App Engine SDK and ignored by
the go tool. Conversely, files that specify
// +build !appengine are ignored by the App Engine SDK, while the go
tool will happily build them.
PS:
Anway, I upvoted other answers. Thank you.
Package import is done during compile/link time. And Go doesn't support runtime conditional imports in distinction from Python.
Feature you are looking for is dynamic library loading (like in C/C++ you can load .so/.dll in runtime), but Go currently doesn't support it.
Related
I have created a GCP Project and using App Engine standard environment for deployment purposes.
I have various developers working with me on the same project and deploying on App engine using various versions, however, we are using a default version to which all the traffic is allocated.
So are there any ways by which we can restrict the deployment on the default version. i.e we want specific people to be able to deploy on default version without removing deployment rights of other people on the same project.
And is there any alternative approach to this situation.
It depends on the runtime where you are deploying the application.
For example, in Python, if you are using the command gcloud app deploy (see the documentation), you can do the following:
gcloud app deploy --no-promote --version=<MY-VERSION-NAME>
The --no promote flag will avoid allocating all of the traffic to the version you are deploying, while the --version=<MY-VERSION-NAME> specifies the name of the version that you will create from the deployment, and replace an older one with the same name if it exists.
AFAIK there is no way to restrict deployment of a specific version. All access control method revolve around a certain identity being allowed access to deploy a certain GAE project or not. The version string used (i.e the version being deployed to in your approach) is irrelevant.
This falls into the "allows you to separate IAM roles" advantage mentioned in the accepted answer to Advantages of implementing CI/CD environments at GAE project/app level vs service/module level?.
As a note: you're attempting to implement environments at the service/module version level, which is IMHO worse than both methods compared in that post, see Continuous integration/deployment/delivery on Google App Engine, too risky?
Using Python 3.4 Google App Engine Flex.
Google documentation on using pull queues with Python says to "from google.appengine.api import taskqueue", but does not explain how to make taskqueue available to Python runtime.
They do link to "Easily Access Google API's from Python", where it explains how to install the api client via "pip install google-api-python-client"
This does not install the taskqueue lib.
From the previous doc, there is a link to "Installation", where it says:
Because the Python client libraries are not installed in the App Engine Python runtime environment, they must be vendored into your application just like third-party libraries.
This links to another page "Using third-party libraries", which states you need to either install a lib to /lib or use requirements.txt. Neither of these make taskueue available.
Searching for taskqueue.py in Google's github shows only an example module with the same name.
There is a documentation page on the module, but no information on how to install it.
There is a Python 2.7 example that google points to here, but it doesn't work. There is no setup of taskqueue, no requirements.txt, no instructions.
There is a stack overflow question on this topic here, and the checked answer says to install the SDK. That takes you to here, which takes you to here, which takes you here, which takes you to here, which provides the gcloud SDK download for deploying and managing gcloud. This does not include the python lib for taskqueue.
There is another similar stackoverflow question here, which says:
... this is now starting to feel like an infinite loop. Yes, it's been made crystal clear you need to import the taskqueue. But how do you make it available?
I've asked the question to Google Support and they haven't been able to answer for 4 days.
I've opened two issues, one here and another here. No answers yet.
Do not want to use Python < 3.4.
Do not want to use HTTP REST API.
Just want a simple pull queue.
Many of the docs you mentioned are standard environment docs and do not apply to the flexible environment.
From the Task Queue section in Migrating Services from the Standard Environment to the Flexible Environment:
The Task Queue service has limited availability outside of the
standard environment. If you want to use the service outside of the
standard environment, you can sign up for the Cloud Tasks alpha.
Outside of the standard environment, you can't add tasks to push
queues, but a service running in the flexible environment can be
the target of a push task. You can specify this using the
target parameter when adding a task to queue or by specifying
the default target for the queue in queue.yaml.
In many cases where you might use pull queues, such as queuing up
tasks or messages that will be pulled and processed by separate
workers, Cloud Pub/Sub can be a good alternative as it offers
similar functionality and delivery guarantees.
The Go SDK currently ships with Go version is 1.6.2 but the most recent is 1.7.1 . I need some enhancements/bugfixes that were released since 1.6.2 . However, when I replace the goroot directory in the SDK directory that contains Go 1.6.2 with a symlink that points to 1.7.1, I get an error that has to do with not being able to find bin/goapp, which looks to be AppEngine-specific and not provided in the standard Go build.
Does anyone know a way to upgrade the Go available in the AppEngine SDK? Does this mean that the Go in production is also 1.6.2?
Unfortunately you're stuck with the Go version that comes bundled in the latest App Engine Go SDK.
Even if you "switch" it locally with Go 1.7.1 and somehow you manage to compile and run your app with Go 1.7.1 (by adding the missing files from the SDK's Go root), the production environment currently also uses Go 1.6.2, so your app and Go code will run into errors in the live environment when code that is missing from 1.6.2 is referenced. Most likely even the deployment would fail.
Also note that when you deploy your app to App Engine, only the source files are uploaded, and your app is compiled in the cloud. So you can't even "trick" it by compiling it locally and somehow "exclude" source files and upload only the binaries (binaries are not even uploaded).
You can't do anything else but wait for Go 1.7.1 (or a newer version) to make it to the SDK. Note that the Go version bundled in the SDK usually lags a few versions behind, because for it to become the "live" version, it usually needs modifications / altering for the sandboxed environment of App Engine (certain restrictions must be applied / implemented), and it needs further / additional testing / strengthening regarding security.
At this point you should be able to upgrade - App Engine has supported Go 1.8 since 2017 and recently announced early support for 1.9.
In general, though, you're pretty much stuck with the versions supported in production - there's no way to link in your own version of Go to the SDK, and I'd argue that it would be extremely ill-advised to do so even if you could.
I'm wondering if there is any way to run and test GAE Go apps using the standard go test || go build etc tools and if it's not possible what's the technical reason.
The Go App Engine SDK contains the standard Go packages and tools, but a modified version of them.
The GAE SDK also contains local versions of the GAE platform services API implementations which are not part of the SDK (not even the API). So you can't just use the standard Go SDK. When you build or test with the GAE SDK, the SDK takes care of a context mockup so your app will have everything (or most of the things) required to "feel" it is running in the GAE environment. The SDK also contains the sandbox restrictions that are in effect in the production environment (e.g. you can't write to local files).
Also note that some features of the GAE SDK also rely on a Python runtime (because the Go GAE SDK was created using the existing Python GAE SDK), not everything is rewritten in go.
So taking all these into consideration it would not be feasible to build/test using the standard Go SDK and it is not even possible.
This question comes from working on my webapp, a feed reader written in Go. The code is available at http://github.com/matthewbauer/rivulet although that should not be necessary to answer the question.
I have a webapp written in Go that is running on Google's App Engine. It uses the usual common App Engine libraries like datastore, memcache, and user.
Is there any way to let my users run this app on their own without breaking App Engine compatibility?
Go now provides build constraints that exclude/include files based on target build platform:
// +build !appengine
So, I know that this is possible. But, my biggest problem is with my libraries that depend on App Engine: datastore, memcache, and user. I know other libraries provide datastore and memcache, and I can implement user on my own, but how would I go about wrapping it up?
I know that I can have them set up a development server with the SDK, but that might be too involved for some users. I want a single executable that normal Go projects provide. IS that possible?
If any examples exist, I haven't found them yet; any examples of App Engine independent webapps would be appreciated. I can learn by example.
You will probably have refactor your code.
The basic rule of thumb will be anywhere you depend on an AppEngine package define your own interface for the way you use it. This will allow you decouple the app from the appengine libraries.
Once you've defined those interfaces then you can start providing alternatives that satisfy the interfaces. You'll be able to plugin any Datastore or Memcache that satisfies the interfaces and your app will be able to both run on appengine or as a standalone with the alternatives.