I need to run some statistical tests in my app, which needs functions from scipy.stats. However I found Google App Engine doesn't trust SciPy. So is there any GAE supported libraries which can do some stats calculations e.g. generate random numbers, estimate CDF, run T-tests, check normality, etc. Thanks!
The Python 2.7 runtime includes NumPy. It's a scientific library that can help you in what you want to do.
Numpy is a language extension that defines the numerical array and matrix type and basic operations on them.
More info for NumPy here
Btw:
However I found Google App Engine doesn't trust SciPy.
I think should be:
GAE only supports native python code
Edit
For random numbers you can use Python's Random
For CDF http://pysclint.sourceforge.net/pycdf/ or http://code.google.com/p/netcdf4-python/ but I am not sure if they just contain native code. You can try if you like.
Also take a look here http://www.astro.cornell.edu/staff/loredo/statpy/
Another approach could be to have a home server running any python module you might please. Then use a PULL QUEUE TO communicate via REST with your "home" server and process the calculations there.
Related
Using Python 3.4 Google App Engine Flex.
Google documentation on using pull queues with Python says to "from google.appengine.api import taskqueue", but does not explain how to make taskqueue available to Python runtime.
They do link to "Easily Access Google API's from Python", where it explains how to install the api client via "pip install google-api-python-client"
This does not install the taskqueue lib.
From the previous doc, there is a link to "Installation", where it says:
Because the Python client libraries are not installed in the App Engine Python runtime environment, they must be vendored into your application just like third-party libraries.
This links to another page "Using third-party libraries", which states you need to either install a lib to /lib or use requirements.txt. Neither of these make taskueue available.
Searching for taskqueue.py in Google's github shows only an example module with the same name.
There is a documentation page on the module, but no information on how to install it.
There is a Python 2.7 example that google points to here, but it doesn't work. There is no setup of taskqueue, no requirements.txt, no instructions.
There is a stack overflow question on this topic here, and the checked answer says to install the SDK. That takes you to here, which takes you to here, which takes you here, which takes you to here, which provides the gcloud SDK download for deploying and managing gcloud. This does not include the python lib for taskqueue.
There is another similar stackoverflow question here, which says:
... this is now starting to feel like an infinite loop. Yes, it's been made crystal clear you need to import the taskqueue. But how do you make it available?
I've asked the question to Google Support and they haven't been able to answer for 4 days.
I've opened two issues, one here and another here. No answers yet.
Do not want to use Python < 3.4.
Do not want to use HTTP REST API.
Just want a simple pull queue.
Many of the docs you mentioned are standard environment docs and do not apply to the flexible environment.
From the Task Queue section in Migrating Services from the Standard Environment to the Flexible Environment:
The Task Queue service has limited availability outside of the
standard environment. If you want to use the service outside of the
standard environment, you can sign up for the Cloud Tasks alpha.
Outside of the standard environment, you can't add tasks to push
queues, but a service running in the flexible environment can be
the target of a push task. You can specify this using the
target parameter when adding a task to queue or by specifying
the default target for the queue in queue.yaml.
In many cases where you might use pull queues, such as queuing up
tasks or messages that will be pulled and processed by separate
workers, Cloud Pub/Sub can be a good alternative as it offers
similar functionality and delivery guarantees.
I have implemented a simple API in Go on Google App Engine Standard using just:
func init() {
http.HandleFunc("/api/v1/resource",submitResource)
}
Nothing special. However I want to port this code to using Cloud Endpoints instead in order to get the better monitoring and diagnostics.
Is it even possible with STANDARD instances or must I move to FLEXIBLE?
I can't find any documentation on this. Nor answers to this seemingly simple question. At the moment I half wish I had chosen Python because its support seems more mature. I chose Go because it seems more appropriate for API-like code because my minimal research suggested Go offered better performance.
If it is possible, are there any pointers to how please?
Only Python and Java are supported on GAE Standard via the Endpoints Frameworks. However, Go is supported on GAE Flexible.
Here is the Go GAE Flexible sample:
https://github.com/GoogleCloudPlatform/golang-samples/tree/master/endpoints/getting-started
After much research and trial and error, the simple answer is "No." - as of Dec 2016.
The longer answer is it's possible if you want to put far too much effort into making up to date libraries of your own. There is basically no support, even in alpha, for the current Google Cloud Endpoints using Go with Google App Engine Standard.
It's possible to run Go+endpoints on GAE Standard environment, however libraries might be outdated now.
Libraries and sample app can be found on github:
https://github.com/GoogleCloudPlatform/go-endpoints
I have successfully deployed "Greetings" as AppEngine SE app, and it works.
I am looking for a solution for a script language like lua to use with Go application running on GAE. I have found golua and luar projects and planned to use them as the solution.
However, once I ran them on GAE environment, I encountered
"o-app-builder: Failed parsing input: parser: bad import "unsafe" in
github.com/stevedonovan/luar/luar.go"
I was confused but finally found that apparently GAE trimmed unsafe package out for a reason. Since luar and golua need the package, I think I have to find a new solution for this.
Is there any way to use luar and golua on GAE? If it is not possible, are there any alternative script languages that will run on GAE environment?
The just announced Managed VMs will allow you to run App Engine applications on Compute Engine virtual machines. This allows access to the full range of libraries, filesystems and sockets. As of today (April 20th, 2014) Managed VMs are in Limited Preview, so you'll need to fill out the form here to get access.
I am developing an application on Google app engine using Python. I want to use editdist feature of Python and for that reason I am importing editdist C python module in my program, but it is showing that module editdist does not exist.
When I import editdist for my local application it is working fine but not for Google app engine application.
Can anyone suggest me a method to import this module?
App Engine is a "pure python" environment, and you cannot use any C extensions or other code that must be compiled.
There is therefore no way to use this program on App Engine, and all of the competing "production quality" python libraries I found were implemented as C modules.
There do exist alternate implementations of the Levenshtein distance algorithm, though none are as nearly as fast as editdist. These more naïve implementations might still be acceptable depending upon your needs.
Here could be couple alternatives that are implemented with Python (I haven't tested them myself):
http://www.korokithakis.net/node/87
http://code.activestate.com/recipes/576874-levenshtein-distance/
I've created a web app in Python 3. It all runs beautifully until I have to upload a file... There's no way to find the path or the file in the environment. I am using wsgi and I am thinking of migrating to another web server, what are your recommendations?
This is all what I receive from the
s = FileWrapper(environ.copy()['wsgi.input'])
for y in s:
print(y)
And the response of that is:
b'-----------------------------1514423166515917395188753897--\\r\\n'
What does this mean?
That's the file data, sent in a format guaranteed to survive to the destination, which is you. You need to decode it, and there are libraries for doing this in Python.
However, since you clearly don't know how these kinds of technical details work in the Web, if you want to write your own webserver you will have to learn. And it's not a trivial question.
I suggest you use some sort of web framework. There are tons of them for Python, I would recommend Pyramid or Django, both which now are ported to Python 3.