I am deploying a Plotly Dash app on Google App Engine but meet some difficulties. The data source to be queried in the dashboard is a Bigquery table, whose content is changing. I hope that the data in the App can always be the latest.
What I tried is at the beginning of the main.py code, I read in the table from Bigquery by Bigquery Python API, but after the App being deployed onto GAE, I found the data was fixed; even I deleted the Bigquery table, the App was not affected. May I know what is the correct way to get data from BigQuery to App Engine? Thanks.
How to connect to BigQuery from GAE using Python might be a bit more of a task than a single question can answer, but here are some hints:
Everything Google Cloud can (in my opinion) be best understood through the repositories on Github. For instance, the python docs samples contain several examples, out of which I think the client example is probably the easiest and most basic. Bigquery Python Samples are here. That will basically answer your question, except for a few gotchas I will mention.
You will of course need to download the client library to do development on a local environment. That is straightforward, but if something seems not to be working make sure you have enabled the API service account for your project--that can be a little confusing.
Something that is critically important to remember is that your GAE app will not be able to easily communicate with BigQuery if it is in a different region, and, in fact, once you set up a GAE app you cannot move or delete it! So, do pay attention to what you are doing as you set up, and if you have a locations mismatch you will need to migrate your BQ instance to the matching location.
Related
We have Java servlets up and running on GAE, using blobstore, datastore and other cloud services.
Currently, we're starting a migration process to cloud Endpoints and we've hit an issue: if we use a different GAE project, we would not be able to query regarding current datastore entities (to the best of my knowledge, Google doesn't want you to do this - see
this question
and the GAE terms of service - section 3.3d), so we need to use the same project for both.
I looked up whether it's possible to have one GAE instance running Java servlets and one instance running Endpoints, but I found no conclusive answer anywhere.
If we try to implement and something goes wrong, we're looking at a potentially major issue for our users, so we need to be sure beforehand.
Has anyone tried something similar, and can assure us that this works?
You have 2 options to run the old and the new code inside the same app (thus with no issues sharing access to the datastore) but as separate engine instances, so they can be developed/deployed/managed independently:
as different versions of the same app/module(s):
the old version remains the default, the new one can be accessed at a different URL during development (possibly via URL routing)
you can use traffic splitting to do live A/B testing on the new code and for gradual final migration until you make the new version the default
as different modules of the same app:
both can run (fully functional) side by side indefinitely, but you need to be more careful during development
traffic is routed to the modules in several possible ways
final migration is done by publishing the new URLs, eventually re-directing the old URLs and finally bringing down the old module code
The 2 approaches can even be combined, if needed, as the final solution described by the OP's in this somehow similar question (for the python environment, but java equivalents exist): Google App Engine upgrading part by part
I was handed an assignment but I don't know where to start.
The aim is to have 2 piece of code running. One will run in Open stack private cloud and perform the task of indexing two sets of text, with another running in EC2 with the task of matching the two indexed tests.
I want to access them via google app engine.
Ideally, I would like to click a button or perform an action on Google app engine, which then sends a request to Openstack to run the code and retrieve the output of a txt file.
That outputted text files will then be forwarded onto EC2 where the matching will occur and the results sent back to Google App Engine.
My question is, how can I send the files between the systems using REST requests?
FrankN --
EC2, GAE and OpenStack are disparate cloud computing platforms. To integrate them might include, say, using one platform while saving backups to another.
CloudU.Rackspace.com is a vendor-neutral education site about cloud computing (note: It'll take six or so hours to finish it all). This might help.
Disclaimer: I work for Rackspace.
Firstly, not really sure what your requirements are, why your code does or why are you trying to mix cloud providers in that way.
That said, I would suggest taking the upload from GAE and push it to AWS S3 where you can then retrieve and use as you please from EC2.
Not sure what functionality you are trying to get out of OpenStack that is not present in AWS; however, I would suggest building whatever you are building in EC2 first, then duplicate in on OpenStack services to avoid future vendor lock in.
I am somewhat new to Web development - specifically Google App Engine and JavaScript/HTML development, but I have an app deployed and working on Google App Engine and it is working ok.
I would like a user of my App to be able to store and retrieve a serialization of the app state in JSON using the GAE Datastore. (Note - This is only a user-initiated action - so channels seem to be overkill)
The examples provided by Google demonstrates one approach that allows the server-side Python implementation to do this. Specifically https://developers.google.com/appengine/docs/python/gettingstartedpython27/usingdatastore. I have this working ok.
But this approach seems rather inelegant especially if as an "app" I want to store and retrieve serialized chunks of data somewhat asynchronously without reloading the page/app each time (again, this is only ever user-initiated).
I have not been able to find any high-level guidance on an approach to do that (assuming it is possible).
Any suggestions/links/examples would be greatly appreciated.
Thank you!
Jeff
As with many things, this depends on your specific needs. If you just want direct access to datastore storage, the datastore is exposed as an independent service with an API.
If you instead want to assert logic over the usage and interact with your app in some fashion, you may also want to look at Google Cloud Endpoints. With an endpoints API, you gain a more structured API you can call directly from javascript, or generate client libraries to be consumed by other languages/platforms.
I would like to have some data fixtures put into data store when my go project is deployed, similar to what can be done in Django. Is it possible? If not, are there any alternatives?
if your are looking for some configuration to put data into datastore, the answer is no. also the python and java versions of appengine don't have such feature.
what i think the django framework for appengine is doing, is the same as you could do.
just write the data you need to the datastore inside your app init method
also inside init you check if the data is already written.
then any time the app is startet, he checks if it should write some fixture data to datastore, and you can be sure that before first request is processed that its there.
I would like to write a client application for Android that uses the Google App Engine as a database backend. My Android client would connect to the App Engine to save information, then it would connect later for reports. Is it possible to use the App Engine as a backend like this?
If you're looking for something like the remote api that the App Engine has in python, then you'll be disappointed to find it missing in Java.
That said, absolutely nothing stops your from hitting your app and posting data either through POST / JSON / XML / any other format you can think of. The same thing goes for getting your reports back.
If security is a concern, the OAuth protocol allows you to authenticate to app engine from your android device.
This is an aside, but as far as reporting is concerned, you might not find the app engine a very suitable platform for reporting type apps. Just make sure you understand its limitations - the lack of joins, 1000 object limit, no sum / average, necessary indexes, etc. It's certainly not impossible, but do think carefully about how you're going to model your data.
Yes, it is possible.
Without more details in your question, any more details in the answer would be speculation.
Yes, its very much possible. It's something I am also currently working on.
My code uses HTTP GET and HTTP POST and I am using a RESTful service on the GAE.
I'm sorry I can't provide any code because I am still learning however the library I'm using is called RESTLET. They have libraries for both GAE and Android however I'm only using RESTLET on the GAE and I'm just using the HTTP library in the Android SDK for the client.
http://www.restlet.org/
The version you require is 2.0 M6 and not the stable release.
No.
In your response to Laurence, you said you want a direct DB connetion. A client cannot connect directly to the GAE datastore. You must write web handlers to interface between the client and your data. It doesn't have to be much, but it must be something.
Yes, it is very possible. You would not connect directly to the GAE database though. A better architecture would be to make your app hit a URL that writes to the DB. For example, you could set up a Struts 2 action that takes the values of your query parameters and then mutates and validates them as necessary before persisting them.