I have a question on using tensorflow on google cloud platform.
I heard that Google cloud tensorflow doesnt support Keras (keras.io). However, now i can see that Tensorflow has its own API to access Keras (https://www.tensorflow.org/api_docs/python/tf/contrib/keras).
Given this, can I use the above mentioned API inside google cloud, since it is coming out along with Tensorflow package? Any idea sir?
I am able to access this API from the tensorflow installed on a anaconda machine.
Option 1# Please try package-path option.
As per the docs...
-package-path=PACKAGE_PATH
"Path to a Python package to build. This should point to a directory containing the Python source for the job"
Try and give a relative path to keras from your main script.
More details here:
https://cloud.google.com/sdk/gcloud/reference/beta/ml-engine/local/train
Option 2# If you have a setup.py file
Inside your setup.py file within setup call pass argument install_requires=['keras']
Google Cloud Machine Learning Engine does support Keras (keras.io), but you have to list it as a dependency when starting a training job. For some specific instructions, see this SO post, or a longer exposition on this blog page. If you'd like to serve your model on Google Cloud Machine Learning or using TensorFlow Serving, then see this SO post about exporting your model.
That said, you can also use tf.contrib.keras, as long as you use the --runtime-version=1.2 flag. Just keep in mind that packages in contrib are experimental and may introduce breaking API changes between versions.
Have a look at this example on git which I saw was recenly added:
Keras Cloud ML Example
Related
There is a python and C++ api available for doing image recognition.The tutorial is provided in tensorflow.org but as tensorflow.js is released few months back only does it support all the api's available in the python and c++ implementation.
Vincent Mühler has created face-api.js, a JavaScript API base on tensorflow.js. You can find out the blog & code on the links below.
https://itnext.io/face-api-js-javascript-api-for-face-recognition-in-the-browser-with-tensorflow-js-bcc2a6c4cf07
https://github.com/justadudewhohacks/face-api.js
Adding to the answers above, tensorflow on JavaScript can be quite slow. Here's a quick comparison:
However, If you do run tensorflow.js on Node, you could make use of the binding directly to the TensorFlow APIs written in C which would be fast. You could also run the CUDA versions if you import the right packages on Node.js
On the browser WebGL is used to run tensorflow. Using the tensorflow or some ML on the browser, opens up whole new opportunities to do things from right within the browser.
As Jirapol suggested, you could take a look at https://github.com/justadudewhohacks/face-api.js which is super easy to use. It actually took a very short while for me to start writing a facial recognition login system on node using face-api.js. Here's a link if you want to take a look at the unfinshed code: https://github.com/WinstonMarvel/face-recognition-authentication
does it support all the api's available in the python and c++ implementation.
No, it still has a limited amount of features. Keep in mind it still in version 0.11.6 so that will change. You can look at the documentation to see what's available.
If you want to port a specific model to tfjs try to get it as a keras model then convert it using tensorflowjs_converter to a tfjs compatible one like this tutorial shows.
There is even a tfjs examples which works with webcam data (Tutorial, Live Demo), so you could look into that to start.
Yes it can.
and also with the help of webassembly and SIMD in the browser.
you can have a smooth experience of image processing and video processing in the browser.
have a look at this link from google v8.
the good news is that with the same api you can run Tensorflow.js in the browser, node.js and React Native all with the native speed and using native capabilities
Using Python 3.4 Google App Engine Flex.
Google documentation on using pull queues with Python says to "from google.appengine.api import taskqueue", but does not explain how to make taskqueue available to Python runtime.
They do link to "Easily Access Google API's from Python", where it explains how to install the api client via "pip install google-api-python-client"
This does not install the taskqueue lib.
From the previous doc, there is a link to "Installation", where it says:
Because the Python client libraries are not installed in the App Engine Python runtime environment, they must be vendored into your application just like third-party libraries.
This links to another page "Using third-party libraries", which states you need to either install a lib to /lib or use requirements.txt. Neither of these make taskueue available.
Searching for taskqueue.py in Google's github shows only an example module with the same name.
There is a documentation page on the module, but no information on how to install it.
There is a Python 2.7 example that google points to here, but it doesn't work. There is no setup of taskqueue, no requirements.txt, no instructions.
There is a stack overflow question on this topic here, and the checked answer says to install the SDK. That takes you to here, which takes you to here, which takes you here, which takes you to here, which provides the gcloud SDK download for deploying and managing gcloud. This does not include the python lib for taskqueue.
There is another similar stackoverflow question here, which says:
... this is now starting to feel like an infinite loop. Yes, it's been made crystal clear you need to import the taskqueue. But how do you make it available?
I've asked the question to Google Support and they haven't been able to answer for 4 days.
I've opened two issues, one here and another here. No answers yet.
Do not want to use Python < 3.4.
Do not want to use HTTP REST API.
Just want a simple pull queue.
Many of the docs you mentioned are standard environment docs and do not apply to the flexible environment.
From the Task Queue section in Migrating Services from the Standard Environment to the Flexible Environment:
The Task Queue service has limited availability outside of the
standard environment. If you want to use the service outside of the
standard environment, you can sign up for the Cloud Tasks alpha.
Outside of the standard environment, you can't add tasks to push
queues, but a service running in the flexible environment can be
the target of a push task. You can specify this using the
target parameter when adding a task to queue or by specifying
the default target for the queue in queue.yaml.
In many cases where you might use pull queues, such as queuing up
tasks or messages that will be pulled and processed by separate
workers, Cloud Pub/Sub can be a good alternative as it offers
similar functionality and delivery guarantees.
Is it possible to run a protorpc based service outside of GAE? I see that there is a project hosted at http://code.google.com/p/google-protorpc/ but it seems to depend on the GAE. The documentation is VERY vague on this point, does anyone have a clear answer?
Thanks!
Yes. The source code available at http://code.google.com/p/google-protorpc/source/browse/ includes all of the necessary libraries (minus protobufs itself, which you can download from the Google Developers site), and contains examples both for AppEngine and for a generic WSGI server.
The protorpc library itself is server-agnostic.
After messing around with this quite a bit, I have determined that ProtoRPC currently has dependencies on the google app engine. It is possible to rip the libraries you need out of the sdk, but you will have to change the protorpc source to not import from relative paths. All in all, it is not worth the hassle, either use the GAE, or don't use protorpc until it can stand on its own.
Thanks.
I've read the GAE docs, and I can't seem to figure out how to download all my entity data.
What I'd love to do is download the whole thing as a big TSV file (or something I can easily munge into one), so I can import my various entities into a spreadsheet and fiddle with them.
But I'm stuck at the starting gate. I don't understand the first few bits of the docs: "This document applies to apps that use the master/slave datastore. If your app uses the High Replication datastore..." -- I'm not even sure which I have, or how I would tell.
Assuming I have the simpler master/slave, the docs continue: "...you can use the Python appcfg.py tool by installing the Java version of the remote_api handler..." but, again, I'm not quite sure what they mean or where I find this appcfg.py tool.
Sorry for such a n00b question, but is there some sort of walk-through? I just want to download my datastore!
Thanks!
Master/Slave is the default (for now), so that's almost certainly what you're using. You can confirm at https://appengine.google.com/ -> app-id -> Administration -> Application Settings -> Datastore Replication Options.
To download your data, first install Remote API for Java, then use appcfg.py to download data:
appcfg.py download_data --application=<app-id> --url=http://<appname>.appspot.com/[remote_api_path] --filename=<data-filename>
There is nothing you need to do other than just follwiong the google's own documentaion. There is no workaround or walkthrough. I am not sure about java. In python , only thing you need to do is enable remote_api in your app.yaml.
appcfg.py can be found inside the root directory of appengine SDK.
Is it possible to access or download my code that is running on Google App Engine? For instance, say my hard drive crashes and I have not saved my code on any kind of external repository. Is there any way to recover?
No, it is not possible. You compile your code on your machine and ship up the compiled code.
If you have a source repository, always make sure that you have a backup.
GAE does not have a mechanism for you to download your binaries, so reverse engineering them is not possible except from your own copies.
Of course you could always host your project on GitHub or SourceForge
see also:
how to download google appengine (uploaded)application files
Can I restore my source code that has been uploaded into Google AppEngine?
How do I download the source code of a google app engine project?
Download source code of deployed GAE application
Get source of appengine published version
I can't find it now, but I remember reading a post where they were pretty specific about NOT being able to do that. The gist of it was that they were not a backup service and that you were responsible for backing up your own code.
There is a "datastore backup and restore" feature on the project roadmap though: http://code.google.com/appengine/docs/roadmap.html I've skimmed over a few posts where people talk about ways to backup the database now without that the official sanctioned feature implemented... of course your datastore isn't your code.