I am developing a web app to run on either Google's AppEngine or a basic server with file storage (it may not stay that way but that's the current status).
How do I detect whether the AppEngine services (most importantly blobstore) are available at runtime?
I have tried using code like the following:
try{
Class.forName( "com.google.appengine.api.blobstore.BlobstoreServiceFactory" );
logger.info( "Using GAE blobstore backend" );
return new GAEBlobService();
}catch( ClassNotFoundException e ){
logger.info( "Using filesystem-based backend" );
return new FileBlobService();
}
but it doesn't work because BlobstoreServiceFactory is available at compile time. What fails if trying to use GAE's blobstore without a GAE server is the following:
com.google.apphosting.api.ApiProxy$CallNotFoundException: The API package 'blobstore' or call 'CreateUploadURL()' was not found.
There's a few things you can use.
You can check the runtime environment to check the running version of App Engine. Check the section about "The Environment" in the runtime docs: https://developers.google.com/appengine/docs/java/runtime
You could also do what you were doing, and attempt to make an call that uses the SDK API functions (instead of just checking for the existence of a class) and catch the exception. This may negatively impact performance since you're making an extra RPC.
You could check request headers for GAE specific request headers too.
Related
Lately I've been deploying apps on App Engine Standard and sometimes I accidentally deploy a version with a typo in my env_variables specified on app.yaml. Let's say that I don't want to deploy a new version because it takes time and this typo should be easily fixed with a "quick patch".
I followed the GAE Admin REST API doc and played around with the API Explorer. It turns out that the request body accepts envVariables so I was thinking that this API can solve my problem but for some reason it fails.
Error:
{
"error": {
"code": 400,
"message": "This operation is only supported on the following field(s): [automatic_scaling.cool_down_period_sec, automatic_scaling.cpu_utilization.target_utilization, automatic_scaling.max_idle_instances, automatic_scaling.max_total_instances, automatic_scaling.min_idle_instances, automatic_scaling.min_total_instances, automatic_scaling.standard_scheduler_settings.max_instances, automatic_scaling.standard_scheduler_settings.min_instances, automatic_scaling.standard_scheduler_settings.target_cpu_utilization, automatic_scaling.standard_scheduler_settings.target_throughput_utilization, instance_class, manual_scaling.instances, serving_status]",
"status": "INVALID_ARGUMENT"
}
}
Request Parameters:
appsId: PROJECT_ID
servicesId: SERVICE_NAME
versionsId: 2021xxxxx
updateMask: envVariables
Request Body:
{
"envVariables": {
"my_key":"my_value"
}
}
Overall question: Is it not possible to patch env_variables on App Engine Standard using REST API or I'm just missing something? Are there any alternatives to avoid redeployment?
Nope, there is no other alternative for this particular use case.
The reason you cannot modify the envVariables using GAE Rest API is that those variables are used when your app is being built in App Engine, so modifying them will imply re-build all the app which implies at the same time you need to redeploy the app.
You can modify the scaling settings since those not depend on the application or how it is built.
Now, that you're able to see envVariables as an option in the API explorer does not mean it is intended to be used with apps.services.versions.patch, in fact that option is shown as well with others just like a generic menu with all the possible options in the API but in the documentation you share it is not mentioned you can use it for the particular method you mention.
I'm following the guidelines and updating my code to use the new Cloud Storage API in GAE, i do need to set the cachecontrol headers, previously it was easy:
files.gs.create(filename, mime_type='image/png', acl='public-read', cache_control='public, max-age=100000, must-revalidate' )
BUT, with the new API, the guidelines says that the "cache_control" is not available...
I get this error when tried to put the cachecontrol inside the Options:
ValueError: option cache_control is not supported.
Tried with Cache-Control and the same error...
As usual, the documentation of the new API is not good.
Can someone help me how to set the cache headers in the new Cloud Storage API using PYTHON. In case is not possible, can I still use the old api for my project?
Thanks.
You are right. As documented here,
the open function only supports x-goog-acl and x-goog-meta headers.
Cache control is likely to be added in the near future to make migration easier. Please note that the main value of the GCS client lib is buffered read, buffered resumable write, and automatically retries to overcome transient errors. Many other simple REST operations on GCS (e.g cache, file copy, create bucket ...) can already be done by Google API Client. The "downside" of Google API Client is that since it doesn't come directly from/for App Engine, it does not have dev appserver support.
I want to put a scraping service using Apache HttpClient to the Cloud. I read problems are possible with Google App Engine, as it's direct network access and threads creation are prohibited. What's about other cloud hosting providers? Have anyone experince with Apache HttpClient + cloud?
AppEngine has threads and direct network access (HTTP only). There is a workaround to make it work with HttpClient.
Also, if you plan to use many parse tasks in parallel, you might check out Task Queue or even mapreduce.
Btw, there is a "misfeature" in GAE that you can not fully set custom User-agent header on your requests - GAE always adds "AppEngine" to the end of it (this breaks requests to certain sites - most notably iTunes).
It's certainly possible to create threads and access other websites from CloudFoundry, you're just time limited for each process. For example, if you take a look at http://rack-scrape.cloudfoundry.com/, it's a simple rack application that inspects the 'a' tags from Google.com;
require 'rubygems'
require 'open-uri'
require 'hpricot'
run Proc.new { |env|
doc = Hpricot(open("http://www.google.com"))
anchors = (doc/"a")
[200, {"Content-Type" => "text/html"}, [anchors.inspect]]
}
As for Apache HttpClient, I have no experience of this but I understand it isn't maintained any more.
I created one google app engine project. There I configured DWR(DirectWebRemoting).
I have created one ajax functionality which will checks the username and password of the user. But it does give me any output rather it gives Error:
dwr is not defined
Source File: http://localhost:8081/dwr.jsp
Line: 16
Where as it works fine in Tomcat web application.
So My question is that does Google app engine not support DWR configuration?
DWR is compatible with GAE according to "Will it play in Java" (broken link. possibly: a list of GAE-compatible java libraries) page.
Several links to threads with proof of success and potential limitations:
thread about "DWR 3 RC1" from "users#dwr.java.net"
StackOverflow question "DWR sometimes die on the GAE server"
Most useful post in "DWR 3 RC1" thread is this one (about potential problems).
Do you require reverse ajax? (Current implementation uses a thread to clean up expired sessions. Current implementation uses javax.swing.event.EventListenerList which is blacklisted by GAE/J.)
Do you require file upload? (Can potentially write to disk)
Do you require file download? (Current implementation uses a thread to clean up expired downloads)
If you don't need anything from the above, then you could replace problematic classes with dummy implementation.
I don't think this is supported, but the same functionality is offered by channels :
https://developers.google.com/appengine/docs/java/channel/
I'm trying to make http requests from my Google App Engine webapp, and discovered I have to use URLConnection since it's the only whitelisted class. The corresponding Clojure library is clojure.contrib.http.agent, and my code is as follows:
(defroutes example
(GET "/" [] (http/string (http/http-agent "http://www.example.com")))
(route/not-found "Page not found"))
This works fine in my development environment- the browser displays the text for example.com. But when I test it out with Google's development app server:
phrygian:example wei$ dev_appserver.sh war
2010-09-28 14:53:36.120 java[43845:903] [Java CocoaComponent compatibility mode]: Enabled
...
INFO: The server is running at http://localhost:8080/
It just hangs when I load the page. No error, or anything. Any idea what might be going on?
http-agent creates threads so that might be why it does not work.
From the API documentation:
Creates (and immediately returns) an Agent representing an HTTP
request running in a new thread.
You could try http-connection, which is a wrapper around HttpURLConnection, so this should work.
Another alternative is to try clj-http. The API seems to be a bit more high-level, but it uses Apache HttpComponents which might be blacklisted.
I am guessing http.async.client is a definite no-go due to its strong asynchronous approach.
You might want to try appengine.urlfetch/fetch from appengine-clj (http://github.com/r0man/appengine-clj, also in clojars)