i tried to use the custom search api ( http://code.google.com/intl/de-DE/apis/websearch/docs ) with java. it works perfectly on eclipse on my local machine.
when i try to do the same from google app engine the reply is: {"responseData": null, "responseDetails": "Quota Exceeded. Please see http://code.google.com/apis/websearch", "responseStatus": 403}
i do not understand. isn't it possible to call search api from GAE apps?
If you look at the very top of that page you linked to, they note that the API has been deprecated and the number of search queries you can make is limited.
However, if you absolutely NEED to use that API instead of the Custom Search API as Google suggests, there are a few troubleshooting steps you can take:
1) Check that your API key is unique to the project, and the limited number of queries you're allowed isn't being consumed by some other application.
2) Google does (did?) hostname filtering so that one computer doesn't use up all the API requests. You may be able to move the queries to Javascript instead of Java -- essentially move the request from the server to the client.
3) Try using a named backend (Java Backends)
Related
I'm trying to switch from basic Google Street View API to Premium. As such, when I'm using:
(space after https is added because I can't post more than two links in the post)
https:/ /maps.googleapis.com/maps/api/streetview?location=40.720032,-73.988354&size=400x400&fov=90&heading=235&pitch=10&key=MY_PREMIUM_API_KEY
I get an image with a Google watermark, which does not scale further than 640x640, same as when using a basic API key.
Okay, to use the advantages of Premium GSV API, I also need to make a digital signature. I've generated a secret key and signed my url (with dropped domain, as said in the tutorial) using python code from there: https:/ /github.com/googlemaps/url-signing/blob/gh-pages/urlsigner.py - it generates just the same signature as one on the Google website: https://developers.google.com/maps/documentation/streetview/get-api-key?hl=en_GB#premium-key
Finally, I add the signature to the URL:
(space after https is added because I can't post more than two links in the post)
https:/ /maps.googleapis.com/maps/api/streetview?location=40.720032,-73.988354&size=400x400&fov=90&heading=235&pitch=10&key=MY_PREMIUM_API_KEY&signature=MY_BASE64_SIGNATURE
However, this is what I get in return instead of an image:
"The Google Maps API server rejected your request. This API project is not authorized to use this API. Please ensure that this API is activated in the APIs Console: https://console.developers.google.com/apis/library?project=_"
It cannot be that API project is not activated, as basic API with this project & Premium key works. Any ideas, why does it happen and Premium image download doesn't work?
It appeared to be a purely technical problem, the quotas were exceeded.
I've exposed a few APIs using go-endpoints. The APIs work fine, but what I'd like to do is restrict usage of the APIs to only a few referers. Since I'm not passing any authentication information, I do not need OAuth (actually, I really do not want to use OAuth as I expect anonymous users to utilize a front-end that uses this API... I just want that front-end and perhaps another one to use my API).
Apparently the way to do this is to make a Public API Key using the Google Developers Console (Project --> APIs and auth --> Credentials --> Create new Key).
I've changed my JavaScript to use this key, by passing it as a param: https://my-app-id.appspot.com/_ah/api/myService/v1/doSomething?key=key_from_developer_console
However, when I make the call, I get a 403 back with this error:
"Access Not Configured. The API () is not enabled for your project. Please use the Google Developers Console to update your configuration."
Well, initially I set the referer to my-app-id.appspot.com/*, which is only place I want my API to be used from. So I figured I'd remove it just to see, but I get the same issue.
There are some old posts here about having to enable Contacts API and Google + API. I tried that, and it didn't work either.
So what gives? There is virtually no documentation from Google on this Public API Key feature. This is really driving me up a wall...
I had this exact same problem yesterday. I decided to generate my own key and added in my own logic to check for the 'key' param from the request. I just added the self-generated key to my env_variables and it works. However, if you try to redeploy after taking this approach, you may still see the access configuration issues..at least I have still.
Ok, i found this link https://code.google.com/p/gwt-platform/wiki/CrawlerSupport#Using_gwtp-crawler-service that explain how you can make your GWTP app crawlable.
I got some GWTP experience, but i know nothing about AppEngine.
Google said its "crawlservice.appspot.com" can parse any Ajax page. Now I have a page "http://mydomain.com#!article" that has an artice that was pulled from Database. Say that page has the text "this is my article". Now I open this link:
crawlservice.appspot.com/?key=123456&url=http://mydomain.com#!article, then i can see all javascript but I couldn't find the text "this is my article".
Why?
Now let check with a real life example
open this link https://groups.google.com/forum/#!topic/google-web-toolkit/Syi04ArKl4k & you will see the text "If i open that url in IE"
Now you open http://crawlservice.appspot.com/?key=123456&url=https://groups.google.com/forum/#!topic/google-web-toolkit/Syi04ArKl4k you can see all javascript but there is no text "If i open that url in IE",
Why is it?
SO if i use http://crawlservice.appspot.com/?key=123456&url=mydomain#!article then Can google crawler be able to see the text in mydomain#!article?
also why the key=123456, it means everyone can use this service? do we have our own key? does google limit the number of calls to their service?
Could you explain all these things?
Extra Info:
Christopher suggested me to use this example
https://github.com/ArcBees/GWTP-Samples/tree/master/gwtp-samples/gwtp-sample-crawler-service
However, I ran into other problem. My app is a pure GWTP, it doesn't have appengine-web.xml in WEB-INF. I have no idea what is appengine or GAE mean or what is Maven.
DO i need to register AppEngine?
My Appp may have a lot of traffic. Also I am using Godaddy VPS. I don't want to register App Engine since I have to pay for Google for extra traffic.
Everything in my GWTP App is ok right now except Crawler Function.
So if I don't use Google App Engine, then how can i build Crawler Function for GWTP?
I tried to use HTMLUnit for my app, but HTMLUnit doesn't work for GWTP (See details in here Why HTMLUnit always shows the HostPage no matter what url I type in (Crawlable GWT APP)? )
I believe you are not allowed to crawl Google Groups. Probably they are actively trying to prevent this, so you do not see the expected content.
There's a couple points I wish to elaborate on:
The Google Code documentation is no longer maintained. You should look on Github instead: https://github.com/ArcBees/GWTP/wiki/Crawler-Support
You shouldn't use http://crawlservice.appspot.com. This isn't a Google service, it's out of date and we may decide to delete it down the road. This only serves as a public example. You should create your own application on App Engine (https://appengine.google.com/)
There is a sample here (https://github.com/ArcBees/GWTP-Samples/tree/master/gwtp-samples/gwtp-sample-crawler-service) using GWTP's Crawler Service. You can basically copy-paste it. Just make sure you update the <application> tag in appengine-web.xml to the name of your application and use your own service key in CrawlerModule.
Finally, if your client uses GWTP and you followed the documentation, it will work. If you want to try it manually, you must encode the Query Parameters.
For example http://crawlservice.appspot.com/?key=123456&url=http://www.arcbees.com#!service will not work because the hash (everything including and after #) is not sent to the server.
On the other hand http://crawlservice.appspot.com/?key=123456&url=http%3A%2F%2Fwww.arcbees.com%2F%23!service will work.
How can we we send a query to the Log API such that it only retrieves logs from the front end and not the backends?
I don't know what runtime you're asking about, but looking at the Python source for SDK 1.8.8 you have the following arguments for the google.appengine.api.logservice.fetch function:
module_versions: A list of tuples of the form (module, version), that
indicate that the logs for the given module/version combination should be
fetched. Duplicate tuples will be ignored. This kwarg may not be used
in conjunction with the 'version_ids' kwarg.
(This isn't yet reflected in the Google Developers site)
This does not mean you can directly access front-end logs, but if you convert your app from using backends to using two named modules, one for front-end requests and another for backend work, you'll be able to fetch the logs of each independently.
There is search support (experimental) for python and Java, and eventually Go also may supported. Till then, how can I do minimal search on my records?
Through the mailing list, I got an idea about proxying the search request to a python backend. I am still evaluating GAE, and not used backends yet. To setup the search with a python backed, do I have to send all the request (from Go) to data store through this backend? How practical is it, and disadvantages? Any tutorial on this.
thanks.
You could make a RESTful Python app that with a few handlers and your Go app would make urlfetches to the Python app. Then you can run the Python app as either a backend or a frontend (with a different version than your Go app). The first handler would receive a key as input, would fetch that entity from the datastore, and then would store the relevant info in the search index. The second handler would receive a query, do a search against the index, and return the results. You would need a handler for removing documents from the search index and any other operations you want.
Instead of the first handler receiving a key and fetching from the datastore you could also just send it the entity data in the fetch.
You could also use a service like IndexDen for now (especially if you don't have many entities to index):
http://indexden.com/
When making urlfetches keep in mind the quotas currently apply even when requesting URLs from your own app. There are two issues in the tracker requesting to have these quotas removed/increased when communicating with your own apps but there is no guarantee that will happen. See here:
http://code.google.com/p/googleappengine/issues/detail?id=8051
http://code.google.com/p/googleappengine/issues/detail?id=8052
There is full text search coming for the Go runtime very very very soon.