I'm using stackoverflow in my meteor application,
I'm calling api directly from my code like below
HTTP.call("GET", urlString,{params:{site:"stackoverflow"}},function(error,result)
{
console.log(result.data);
});
I'm not using any oauth or client id secret id in my calls.
In the response, I'm getting a variable called quota. with maximum pings 300
Is that means I can only call the api for 300 times, I want more than that, I'm even ready to pay for it.
Is there a way to increase that number.
Thanks
You are throttled because you have not registered your application. If you register your application, you will receive a quota increase to 10,000 hits per day. You will want to read the authentication documentation on how to utilize the keys you receive from registering your application.
Related
In Coinbase API doc, it describes
"By default, each API key or app is rate limited at 10,000 requests per hour if your requests are being rate limited. HTTP response code 429 will be returned with an rate_limit_exceeded error"
[Question] I'd like to know that whether the current API restrictions are for a single app or a single user.
Thanks in advance
As you already mentioned, the limit is linked to the API key.
So if you have multiple apps using the same API key(ideally you should not), the limit will apply to the cumulative calls from all the apps. If you have separate apps using different keys, the limit will apply to each app.
In addition to Anupam's answer, if you use the exchange api there are different rates and limited by IP.
REST API
When a rate limit is exceeded, a status of 429 Too Many Requests will be returned.
Public endpoints
We throttle public endpoints by IP: 10 requests per second, up to 15 requests per second in bursts. Some endpoints may have custom rate limits.
Private endpoints
We throttle private endpoints by profile ID: 15 requests per second, up to 30 requests per second in bursts. Some endpoints may have custom rate limits.
/fills endpoint has a custom rate limit of 10 requests per second, up to 20 requests per second in bursts.
Since requesting the Inventory from https://api.steampowered.com/IEconItems_730/GetPlayerItems/v1/ is permanently disabled I have to use https://steamcommunity.com/profiles/<steam_id>/inventory/json/2
Because of CORS I have to use my backend to provide the requested data. But when I do requests too often, my requests get rejected and my app cannot work on a large scale.
So the question should be simple: How can I avoid the rejection of my requests?
Any ideas and suggestions welcome.
Steam inventory endpoints are pretty heavily rate-limited, but there are a few different endpoints that you can use.
Trade offer endpoint
https://steamcommunity.com/tradeoffer/new/partnerinventory
This is the endpoint that's used when you open a trade offer with someone. It can be used to fetch both your own inventory as well as a trade partners. Required parameters are partner which is the user's Steam 64 Id, appid which is 730 in the case of CS:GO, and contextid which is 2 for most valve games. I don't know the exact limit, but I've been hitting this endpoint about once a second for a month with minimal 429 responses. To use this endpoint, you need to have a valid steam session and send the proper cookies along with the request. This will also only return tradable items.
Inventory endpoint #1
http://steamcommunity.com/inventory/STEAM64ID/APPID/CONTEXTID
Another inventory endpoint which has the same parameters but in the URL. I use this endpoint as a fallback to the first, but I've found that if the first endpoint is rate limited then this one will be too. That said, this one becomes limited much faster so it's best to use the first one instead.
Inventory endpoint #2
http://steamcommunity.com/profiles/STEAM64ID/inventory/json/APPID/CONTEXTID
The endpoint you're using. I don't use this one at all, but it could be worth knowing as another fallback.
Not all of them return the same data format, so be mindful. One inventory a second is a pretty solid rate for any decently sized site, especially if you limit user's ability to refresh inventories. If you need more though, you'll have to start looking into proxies.
I'm writing an app on top of google app engine. Using Java as language.
I'm new to GAE. I want to use URL fetch service for async calls in application.
I'm having few doubts
1. How many async calls I can call in one appengine request ?
2. What happens if I call more async calls in one request, let's say 50
3. What happens these async calls takes more than 1 minute (the appengine request timeout)
4. What is default timeout for each async call (I'm assuming it is 60 sec.Is it correct)
There is no limit on the number of calls that App Engine can process in general. There are limits (quotas) set for different APIs: e.g. number of Mail API, Task API, URLFetch, etc. calls. Often quotas are different for paying customers, and in most cases you can request to increase a quota if your app requires it. In other words, these quotas do not reflect a technological limitation - they are in place to prevent abuse.
The number of requests that your app can process depends on how many instances you have running, and how fast each call can be processed. You can enable multithreading, so each instance can process multiple calls in parallel.
As for the timeouts, they depend on the instance type and the type of calls that you are making (Datastore API, Cloud Storage API, URLFetch, etc.)
I have been using Google app engine for ~2 years now and love it. I am tasked with making a game and I wanted to use GAE as the backend. From what I understand the Channels API is meant for this kind of application. I got the tic-tac-toe demo working and noticed that each client got its own channel. After reading the documentation a little closer I noticed that this is how the system is meant to be used. I then checked the quotas and limits in the docs and... was devastated. GAE will let me at a maximum create 60 new channels a minute. That means I can only have 60 new users a minute. This is so limiting that I cannot use this API.
So my questions are:
Am I doing it right (1 client = 1 channel)
Is the channel API the best API to use to update multiple clients about the current state of a game.
It can't really be only 60!? Can I pay for more per minute?
Keep in mind that channel tokens do not expire for two hours. This
means that if one client connects multiple times within the life of
that token, you can serve that same channel token again, thus not
counting as a creation.
But otherwise, yes, you are correct; 1 channel per 1 client per 2
hours.
The channel API is the best AppEngine API to use for this,
currently, although some users are pushing for integrated WebSocket
support.
60 is the max for paid billing. A whopping 6 per minute is the max
for free apps. They say that you can request an increase for
pretty much any resource, but the page they link to (and which I've
linked to here) specifically talks about CPU usage and QPS.
I'm using urlfetch in my app and while everything works perfectly fine in the development environment, i'm finding urlfetch to be VERY unreliable when it's actually deployed. Sometimes it works as it should (retrieving data), but then a few minutes later it might return nothing, then it'll be working fine again a few minutes after that. This is very unacceptable. I've checked to make sure it's NOT the source URL that's the problem (YQL) and, again, everything works as it should in the development environment.
Are there any third-party libraries I could try?
Example code:
url = "http://query.yahooapis.com/v1/public/yql?q=%s&format=json" % urllib.quote_plus(query)
result = urlfetch.fetch(url, deadline=10)
if result.status_code == 200:
r = json.loads(result.content)
else:
return
a = r['query']['results']
# Do stuff with 'a'
Sometimes it'll work as it should, but other times - completely randomly with no code changes - i'll get this this error:
a = r['query']['results']
TypeError: 'NoneType' object is unsubscriptable
Sometimes it'll work as it should,
but other times completely randomly with no code changes
This is a common symptom that your application's requests have exceeded the Yahoo API calls rate limit.
Quoting Yahoo developer documentations rate limit:
IP Based Limits
Our service rate limits are imposed as
a limit on the number of API calls
made per IP address during a specific
time window. If your IP address
changes during that time period, you
may find yourself with more "credit"
available. However, if someone else
had been using the address and hit the
limit, you'll need to wait until the
end of the time period to be allowed
to make more API calls.
Google App Engine uses a pool of IP addresses for outgoing urlfetch requests and your application is sharing these IP addresses with other applications that are calling the same Yahoo endpoint; when the rate limit is exceeded, the endpoint replies with a limit exceeded error causing UrlFetch to fail.
Here another case using the Twitter search API.
When you mix Google App Engine+Third party web APIs, you need to be sure that the API provides authenticated calls allowing your application to have its own quota (StackApps API for example).
import urllib2
response = urllib2.urlopen('http://python.org/')
html = response.read()
This isn't an error in URLFetch - it's an issue with the JSON being returned. Either json.loads is returning None, or r['query'] is - I'm guessing it's probably the latter. Try logging result.content to see what the service is returning. You probably also want to cehck result.status.
One possibility is that your request is being denied or ratelimited by Yahoo in production, but not on your development machine.