I'm trying to structure a piece of code and I'm having trouble with the logic. I want to do a http request which will give an output. For certain outputs, I want to loop back and repeat the request, but I don't know how many times before I've done each request. The following might give an indication of the structure:
stored_data = []
request = response(url)
data = request["Some_info_from_json"]
if data == "Some response that needs a repeat":
request = response(url+"some appended string")
data = request["Some_info_from_json"
elif data == "Some request that doesn't need a repeat":
stored_data.append(data)
if data == "Some response that needs a repeat":
request = response(url+"some other appended string") #This could in theory have any number of nestings
data = request["Some_info_from_json"]
elif data == "Some request that doesn't need a repeat":
stored_data.append(data)
How can I do this as a loop such that for one response it loops back to the request and for the other it carries on and stores the data?
I have attempted to do the initial request and loop back but I can't work out a way of getting the iterable correct.
Related
I'm using Apollo React Native client working with a query for which my request body has become too large to use (it's being rejected by our CDN for a request-too-large rule). So, I'm hoping to split/chunk this request into smaller requests and particularly curious if it's possible to do parallelized.
I think this is better illustrated with an example, so we can imagine I'm building a WhatsApp challenger -- WhoseApp -- for which we want users to be able to see who of their contacts have a WhoseApp account upon signup.
For our implementation, we'll take all of the phone numbers stored on our user's device and send them to our GraphQL query GetPhoneNumberAccountStatus which accepts an array of phone numbers and which returns an Account for each number associated to an account (and nothing for those that are not).
If we send the contacts as one request, we'll have a request body that looks something like this:
[
"+15558675309",
"+15558675308",
"+15558675307"
"+15558675306"
...
// 500+ numbers for some users
]
What's the correct way to split this request into multiple?
I'm curious of both:
What's the 'optimal' way to approach this using a sequential approach (e.g., send one group, wait for response, send next group), or
Is there a way to do this parallelized (e.g., send all groups at beginning and then receive responses as they arrive)?
I initially figured it might be possible to use useLazyQuery and send tranches of ~50 numbers at a time, firing each group and then awaiting the responses but this GitHub thread for the library makes it clear that that's not the correct approach.
I think it's readable
const promises = [];
const chunkSize = 50;
for (let i = 0; i <= contacts.length; i += chunkSize) {
const promise = apollo.query({...dataHere});
promises.push(promise);
}
await Promise.all(promises);
#slash.slash(name='spam', description='I will spam your content for times!', options=optionsspam, guild_ids=[847769978308526090])
async def spam(ctx, text: str, times: int="15"):
if bool(times):
Times = 15
else:
Times = times
for i in range(int(Times)):
await ctx.send(text)
await asyncio.sleep(.7)
And the result is:
It keeps replying to the first message that the bot sent. I don’t want the bot to reply. I want it to just send a normal message. How?
An interaction (slash-command) will always require a direct response towards the user. If you do not use ctx.send(str), the interaction will fail.
You've got 2 options to make it seem, like you are not responding to the slash command
Hide the response
You can post a hidden answer ctx.send('ok', hidden=True) and then send the intented message into the channel ctx.channel.send(str).
This will make the initial 'ok' only visible for the invoking users and all other members of the server will neither see the request, nor the first response.
Delete the response
Your second option is to automatically delete the answer after a very short period (ctx.send('ok', delete_after=1)), followed by a normal message into the channel ctx.channel.send(str).
Defering the response
You might need to defer your response if you can't respond within 3 seconds of the invocation. Defering an interaction (ctx.defer(hidden=True) or ctx.defer()) must be called with the same hidden attribute as your future ctx.send().
If you want to hide your respons ctx.send('ok', hidden=True), you need to defer in the same state ctx.defer(hidden=True).
You could get the channel and send message to the channel directly. However, you then must use something like ctx.defer() so that the interaction doesn't get displayed as failed.
#slash.slash(name='spam', description='I will spam your content for times!', options=optionsspam, guild_ids=[847769978308526090])
async def spam(ctx, text: str, times: int="15"):
channel = ctx.channel
if bool(times):
Times = 15
else:
Times = times
for i in range(int(Times)):
if channel != None:
await channel.send(text)
await asyncio.sleep(.7)
await ctx.send("Done")
I am trying to get scrape information.
I goto the introduction page to determine the number of search results. Often, the results occur over >1 page, and as such, I need to refresh and run another requests in a loop. On a few occasions an error occurs in the extra requests, or it hangs.
I am curious if there is a way to check a request, if it fails, then try again, and if it still fails, log it and go to the next one
Here is a sample script:
t=.3
urls=['https://stackoverflow.com/','https://www.google.com/'] #list of upto 200 urls
for url in urls:
print(url)
response = requests.get(url, timeout=t)
t=t+10
i=i-1
running this returns a timeout occasionally, and the processing stops. My workaround is to print the url that failed, and then rerun, updating the list manually.
I would like to find a way that if a request error occurs, the response retries 5x, and if it fails, logs and stores the failed url, then goes onto the next one, so that I can try the failed urls at a later stage
Any suggestions?
I haven't used Python in a while, but I'm pretty sure that this will work.
t=.3
urls=['https://stackoverflow.com/','https://www.google.com/'] #list of upto 200 urls
for url in urls:
print(url)
try:
response = requests.get(url, timeout=t)
except:
# if the request fails: try again
try:
response = requests.get(url, timeout=t)
except:
# if the request fails again: do nothing (continue to the next url)
pass
t=t+10
i=i-1
I used to interact with the Gmail API since past year using these tests https://developers.google.com/gmail/api/v1/reference/users/messages/list#try-it but now this examples are failing because seems there are more messages but the next iteration is coming empty.
Problem is in this part of the code:
while 'nextPageToken' in response:
page_token = response['nextPageToken']
response = service.users().messages().list(userId=user_id, q=query,
pageToken=page_token).execute()
messages.extend(response['messages'])
The error is raised when trying to access the response['messages'] as the unique key in the reponse is 'resultSizeEstimate' and is 0. Sounds like the page_token is pointing to a next empty page.
Is someone experiencing this issue as well?
If your last page perfectly contains the last email with that particular query, you will get a nextPageToken to a page with a response like this:
{
"resultSizeEstimate": 0
}
The easiest way around this is to just add a check if messages is part of the response:
while 'nextPageToken' in response:
page_token = response['nextPageToken']
response = service.users().messages().list(userId=user_id, q=query, pageToken=page_token).execute()
if 'messages' in response:
messages.extend(response['messages'])
I have written an apache 2.x module that attempts to scan request bodies, and conditionally return 403 Forbidden if certain patterns match.
My first attempt used ap_hook_handler to intercept the request, scan it and then returned DECLINED to the real handler could take over (or 403 if conditions were met).
Problem with that approach is when I read the POST body of the request (using ap_get_client_block and friends), it apparently consumed body so that if the request was subsequently handled by mod_proxy, the body was gone.
I think the right way to scan the body would be to use an input filter, except an input filter can only return APR_SUCCESS or fail. Any return codes other than APR_SUCCESS get translated into HTTP 400 Bad Request.
I think maybe I can store a flag in the request notes if the input filter wants to fail the request, but I'm not sure which later hook to get that.
turned out to be pretty easy - just drop an error bucket into the brigade:
apr_bucket_brigade *brigade = apr_brigade_create(f->r->pool, f->r->connection->bucket_alloc);
apr_bucket *bucket = ap_bucket_error_create(403, NULL, f->r->pool,
f->r->connection->bucket_alloc);
APR_BRIGADE_INSERT_TAIL(brigade, bucket);
bucket = apr_bucket_eos_create(f->r->connection->bucket_alloc);
APR_BRIGADE_INSERT_TAIL(brigade, bucket);
ap_pass_brigade(f->next, brigade);