I am looking for a simple example of how to use aiohttp.server error logging in request handlers. For e.g: how can I log an error when a request is made to non-existent route (e.g: http://127.0.0.1/none).I know I can see 404 errors in access logs but want to log errors within handlers
Can I use aiohttp format specification macros in the error logs?
Related
I am using fetch in a NodeJS application. Technically, I have a ReactJS front-end calling the NodeJS backend (as a proxy), and then the proxy calls out to backend services on a different domain.
However, from logging errors from consumers (I haven't been able to reproduce this issue myself) I see that a lot of these proxy calls (using fetch) throw an error that just says Network Request Failed, which is of no help. Some context:
This only occurs on a subset of all total calls (lets say 5% of traffic)
Users that encounter this error can often make the same call again some time later (next couple minutes/hours/days) and it will go through
From Application Insights, I can see no correlation between browsers, locations, etc
Calls often return fast, like < 100 ms
All calls are HTTPS, non are HTTP
We have a fetch polyfill from fetch-ponyfill that will take over if fetch is not available (Internet Explorer). I did test this package itself and the calls went through fine. I also mentioned that this error does occur on browsers that do support fetch, so I don't think this is the error.
Fetch settings for all requests
Method is set per request, but I've seen it fail on different types (GET, POST, etc)
Mode is set to 'same-origin'. I thought this was odd, since we were sending a request from one domain to another, but I tried to set it differently and it didn't affect anything. Also, why would some requests work for some, but not for others?
Body is set per request, based on the data being sent.
Headers is usually just Accept and Content-Type, both set to JSON.
I have tried researching this topic before, but most posts I found referenced React native applications running on iOS, where you have to set some security permissions in the plist file to allow HTTP requests or something to do with transport security.
I have implement logging specific points for the data in Application Insights, and I can see that fetch() was called, but then() was never reached; it went straight to the .catch(). So it's not even reaching code that parses the request, because apparently no request came back (we then parse the JSON response and call other functions, but like I said, it doesn't even reach this point).
Which is also odd, since the request never comes back, but it fails (often) within 100 ms.
My suspicions:
Some consumers have some sort of add-on for there browser that is messing with the request. Although, I run with uBlock Origin and HTTPS Everywhere and I have not seen this error. I'm not sure what else could be modifying requests that would cause it to immediately fail.
The call goes through, which then reaches an Azure Application Gateway, which might fail for some reason (too many connected clients, not enough ports, etc) and returns a response that immediately fails the fetch call without running the .then() on the response.
For #2, I remember I had traced a network call that failed and returned Network Request Failed: Made it through the proxy -> made it through the Application Gateway -> hit the backend services -> backend services sent a response. I am currently requesting access to backend service logs in order to verify this on some more recent calls (last time I did this, I did it through a screenshare with a backend developer), and hopefully clear up the path back to the client (the ReactJS application). I do remember though that it made it to the backend services successfully.
So I'm honestly not sure what's going on here. Does anyone have any insight?
Based on your excellent description and detective work, it's clear that the problem is between your Node app and the other domain. The other domain is throwing an error and your proxy has no choice but to say that there's an error on the server. That's why it's always throwing a 500-series error, the Network Request Failed error that you're seeing.
It's an intermittent problem, so the error is inconsistent. It's a waste of your time to continue to look at the browser because the problem will have been created beyond that, either in your proxy translating that request or on the remote server. You have to find that error.
Here's what I'd do...
Implement brute-force logging in your Node app. You can use Bunyan, or Winston or just require(fs) and write out to some file when an error occurs. Then look at the results. Only log it out when the response code from the other server is in the 400 or 500 ranges. Log the request object and the response object.
Something like this with Bunyan:
fetch(urlToRemoteServer)
.then(res => res.json())
.then(res => whateverElseYoureDoing(res))
.catch(err => {
// Get the request & response to the remote server
log.info({request: req, response: res, err: err});
});
where the res in this case is the response we just got from the other domain and req is our request to them.
The logs on your Azure server will then have the entire request and response. From this you can find commonalities. and (🤞) the cause of the problem.
I am using React to create an application and using a PHP server for the API. I realized when there is a PHP error i am unable to see what the error is. The only thing i get back is
'http://localhost:3000' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: It does not have HTTP ok status.
Is there anyway where I can see what is the error message from the php side?
Actually, the current error message is pretty comprehensive. You need to configure CORS correctly by adding appropriate headers.
Cross-Origin Request Headers(CORS) with PHP headers
You are getting the CORS issue because your PHP server is running on a different port compared to your react server (react dev server, running on port 3000).
There are two action items to be done in this case:
You may whitelist localhost:3000 on your PHP server (you can check this answer https://stackoverflow.com/a/33090515/13142033).
Use a logger service in your PHP code. For example monolog (https://github.com/Seldaek/monolog). So, you may use a try-catch or generic exception handlers in your PHP code, and whenever there is an error use the logger to write to a file on the disk.
Actually, when you fix the CORS issue, you will see PHP error logs in your react API responses, but having a logger service is a good practice.
I'm trying to implement a simple interceptor that allows me to display a message along the lines of "cannot contact the server" in my Angular app. However as the API is on a different host I'm dealing with CORS pre-flight OPTIONS requests.
I've found that if the API is unavailable Chrome dev tools shows a 503 on the OPTIONS request but Angular's $http interceptor catches a 404 response to the subsequent GET request. I believe this is because the OPTIONS response did not contain the required CORS headers so the GET is actually never performed.
Is is possible to intercept the OPTIONS response? If all I see is a 404 I can't distinguish "server down" from "no such resource".
You can't intercept this request by design - the browser is "checking up" on you, making sure YOU should be allowed to make the request.
We've used three solutions to work around this:
If the problem is that you're using a development environment like NodeJS, and your domain names aren't matching (that is, if you normally wouldn't need to deal with this in Production) you can use a proxy. The https://github.com/substack/bouncyBounceJS NodeJS Module is an easy to use option. Then your Web service request domain will match the domain your page is on, and the check won't be triggered. (You can also use tricks like this in Production, although it can be easily abused!)
Also for temporary use, you can use something like Fiddler or Charles to manipulate the request by faking the required headers, or tell your browser not to check them (--disable-web-security in Chrome).
If you have this problem in Production, you either need to legitimately fix it (adjust the Web service handler to add the required headers - there are only two), or find a way to make the request in a way that doesn't trigger the check. For instance, if you control both the source and target domains, you can put a script on the target that makes the requests to itself. Run this in an IFRAME, invisibly. Then you can use things like postMessage() to communicate back and forth. Large services like Facebook use "XHR bridges" like this for the same reason.
can someone point me in the right direction re: redirect uri
right now i have app hosted on appspot (nothing done or uploaded to it at this point)
"Error: Server Error
The server encountered an error and could not complete your request.
If the problem persists, please report your problem and mention this error message and the query that caused it."
do i need to upload anything for this to work?
https://developers.google.com/accounts/docs/OAuth2InstalledApp#choosingredirecturi
POST /o/oauth2/token HTTP/1.1
Host: accounts.google.com
Content-Type: application/x-www-form-urlencoded
code=4/v6xr77ewYqhvHSyW6UJ1w7jKwAzu&
client_id=8819981768.apps.googleusercontent.com&
client_secret={client_secret}&
redirect_uri=https://oauth2-login-demo.appspot.com/code&
grant_type=authorization_code
i just want to authenticate so that i can use the calendar api
Do upload the application. I am not sure what you mean by saying that the app is hosted on appspot but nothing done or uploaded at this point. So, first make sure that you have uploaded a version of your application.
The OAuth process is redirecting the flow if I understand correct to oauth2-login-demo.appspot.com and you must have registered this callback url when setting up your application.
So it seems that the redirection is probably taking place but you are seeing the General Exception that is normally thrown by App Engine.
You should look into the Admin Console -> Logs for your application to understand the reason for the failure. You might get some information there about the cause.
If there is a problem with your code, it will point that in the logs. Alternately, put in some exception handlers and bump up the Log Level to INFO in your logging.properties to have a better chance of tracking down the root case.
I'm trying to setup an dataimport-scheduler for solr, everything's working and the deltaimport url is called every 30 minutes, the only problem is I'm using jetty and activated authentication in jetty.xml so the dataimport_scheduler gets:
<index update process> Response message Unauthorized
(saw in log file), How can I solve this?
The DataImportScheduler needs to have access to your solr/dataimport url via http. The error you see in the log file is because of the authentication you added. As far as I know the DataImportScheduler doesn't support authentication out of the box, but it should be easy to add it to the code.
Unfortunately it doesn't use http-client, which would have made things a bit easier and flexible I guess, but you can have a look at this answer to find out how to add http basic authentication to http calls made through the HttpURLConnection class.