I am running a website on Google App Engine. From time to time I get out-of-control bots or perhaps brute force hacking attempts that I see in my logs. Recently I've had a bot (I presume) trying to access administrator/index.php several times a second. That file doesn't exist on my site. If I try to access it, I get the standard 404 and this in my logs:
But for the bot I am seeing HTTP 301 in the logs and I'm wondering why. Does Google interpret the requests as a denial of service or other attack and automatically intervene? I haven't seen documentation stating as much, but I'm not sure why else I would be seeing the 301 instead of 404 for the same URL:
Does anyone have an explanation for this?
The log entires shown on the screenshots can be clicked & expanded to view additional information. As mentioned in the comment above two things could be checked there for further analysis of what's going on:
check the hostname of where the request came to & see if it's not the expected behaviour for that hostname.
if the json object is shown, navigate to protoPayload -> line -> [0] -> logMessage where something like redirecting "http://example.com/" to "https://www.example.com/" should be shown which could also clear things up a bit.
Related
I've been studying logs in Logs Explorer in Google Cloud Platform and noticed that available logs don't show the body of a POST request, which is crucial for my task (I run Google Tag Manager Server-Side and need to study logs of all inbound Measurement Protocol requests). Is there a way to enable the body contents for requests like this one?
There's no built in feature in GCP Logging to do this just as norbjd said.
Only thing that's being logged by default are GET requests and only the first line:
First line of the request, containing method, path, and HTTP version. Example: GET / HTTP/1.1
Have a look at the answer here and then the fallowing comments fallowing:
A logging.info() shows up (link)
I have, and it does show up (with an i left to it) in the Stackdriver request logs and that is the only way to see POST arguments (link)
You may also try a solution from the next answer:
post request data isn't shown in request logs.
In your app, you must explicitly log it
logging.log(myRequest.PostData)
Edit: just be careful logging sensitive customer information.
I've deployed an app using googles cloud compute engine service.
I get an error when I try to register a new user:
We're sorry, but something went wrong. If you are the application owner check the logs for more information.
I'm in the logs section of the google cloud console and it has all sorts of random logs I can look at but I cant find where the errors that the app coding itself caused caused are put, e.g. I'm sure its a mail configuration problem that is causing the error message for new registered users but I cant fix the problem until I find out what the actual error is.
Where exactly are actual app errors put, because they aren't displayed on the page the like in production on my local computer. (I'm using a rails app if that makes a difference to anything)
All errors are logged in the logging section, try applying filters or sorting the logs by date. Sometimes they take a little while to appear.
Well, I'm having a weird error here:
I'm developing one GAE app to read some Twitter Data, and after read a lot of docs, I have it working on my test server (Running on my pc) but after deploy and test on the real (my appspot domain) it shows this message:
401:Authentication credentials (https://dev.twitter.com/pages/auth) were missing or >incorrect. Ensure that you have set valid consumer key/secret, access token/secret, and the >system clock is in sync.
message - Could not authenticate you
code - 32
I've tried to recreate my OAuthAppToken and OAuthAppTokenSecret keys, even changing the permissions to "Write, Read and Direct Messages" and even assingning one Callback URL but nothing seems to work...
I've tried using twitter4j.properties OR using setOAuthConsumer(TW_CONSUMER_KEY, TW_CONSUMER_SECRET) OR a ConfigurationBuilder whith the correct constants and I'm experimenting the same Issue.
I'm working with AppEngine 1.8.3 and Twitter4j 3.0.4
Iv'e been writing on log and the Twitter object seems to be well created... I dont understand why is working on my PC but not on the real app.
On some other post someone says that could be because it needs to use Sync clock.. but he doesn't explains where to change that property...
Did someone had a clue?
Ok, the problem was me (and Twitter.... well..... I really think it was Twitter problem for being so dark on his api messages)...
On testing server I was looking for an existing account and on the cloud I was looking for an inexistent one. So, It was my mistake. But seriously, what about Twitter saying: "Access Forbidden"? That doesn't have any sense...
I have a fairly big application which went over a major overhaul.
The newer version uses lot of JSONP calls and I notice 500 server errors. Nothing is logged in the logs section to determine the error cause. It happens on JS, png and even jersey (servlets) too.
Searching SO and groups suggested that these errors are common during deployment. But it happens even after hours after deployment.
BTW, the application has become slightly bigger and it even causes deadline exception while starting few instances in few rare cases. Sometimes, it starts & serves within 6-10secs. Sometimes it goes to more than 75secs thereby causing a timeout for the similar request. I see the same behavior for warmup requests too. Nothing custom is loaded during app warmup.
I feel like you should be seeing the errors in your logs. Are you exceeding quotas or having deadline errors? Perhaps you have an error in your error handler like your file cannot be found, or the path to the error handler overlaps with another static file route?
To troubleshoot, I would implement custom error pages so you could determine the actual error code. I'm assuming Python since you never specified what language you are using. Add the following to your app.yaml and create static html pages that will give the recipient some idea of what's going on and then report back with your findings:
error_handlers:
- file: default_error.html
- error_code: over_quota
file: over_quota.html
- error_code: dos_api_denial
file: dos_api_denial.html
- error_code: timeout
file: timeout.html
If you already have custom error handlers, can you provide some of your app.yaml so we can help you?
Some 500s are not logged in your application logs. They are failures at the front-end of GAE. If, for some reason, you have a spike in requests and new instances of your application cannot be started fast enough to serve those requests, your client may see 500s even though those 500s do not appear in your application's logs. GAE team is working to provide visibility into those front-end logs.
I just saw this myself... I was researching some logs of visitors who only loaded half of the graphics files on a page. I tried clicking on the same link on a blog that they did to get to our site. In my case, I saw a 500 error in the chrome browser developer console for a js file. Yet when I looked at the GAE logs it said it served the file correctly with a 200 status. That js file loads other images which were not. In my case, it was an https request.
It is really important for us to know our customer experience (obviously). I wanted to let you know that this problem is still occurring. Just having it show up in the logs would be great, even attach a warm-up error to it or something so we know it is an unavoidable artefact of a complex server system (totally understandable). I just need to know if I should be adding instances or something else. This error did not wait for 60 seconds, maybe 5 to 10 seconds. It is like the round trip for SSL handshaking failed in the middle but the logs showed it as success.
So can I increase any timeout for the handshake or is that done on the browser side?
I'm getting a strange error in my newly deployed application in appengine. In the error log it tells me that PageRank, TwitterBot and a couple of others. I would guess this is due to these try to get data using ajax or another async service resulting in "same origin policy"-problem.
My question is does anyone know what these bots are trying to get? For example if pagerank (google page rank I would guess) can't get any info about my application would this effect my page rank. And anyone know what the twitterbot does? And if there is away to handle to provide a proper response?
Most likely, your RequestHandlers (I'm assuming you're using python with webapp from the tags on your earlier questions) aren't implementing a method for whichever request method the bots are using. I'd guess they're HEAD requests, and you have no head() methods defined.