I want to know how retrieve metrics pubsub from an API. (I saw this topic Google PubSub - Counting messages in topic, but it's not the same problem).
When I attempt to retrieve them from Logging API, but it returns a 404 error.
It's not possible or it just lacks rights ?
Client error: `GET https://logging.googleapis.com/v2/projects/my-project/metrics/pubsub.googleapis.com/topic/send_message_operation_count` resulted in a `404 Not Found`
Thanks for your time !
Stackdriver Monitoring and Stackdriver Logging are different. The metrics from Pub/Sub are collected as part of Stackdriver Monitoring. The logging.googleapis.com API is part of Stackdriver Logging, which is used for searching, analyzing, and monitoring your logs.
To access the send_message_operation_count metric, you'll want to use the timeSeries.list method. You would set filter to metric.type = "pubsub.googleapis.com/topic/send_message_operation_count", set interval.startTime to the start of the interval for which you want data, e.g., 2017-02-24T21:01:23.00Z, and set interval.endTime to the end of the interval for which you want data, e.g., 2017-02-24T22:01:23.00Z. There are other properties you can set to alter what data is returned and how. If you want to return the data for a specific topic, you would add AND resource.label.topic_id = "<topic name>" to the filter.
A complete request URL (without the necessary authorization info) would look something like:
https://monitoring.googleapis.com/v3/projects/my-project/timeSeries?interval.endTime=2017-02-24T22%3A01%3A23.00Z&filter=metric.type%20%3D%20%22pubsub.googleapis.com%2Ftopic%2Fsend_message_operation_count%22%20AND%20resource.label.topic_id%20%3D%20%22my-topic%22&interval.startTime=2017-02-23T21%3A01%3A23.00Z
Related
I want to get notified (through email of slack channel) for every Exception occurence in my GAP services.
When I'm trying to create a notification through GCP Error Reporting, it seems like I'm being able to get one notification per incident (and not per occurrence), and that it is also being queried only once a day/hour/month.
I've also tried to create a policy in GCP Logs Monitoring based on log severity, but of course I don't get notified only for exceptions, and the slack notifications just announce that a threshold is being passed, without the actual data I want to get by push.
Any way I can make Error Reporting notify me per each occurance?
Is there any other internal tool by GCP that notify when event occurs? or should I use an external tool such as Epsagon/Operations (formally Stackdriver)?
So it seems like there is no availability for getting a pushed event on each exception occurrence, and since I didn't want to query it by pull (with ereport) or to get only new exceptions (through Stackdriver), I had to go with an external service such as Datadog.
I've been studying logs in Logs Explorer in Google Cloud Platform and noticed that available logs don't show the body of a POST request, which is crucial for my task (I run Google Tag Manager Server-Side and need to study logs of all inbound Measurement Protocol requests). Is there a way to enable the body contents for requests like this one?
There's no built in feature in GCP Logging to do this just as norbjd said.
Only thing that's being logged by default are GET requests and only the first line:
First line of the request, containing method, path, and HTTP version. Example: GET / HTTP/1.1
Have a look at the answer here and then the fallowing comments fallowing:
A logging.info() shows up (link)
I have, and it does show up (with an i left to it) in the Stackdriver request logs and that is the only way to see POST arguments (link)
You may also try a solution from the next answer:
post request data isn't shown in request logs.
In your app, you must explicitly log it
logging.log(myRequest.PostData)
Edit: just be careful logging sensitive customer information.
I have watch/subscribed to the topic using the following code.
request = {
'labelIds': ['INBOX'],
'topicName': 'projects/myproject/topics/mytopic'
}
gmail.users().watch(userId='me', body=request).execute()
How can I get the status of the topic at any given point in time? The problem is, sometimes I am not getting the push from Gmail for any incoming emails.
From the Cloud Pub/Sub perspective, if you want to check on the status of messages, you could look at metrics via Stackdriver. There are many Cloud Pub/Sub metrics that are available. You can create graphs on any of the metrics that will be mentioned later by going to Stackdriver, creating a new dashboard, clicking on "Add Chart," and then typing in the name of the metric in the "Find resource type and metric box:
The first thing you have to determine is whether the issue is on the publish side (from Gmail into your topic) or on the subscribe side (from the subscription to your push endpoint). To determine if the topic is receiving messages, look at the topic/send_message_operation_count metric. This should be non-zero at points where messages were sent from Gmail to the topic. If it is always zero, then it is likely that the connection from Gmail to Cloud Pub/Sub is not set up properly, e.g., you need to grant publish rights to the topic. Note that results are delayed, so from the time you expect a message to have been sent to when it would be reflected on the graph could be up to 5 minutes.
If the messages are successfully being sent to Pub/Sub, then you'll want to see the status of attempts to receive those messages. If your subscription is a push subscription, then you'll want to look at subscription/push_request_count for the subscription. Results are grouped by response code. If the responses are in the 400 or 500 ranges, then Cloud Pub/Sub is attempting to deliver messages to your subscriber, but the subscriber is returning errors. In this case, it is likely an issue with your subscriber itself.
If you are using the Cloud Pub/Sub client libraries, then you'll want to look at properties like subscription/streaming_pull_message_operation_count to determine if your subscriber is managing to try to fetch messages for a subscription. If you are calling the pull method directly in your subscriber, then you'll want to look at subscription/pull_message_operation_count to see if there are pull requests returning successfully to your subscriber.
If the metrics for push, pull, or streaming pull indicate errors, that should help to narrow down the problem. If there are no requests at all, then it indicates that the subscribers may not There could be permission problems, e.g., the subscriber is running as a user that doesn't have permission to read from subscriptions.
It was working fine till last day and suddenly stopped pushing to endpoint. Checked all settings including endpoint URL and found everything remains unchanged. Can you guys suggest possible causes.
Not receiving a message on a push endpoint could happen for many reasons. The first thing to do would be to go to Stackdriver and create a graph for the subscription/push_request_count metric. You can break this down by response_code to see how many requests Cloud Pub/Sub is sending to your push endpoint and what response codes it is returning. If there are requests being delivered that are returning errors, this graph will show that.
It might also be worth checking the publish side to ensure messages are still being published as expected. You can look at the topic/send_message_operation_count metric, which can also be broken down by response_code, to make sure the publish requests are all returning success.
You should also check to ensure the subscription still exists using the Pub/Sub Subscriptions page in the Cloud console. After 30 days of inactivity (including inability to successfully deliver a message to a push endpoint), subscriptions are potentially deleted.
If the issue still unsolved after those steps, it is best to contact Google Cloud support with your project ID and subscription name so that things can be investigated for your specific case.
I want to check latencies of RPC every day about CakePHP Application each endpoints running in GKE cluster. I found it is possible using php google client or zipkin server by reading documents , but I don't know how easy to introduce to our app though both seem tough for me.
In addition, I'm concerned about GKE cluster configuration has StackDriver Trace option though our cluster it sets disabled.Can we trace span if it sets enable?
Could you give some advices?
I succeeded to send gcp's trace api in php client via REST. It can see trace set by php client parameters , but my endpoint for trace api has stopped though I don't know why.Maybe ,it is not still supported well because the document have many ambiguous expression so, I realized watching server response by BigQuery with fluentd and DataStudio and it seem best solution because auto span can be set by table name with yyyymmdd and we can watch arbitrary metrics with custom query or calculation field.