As a subscriber to a pubsub topic, is it possible for me to find out what time a message was received by pubsub? That is, can a subscriber that has just received a message find out what time the corresponding publisher published the message?
This is possible with a pull subscriber. Use the publishTime field of PubsubMessage.
If you are using a client library, read the library docs on how to access this. For example, with the python client lib, it is accessible via the publish_time field on the Message class.
Related
I have watch/subscribed to the topic using the following code.
request = {
'labelIds': ['INBOX'],
'topicName': 'projects/myproject/topics/mytopic'
}
gmail.users().watch(userId='me', body=request).execute()
How can I get the status of the topic at any given point in time? The problem is, sometimes I am not getting the push from Gmail for any incoming emails.
From the Cloud Pub/Sub perspective, if you want to check on the status of messages, you could look at metrics via Stackdriver. There are many Cloud Pub/Sub metrics that are available. You can create graphs on any of the metrics that will be mentioned later by going to Stackdriver, creating a new dashboard, clicking on "Add Chart," and then typing in the name of the metric in the "Find resource type and metric box:
The first thing you have to determine is whether the issue is on the publish side (from Gmail into your topic) or on the subscribe side (from the subscription to your push endpoint). To determine if the topic is receiving messages, look at the topic/send_message_operation_count metric. This should be non-zero at points where messages were sent from Gmail to the topic. If it is always zero, then it is likely that the connection from Gmail to Cloud Pub/Sub is not set up properly, e.g., you need to grant publish rights to the topic. Note that results are delayed, so from the time you expect a message to have been sent to when it would be reflected on the graph could be up to 5 minutes.
If the messages are successfully being sent to Pub/Sub, then you'll want to see the status of attempts to receive those messages. If your subscription is a push subscription, then you'll want to look at subscription/push_request_count for the subscription. Results are grouped by response code. If the responses are in the 400 or 500 ranges, then Cloud Pub/Sub is attempting to deliver messages to your subscriber, but the subscriber is returning errors. In this case, it is likely an issue with your subscriber itself.
If you are using the Cloud Pub/Sub client libraries, then you'll want to look at properties like subscription/streaming_pull_message_operation_count to determine if your subscriber is managing to try to fetch messages for a subscription. If you are calling the pull method directly in your subscriber, then you'll want to look at subscription/pull_message_operation_count to see if there are pull requests returning successfully to your subscriber.
If the metrics for push, pull, or streaming pull indicate errors, that should help to narrow down the problem. If there are no requests at all, then it indicates that the subscribers may not There could be permission problems, e.g., the subscriber is running as a user that doesn't have permission to read from subscriptions.
According to here: https://cloud.google.com/pubsub/docs/reference/rest/v1/PubsubMessage
The timestamp field should not be populated when a publisher sends a message to the queue. So this leads me to think PubSub attaches a timestamp automatically whenever it gets a message from a publisher.
Is this correct?
Yes, you got that correctly. This is what is implied with the following sentence (from the docs you linked):
publishTime: The time at which the message was published, populated by
the server when it receives the topics.publish call.
You can test that yourself: if you go to the Explorer's API and publish to a topic using the pubsub.projects.topics.publish method, without giving a publishTime and then you pull using a subscription from that same topic (pubsub.projects.subscriptions.pull), the pulled message will have a publishTime.
Now, there is also a sentence in the docs regarding publishTime which seems a bit unclear to me:
It must not be populated by the publisher in a topics.publish call.
If you actually try to add a (correctly formatted) publishTime in your publish call, you will not get an error. Still, the actual publishTime attached to the message that you later pull is the one provided by the pub/sub service (i.e. the publish time you gave will simply be ignored).
They do generate using the publishTime. However on cases where there are multiple publishers publishing to Pub-Sub, you could attach a timestamp to every event in the publisher.
https://cloud.google.com/pubsub/docs/ordering
I know that in google cloud pub/sub, message will be lost after 7 days regardless of their acknowledgement state. Is there anyway we can send and store those messages in a file or csv or mq even after 7 days. My aim is whenever publisher publishes message this message should store in other place also.
Thanks,
santosh
There is no automatic way to store the messages that are published into Google Cloud Pub/Sub, but you could set up a subscriber that would store the messages as they are published. You would create a separate subscription on your topic that would be used for making the backups. Then, you would write a subscriber that reads messages using this subscription and immediately persists them in the desired place and format. You could use Cloud Dataflow to solve this by connecting a PubSubIO on the input side with a TextIO on the output side.
My question is related to this description of the pub/sub message flow from The Basics of a Publish/Subscribe Service:
The description apears to suggest that it's possible for a subscriber to only receive some of the messages hitting a subscription point: Subscriber 1 appears to be getting just the B message and Subscriber 2 getting just the A message, despite the fact that both A and B messages are coming from Subscription 1.
Nowhere else in the docs I encountered such concept, message receiving appears to be done based on particular subscription and a subscription appears to be done for a particular topic, but not for a particular publisher.
Am I misinterpreting the above description or is it really possible for a subscriber to select only some of the messages it receives (based on the publisher)?
The subscribers themselves do not choose which messages they get. When there are multiple subscribers for a single subscription, they can both pull from the same subscription and receive an arbitrary subset of the messages. This can be used to load balance across multiple subscribers and process more messages in parallel by increasing the number of subscribers.
I created a working Google Channel AP and now I would like to send a message to all clients.
I have two servlets. The first creates the channel and tells the clients the userid and token. The second one is called by an http post and should send the message.
To send a message to a client, I use:
channelService.sendMessage(new ChannelMessage(channelUserId, "This is a server message!"));
This sends the message just to one client. How could I send this to all?
Have I to store every Id which I use to create a channel and send the message for every id? How could I pass the Ids to the second servlet?
Using Channel API it is not possible to create one channel and then having many subscribers to it. The server creates a unique channel for individual JavaScript clients, so if you have the same Client ID the messages will be received only by one.
If you want to send the same message to multiple clients, in short, you will have to keep a track of active clients and send the same message to all of them.
If that approach sounds scary and messy, consider using PubNub for your push notification messages, where you can easily create one channel and have many subscribers. To make it run on Google App Engine is not that hard, since they support almost any platform or device.
I know this is an old question, but I just finished an open source project that uses the Channel API to implement a publish/subscribe model, i.e. you can have multiple users subscribe to a single topic, and then all those subscribers will be notified when anyone publishes a message to the topic. It also has some nice features like automatic message persistence if desired, and "return receipts", where a subscriber can be notified whenever OTHER subscribers receive that message. See https://github.com/adevine/gaewebpubsub#gae-web-pubsub. Licensed under Apache 2.0 license.