Subscribe AWS SQS to IoT topic - aws-iot

Is there a way to subscribe AWS sqs to AWS IoT topic ? I see option only for subscribing sqs to SNS topic . We are using IoT device gateway as the message broker and would like SQS subscribe to IoT topic .

Create a IoT Topic rule with an SQS action. Your rule will look something like this
SELECT * FROM 'your topic here'
Add an SQS action to the rule targeting your SQS queue
You can dynamically select the SQS queue if you can somehow determine the SQS queue url from the topic or data. Use a substitution template in the Action. if the queue name was the 3rd segment of the topic you could do
{
"rule": {
"sql": "SELECT * FROM 'some/topic/+'",
"ruleDisabled": false,
"actions": [{
"sqs": {
"queueUrl": "https://sqs.us-east-1.amazonaws.com/123456789012/${topic(3)}",
"roleArn": "arn:aws:iam::123456789012:role/aws_iot_sqs",
"useBase64": false
}
}]
}
}

Related

How can I know if Snowflake will create a new SQS Queue?

According to the Snowflake documentation here:
Following AWS guidelines, Snowflake designates no more than one SQS
queue per S3 bucket. This SQS queue may be shared among multiple
buckets in the same AWS account. The SQS queue coordinates
notifications for all pipes connecting the external stages for the S3
bucket to the target tables. When a data file is uploaded into the
bucket, all pipes that match the stage directory path perform a
one-time load of the file into their corresponding target tables.
I am configuring Snowpipe and relying on the ARN of the SQS Queue provided by snowflake (which can be queried via DESCRIBE PIPE <pipe name>). But I am confused regarding the statement:
This SQS queue may be shared among multiple
buckets in the same AWS account
Does Snowflake use a single SQS Queue for all buckets? How do I know whether to use the same SQS Queue or if Snowflake will create a new one?
You can check the SQS queues used by the pipes:
select pipe_name, notification_channel_name from information_schema.pipes;
I created 3 pipes for testing which uses 2 buckets in same account, and I see that all of them uses the same SQS queue.
https://docs.snowflake.com/en/sql-reference/info-schema/pipes.html

Snowflake - Snowpipe - SNS - SQS Subscriptions - Pending confirmation

As part of SNS configuration, I followed all the steps mentioned in the link https://docs.snowflake.com/en/user-guide/data-load-snowpipe-auto-s3.html#option-2-configuring-amazon-sns-to-automate-snowpipe-using-sqs-notifications. But still I don't see any message on SQS queue even though new file is dropped in S3 bucket. One thing I noticed that "Create subscriptions" steps are missing to add Snowflake SQS. I went ahead and added Snowflake SQS ARN in the subscription of the SNS topic but the status shows "Pending confirmation". I am not sure where we can approve this confirmation?? Any help on this greatly appreciated.

Fiware cepheus cep subscription to context broker issue

I wanted to make the cepheus cep receive incoming events from the context broker rather than the cepheus broker but then I checked the cepheus-cep logs when I uploaded the config.json file and found that it doesn't subscribe to the entities in the context broker as there's no subscription launched. Also when I tried send the subscription manually on behalf of the cepheus-cep and started to send messages to the entities in the context broker, the Cepheus-cep treated the notifications as mere post request not as eventin messages that should be processed and it displays a message stating "POST method not supported".
So, what could be the solution for both issues regarding the Cep subscription to context broker and the manual subscription I made?

Websphere MQ - Topic Subscription with multiple consumers

I have a micro-service which subscribes to a topic in WebSphere MQ. The subscription is managed and durable. I explicitly set the subscription name, so that it can be used to connect back to the queue, after recovering from any micro service failure. The subscription works as expected.
But I might have to scale up the micro service and run multiple instances. In this case I will end up with having multiple consumers to the same topic. But here it fails with error 2429 : MQRC_SUBSCRIPTION_IN_USE. I am not able to run more than one consumer to the topic subscription. Note : A message should be sent only to one of the consumers.
Any thought ?
IBM Websphere Version : 7.5
I use the C-client API to connect to the MQ.
When using a subscriber what you describe is only supported via the IBM MQ Classes for JMS API. In v7.0 and later you can use Cloned subscriptions (this is a IBM extension to the JMS spec), in addition in MQ v8.0 and later you can alternately use Shared subscriptions which is part of the JMS 2.0 spec. With these two options, multiple subscribers can be connected to the same subscription and only one of them will receive each published message.
UPDATE 20170710
According to this APAR IV96489: XMS.NET DOESN'T ALLOW SHARED SUBSCRIPTIONS EVEN WHEN CLONESUP PROPERTY IS ENABLED, XMS.NET is also supposed to support Cloned subscriptions but due to a defect this is will not be supported until 8.0.0.8 or 9.0.0.2 or if you request the IFIX for the APAR above.
You can accomplish something similar with other APIs like C by converting your micro-service to get from a queue instead of subscribing to a topic.
To get the published messages to the queue you have two options:
Setup a administrative subscription on the queue manager. You can do this a few different ways. The example below would be using a MQSC command.
DEFINE SUB('XYZ') TOPICSTR('SOME/TOPIC') DEST(SOME.QUEUE)
Create a utility app that can open a queue and create a durable subscription with that provided queue, the only purpose of this app would be to subscribe and unsubscribe a provided queue, it would not be used to consume any of the published messages.
Using the above method, each published message can only be read (GET) from the queue by one process or thread.

Camel-RabbitMQ : how to define an endpoint from a durable topic subscription?

Using Camel and its rabbitMQ module, how would I define an endpoint URL to a durable topic subscription? What options need to be set? what are corresponding options for what in camel JMS would be clientId and durableSubscriptionName?
You can use the rabbit-mq component found here:
http://camel.apache.org/rabbitmq.html
The parameter exchangeType can be set to topic.
Very simple example:
from("rabbitmq://localhost/A?routingKey=B&exchangeType=topic")
I found by experimentation that the exchange name to be used is "amq.topic", the routing key would be what would be the topic name and the queue name would be something unique based on the client id and the durable subscription name I would have used in JMS.
Souciance added correctly that the exchange type needs to be topic and autoDelete must be false. but there is no appid property on the Camel RabbitMQ consumer properties.

Resources