Unble to open multiple simultaneous websockets to the same endpoint in case where the consumer is busy for the first websocket - reactjs

I am using Django channels with React websockets. I am unable to open multiple simulataneous websockets if the consumer of the first websocket channel is busy with some activity. I want to open multiple simultaneous websockets where consumers is doing something for each individual websocket.
The module versions are:
asgiref 3.6.0
daphne 3.0.2
django-cors-headers 3.13.0
djangorestframework 3.14.0
djangorestframework-simplejwt 5.2.2
From the snippet below, once one websocket completes the task the second websocket can be connected (in a separate tab) but while 1st websocket is busy (sleeping), the other websockets in the new browser tabs fails after handshake with the following error:
**
WebSocket HANDSHAKING /ws/socket-server/monitor-rack-kpis/ushoeuhrti/ [127.0.0.1:49228]
django.channels.server INFO WebSocket HANDSHAKING /ws/socket-server/monitor-rack-kpis/ushoeuhrti/ [127.0.0.1:49228]
daphne.http_protocol DEBUG Upgraded connection ['127.0.0.1', 49228] to WebSocket
daphne.ws_protocol DEBUG WebSocket closed for ['127.0.0.1', 49228]
WebSocket DISCONNECT /ws/socket-server/monitor-rack-kpis/ushoeuhrti/ [127.0.0.1:49228]
django.channels.server INFO WebSocket DISCONNECT /ws/socket-server/monitor-rack-kpis/ushoeuhrti/ [127.0.0.1:49228]
daphne.server WARNING Application instance <Task pending name='Task-56' coro=<StaticFilesWrapper.__call__() running at /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/channels/staticfiles.py:44> wait_for=<Future pending cb=[_chain_future.<locals>._call_check_cancel() at /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/futures.py:387, Task.task_wakeup()]>> for connection <WebSocketProtocol client=['127.0.0.1', 49216] path=b'/ws/socket-server/monitor-rack-kpis/vhofoefwmr/'> took too long to shut down and was killed.
**
I tried using url in routing instead of re_path but that didn't help either
settings.py
`
ASGI_APPLICATION = 'backend.asgi.application'
CHANNEL_LAYERS={'default':{'BACKEND':'channels.layers.InMemoryChannelLayer'}}
`
asgi.py
application = ProtocolTypeRouter({
'http':get_asgi_application(),
'websocket':AuthMiddlewareStack(
URLRouter(logstatus.routing.websocket_urlpatterns)
)
})
routing.py
websocket_urlpatterns=[
re_path(r'^ws/socket-server/monitor-rack-kpis/(?P<username>[A-Za-z]+)/', consumers.InfluxWritePromesthusConsumer.as_asgi())
]
consumer.py
class InfluxWritePromesthusConsumer(WebsocketConsumer):
def ws_connect(message):
message.reply_channel.send({"accept": True})
def receive(self, text_data):
print(f"\nReceiving from: {self.channel_name}\n")
t = 0
while t<=100:
self.send(text_data=json.dumps({
'type':"LearnSocket",
'message': "Received Messages"
}))
t += 10
time.sleep(10)
Frontend- React.JS
const randN = generate()
console.log(randN)
const socket = new WebSocket('ws://127.0.0.1:8000/ws/socket-server/monitor-rack-kpis/'+ randN+ '/');
console.log(socket)
socket.onopen=function(e){
console.log(socket.readyState)
if (user && user.access) {
socket.send("Hi");
//Receiving response from WebSocket
socket.onmessage=function(e){
//let data1=JSON.parse(e.data);
console.log(e.data, new Date());
//setOutputResponse(data1['message']);
}
}
}

Related

How to get retry count from ActiveMQ Artemis message

I'm taking the message from an ActiveMQ Artemis queue and trying to send it to an invalid HTTP endpoint (through a Camel route) it fails and the message gets retried. Whenever message delivery fails it gets retried at ActiveMQ Artemis level. Retry configuration is available in broker. Is there any way to get current retry count for a JMS message in client applications?
I can maintain retry count in DB per message - looking for better solution / existing methods etc.
If I try to maintain the count and set it in the message header then during retry those get lost. Retry happens as if it's a new message. So can't use it.
We are using org.apache.camel.component.jms.JmsComponent with the Qpid JMS client.
You should inspect the JMSXDeliveryCount property on the message. Section 3.5.9 of the JMS 1.1 specification says this about JMSXDeliveryCount:
The number of message delivery attempts; the first is 1, the second 2,...
I just tested this with Qpid JMS 1.7.0 and ActiveMQ Artemis 2.27.1 and it worked fine in both transacted and non-transacted use-cases. In both cases the JMSRedelivered header and the JMSXDeliveryCount property were set appropriately.
Here's the code for the transacted use-case:
try (Connection connection = new JmsConnectionFactory("amqp://127.0.0.1:5672").createConnection()) {
Session session = connection.createSession(true, Session.CLIENT_ACKNOWLEDGE);
Queue queue = session.createQueue("myQueue");
MessageConsumer consumer = session.createConsumer(queue);
MessageProducer producer = session.createProducer(queue);
TextMessage message = session.createTextMessage("test-message");
producer.send(message);
session.commit();
producer.close();
connection.start();
message = (TextMessage) consumer.receive(1000);
System.out.println("Redelivered? " + message.getJMSRedelivered());
System.out.println("Delivery count: " + message.getIntProperty("JMSXDeliveryCount"));
message.acknowledge();
session.rollback();
message = (TextMessage) consumer.receive(1000);
System.out.println("Redelivered? " + message.getJMSRedelivered());
System.out.println("Delivery count: " + message.getIntProperty("JMSXDeliveryCount"));
}
Here's the code for the non-transacted use-case:
try (Connection connection = new JmsConnectionFactory("amqp://127.0.0.1:5672").createConnection()) {
Session session = connection.createSession(false, Session.CLIENT_ACKNOWLEDGE);
Queue queue = session.createQueue("myQueue");
MessageConsumer consumer = session.createConsumer(queue);
MessageProducer producer = session.createProducer(queue);
TextMessage message = session.createTextMessage("test-message");
producer.send(message);
producer.close();
connection.start();
message = (TextMessage) consumer.receive(1000);
System.out.println("Redelivered? " + message.getJMSRedelivered());
System.out.println("Delivery count: " + message.getIntProperty("JMSXDeliveryCount"));
session.close();
session = connection.createSession(false, Session.CLIENT_ACKNOWLEDGE);
consumer = session.createConsumer(queue);
message = (TextMessage) consumer.receive(1000);
System.out.println("Redelivered? " + message.getJMSRedelivered());
System.out.println("Delivery count: " + message.getIntProperty("JMSXDeliveryCount"));
}
In both cases this is what was printed:
Redelivered? false
Delivery count: 1
Redelivered? true
Delivery count: 2
The behavior you're seeing with the message being redelivered as "a new message" is expected. As noted in section 3.10 of the JMS 1.1 specification:
A consumer can modify a received message after calling either the clearBody or clearProperties method to make the body or properties writable. If the consumer modifies a received message, and the message is subsequently redelivered, the redelivered message must be the original, unmodified message (except for headers and properties modified by the JMS provider as a result of the redelivery, such as the JMSRedelivered header and the JMSXDeliveryCount property).
I'm not a Camel expert so I can't say exactly is Camel doing with the JMS message when delivery to the HTTP endpoint fails. All I know is that this works fine with straight JMS. I would expect this to work with Camel also, but Camel may be doing their own form of retry rather than using the redelivery which JMS provides.
This would be relatively trivial to validate by turning on AMQP frame tracing from the Qpid JMS client and inspecting the frames around message delivery and the resulting disposition.

PAHO MQTT 5 throwing exception when using same clientId in routes

When using paho-mqtt5:test more than once with same clientId then it throw exception Client not connected but if i will use different clientId for each to and from then it will work fine
2021-10-05 19:25:28,650 ERROR [org.apa.cam.pro.err.DefaultErrorHandler] (Camel (camel-1) thread #0 - timer://test) Failed delivery for (MessageId: 871E4623819E4FB-000000000000001B on ExchangeId: 871E4623819E4FB-000000000000001B). Exhausted after delivery attempt: 1 caught: Client is not connected (32104)
Message History (complete message history is disabled)
---------------------------------------------------------------------------------------------------------------------------------------
RouteId ProcessorId Processor Elapsed (ms)
[route1 ] [route1 ] [from[timer://test?period=1000] ] [ 0]
...
[route1 ] [to1 ] [paho:test ] [ 0]
Stacktrace
---------------------------------------------------------------------------------------------------------------------------------------
: Client is not connected (32104)
at org.eclipse.paho.mqttv5.client.internal.ExceptionHelper.createMqttException(ExceptionHelper.java:32)
at org.eclipse.paho.mqttv5.client.internal.ClientComms.sendNoWait(ClientComms.java:231)
at org.eclipse.paho.mqttv5.client.MqttAsyncClient.publish(MqttAsyncClient.java:1530)
at org.eclipse.paho.mqttv5.client.MqttClient.publish(MqttClient.java:564)
at org.apache.camel.component.paho.mqtt5.PahoMqtt5Producer.process(PahoMqtt5Producer.java:55)
at org.apache.camel.support.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:66)
at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:172)
at org.apache.camel.processor.errorhandler.RedeliveryErrorHandler$SimpleTask.run(RedeliveryErrorHandler.java:463)
at org.apache.camel.impl.engine.DefaultReactiveExecutor$Worker.schedule(DefaultReactiveExecutor.java:179)
at org.apache.camel.impl.engine.DefaultReactiveExecutor.scheduleMain(DefaultReactiveExecutor.java:64)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:184)
at org.apache.camel.impl.engine.CamelInternalProcessor.process(CamelInternalProcessor.java:398)
at org.apache.camel.component.timer.TimerConsumer.sendTimerExchange(TimerConsumer.java:210)
at org.apache.camel.component.timer.TimerConsumer$1.run(TimerConsumer.java:76)
at java.base/java.util.TimerThread.mainLoop(Timer.java:556)
at java.base/java.util.TimerThread.run(Timer.java:506)
Here is my code which is throwing exception
#ApplicationScoped
class TestRouter : RouteBuilder() {
override fun configure() {
val mqtt5Component = PahoMqtt5Component()
mqtt5Component.configuration = PahoMqtt5Configuration().apply {
brokerUrl = "tcp://192.168.99.101:1883"
clientId = "paho123"
isCleanStart = true
}
context.addComponent("paho-mqtt5", mqtt5Component)
from("timer:test?period=1000").setBody(constant("Testing timer2")).to("paho-mqtt5:test")
from("paho-mqtt5:test").process { e ->
val body = (e.`in`?.body as? ByteArray)?.let { String(it) }
println("test body 1 => $body")
}
}
}
#William, this is expected behavior
The message broker uses the client id to differentiate between clients so it can perform housekeeping for a client connection that is no longer used
In addition, a client may have a "Last Will and Testament" that the broker keeps track of
It is acceptable to append a random number to the end of your current 'clientId' since it is likely no one but you will care about this
If you have access to the individuals login, you could use that as well but you would still want to make each session unique in case they run multiple sessions
Maybe I don't understand what your problem is
Each client must have a unique Id
What are you observing that makes you think that it is creating multiple connections for a single client?
Is there a chance you are opening multiple windows and each is generating a different clientId?
This is a good way to diagnose issues by monitoring what the server is seeing
My paho-mqtt client (Javascript) is connecting as "webclient" and I append a randome number (webclient173) to identify this client
To troubleshoot, I would suggest you close all connections on the client and monitor the log of the MQTT process
When the monitor is in place, open a connection from a client that currently has no connections
This is an example connection to my Mosquitto log file
$ tail -f /var/log/mosquitto/mosquitto.log
1635169943: No will message specified.
1635169943: Sending CONNACK to webclient173 (0, 0)
1635169943: Received SUBSCRIBE from webclient173
1635169943: testtopic (QoS 0)
1635169943: Sending SUBACK to webclient173
1635170003: Received PINGREQ from webclient173
1635170003: Sending PINGRESP to webclient173
1635170003: Received PINGREQ from webclient173
1635170003: Sending PINGRESP to webclient173
What does your log show?

Google Cloud Run pubsub pull listener app fails to start

I'm testing pubsub "pull" subscriber on Cloud Run using just listener part of this sample java code (SubscribeAsyncExample...reworked slightly to fit in my SpringBoot app):
https://cloud.google.com/pubsub/docs/quickstart-client-libraries#java_1
It fails to startup during deploy...but while it's trying to start, it does pull items from the pubsub queue. Originally, I had an HTTP "push" receiver (a #RestController) on a different pubsub topic and that worked fine. Any suggestions? I'm new to Cloud Run. Thanks.
Deploying...
Creating Revision... Cloud Run error: Container failed to start. Failed to start and then listen on the port defined
by the PORT environment variable. Logs for this revision might contain more information....failed
Deployment failed
In logs:
2020-08-11 18:43:22.688 INFO 1 --- [ main] o.s.web.context.ContextLoader : Root WebApplicationContext: initialization completed in 4606 ms
2020-08-11T18:43:25.287759Z Listening for messages on projects/ce-cxmo-dev/subscriptions/AndySubscriptionPull:
2020-08-11T18:43:25.351650801Z Container Sandbox: Unsupported syscall setsockopt(0x18,0x29,0x31,0x3eca02dfd974,0x4,0x28). It is very likely that you can safely ignore this message and that this is not the cause of any error you might be troubleshooting. Please, refer to https://gvisor.dev/c/linux/amd64/setsockopt for more information.
2020-08-11T18:43:25.351770555Z Container Sandbox: Unsupported syscall setsockopt(0x18,0x29,0x12,0x3eca02dfd97c,0x4,0x28). It is very likely that you can safely ignore this message and that this is not the cause of any error you might be troubleshooting. Please, refer to https://gvisor.dev/c/linux/amd64/setsockopt for more information.
2020-08-11 18:43:25.680 WARN 1 --- [ault-executor-0] i.g.n.s.i.n.u.internal.MacAddressUtil : Failed to find a usable hardware address from the network interfaces; using random bytes: ae:2c:fb:e7:92:9c:2b:24
2020-08-11T18:45:36.282714Z Id: 1421389098497572
2020-08-11T18:45:36.282763Z Data: We be pub-sub'n in pull mode2!!
Nothing else after this and the app stops running.
#Component
public class AndyTopicPullRecv {
public AndyTopicPullRecv()
{
subscribeAsyncExample("ce-cxmo-dev", "AndySubscriptionPull");
}
public static void subscribeAsyncExample(String projectId, String subscriptionId) {
ProjectSubscriptionName subscriptionName =
ProjectSubscriptionName.of(projectId, subscriptionId);
// Instantiate an asynchronous message receiver.
MessageReceiver receiver =
(PubsubMessage message, AckReplyConsumer consumer) -> {
// Handle incoming message, then ack the received message.
System.out.println("Id: " + message.getMessageId());
System.out.println("Data: " + message.getData().toStringUtf8());
consumer.ack();
};
Subscriber subscriber = null;
try {
subscriber = Subscriber.newBuilder(subscriptionName, receiver).build();
// Start the subscriber.
subscriber.startAsync().awaitRunning();
System.out.printf("Listening for messages on %s:\n", subscriptionName.toString());
// Allow the subscriber to run for 30s unless an unrecoverable error occurs.
// subscriber.awaitTerminated(30, TimeUnit.SECONDS);
subscriber.awaitTerminated();
System.out.printf("Async subscribe terminated on %s:\n", subscriptionName.toString());
// } catch (TimeoutException timeoutException) {
} catch (Exception e) {
// Shut down the subscriber after 30s. Stop receiving messages.
subscriber.stopAsync();
System.out.printf("Async subscriber exception: " + e);
}
}
}
Kolban question is very important!! With the shared code, I would like to say "No". The Cloud Run contract is clear:
Your service must answer to HTTP request. Out of request, you pay nothing and no CPU is dedicated to your instance (the instance is like a daemon when no request is processing)
Your service must be stateless (not your case here, I won't take time on this)
If you want to pull your PubSub subscription, create an endpoint in your code with a Rest controller. While you are processing this request, run your pull mechanism and process messages.
This endpoint can be called by Cloud Scheduler regularly to keep the process up.
Be careful, you have a max request processing timeout at 15 minutes (today, subject to change in a near future). So, you can't run your process more than 15 minutes. Make it resilient to fail and set your scheduler to call your service every 15 minutes

ESP-32 can't connect to MQTT broker: mqtt_client: Error network response

I am trying to connect my ESP32 which runs using the ESP-IDF framework to MQTT. I have imported this MQTT library successfully and have set up the configuration to look like this:
static void mqtt_app_start(void)
{
const esp_mqtt_client_config_t mqtt_cfg = {
// .host = "m15.cloudmqtt.com",
.uri = "mqtt://rxarkckf:smNb81Ppfe7T#m15.cloudmqtt.com:10793", // uri in the format (username:password#domain:port)
// .host = "m15.cloudmqtt.com", // config with host, port, user, password seperated
// .port = 10793,
// .username = "rxarkckf",
// .password = "smNb81Ppfe7T",
.event_handle = mqtt_event_handler,
// .user_context = (void *)your_context
};
esp_mqtt_client_handle_t client = esp_mqtt_client_init(&mqtt_cfg);
esp_mqtt_client_start(client);
}
I call mqtt_app_start(); in my app_main function. After uploading the code my ESP-32 doesn't connect to the MQTT broker and outputs this:
␛[0;32mI (12633410) MQTT_CLIENT: Sending MQTT CONNECT message, type: 1, id: 0000␛[0m
␛[0;31mE (12633710) MQTT_CLIENT: Error network response␛[0m
␛[0;32mI (12633710) MQTT_CLIENT: Error MQTT Connected␛[0m
␛[0;32mI (12633710) MQTT_CLIENT: Reconnect after 10000 ms␛[0m
␛[0;32mI (12633710) MQTT_SAMPLE: MQTT_EVENT_DISCONNECTED␛[0m
I have double checked that the values for the host, username, password, and port are all correct. When I look at the logs on the web interface hosted at cloudmqtt.com, I can see this output:
2018-11-17 03:50:53: New connection from 73.94.66.49 on port 10793.
2018-11-17 03:50:53: Invalid protocol "MQIs�" in CONNECT from 73.94.66.49.
2018-11-17 03:50:53: Socket error on client <unknown>, disconnecting.
2018-11-17 03:51:20: New connection from 73.94.66.49 on port 10793.
I had similar experience using mosquitto.
Adding this line to mqtt_config.h made my mqtt working.
#define CONFIG_MQTT_PROTOCOL_311
I think the more correct way to set this configuration is in sdkconfig.h, either manually or using "make menuconfig"
The problem is very simple. The library you are using implements the MQTT 3.1 protocol. The server you are trying to connect to implements the MQTT 3.1.1 protocol or higher.
As specified in the document (https://www.oasis-open.org/committees/download.php/55095/mqtt-diffs-v1.0-wd01.doc):
4.1 Protocol Name
The Protocol Name is present in the variable header of a MQTT CONNECT control packet. The Protocol Name is a UTF-8 encoded
string. In MQTT 3.1 the protocol name is "MQISDP". In MQTT 3.1.1 the
protocol name is represented as "MQTT".
For technical info:
https://mqtt.org/mqtt-specification/

akka http with connecting http server to outgoing tcp server throught nginx

Problem when try to using akka http like proxy to tcp.
Start http server, which responsibility only to get json request from the client and send message on Tcp server through nginx, take response from it and send response back to the client. So, only like proxy.
The problem is that server try to response before tcp server response. But when I remove nginx proxy between http and tcp servers everything all right. In tcp level akka http server send RST packet and right after that get RST from nginx and connection close. Tcp respond to nginx but connection already closed. With simple tcp server the same situation.
Akka version 2.4.17, akka http 10.0.10
The code below
val connections: Flow[ByteString, ByteString,
Future[Tcp.OutgoingConnection]] =
Tcp().outgoingConnection(new InetSocketAddress(settings.tcpServerHost,
settings.tcpServerPort))
def start(): Unit ={
val requestHandler: HttpRequest => Future[HttpResponse] = {
case HttpRequest(POST, uri#Uri.Path("/v1/doc"), _, entry, _) =>
proxyTcp(100, uri, entry)
case HttpRequest(POST, uri#Uri.Path("/v5/doc"), _, entry, _) =>
proxyTcp(105, uri, entry)
case _ => Future(HttpResponse(status = StatusCodes.NotFound))
}
Http().bind(settings.host, settings.port).runForeach(connection => {
connection.handleWithAsyncHandler(requestHandler)
}) }
private def proxyTcp(version: Short, uri: Uri, entry: RequestEntity):
Future[HttpResponse] = {
val transform = Transformations(connections, version)
entry.withoutSizeLimit().dataBytes
.via(transform.json2tlv)
.via(transform.connections)
.runWith(Sink.head)
.map(result => HttpResponse(status = StatusCodes.OK, entity =
HttpEntity(ContentTypes.`application/json`, result.get)))
}

Resources