I'm testing a simple MQTT client-server model where a client is repeatedly sending requests to a server and awaits replies to each request. The flow is like this:
Client subscribes to topic 'server/client' with QoS 1
Server subscribes to topic 'client/server' with Qos 1
Client sends 1000 messages to server, on message delivery, client increments its messagesOut counter by 1.
Server receives a new message and invoke a registered callback. In the callback, the server sends a simple message to 'server/client'. On message delivery, the server increments its messagesOut counter.
On new messages to topic 'server/client', the client will increment its messagesIn counter by 1.
Both server and client will stop when client's messagesIn and server's messagesOut are 1000.
With QoS level 1, I expect the server to fully receives 1000 messages sent from the client, and sends 1000 replies back. However, my program is always stuck when the server has received and sent out about 200-300 messages. When I comment out the call to MQTTAsync_sendMessage(), the server can receive all 1000 messages. I suspect some kind of race condition or threads limit is causing this, but I'm not too sure.
The callback for new messages in the server looks like this:
#define TOPIC_IN "client/server"
#define TOPIC_OUT "server/client"
#define PAYLOAD "hello client\n"
unsigned long long messagesIn = 0, messagesOut = 0;
int on_message_arrival (void *context, char *topic_name,
int topic_length, MQTTAsync_message *message) {
if (!strcmp(topic_name, TOPIC_IN)) {
MQTTAsync_freeMessage(&message);
MQTTAsync_free(topic_name);
messagesIn++;
MQTTAsync_message pub_message = MQTTAsync_message_initializer;
MQTTAsync_responseOptions opts = MQTTAsync_responseOptions_initializer;
pub_message.payload = PAYLOAD;
pub_message.payloadlen = strlen(PAYLOAD);
pub_message.qos = QOS1;
pub_message.retained = 0;
delivered_token = 0;
opts.context = &message;
MQTTAsync_sendMessage((MQTTAsync)context, TOPIC_OUT, &pub_message, &opts);
} else {
MQTTAsync_freeMessage(&message);
MQTTAsync_free(topic_name);
}
return 1;
}
Any guide to help me debug this is appreciated.
Related
I'm trying to implement a simple firewall which filters network connections made by Windows processes.
The firewall should either allow/block the connection.
In order to intercept connections by any process, I created a kernel driver which makes use of Windows Filtering Platform.
I registered a ClassifyFn (FWPS_CALLOUT_CLASSIFY_FN1) callback at the filtering layer FWPM_LAYER_ALE_AUTH_CONNECT_V4:
FWPM_CALLOUT m_callout = { 0 };
m_callout.applicableLayer = FWPM_LAYER_ALE_AUTH_CONNECT_V4;
...
status = FwpmCalloutAdd(filter_engine_handle, &m_callout, NULL, NULL);
The decision regarding connection allow/block should be taken by userlevel.
I communicate with Userlevel using FltSendMessage,
which cannot be used at IRQL DISPATCH_LEVEL.
Following the instructions of the Microsoft documentation regarding how to process callouts asynchronously,
I do call FwpsPendOperation0 before calling FltSendMessage.
After the call to FltSendMessage, I resume packet processing by calling FwpsCompleteOperation0.
FwpsPendOperation0 documentation states that calling this function should make possible to operate calls at PASSIVE_LEVEL:
A callout can pend the current processing operation on a packet when
the callout must perform processing on one of these layers that may
take a long interval to complete or that should occur at IRQL =
PASSIVE_LEVEL if the current IRQL > PASSIVE_LEVEL.
However, when the ClassifyFn callback is called at DISPATCH_LEVEL, I do sometimes still get a BSOD on FltSendMessage (INVALID_PROCESS_ATTACH_ATTEMPT).
I don't understand what's wrong.
Thank you in advance for any advice which could point me to the right direction.
Here is the relevant code of the ClassifyFn callback:
/*************************
ClassifyFn Function
**************************/
void example_classify(
const FWPS_INCOMING_VALUES * inFixedValues,
const FWPS_INCOMING_METADATA_VALUES * inMetaValues,
void * layerData,
const void * classifyContext,
const FWPS_FILTER * filter,
UINT64 flowContext,
FWPS_CLASSIFY_OUT * classifyOut)
{
NTSTATUS status;
BOOLEAN bIsReauthorize = FALSE;
BOOLEAN SafeToOpen = TRUE; // Value returned by userlevel which signals to allow/deny packet
classifyOut->actionType = FWP_ACTION_PERMIT;
remote_address = inFixedValues->incomingValue[FWPS_FIELD_ALE_AUTH_CONNECT_V4_IP_REMOTE_ADDRESS].value.uint32;
remote_port = inFixedValues->incomingValue[FWPS_FIELD_ALE_AUTH_CONNECT_V4_IP_REMOTE_PORT].value.uint16;
bIsReauthorize = IsAleReauthorize(inFixedValues);
if (!bIsReauthorize)
{
// First time receiving packet (not a reauthorized packet)
// Communicate with userlevel asynchronously
HANDLE hCompletion;
status = FwpsPendOperation0(inMetaValues->completionHandle, &hCompletion);
//
// FltSendMessage call here
// ERROR HERE:
// INVALID_PROCESS_ATTACH_ATTEMP BSOD on FltMessage call when at IRQL DISPATCH_LEVEL
//
FwpsCompleteOperation0(hCompletion, NULL);
}
if (!SafeToOpen) {
// Packet blocked
classifyOut->actionType = FWP_ACTION_BLOCK;
}
else {
// Packet allowed
}
return;
}
You need to invoke FltSendMessage() on another thread running at PASSIVE_LEVEL. You can use IoQueueWorkItem() or implement your own mechanism to process it on a system worker thread created via PsCreateSystemThread().
I have ported the kernel's synproxy code to user space. I use it as a transparent proxy between the client and the web server.
When I request a web page, it works fine (the web page is below 512kbit). However, if I access a huge file (4Gbit or more), it will not continue to transfer after 3Gbits is transmitted.
I have adjusted the "confirmation" sent by the server to the client and the "seq" sent from the client to the server. At the same time, the tcp option "sack" sent from the client to the server is also adjusted.
if(tcpinp->state == CONNTRACK_SYN_SENT){
if(tcphdr->tcp_flags == (TCP_SYN_FLAG|TCP_ACK_FLAG)){
synproxy_parse_options(tcphdr, &opts);
tcpinp->tsoff = opts.tsval - tcpinp->its;
}
swap(opts.tsval, opts.tsecr);
synproxy_send_server_ack(iphdr,tcphdr, &opts);
/* send client ack,update tcp window */
swap(opts.tsval, opts.tsecr);
/*
* update window to client
*
* dir:server -> client
*
* save isn_off = client.ISN1 - server.ISN2
*/
tcpinp->isn_off = tcpinp->isn - tcphdr->seq;
synproxy_send_client_ack(tcpinp, &opts);
tcpinp->state = CONNTRACK_ESTABLISHED;
return 0;
}
if(tcpinp->dir == IP_CT_DIR_REPLY){
tcphdr->sent_seq = htonl(ntohl(tcphdr->sent_seq) + tcpinp->isn_off);
}else if(tcpinp->dir == IP_CT_DIR_ORIGINAL){
tcphdr->recv_ack = htonl(ntohl(tcphdr->recv_ack) - tcpinp->isn_off);
nf_ct_sack_adjust(tcph, other_way);
}
What should I adjust besides seq / ack and sack?
---I found that the client tcp'win' option has been increased to '828800' and then it does not change.
I have solved ,that it need 'mss' is always same.
I have created a little program to send an email in c.
I did 3 functions :
Connect : to connect to a server by using socket,
SendAndRecieve : to send and recieve message to the mail server,
SendAnEmail : to write the text that will be send by SendAndRecieve.
Connect and SendAndRecieve are working perfectly.
My problem is in SendAnEmail. I don't know how to do finish the data step.
For each command, i recieve the answer of the server : 250 ******
But for the data one, it reply in first 354 but never 250 after the final full stop. I though it is the same protocol as telnet.
The final full stop = end of the body.
But it don't work. The mail server never reply.
Anyone know the solution to tell to the server that i finish to write the body ?
Code
SOCKET connexion(char* server_name,unsigned short port)
{
char buf[1024]={0};
int res_l= 0;
int nbrecv;
struct sockaddr_in serverSockAddr;
struct hostent * serverHostEnt;
SOCKET to_server_socket = 0;
serverHostEnt = gethostbyname( server_name );
if ( serverHostEnt == NULL ) {
res_l = h_errno;
return (SOCKET)-1; }
memcpy(&serverSockAddr.sin_addr,serverHostEnt->h_addr, serverHostEnt->h_length );
serverSockAddr.sin_port = htons( port );/
serverSockAddr.sin_family = AF_INET;
/* Create socket */
to_server_socket = socket( AF_INET, SOCK_STREAM, 0 );
/* connexion request*/
if( connect( to_server_socket, ( struct sockaddr * ) &serverSockAddr,sizeof( serverSockAddr ) ) < 0 )
return (SOCKET)-3;
while( !buf[0] ) nbrecv = recv( to_server_socket, buf, 1024, 0 );
printf("Welcome message : %s\n",buf);
return to_server_socket;
}
void SendAndReceive(SOCKET to_server_socket, char * messagesend)
{
char bufreceive[1024];
int size,retVal,nbrecv;
size = (int)strlen( messagesend );
// SEND THE MESSAGE
retVal = send( to_server_socket, messagesend, size, 0 );
printf("Envoye : %s\n",messagesend) ;
// WAIT THE ANSWER
memset(bufreceive,0,1024);
while(!bufreceive[0])
nbrecv = recv( to_server_socket, bufreceive, 1024, 0 );
printf("Recu : %s\n",bufreceive);
}
void SendAnEmail(SOCKET sock)
{
//ehlo
SendAndReceive(sock, "EHLO localhost\r\n");
//mail from
SendAndReceive(sock, "MAIL FROM:<user#adresse.com\r\n");
//rcpt to
SendAndReceive(sock, "RCPT TO:<user#adress.com>\r\n");
/data
SendAndReceive(sock, "DATA\r\n");
//Subject : Subject
//
//Body
//.
SendAndReceive(sock, "Subject : Subject\r\n\r\nBody\r\n.\r\n");
//quit
SendAndReceive(sock, "QUIT\r\n");
}
In your main, you must set the port =25, use a real smtp server for server_name, and add a library that contain Winsock2.h
CMD :
Welcome message : 220 mwinf5d05 ME ESMTP server ready
S: EHLO localhost
R: 250-mwinf5d05 hello [10.162.66.36], pleased to meet you
250-HELP
250-AUTH LOGIN PLAIN
250-SIZE 44000000
250-ENHANCEDSTATUSCODES
250-8BITMIME
250 OK
S: MAIL FROM: <blablabla#orange.fr>
R: 250 2.1.0 <blablabla#orange.fr> sender ok
S: RCPT TO: <xapx35#gmail.com>
R: 250 2.1.5 <xapx35#gmail.com> recipient ok
S: DATA
R: 354 enter mail, end with "." on a line by itself
S: Subject: FlashRad
Ceci est un test.
.
I expected to be able to do like when i use telnet by myself :
Like it's explain in this website.
So anyone know what how to say that i finish to write the body ?
Like the final full stop with telnet.
RFC 821 says
The third step in the procedure is the DATA command.
DATA <CRLF>
If accepted, the receiver-SMTP returns a 354 Intermediate reply
and considers all succeeding lines to be the message text.
When the end of text is received and stored the SMTP-receiver
sends a 250 OK reply.
Since the mail data is sent on the transmission channel the end
of the mail data must be indicated so that the command and
reply dialog can be resumed. SMTP indicates the end of the
mail data by sending a line containing only a period. A
transparency procedure is used to prevent this from interfering
with the user's text (see Section 4.5.2).
So you need to send a line with a only a single period after your body.
If you are waiting for a 250 after DATA, that would be a mistake too. You get 354 first, and then the 250 after the period only line
RFC 821
On the Eclipse Paho MQTT website, the developers provide a client example (http://www.eclipse.org/paho/files/mqttdoc/MQTTClient/html/pubsync.html) that does the following:
Create a client object with the specified parameters
Connect the client with the specified connection options
Publish a MQTT message
Disconnect the client
Destroy the client object
This works well if all you want is to publish one single message.
In my code, I have a function that contains pretty much the same code as in the aforementioned example, however, the function is called repeatedly from main() as I need to publish a large number of messages one after another. The problem is, if I use the code exactly as in the example, every time my function is called a new connection is created and shortly after destroyed. This happens again and again and again as long as the function is called repeatedly, causing a huge overhead.
Is there a way to check whether a client object has already been created, and if so, don't do it again but use the existing one?
In my understanding, the MQTTClient_isConnected() function is supposed to do that: https://www.eclipse.org/paho/files/mqttdoc/MQTTClient/html/_m_q_t_t_client_8h.html#ad9e40bdb7149ee3e5d075db7f51a735f
But if I try it like this, I get a Segmentation fault:
if (!MQTTClient_isConnected(client)) {
MQTTClient_create(&client, mqtt.addr, CLIENT_ID, MQTTCLIENT_PERSISTENCE_NONE, NULL);
conn_opts.keepAliveInterval = 20;
conn_opts.cleansession = 1;
conn_opts.username = TOKEN;
if (MQTTClient_connect(client, &conn_opts) != MQTTCLIENT_SUCCESS) {
printf("\n==> Connection to MQTT Broker failed.\n");
MQTTClient_destroy(&client);
exit(EXIT_FAILURE);
}
}
[EDIT]
Here is a simple demo code that better illustrates what I'm trying to accomplish:
#include <stdio.h>
#include <MQTTClient.h>
MQTTClient client;
void publish_MQTT() {
MQTTClient_connectOptions conn_opts = MQTTClient_connectOptions_initializer;
MQTTClient_message pubmsg = MQTTClient_message_initializer;
MQTTClient_deliveryToken token;
char *payload = (char *)calloc(1024, sizeof(char));
strcpy(payload, "hello");
printf("DEBUG_BEFORE >> MQTTClient_isConnected(client) = %d\n", MQTTClient_isConnected(client)); // DEBUG OUTPUT
if (!MQTTClient_isConnected(client)) {
MQTTClient_create(&client, addr, CLIENT_ID, MQTTCLIENT_PERSISTENCE_NONE, NULL);
conn_opts.keepAliveInterval = 20;
conn_opts.cleansession = 1;
conn_opts.username = TOKEN;
if (MQTTClient_connect(client, &conn_opts) != MQTTCLIENT_SUCCESS) {
fprintf(stderr, RED "\n==> Connection to MQTT Broker failed.\n" RESET_CL);
MQTTClient_destroy(&client);
free(payload);
exit(EXIT_FAILURE);
}
}
printf("DEBUG_AFTER >> MQTTClient_isConnected(client) = %d\n", MQTTClient_isConnected(client)); // DEBUG OUTPUT
pubmsg.payload = payload;
pubmsg.payloadlen = strlen(payload);
pubmsg.qos = QOS;
pubmsg.retained = 0;
MQTTClient_publishMessage(client, TOPIC, &pubmsg, &token);
MQTTClient_waitForCompletion(client, token, TIMEOUT);
//MQTTClient_disconnect(client, 10000);
//MQTTClient_destroy(&client);
free(payload);
}
int main(void) {
for (i=0; i<1000; i++) {
publish_MQTT();
}
return 0;
}
Please ignore the fact that the addr parameter is never specified (in my real code it is) or that it is pretty useless specifying a message in the publish_MQTT() function (in my real code, data is passed from main() to that function).
I figured it out: Apparently, there is absolutely nothing wrong with the example codes in the original posting.
It turns out I was appending the port of the MQTT server to the addr parameter again and again (in a section of the code not shown here as I didn't suspect the source of the error to be there), every time the publish_MQTT() function was called. This made the addr char string grow and eventually exceed the specified length, thus causing the SegFault.
This way everything works just as intended:
printf("\nADDR = %s\n\n", addr); // DEBUG OUTPUT
if (!MQTTClient_isConnected(client)) {
strcat(strcat(addr, ":"), pt); // This line needed to be placed here, not before that if block
MQTTClient_create(&client, addr, CLIENT_ID, MQTTCLIENT_PERSISTENCE_NONE, NULL);
conn_opts.keepAliveInterval = 20;
conn_opts.cleansession = 1;
conn_opts.username = TOKEN;
if (MQTTClient_connect(client, &conn_opts) != MQTTCLIENT_SUCCESS) {
printf("\n==> Connection to MQTT Broker failed.\n");
MQTTClient_destroy(&client);
free(payload);
exit(EXIT_FAILURE);
}
}
Probably you are setting up "clean session flag", what's mean: "
If the ClientId represents a Client already connected to the Server then the Server MUST disconnect the existing Client [MQTT-3.1.4-2]." (from mqtt standard). So you client is disconnected (the existing one).
Code from example seems to be reasobable. It looks like there is problem with passing function argument. For example if function needed address, and you are giving objects itself.
Morze from standard:
"3.2.2.2 Session Present
Position: bit 0 of the Connect Acknowledge Flags.
If the Server accepts a connection with CleanSession set to 1, the Server MUST set Session Present to 0 in the CONNACK packet in addition to setting a zero return code in the CONNACK packet [MQTT-3.2.2-1].
If the Server accepts a connection with CleanSession set to 0, the value set in Session Present depends on whether the Server already has stored Session state for the supplied client ID. If the Server has stored Session state, it MUST set Session Present to 1 in the CONNACK packet [MQTT-3.2.2-2]. If the Server does not have stored Session state, it MUST set Session Present to 0 in the CONNACK packet. This is in addition to setting a zero return code in the CONNACK packet".
I have a tiny C proxy server which I just want to get one request at a time from the client and send back the response from the server. No pipelining, no anything advanced, just a persistent http connection.
Structs:
typedef struct http_request {
char* h_data; // Header raw data
int h_size; // Header size
char host[5000]; // Host to connect to
char resource[5000]; // Resource to get
} http_request;
typedef struct http_response {
char* h_data; // Header raw data
int h_size; // Header size
char* b_data; // Body raw data
int b_size; // Content-length of the body
} http_response;
Code:
while(1){
// Waiting for user to connect
int sock_user = accept(sock, (struct sockaddr*)NULL, NULL);
int sock_host=-1;
// Accept 1 request at a time and respond
while(1){
http_request req;
http_response resp;
// 1. Client ==> Proxy Server
http_parse_request(sock_user, &req); // uses recv(sock_user)
// 2. Client Proxy ==> Server
if (sock_host < 0)
sock_host=proxy_connect_host(req.host);
write(sock_host, req.h_data, req.h_size);
// 3. Client Proxy <== Server
http_parse_response(sock_host, &resp); // uses send(sock_host)
// 4. Client <== Proxy Server
write(sock_user, resp.h_data, resp.h_size);
write(sock_user, resp.b_data, resp.b_size);
}
}
Now this works good for a few first pages. Then the program blocks at step 1 and the browser just shows Waiting for www.calcoolate.com... all the time.
Firebug:
All of those GET, are requests sent to my proxy. I however receive only the first two of them. I double checked the return value of each write() and recv() and they seem to match exactly with what is expected. I checked for both -1s and 0s.
There must be something wrong with the logic of my proxy.. Any ideas?