I am trying to send this JSON packet to AWS IoT, but it is not recognized by AWS. I am using the example ESP32 AWS FreeRTOS code, but cannot understand what would be the correct format for the JSON packet with the following code:
#define echoMAX_DATA_LENGTH 20
char cDataBuffer[ echoMAX_DATA_LENGTH ];
(void) snprintf(cDataBuffer, echoMAX_DATA_LENGTH, "{\"state\":{\"reported\":%.*d}, \"clientToken\":\"%d\"}", x, x, x);
/* Setup the publish parameters. */
memset( &( xPublishParameters ), 0x00, sizeof( xPublishParameters ) );
xPublishParameters.pucTopic = echoTOPIC_NAME;
xPublishParameters.pvData = cDataBuffer;
xPublishParameters.usTopicLength = ( uint16_t ) strlen( ( const char * ) echoTOPIC_NAME );
xPublishParameters.ulDataLength = ( uint32_t ) strlen( cDataBuffer );
xPublishParameters.xQoS = eMQTTQoS1;
The AWS test page, cannot display the message and has converted it to UTF-8 (this error message is below)
Increase echoMAX_DATA_LENGTH to be large enough to fit your entire JSON message.
The static part of the JSON in your code (without the values filled in by snprintf()) is 34 characters, so there's no way this could ever work with echoMAX_DATA_LENGTH set to 20 - it would always produce a fragment of JSON instead of an entire JSON object.
Remember that the length that snprintf() uses includes a byte for the C string terminating character '\0', so you'll want to make echoMAX_DATA_LENGTH be one greater than the maximum total JSON message length.
When you increase echoMAX_DATA_LENGTH, try adding a debug message after the snprintf() so that you can see the JSON you're generating. If your code is set up to use Serial already, add:
Serial.println(cDataBuffer);
after the snprintf() so you can confirm that you've generated the JSON properly.
Related
I'm currently trying to build a library similar to ExpressJS in C. I have the ability to send any text (with res.send() functionality) or textually formatted file (.html, .txt, .css, etc.).
However, sending image data seems to cause a lot more trouble! I'm trying to use pretty much the exact same process I used for reading textual files. I saw this post and answer which uses a MAXLEN variable, which I would like to avoid. First, here's how I'm reading the data in:
// fread char *, goes 64 chars at a time
char *read_64 = malloc(sizeof(char) * 64);
// the entirety of the file data is placed in full_data
int *full_data_max = malloc(sizeof(int)), full_data_index = 0;
*full_data_max = 64;
char *full_data = malloc(sizeof(char) * *full_data_max);
full_data[0] = '\0';
// start reading 64 characters at a time from the file while fread gives positive feedback
size_t fread_response_length = 0;
while ((fread_response_length = fread(read_64, sizeof(char), 64, f_pt)) > 0) {
// internal array checker to make sure full_data has enough space
full_data = resize_array(full_data, full_data_max, full_data_index + 65, sizeof(char));
// copy contents of read_64 into full_data
for (int read_data_in = 0; read_data_in < fread_response_length / sizeof(char); read_data_in++) {
full_data[full_data_index + read_data_in] = read_64[read_data_in];
}
// update the entirety data current index pointer
full_data_index += fread_response_length / sizeof(char);
}
full_data[full_data_index] = '\0';
I believe the error is related to this component here. Likely something with calculating data length with fread() responses perhaps? I'll take you through the HTTP response creating as well.
I split the response sending into two components (as per the response on this question here). First I send my header, which looks good (29834 seems a bit large for image data, but that is an unjustified thought):
HTTP/1.1 200 OK
Content-Length: 29834
Content-Type: image/jpg
Connection: Keep-Alive
Access-Control-Allow-Origin: *
I send this first using the following code:
int *head_msg_len = malloc(sizeof(int));
// internal header builder that builds the aforementioned header
char *main_head_msg = create_header(status, head_msg_len, status_code, headers, data_length);
// send header
int bytes_sent = 0;
while ((bytes_sent = send(sock, main_head_msg + bytes_sent, *head_msg_len - bytes_sent / sizeof(char), 0)) < sizeof(char) * *head_msg_len);
Sending the image data (body)
Then I use a similar setup to try sending the full_data element that has the image data in it:
bytes_sent = 0;
while ((bytes_sent = send(sock, full_data + bytes_sent, full_data_index - bytes_sent, 0)) < full_data_index);
So, this all seems reasonable to me! I've even taken a look at the file original file and the file post curling, and they each start and end with the exact same sequence:
Original (| implies a skip for easy reading):
�PNG
�
IHDR��X��d�IT pHYs
|
|
|
RU�X�^Q�����땵I1`��-���
#QEQEQEQEQE~��#��&IEND�B`�
Post using curl:
�PNG
�
IHDR��X��d�IT pHYs
|
|
|
RU�X�^Q�����땵I1`��-���
#QEQEQEQEQE~��#��&IEND�B`
However, trying to open the file that was created after curling results in corruption errors. Similar issues occur on the browser as well. I'm curious if this could be an off by one or something small.
Edit:
If you would like to see the full code, check out this branch on Github.
I want to parse the actual payload from the output of AT commands.
For instance: in the example below, I'd want to read only "2021/11/16,11:12:14-32,0"
AT+QLTS=1 // command
+QLTS: "2021/11/16,11:12:14-32,0" // response
OK
In the following case, I'd need to only read 12345678.
AT+CIMI // command
12345678 // example response
So the point is: not all commands have the same format for the output. We can assume the response is stored in a string array.
I have GetAtCmdRsp() already implemented which stores the response in a char array.
void GetPayload()
{
char rsp[100] = {0};
GetAtCmdRsp("AT+QLTS=1", rsp);
// rsp now contains +QLTS: "2021/11/16,11:12:14-32,0"
// now, I need to parse "2021/11/16,11:12:14-32,0" out of the response
memset(rsp, 0, sizeof(rsp));
GetAtCmdRsp("AT+CIMI", rsp);
// rsp now contains 12345678
// no need to do additional parsing since the output already contains the value I need
}
I was thinking of doing char *start = strstr(rsp, ":") + 1; to get the start of the payload but some responses may only contain the payload as it's the case with AT+CIMI
Perhaps could regex be a good idea to determine the pattern +<COMMAND>: in a string?
In order to parse AT command responses a good starting point is understanding all the possible formats they can have. So, rather than implementing a command specific routine, I would discriminate commands by "type of response":
Commands with no payload in their answers, for example
AT
OK
Commands with no header in their answers, such as
AT+CIMI
12345678
OK
Commands with a single header in their answers
AT+QLTS=1
+QLTS: "2021/11/16,11:12:14-32,0"
OK
Command with multi-line responses.Every line could of "single header" type, like in +CGDCONT:
AT+CDGCONT?
+CGDCONT: 1,"IP","epc.tmobile.com","0.0.0.0",0,0
+CGDCONT: 2,"IP","isp.cingular","0.0.0.0",0,0
+CGDCONT: 3,"IP","","0.0.0.0",0,0
OK
Or we could even have mixed types, like in +CGML:
AT+CMGL="ALL"
+CMGL: 1,"REC READ","+XXXXXXXXXX","","21/11/25,10:20:00+00"
Good morning! How are you?
+CMGL: 2,"REC READ","+XXXXXXXXXX","","21/11/25,10:33:33+00"
I'll come a little late. See you. Bruce Wayne
OK
(please note how it could have also "empty" lines, that is \r\n).
At the moment I cannot think about any other scenario.In this way you'll be able to define an enum like
typedef enum
{
AT_RESPONSE_TYPE_NO_RESPONSE,
AT_RESPONSE_TYPE_NO_HEADER,
AT_RESPONSE_TYPE_SINGLE_HEADER,
AT_RESPONSE_TYPE_MULTILINE,
AT_RESPONSE_TYPE_MAX
}
and pass it to your GetAtCmdRsp( ) function in order to parser the response accordingly. If implement the differentiation in that function, or after it (or in an external function is your choice.
A solution without explicit categorization
Once you have clear all the scenarios that might ever occur, you can think about a general algorithm working for all of them:
Get the full response resp after the command echo and before the closing OK or ERROR. Make sure that the trailing \r\n\r\nOK is removed (or \r\nERROR. Or \r\nNO CARRIER. Or whatever the terminating message of the response might be).Make also sure to remove the command echo
If strlen( resp ) == 0 we belong to the NO_RESPONSE category, and the job is done
If the response contains \r\ns in it, we have a MULTILINE answer. So, tokenize it and place every line into an array element resp_arr[i]. Make sure to remove trailing \r\n
For every line in the response (for every resp_arr[i] element), search for <CMD> : pattern (not only :, that might be contained in the payload as well!). Something like that:
size_t len = strlen( resp_cur_line );
char *payload;
if( strstr( "+YOURCMD: ", resp_cur_line) == NULL )
{
// We are in "NO_HEADER" case
payload = resp_cur_line;
}
else
{
// We are in "HEADER" case
payload = resp_cur_line + strlen( "+YOURCMD: " );
}
Now payload pointer points to the actual payload.
Please note how, in case of MULTILINE answer, after splitting the lines into array elements every loop will handle correctly also the mixed scenarios like the one in +CMGL, as you'll be able to distinguish the lines containing the header from those containing data (and from the empty lines, of course). For a deeper analysis about +CMGL response parsing have a look to this answer.
I was able to transform 95% of a dedicated MIB to C code and make it run in a sub-agent like described in the last part of this Net-SNMP tutorial
For this I naturally use the mib2c.mfd.conf (I just read that mfd stands for Mib For Dummies ... all is said ...)
mib2c -I -c mib2c.mfd.conf my_mib_node
It generated a long .c file with almost all the oids like the one below.
Almost no lines were generated for the VideoInetAddr OID
//ABSTRACT OF SOURCE FILE GENERATED BY MIB2C
//...
long VideoFormat = 0; /* XXX: set default value */
// <<<=== NOTHING GENERATED HERE FOR VideoInetAddr OF TYPE INETADDRESS
// WHEREAS OTHER INTEGERS ARE NORMALLY PRESENT
long VideoInetPort = 0; /* XXX: set default value */
//...
void init_my_mib_node(void)
{
//...
const oid VideoFormat_oid[] = { 1,3,6,1,4,1,a,b,c,d,e };
static netsnmp_watcher_info VideoFormat_winfo;
// <<<=== NO OID GENERATED for VideoInetAddr OF TYPE INETADDRESS
// WHEREAS OTHER OIDs ARE NORMALLY GENERATED
static netsnmp_watcher_info VideoInetAddr_winfo; //We have the winfo after all
const oid VideoInetPort_oid[] = { 1,3,6,1,4,1,a,b,c,d,g };
static netsnmp_watcher_info VideoInetPort_winfo;
DEBUGMSGTL(("my_mib_node",
"Initializing VideoFormat scalar integer. Default value = %d\n",
VideoFormat));
reg = netsnmp_create_handler_registration(
"VideoFormat", NULL,
VideoFormat_oid, OID_LENGTH(VideoFormat_oid),
HANDLER_CAN_RWRITE);
netsnmp_init_watcher_info(&VideoFormat_winfo, &VideoFormat,
sizeof(long),ASN_INTEGER, WATCHER_FIXED_SIZE);
if (netsnmp_register_watched_scalar( reg, &VideoFormat_winfo ) < 0 ) {
snmp_log( LOG_ERR, "Failed to register watched VideoFormat" );
//...
}
This worked fine and needed 5 minutes (no code to write, just call the init() function), I was able to GET and SET all ... integers ...
Some oids are of Type InetAddress were not generated, neither were strings
Question
Is there a mib conf file able to generate code for every type
I tried the mib2c.old-api.conf which generates code also for the non-integer oids but I find it not as convenient. There is more boilerplate code to write.
Yes, mib2c could generate code for IP addresses. I cannot say that mfd does this, but, definitely, some mib2c.iterate.conf (for tables) does this.
The type of IP in SNMP is ASN_IPADDRESS represented by unint32_t in C.
Also,You need to make sure that in MIB-file for object, which represents IP, you have "SYNTAX IpAddress".
Have a look:
at the MIB file with IP object and implementation in C
Piece of answer but I am very far from comprehension and so side problems persist
Very pragmatically I managed to add by hand
//I put here ONLY what I added, see question above to complete code
#define STR_LENGTH_IPV4 sizeof("xxx.yyy.zzz.www")
char VideoInetAddr[STR_LENGTH_IPV4] = "192.168.2.3";
//...
const oid VideoInetAddr_oid[] = { 1,3,6,1,4,1,a,b,c,d,f };
reg = netsnmp_create_handler_registration(
"VideoInetAddr", NULL,
VideoInetAddr_oid, OID_LENGTH(VideoInetAddr_oid),
HANDLER_CAN_RWRITE);
netsnmp_init_watcher_info(&VideoInetAddr_winfo, &VideoInetAddr, sizeof(VideoInetAddr),
ASN_OCTET_STR, WATCHER_MAX_SIZE );
if (netsnmp_register_watched_scalar( reg, &VideoInetAddr_winfo ) < 0 ) {
snmp_log( LOG_ERR, "Failed to register watched VideoInetAddr" );
}
It still need to understand exactly the option like WATCHER_MAX_SIZE (is-it the good one ?)
I am working on a IOT project using MSP430F5529LP and CC3100Boost. The hardware is successfully connecting to the cloud and exchanging data. The response to the IOT devices is XML based. I am trying to parse the data. The following printf("\n%.*s\n", pch2-pch1-8, pch1 +8); extracts the data and prints to the console. Now in need to save this data to a variable. Here is my code snippet. The answer might be obvious, unfortunately I am failing to see it.
_i8 * databuffer;
char * pch0;
char * pch1;
char * pch2;
char data[7];
pch0 = strstr((char *)dataBuffer,"textResponse");
pch1 = strstr(pch0,"<text_1>");
pch2 = strstr(pch1,"</text_1>");
printf("\n%.*s\n", pch2-pch1-8, pch1 +8);
References:
Extract data between two delimiters
parsing the value in between two XML tags
MSP430G2121: looking for a xml library to parse xml text
Ensure that the data received is valid and of a length that will fit.
Print it to a string using sprintf() or equivalent function.
Print this string to the console with puts(the_string).
I need to parse manually, without external libraries, a JSON message coming from a server, in C language.
The message coming from server would be like:
{[CR+LF]
"Tmg": "R",[CR+LF]
"STP": 72[CR+LF]
}[CR+LF]
or
{[CR+LF]
"Tmg": "R",[CR+LF]
"STP": 150[CR+LF]
}[CR+LF]
I need the number after STP:. The number is different in each message structure, so I need to get that number from the JSON structure. I can't use external libraries because this code is in an embedded system and exernal code is not allowed.
I tried this following:
int main (){
const char response_message[35] = "{\r\n\"Tmg\":\"R\",\r\n\"STP\":72,\r\n}";
const char needle[8] = "P\":";
char *ret;
ret = strstr(response_message, needle);
printf("The number is: %s\n", ret);
return 0;
}
But obviously, I am getting this result:
The number is: P":72,
}
So I need to only get the number, how can I get this?
Thanks
You can use a hacked solution. Use strstr () to find "STP": then find the following , or } and extract the digits in between.
And that's a hack. Not guaranteed to work. For something that's guaranteed to work, you use a JSON parser.