I was debugging ganglia and came through sending data using XDR through UDP channel.
I found the second argument of the function xdrmem_create ( xdrs, addr, size, op) strange.
Where the syntax of the arguments are given as:
XDR *xdrs;
char *addr;
u_int size;
enum xdr_op op;
The reference of this function is here.
As you can see, the second argument (xdrs) of this function is a character array. And this is similarly declared in one of the ganglia's function as char msgbuf[GANGLIA_MAX_MESSAGE_LEN];.
After calling above function as xdrmem_create(&x, msgbuf, GANGLIA_MAX_MESSAGE_LEN, XDR_ENCODE); in ganglia , appropriate data in the ganglia's specific structure (cb->msg) is encoded to XDR format by calling the function xdr_Ganglia_value_msg(&x, &(cb->msg)) where x is an XDR type of variable.
Later, to send the encoded data through UDP channel, the function Ganglia_udp_send_message( udp_send_channels, msgbuf, len); is called.
To understand how this XDR data is sent, I tried to print the output of the content of msgbuf using fprintf but it always print nothing in spite of the fact that it is a character array . And it is also evident that the encoded data is sent successfully.
So, my question is, How data encoded into XDR format is being sent through UDP channel here?
I have pasted a part of code from ganglia here. You can see from the line 131 to 136 for your reference.
The messages are encoded with XDR, a binary format. The XDR libraries used are old, and a modern version of the API probably would have used uint8_t instead of char. Despite it using char, it is binary data - i.e. you cannot print the data as a string.
If you want to print this data, use a loop where you print each byte as hex e.g. by doing printf("%02X ", msgbuf[i]);
Read RFC4506 to learn about the XDR encoding. The actual messages are described in an XDR language specification, and run through a code generation tool (e.g. rpcgen for C code) to generate code for encoding/decoding. See https://github.com/fastly/ganglia/blob/master/lib/gm_protocol.x for the message definitions that Ganglia defines.
Related
I have to implement a relatively simple communication protocol on top of RS-232.
It's an ASCII based text protocol with a couple of frame types.
Each frame looks something like this:
* ___________________________________
* | | | | |
* | SOH | Data | CRC-16 | EOT |
* |_____|_________|_________|________|
* 1B nBytes 2B 1B
Start Of Header (1 Byte)
Data (n-Bytes)
CRC-16 (2 Bytes)
EOT (End Of Transmission)
Each data-field needs to be separated by semicolon ";":
for example, for HEADER type data (contains code,ver,time,date,src,id1,id2 values):
{code};{ver};{time};{date};{src};{id1};{id2}
what is the most elegant way of implementing this in C is my question?
I have tried defining multiple structs for each type of frame, for example:
typedef struct {
uint8_t soh;
char code;
char ver;
Time_t time;
Date_t date;
char src; // Unsigned char
char id1[20]; // STRING_20
char id2[20]; // STRING_20
char crlf;
uint16_t crc;
uint8_t eot;
} stdHeader_t;
I have declared a global buffer:
uint8_t DATA_BUFF[BUFF_SIZE];
I then have a function sendHeader() in which I want to use RS-232 send function to send everything byte by byte by casting the dataBuffer to header struct and filling out the struct:
static enum_status sendHeader(handle_t *handle)
{
uint16_t len;
enum_RETURN_VALUE rs232_err = OK;
enum_status err = STATUS_OK;
stdHeader_t *header = (stdHeader_t *)DATA_BUFF;
memset(DATA_BUFF, 0, size);
header ->soh= SOH,
header ->code= HEADER,
header ->ver= 10, // TODO
header ->time= handle->time,
header ->date= handle->date,
header ->src= handle->config->source,
memset(header ->id1,handle->config->id1, strlen(handle->config->id1));
memset(header ->id2,handle->config->id2, strlen(handle->config->id1));
header ->crlf = '\r\n',
header ->crc = calcCRC();
header ->eot = EOT;
len = sizeof(stdHeader_t );
do
{
for (uint16_t i = 0; i < len; i++)
{
rs232_err= rs232_tx_send(DATA_BUFF[i], 1); // Send one byte
if (rs232_err!= OK)
{
err = STATUS_ERR;
break;
}
}
// Break do-while loop if there is an error
if (err == STATUS_ERR)
{
break;
}
} while (conditions);
return err;
}
My problem is that I do not know how to approach the problem of handling ascii text based protocol,
the above principle would work very well for byte based protocols.
Also, I do not know how to implement semicolon ";" seperation of data in the above snippet, as everything is sent byte by byte, I would need aditional logic to know when it is needed to send ";" and with current implementation, that would not look very good.
For fields id1 and id2, I am receiveing string values as a part of handle->config, they can be of any lenght, but max is 20. Because of that, with current implementation, I would be sending more than needed in case actual lenght is less than 20, but I cannot use pointers to char inside the struct, because in that case, only the pointer value would get sent.
So to sumarize, the main question is:
How to implement the above described text based protocol for rs-232 in a nice and proper way?
what is the most elegant way of implementing this (ASCII Text Based protocol) in C is my question?
Since this is ASCII, avoid endian issues of trying to map a multi-byte integer. Simply send an integer (including char) as decimal text. Likewise for floating point, use exponential notation and sufficient precision. E.g. sprintf(buf, "%.*e", DBL_DECIMAL_DIG-1, some_double);. Allow "%a" notation.
Do not use the same code for SOH and EOT. Different values reduce receiver confusion.
Send date and time using ISO 8601 as your guide. E.g. "2022-11-10", "23:38:42".
Send string with a leading/trailing ". Escape non-printable ASCII characters, and ", \, ;. Example for 10 long string 123\\;\"\xFF456 --> "123\\\;\"\xFF456".
Error check, like crazy, the received data. Reject packets of data for all sorts of reasons: field count wrong, string too long, value outside field range, bad CRC, timeout, any non-ASCII character received.
Use ASCII hex characters for CRC: 4 hex characters instead of 2 bytes.
Consider a CRC 32 or 64.
Any out-of-band input, (bytes before receiving a SOF) are silently dropped. This nicely allows an optional LF after each command.
Consider the only characters between SOH/EOT should be printable ASCII: 32-126. Escape others as needed.
Since "it's an ASCII based text protocol with a couple of frame types.", I'd expect a type field.
See What type of framing to use in serial communication for more ideas.
First of all, structs are really not good for representing data protocols. The struct in your example will be filled to the brim with padding bytes everywhere, so it is not a proper nor portable representation of the protocol. In particular, forget all about casting a struct to/from a raw uint8_t array - that's problematic for even more reasons: the first address alignment and pointer aliasing.
In case you insist on using a struct, you must write serialization/deserialization routines that manually copy to/from each member into the raw uint8_t buffer, which is the one that must be used for the actual transmission.
(De)serialization routines might not be such a bad idea anyway, because of another issue not addressed by your post: network endianess. RS-232 protocols are by tradition almost always Big Endian, but don't count on it - endianess must be documented explicitly.
My problem is that I do not know how to approach the problem of handling ascii text based protocol, the above principle would work very well for byte based protocols.
That is a minor problem compared to the above. Often it is acceptable to have a mix of raw data (essentially everything but the data payload) and ASCII text. If you want a pure ASCII protocol you could consider something like "AT commands", but they don't have much in the way of error handling. You really should have a CRC16 as well as sync bytes. Hint: preferably pick the first sync byte as something that don't match 7 bit ASCII. That is something with MSB set. 0xAA is popular.
Once you've sorted out data serialization, endianess and protocol structure, you can start to worry about details such as string handling in the payload part.
And finally, RS232 is dinosaur stuff. There's not many reasons why one shouldn't use RS422/RS485. The last argument for using RS232, "computers come with RS232 COM ports", went obsolete some 15-20 years back.
One thing your struct implementation is missing is packing. For efficiency reasons, depending on which processor your code is running on, the compiler will add padding to the structure to align on certain byte boundaries. Normally this doesn't effect you code that much, but if you are sending this data across a serial stream where every byte matters, then you will be sending random zeros across as well.
This article explains padding well, and how to pack your structures for use cases like yours
Structure Padding
I need help in sending a specific escape sequence, using Embedded C.
This is my very first topic at stackoverflow!
I use this function to write commands through UART:
void UART_Write(UARTChannel* channel, uint8_t* data, uint32_t length)
The inputs, channel, data and length, correspond with the UART channel, the command to be send and the length of the command, respectively.
This works great in general!
However, I have some difficulties in generating the correct escape sequence in C. I need to write the following escape sequence, using the UART_Write function:
EscR0,1,2,7;
Esc being the Escape character (0x1b), R0 being the character command designator, 1,2,7 being the context specific parameters and ; being the termination sign.
How can I make the input "data" to the function "UART_Write" be equal to the escape sequence EscR0,1,2,7; in Embedded C?
I suppose it can be done in many different ways, but any suggestions will do.
do you want to send 'esc' or do you want to send '1b' as a hex value?
just send the characters, just like you would any other character string
for instance for the example: EscR0,1,2,7;:
char buffer[20] = {'\0'};
buffer[0] = 0x1b;
strcat( buffer, "R0,1,2,6;" );
....
i need to read an integer through a socket and the sens it to a function. i do
strcpy(out,"insert id messagge\n");
if (write(sd_client,out, sizeof(out))==-1){
printf("Error.\n");
}
while ((read(sd_client, &id, sizeof(int)))==-1){ //id is an integer
if(errno!=EINTR){
printf("Error.\n");
exit(-1);
}
}
messaggio2(sd_client, logi, atoi(id)); //atoi(id) try to send int to func
someone can help me please? :D
The second parameter of read and write is a pointer to the data.
When you say:
write(sd_client,out, sizeof(out))
you're passing the value of out. That should be:
write(sd_client, &out, sizeof(out))
Also, I think that you've declared id as int (which is correct), so why are you passing it to atoi? That function is for parsing a int from a string.
The problem you're having seems a bit vague since you haven't actually mentioned the errors you're getting, but from the code you have posted, it seems like you are running into two different problems:
atoi() is for converting a string value to an integer ... now it seems you already have an integer in the value of id, so the argument to atoi() is incorrect. You would simply need to use something like sprintf() or snprintf() to convert your integer value to a string value and then copy that into a user-defined string-buffer if you are wanting a string-representation of your integer.
You are working with sockets, therefore any information transferred over the network will be coming in network-byte-order. If you are on a little-endian platform, then the integer will be coming in a big-endian format, and you will be interpreting the value of your integer incorrect. Therefore you should be converting your integer to the native platform endian format using a function like ntohl() which can be found inside of the header file netinet/in.h
So for instance:
#include <netinet/in.h>
int id;
int sd_client;
//... code to open socket, etc.
//add your own error-checking ... stripped out here for simplicity's sake
read(sd_client, &id, sizeof(int));
id = ntohl(id);
char buffer[32];
snprintf(buffer, sizeof(buffer), "%d", id);
messaggio2(sd_client, logi, buffer);
I am trying to send an integer with pipe in a POSIX system but write() function is working for sending string or character data. Is there any way to send integer with a pipe?
Regards
The safe way is to use snprintf and strtol.
But if you know both processes were created using the same version of compiler (for example, they're the same executable which forked), you can take advantage of the fact that anything in C can be read or written as an array of char:
int n = something();
write(pipe_w, &n, sizeof(n));
int n;
read(pipe_r, &n, sizeof(n));
Either send a string containing the ASCII representation of integer e.g., 12345679, or send four bytes containing the binary representation of int, e.g., 0x00, 0xbc, 0x61, 0x4f.
In the first case, you will use a function such as atoi() to get the integer back.
Aschelpler's answer is right, but if this is something that can grow later I recommend you use some kind of simple protocol library like Google's Protocol Buffers or just JSON or XML with some basic schema.
Below one works fine for writing to pipe and reading from pipe as:
stop_daemon =123;
res = write(cli_pipe_fd_wr, &stop_daemon, sizeof(stop_daemon));
....
res = read(pipe_fd_rd, buffer, sizeof(int));
memcpy(&stop_daemon,buffer,sizeof(int));
printf("CLI process read from res:%d status:%d\n", res, stop_daemon);
output:
CLI process read from res:4 status:123
I'm lead to believe that write() can only send data buffers of byte (i.e. signed char), so how do I send an array of long integers using the C write() function in the sys/socket.h library?
Obviously I can't just cast or convert long to char, as any numbers over 127 would be malformed.
I took a look at the question, how to decompose integer array to a byte array (pixel codings), but couldn't understand it - please could someone dumb it down a little if this is what I'm looking for?
Follow up question:
Why do I get weird results when reading an array of integers from a TCP socket?
the prototype for write is:
ssize_t write(int fd, const void *buf, size_t count);
so while it writes in units of bytes, it can take a pointer of any type. Passing an int* will be no problem at all.
EDIT:
I would however, recomend that you also send the amount of integers you plan to send first so the reciever knows how much to read. Something like this (error checking omitted for brevity):
int x[10] = { ... };
int count = 10;
write(sock, &count, sizeof(count));
write(sock, x, sizeof(x));
NOTE: if the array is from dynamic memory (like you malloced it), you cannot use sizeof on it. In this case count would be equal to: sizeof(int) * element_count
EDIT:
As Brian Mitchell noted, you will likely need to be careful of endian issues as well. This is the case when sending any multibyte value (as in the count I recommended as well as each element of the array). This is done with the: htons/htonl and ntohs/ntohl functions.
Write can do what you want it to, but there's some things to be aware of:
1: You may get a partial write that's not on an int boundary, so you have to be prepared to handle that situation
2: If the code needs to be portable, you should convert your array to a specific endianess, or encode the endianess in the message.
The simplest way to send a single int (assuming 4-byte ints) is :
int tmp = htonl(myInt);
write(socket, &tmp, 4);
where htonl is a function that converts the int to network byte order. (Similarly,. when you read from the socket, the function ntohl can be used to convert back to host byte order.)
For an array of ints, you would first want to send the count of array members as an int (in network byte order), then send the int values.
Yes, you can just cast a pointer to your buffer to a pointer to char, and call write() with that. Casting a pointer to a different type in C doesn't affect the contents of the memory being pointed to -- all it does is indicate the programmer's intention that the contents of memory at that address be interpreted in a different way.
Just make sure that you supply write() with the correct size in bytes of your array -- that would be the number of elements times sizeof (long) in your case.
It would be better to have serialize/de-serialize functionality in your client /server program.
Whenever you want to send data, serialize the data into a byte buffer and send it over TCP with byte count.
When receiving data, de-serialize the data from buffer to your own interpretation .
You can interpret byte buffer in any form as you like. It can contain basic data type, objects etc.
Just make sure to take care of endianess and also alignment stuff.
Declare a character array. In each location of the array, store integer numbers, not characters.
Then you just send that.
For example:
char tcp[100];
tcp[0] = 0;
tcp[1] = 0xA;
tcp[2] = 0xB;
tcp[3] = 0xC;
.
.
// Send the character array
write(sock, tcp, sizeof(tcp));
I think what you need to come up with here is a protocol.
Suppose your integer array is:
100, 99, 98, 97
Instead of writing the ints directly to the buffer, I would "serialize" the array by turning it into a string representation. The string might be:
"100,99,98,97"
That's what would be sent over the wire. On the receiving end, you'd split the string by the commas and build the array back up.
This is more standardised, is human readable, and means people don't have to think about hi/lo byte orders and other silly things.
// Sarcasm
If you were working in .NET or Java, you'd probably encode it in XML, like this:
<ArrayOfInt><Int>100</Int><Int>99</Int><Int>98</Int><Int>97</Int></ArrayOfInt>
:)