I am trying to make a communication between a Client and a Sever through TCP.
Let's say the client send 10 Hex numbers as bitstream to Sever. send(socketID, pSend, 20, 0); Where pSend = &ArrayClient; and unsigned short ArrayClient[] = { 0A, 0B, BA, B1...., FA }.
The sever receive the bytes stream with recv(acceptID, pRecv, 20, 0). How can I print out the content in my pointer pRecv correctly. As Sever I won't know how many bytes did the Client send, therefore I don't know how many to print.
Because the datas were sent as bitstream, not a string. I can not know the end of the message by finding \0 like working with string.
So is there a way that I can know how many bytes that I received as sever, or any way to print out the bitstream in my pRecv as Hex numbers.
I have tried pointer to a pointer and something similar like this:
while( ((unsigned int*)pRecv)[i] != 0){
printf("%X", ((unsigned int*)pRecv)[i]);
i++;
}
When dealing with TCP, there is no built-in way of separating messages, so your protocol needs to delimit the message boundaries in some known way.
A simple way of doing this is by first sending single byte denoting the length of the message, then sending that number of bytes.
For example, before sending the above message, do this:
char mlen = 20;
send(socketID, &mlen, 1, 0);
Related
I have to implement a relatively simple communication protocol on top of RS-232.
It's an ASCII based text protocol with a couple of frame types.
Each frame looks something like this:
* ___________________________________
* | | | | |
* | SOH | Data | CRC-16 | EOT |
* |_____|_________|_________|________|
* 1B nBytes 2B 1B
Start Of Header (1 Byte)
Data (n-Bytes)
CRC-16 (2 Bytes)
EOT (End Of Transmission)
Each data-field needs to be separated by semicolon ";":
for example, for HEADER type data (contains code,ver,time,date,src,id1,id2 values):
{code};{ver};{time};{date};{src};{id1};{id2}
what is the most elegant way of implementing this in C is my question?
I have tried defining multiple structs for each type of frame, for example:
typedef struct {
uint8_t soh;
char code;
char ver;
Time_t time;
Date_t date;
char src; // Unsigned char
char id1[20]; // STRING_20
char id2[20]; // STRING_20
char crlf;
uint16_t crc;
uint8_t eot;
} stdHeader_t;
I have declared a global buffer:
uint8_t DATA_BUFF[BUFF_SIZE];
I then have a function sendHeader() in which I want to use RS-232 send function to send everything byte by byte by casting the dataBuffer to header struct and filling out the struct:
static enum_status sendHeader(handle_t *handle)
{
uint16_t len;
enum_RETURN_VALUE rs232_err = OK;
enum_status err = STATUS_OK;
stdHeader_t *header = (stdHeader_t *)DATA_BUFF;
memset(DATA_BUFF, 0, size);
header ->soh= SOH,
header ->code= HEADER,
header ->ver= 10, // TODO
header ->time= handle->time,
header ->date= handle->date,
header ->src= handle->config->source,
memset(header ->id1,handle->config->id1, strlen(handle->config->id1));
memset(header ->id2,handle->config->id2, strlen(handle->config->id1));
header ->crlf = '\r\n',
header ->crc = calcCRC();
header ->eot = EOT;
len = sizeof(stdHeader_t );
do
{
for (uint16_t i = 0; i < len; i++)
{
rs232_err= rs232_tx_send(DATA_BUFF[i], 1); // Send one byte
if (rs232_err!= OK)
{
err = STATUS_ERR;
break;
}
}
// Break do-while loop if there is an error
if (err == STATUS_ERR)
{
break;
}
} while (conditions);
return err;
}
My problem is that I do not know how to approach the problem of handling ascii text based protocol,
the above principle would work very well for byte based protocols.
Also, I do not know how to implement semicolon ";" seperation of data in the above snippet, as everything is sent byte by byte, I would need aditional logic to know when it is needed to send ";" and with current implementation, that would not look very good.
For fields id1 and id2, I am receiveing string values as a part of handle->config, they can be of any lenght, but max is 20. Because of that, with current implementation, I would be sending more than needed in case actual lenght is less than 20, but I cannot use pointers to char inside the struct, because in that case, only the pointer value would get sent.
So to sumarize, the main question is:
How to implement the above described text based protocol for rs-232 in a nice and proper way?
what is the most elegant way of implementing this (ASCII Text Based protocol) in C is my question?
Since this is ASCII, avoid endian issues of trying to map a multi-byte integer. Simply send an integer (including char) as decimal text. Likewise for floating point, use exponential notation and sufficient precision. E.g. sprintf(buf, "%.*e", DBL_DECIMAL_DIG-1, some_double);. Allow "%a" notation.
Do not use the same code for SOH and EOT. Different values reduce receiver confusion.
Send date and time using ISO 8601 as your guide. E.g. "2022-11-10", "23:38:42".
Send string with a leading/trailing ". Escape non-printable ASCII characters, and ", \, ;. Example for 10 long string 123\\;\"\xFF456 --> "123\\\;\"\xFF456".
Error check, like crazy, the received data. Reject packets of data for all sorts of reasons: field count wrong, string too long, value outside field range, bad CRC, timeout, any non-ASCII character received.
Use ASCII hex characters for CRC: 4 hex characters instead of 2 bytes.
Consider a CRC 32 or 64.
Any out-of-band input, (bytes before receiving a SOF) are silently dropped. This nicely allows an optional LF after each command.
Consider the only characters between SOH/EOT should be printable ASCII: 32-126. Escape others as needed.
Since "it's an ASCII based text protocol with a couple of frame types.", I'd expect a type field.
See What type of framing to use in serial communication for more ideas.
First of all, structs are really not good for representing data protocols. The struct in your example will be filled to the brim with padding bytes everywhere, so it is not a proper nor portable representation of the protocol. In particular, forget all about casting a struct to/from a raw uint8_t array - that's problematic for even more reasons: the first address alignment and pointer aliasing.
In case you insist on using a struct, you must write serialization/deserialization routines that manually copy to/from each member into the raw uint8_t buffer, which is the one that must be used for the actual transmission.
(De)serialization routines might not be such a bad idea anyway, because of another issue not addressed by your post: network endianess. RS-232 protocols are by tradition almost always Big Endian, but don't count on it - endianess must be documented explicitly.
My problem is that I do not know how to approach the problem of handling ascii text based protocol, the above principle would work very well for byte based protocols.
That is a minor problem compared to the above. Often it is acceptable to have a mix of raw data (essentially everything but the data payload) and ASCII text. If you want a pure ASCII protocol you could consider something like "AT commands", but they don't have much in the way of error handling. You really should have a CRC16 as well as sync bytes. Hint: preferably pick the first sync byte as something that don't match 7 bit ASCII. That is something with MSB set. 0xAA is popular.
Once you've sorted out data serialization, endianess and protocol structure, you can start to worry about details such as string handling in the payload part.
And finally, RS232 is dinosaur stuff. There's not many reasons why one shouldn't use RS422/RS485. The last argument for using RS232, "computers come with RS232 COM ports", went obsolete some 15-20 years back.
One thing your struct implementation is missing is packing. For efficiency reasons, depending on which processor your code is running on, the compiler will add padding to the structure to align on certain byte boundaries. Normally this doesn't effect you code that much, but if you are sending this data across a serial stream where every byte matters, then you will be sending random zeros across as well.
This article explains padding well, and how to pack your structures for use cases like yours
Structure Padding
I read the documentation regarding send() function, when it was said that the third parameter (len) is "The length, in bytes, of the data in buffer pointed to by the buf parameter".
I can't seem to understand if it sends the number of bytes I pass, or I nedd to pass the size of the buffer and it sends all the data included there.
Exmaple:
#define X 256
// main :
char test[X] = {0};
memcpy(test, "hello", 6);
send(sockfd, test, 6, 0)
send(sockfd, test, 256,0)
// will the first opetion send only hello? or hello000000....?
Thanks!
The send function sends precisely the number of bytes you tell it to (assuming it's not interrupted and doesn't otherwise fail). The number of bytes you need to send is determined by the protocol you are implementing. For example, if the protocol says you should send "FOO\r\n", then you need to send 5 bytes. If the protocol specifies that integers are represented as 4 bytes in network byte order and you're sending an integer, the buffer should contain an integer in network byte order and you should send 4 bytes. The size of the buffer doesn't matter to send.
As a complement to David Schwartz proper answer:
Depending on if the socket is non-blocking,or not, it is NOT guaranteed that a single send will actually send all data. You must check return value and might have to call send again (with correct buffer offsets).
For instance if you want to send 10 bytes of data (len=10), you call send(sock, buf, len, 0). However lets say it only manages to send 5 bytes, then send(..) will return 5, meaning that you will have to call it again later like send(sock, (buf + 5), (len - 5), 0). Meaning, skip first five bytes in buffer, they're already sent, and withdraw five bytes from the total number of bytes (len) we want to send.
(Note that I used parenthesis to make it easier to read only, and it assumes that buf is a pointer to 1 byte type.)
Error While Sending byte array serialy using Serial.write.
byte buf[] = {125, 126, 127, 2000, 5000};
int i = Serial.write(buf, sizeof(buf));
for(int i = 0; i < (sizeof(buf) / sizeof(buf[0])); i++)
{
Serial.println(buf[i]);
}
output :
}~??125
126
127
208
136
Any while for Unknown charcters at start. I am using Arduino 1.0.5 version
They are not Unknow characters, that's what you printed with:
int i = Serial.write(buf, sizeof(buf));
Just check an ASCII table buf[0] = 125 = '{'
With write() you are writing raw data without any kind of format. Your first byte is the value 125, in binary 01111101. This byte correspond to the character { if it is intepreted as char. Your serial communication interprets the incoming byte as char, so it prints '{`.
If you want to print 125 as string on a serial communication, you have to send buf[] = {49, 50, 53}. Or you have to convert your interget into a string.
what's also wrong is that you are using the byte type with values higher than 255. Try changing to int16_t.
Characters at the start is the ASCII representation of the buff numbers you send. The arduino serial monitor monitors all the activities, and it prints out also your .write commands. .writeln which you do later, gets printed additionally after the original .write.
So what you see, is the ASCII representation of arduino sending your commands.
PS:
The numbers 2000 and 5000 don't fit into the byte, so the last two bytes that you send are probably corrupted.
I'm in the middle of of reading Internetworking with TCP/IP Vol III, by Comer.
I am looking at a some sample code for a "TIME" client for UDP.
The code gets to the point where it does the read of the response, and it takes what should be a 4 bytes and converts it to a 32 bit unsigned integer, so it can be converted to UNIX time.
"n" is a file descriptor that points to a socket that listens for UDP.
n = read (s, (char *)&now, sizeof(now));
if (n < 0)
errexit("read failed: %s\n", strerror(errno));
now = ntohl((u_long)now); /* put in host byte order */
What I am wondering is:
Are there some assumptions that should be checked before making the conversion? This is in C, and I am wondering if there are situations where read would pass a number of bytes that is not 4. If so, it seems like "now" would be a mess.
"Now" is defined as:
time_t now; /* 32-bit integer to hold time */
So maybe I don't understand the nature of "time_t", or how the bytes are passed around in C, or what situations UDP would return the wrong number of bytes to the file descriptor...
Thanks in advance.
With UDP, as long as the recieve buffer you pass to read is long enough, a single UDP packet won't be broken up between read calls.
However, there's no guarantee that the other side sent a packet of at least 4 bytes - you're quite right, if a server sent only a 2 byte response then that code would leave now containing garbage.
That probably doesn't matter too much in this precise situation - after all, the server is just as free to send 4 bytes of garbage as it is to send only 2 bytes. If you want to check for it, just check that the n returned by read is as long as you were expecting.
I'm lead to believe that write() can only send data buffers of byte (i.e. signed char), so how do I send an array of long integers using the C write() function in the sys/socket.h library?
Obviously I can't just cast or convert long to char, as any numbers over 127 would be malformed.
I took a look at the question, how to decompose integer array to a byte array (pixel codings), but couldn't understand it - please could someone dumb it down a little if this is what I'm looking for?
Follow up question:
Why do I get weird results when reading an array of integers from a TCP socket?
the prototype for write is:
ssize_t write(int fd, const void *buf, size_t count);
so while it writes in units of bytes, it can take a pointer of any type. Passing an int* will be no problem at all.
EDIT:
I would however, recomend that you also send the amount of integers you plan to send first so the reciever knows how much to read. Something like this (error checking omitted for brevity):
int x[10] = { ... };
int count = 10;
write(sock, &count, sizeof(count));
write(sock, x, sizeof(x));
NOTE: if the array is from dynamic memory (like you malloced it), you cannot use sizeof on it. In this case count would be equal to: sizeof(int) * element_count
EDIT:
As Brian Mitchell noted, you will likely need to be careful of endian issues as well. This is the case when sending any multibyte value (as in the count I recommended as well as each element of the array). This is done with the: htons/htonl and ntohs/ntohl functions.
Write can do what you want it to, but there's some things to be aware of:
1: You may get a partial write that's not on an int boundary, so you have to be prepared to handle that situation
2: If the code needs to be portable, you should convert your array to a specific endianess, or encode the endianess in the message.
The simplest way to send a single int (assuming 4-byte ints) is :
int tmp = htonl(myInt);
write(socket, &tmp, 4);
where htonl is a function that converts the int to network byte order. (Similarly,. when you read from the socket, the function ntohl can be used to convert back to host byte order.)
For an array of ints, you would first want to send the count of array members as an int (in network byte order), then send the int values.
Yes, you can just cast a pointer to your buffer to a pointer to char, and call write() with that. Casting a pointer to a different type in C doesn't affect the contents of the memory being pointed to -- all it does is indicate the programmer's intention that the contents of memory at that address be interpreted in a different way.
Just make sure that you supply write() with the correct size in bytes of your array -- that would be the number of elements times sizeof (long) in your case.
It would be better to have serialize/de-serialize functionality in your client /server program.
Whenever you want to send data, serialize the data into a byte buffer and send it over TCP with byte count.
When receiving data, de-serialize the data from buffer to your own interpretation .
You can interpret byte buffer in any form as you like. It can contain basic data type, objects etc.
Just make sure to take care of endianess and also alignment stuff.
Declare a character array. In each location of the array, store integer numbers, not characters.
Then you just send that.
For example:
char tcp[100];
tcp[0] = 0;
tcp[1] = 0xA;
tcp[2] = 0xB;
tcp[3] = 0xC;
.
.
// Send the character array
write(sock, tcp, sizeof(tcp));
I think what you need to come up with here is a protocol.
Suppose your integer array is:
100, 99, 98, 97
Instead of writing the ints directly to the buffer, I would "serialize" the array by turning it into a string representation. The string might be:
"100,99,98,97"
That's what would be sent over the wire. On the receiving end, you'd split the string by the commas and build the array back up.
This is more standardised, is human readable, and means people don't have to think about hi/lo byte orders and other silly things.
// Sarcasm
If you were working in .NET or Java, you'd probably encode it in XML, like this:
<ArrayOfInt><Int>100</Int><Int>99</Int><Int>98</Int><Int>97</Int></ArrayOfInt>
:)