I am trying to write from client a struct to the server with socket.
the struct is :
typedef struct R
{
int a;
int b;
double c;
double d;
double result[4];
}R;
The struct is the same at the 2 programs(server,client) and i malloc for the struct in both.
the client program:
struct R* r;
malloc..
...(fill the struct with data)
write(socket_fd,(void*)r,size of(R));
the Server program:
struct R* r;
malloc..
read(client_fd,(R*)r,size of(R));
This is not passing the struct from client to server..
How to write the struct to the server by socket??
Some basic elements of network programming are:
One read or write call might not write the total bytes you intend to
read/write. Check the return value of call. It would return number
of bytes read/written. If less bytes have been written, you should
call write in a loop until all data has been written. Same applies
to read.
Endianess of machine also matters. If you wrote an int which was
little endian (e.g. x86), when travelling on the network it would be converted into a
big endian value. You need to use apis such as htons, ntohs in POSIX to
accommodate that.
These are just starting points, but the most likely reasons of data not reaching destination in the form as you expected.
I'm assuming you are getting some data, but not in the form you are expecting. If so, it might also help to add a breakpoint in gdb and inspect the memory of r, in the client code.
You could fill the struct in sender code with 0xdeadbeef or similar debug strings(http://en.wikipedia.org/wiki/Magic_number_%28programming%29#Magic_debug_values), to identify your data in the client memory more easily. I have found that very helpful for debugging. Like some of the other answers mentioned, endianess and partial data might be the problem. Checking the return values and error codes will help too.
Related
I am porting an application to an ARM platform in C, the application also runs on an x86 processor, and must be backward compatible.
I am now having some issues with variable alignment. I have read the gcc manual for
__attribute__((aligned(4),packed)) I interpret what is being said as the start of the struct is aligned to the 4 byte boundry and the inside remains untouched because of the packed statement.
originally I had this but occasionally it gets placed unaligned with the 4 byte boundary.
typedef struct
{
unsigned int code;
unsigned int length;
unsigned int seq;
unsigned int request;
unsigned char nonce[16];
unsigned short crc;
} __attribute__((packed)) CHALLENGE;
so I change it to this.
typedef struct
{
unsigned int code;
unsigned int length;
unsigned int seq;
unsigned int request;
unsigned char nonce[16];
unsigned short crc;
} __attribute__((aligned(4),packed)) CHALLENGE;
The understand I stated earlier seems to be incorrect as both the struct is now aligned to a 4 byte boundary, and and the inside data is now aligned to a four byte boundary, but because of the endianess, the size of the struct has increased in size from 42 to 44 bytes. This size is critical as we have other applications that depend on the struct being 42 bytes.
Could some describe to me how to perform the operation that I require. Any help is much appreciated.
If you're depending on sizeof(yourstruct) being 42 bytes, you're about to be bitten by a world of non-portable assumptions. You haven't said what this is for, but it seems likely that the endianness of the struct contents matters as well, so you may also have a mismatch with the x86 there too.
In this situation I think the only sure-fire way to cope is to use unsigned char[42] in the parts where it matters. Start by writing a precise specification of exactly what fields are where in this 42-byte block, and what endian, then use that definition to write some code to translate between that and a struct you can interact with. The code will likely be either all-at-once serialisation code (aka marshalling), or a bunch of getters and setters.
This is one reason why reading whole structs instead of memberwise fails, and should be avoided.
In this case, packing plus aligning at 4 means there will be two bytes of padding. This happens because the size must be compatible for storing the type in an array with all items still aligned at 4.
I imagine you have something like:
read(fd, &obj, sizeof obj)
Because you don't want to read those 2 padding bytes which belong to different data, you have to specify the size explicitly:
read(fd, &obj, 42)
Which you can keep maintainable:
typedef struct {
//...
enum { read_size = 42 };
} __attribute__((aligned(4),packed)) CHALLENGE;
// ...
read(fd, &obj, obj.read_size)
Or, if you can't use some features of C++ in your C:
typedef struct {
//...
} __attribute__((aligned(4),packed)) CHALLENGE;
enum { CHALLENGE_read_size = 42 };
// ...
read(fd, &obj, CHALLENGE_read_size)
At the next refactoring opportunity, I would strongly suggest you start reading each member individually, which can easily be encapsulated within a function.
I've been moving structures back and forth from Linux, Windows, Mac, C, Swift, Assembly, etc.
The problem is NOT that it can't be done, the problem is that you can't be lazy and must understand your tools.
I don't see why you can't use:
typedef struct
{
unsigned int code;
unsigned int length;
unsigned int seq;
unsigned int request;
unsigned char nonce[16];
unsigned short crc;
} __attribute__((packed)) CHALLENGE;
You can use it and it doesn't require any special or clever code. I write a LOT of code that communicates to ARM. Structures are what make things work. __attribute__ ((packed)) is my friend.
The odds of being in a "world of hurt" are nil if you understand what is going on with both.
Finally, I can't for the life make out how you get 42 or 44. Int is either 4 or 8 bytes (depending on the compiler). That puts the number at either 16+16+2=34 or 32+16+2=50 -- assuming it is truly packed.
As I say, knowing your tools is part of your problem.
What is your true goal?
If it's to deal with data that's in a file or on the wire in a particular format what you should do is write up some marshaling/serialization routines that move the data between the compiler struct that represents how you want to deal with the data inside the program and a char array that deals with how the data looks on the wire/file.
Then all that needs to be dealt with carefully and possibly have platform specific code is the marshaling routines. And you can write some nice-n-nasty unit tests to ensure that the marshaled data gets to and from the struct properly no matter what platform you might have to port to today and in the future.
I would guess that the problem is that 42 isn't divisible by 4, and so they get out of alignment if you put several of these structs back to back (e.g. allocate memory for several of them, determining the size with sizeof). Having the size as 44 forces the alignment in these cases as you requested. However, if the internal offset of each struct member remains the same, you can treat the 44 byte struct as though it was 42 bytes (as long as you take care to align any following data at the correct boundary).
One trick to try might be putting both of these structs inside a single union type and only use 42-byte version from within each such union.
As I am using linux, I have found that by echo 3 > /proc/cpu/alignment it will issue me with a warning, and fix the alignment issue. This is a work around but it is very helpful with locating where the structures are failing to be misaligned.
Basicly i have a custom structure that contains different kind of data. For example:
typedef struct example_structure{
uint8_t* example_1[4];
int example_2[4];
int example_3;
} example_structure;
What i need to do is to copy context of this structure to a const char* buffer so i can send that copied data (buffer) using winsock2's send(SOCKET s, const char* buffer, int len, int flags) function. I tried using memcpy() but wouldn't i just copy address of pointers and not the data?
Yes, if you copied or sent that structure through a socket you would end up copying/sending pointers, which would obviously be meaningless to the recipient, however, if the recipient is running on different hardware (e.g. not the same endian), all of the data may be meaningless anyway. On top of that, differences in the amount of padding between structure members may also become a problem.
For non-trivial situations it is best to use an existing protocol (such as protobuf), or roll your own protocol, keeping in mind the potential differences in hardware representation of your data.
You need to design a protocol before you can encode the data in accord with that protocol. Decide exactly how the data will be encoded at the byte level. Then write code to encode and decode to that format that you decided on.
Do not skip the step of actually documenting the wire protocol at the byte level. It will save you pain later, I promise.
See this answer for a bit more detail.
const char* buffer
This buffer has a constant value so u cant copy anything to it. You probably don't need to copy anything. Just use send function in such a way:
send(s, (char*)&example_structure, sizeof(structure), flags)
But here is the problem with pointers in your structure (uint8_t* example_1[4];).
Sending pointers between different applications / machine does not make sense.
Hmm, your struct contains uint8_t * fields, what looks like C strings... It does not make sense copying or sending a pointer which is just a mere memory address in sending process user space.
If your struct has been (note, no pointers):
typedef struct example_structure{
uint8_t example_1[4];
int example_2[4];
int example_3;
} example_structure;
and provided you transfer it on exactly same architecture (same hardware, same compiler, same compiler options), you could do simply:
example_structure ex_struc;
// initialize the struct
...
send(s, &ex_struc, sizeof(ex_struc), flags);
And even in that case, I would strongly advise you to define and use a protocol - as already said by #DavidSchwartz, it could save you time and headaches later...
But as you have pointers, you cannot do that and must define a protocol.
it could be (but you are free to prefere little endian order, or 2 or 8 bytes for each int depending on your actual data):
one byte (or two) for length of first uint8_t array, followed by the array
above repeated 3 more times
four bytes in big endian order for first int of example_2
repeated 3 times
four bytes in big endian order for int of example_3
This clearly defines the format of a message.
I have several structures defined to send over different Operating Systems (tcp networks).
Defined structures are:
struct Struct1 { uint32_t num; char str[10]; char str2[10];}
struct Struct2 { uint16_t num; char str[10];}
typedef Struct1 a;
typedef Struct2 b;
The data is stored in a text file.
Data Format is as such:
123
Pie
Crust
Struct1 a is stored as 3 separate parameters. However, struct2 is two separate parameters with both 2nd and 3rd line stored to the char str[] . The problem is when I write to a server over the multiple networks, the data is not received correctly. There are numerous spaces that separate the different parameters in the structures. How do I ensure proper sending and padding when I write to server? How do I store the data correctly (dynamic buffer or fixed buffer)?
Example of write: write(fd,&a, sizeof(typedef struct a)); Is this correct?
Problem Receive Side Output for struct2:
123( , )
0 (, Pie)
0 (Crust,)
Correct Output
123(Pie, Crust)
write(fd,&a, sizeof(a)); is not correct; at least not portably, since the C compiler may introduce padding between the elements to ensure correct alignment. sizeof(typedef struct a) doesn't even make sense.
How you should send the data depends on the specs of your protocol. In particular, protocols define widely varying ways of sending strings. It is generally safest to send the struct members separately; either by multiple calls to write or writev(2). For instance, to send
struct { uint32_t a; uint16_t b; } foo;
over the network, where foo.a and foo.b already have the correct endianness, you would do something like:
struct iovec v[2];
v[0].iov_base = &foo.a;
v[0].iov_len = sizeof(uint32_t);
v[1].iov_base = &foo.b;
v[1].iov_len = sizeof(uint16_t);
writev(fp, v, 2);
Sending structures over the network is tricky. The following problems you might have
Byte endiannes issues with integers.
Padding introduced by your compiler.
String parsing (i.e. detecting string boundaries).
If performance is not your goal, I'd suggest to create encoders and decoders for each struct to be send and received (ASN.1, XML or custom). If performance is really required you can still use structures and solve (1), by fixing an endianness (i.e. network byte
order) and ensure your integers are stored as such in those structures, and (2) by fixing a compiler and using the pragmas or attributes to enforce a "packed" structure.
Gcc for example uses attribute((packed)) as such:
struct mystruct {
uint32_t a;
uint16_t b;
unsigned char text[24];
} __attribute__((__packed__));
(3) is not easy to solve. Using null terminated strings at a network protocol
and depending on them being present would make your code vulnerable to several attacks. If strings need to be involved I'd use an proper encoding method such as the ones suggested above.
The easy way would be to write two functions for each structure: one to convert from textual representation to the struct and one to convert a struct back to text. Then you just send the text over the network and on the receiving side convert it to your structures. That way endianness does not matter.
There are conversion functions to ensure portability of binary integers across a network. Use htons, htonl, ntohs and ntohl to convert 16 and 32 bit integers from host to network byte order and vice versa.
I am reading a file to the structs, then write them to the server.
Here is a snippet of code:
struct b{
uint16_t num;
char str[10];
} x;
struct a{
uint32_t pid;
char str1[10];
char str2[10];
} y;
while(fscanf(fp,"%s",buff) != EOF)
while(1){
c = getchar();
if (c == '\n')
break;
else
buff[i]= c ;
i++;
write(fd, &b,sizeof(b))
Text File Format
123
George
Lee
How do I read and store the file to each struct?
When I write to a server, does it look like
write(fd, &a,sizeof(struct a)).
write(fd,&b,sizeof(struct b))?
How to ensure correct padding, endianness
This is how I run the file : /a.out IP PORT < file.txt
When you say "read and store the file to each struct", could you clarify that question?
To ensure the right padding and endian-ness, you have to do two things:
Send the struct one field at a time. It is a pain. Since you are declaring each struct member as a uint16_t type (and from that family), sizeof() will give you the same answer across all platforms.
TO ensure endianness, you have to use the host-to-network-order family. See: http://www.gnu.org/s/hello/manual/libc/Byte-Order.html
When you do this, you have to use htons() (host to network short) when sending and then ntohs() (network to host short) when receiving. For each member of your struct.
Often people ignore this; if both machines are modern Intel boxes, which is true for the vast majority of us, you can get away with the code you have for sending structs. But like you have mentioned, that doesn't gaurantee padding and endinaness!
You could also use the #pragma pack() GNU directives for specifying how padding should be handled. This works if you can rely on the GNU toolchain for both your client and server, as that is not part of the C specification.
Don't use send/recv (sockets) or read/write (file descriptors) with structures. It's virtually guaranteed to break later on. Among other problems, the padding and/or alignment of the structure members can change depending on the compiler, the compilation options, and the environment.
Instead, marshal the data into an independent format (like text) and send that. If the data must be absolutely the same on both sides, encode it into base 64 and prepend a checksum.
If you absolutely must transmit binary data, remember to convert each of the structure members to network byte order (man byteorder) and send each one individually.
Or maybe, I shouldn't cast. Here's what I'm doing:
I'm writing a piece of code that links a Linux device driver to a higher level library. The authors of the library use void * (under a new name via typedef) to store handles to an implementation specific object that describes a communication channel.
The driver I want to connect with the library uses int to store handles to its channels (because they are file descriptors as returned by calls to open()). So, in my code, I get void * passed in from the library and need to call stuff from the driver using an int and vice versa. I. e.:
// somewhere in the library ...
typedef void* CAN_HANDLE;
// ... in my code
CAN_HANDLE canOpen_driver(s_BOARD *board)
{
int fd;
// ...
fd = open(busname, O_RDWR);
// ...
return (CAN_HANDLE) fd; // <-- not safe, especially not when converting back.
}
The adapters that others have written actually store some struct etc. somewhere and just cast between pointers, so no size issues arise. In my case, I don't really want to manage file descriptors, as the OS already does.
On my PC, I think the pointer is larger than the int, so I could bit-twiddle my way out of this, but the code goes into embedded systems, too, and I'm not experienced enough to make any assumptions about the size of types on those machines.
Edit: Of course you don't need a struct, you can just allocate memory for a plain int.
CAN_HANDLE canOpen_driver(s_BOARD *board)
{
int *fd = malloc(sizeof(int));
if (fd)
{
// ...
*fd = open(busname, O_RDWR);
// ...
return (CAN_HANDLE) fd;
}
// failure
return NULL;
}
This assumes there's a matching call to clean up. Something like:
void canClose_driver(CAN_HANDLE handle)
{
int *fd = handle;
free(fd);
}
Depending on the architecture, you might get away with that. If I understand correctly, the driver never actually uses the void* that you provide to it. It simply stores it to pass it back to your code later on.
Based on that assumption, as long as sizeof(void*) >= sizeof(int), it will be safe to cast between those types because you are sure that it is really a int.
If you cannot guarantee the size condition, or do not want to rely on a hack, you should allocate memory for the int and return the address of that memory. You might use malloc() or allocate a int in a fixed-size array, for example. The downside is that you will need to free that memory when it is no longer needed. I imagine that the driver has some kind of notification that signals your code when the data structure is no longer needed.
Generally on embedded systems, the following CPU models are most common:
Data bus Address bus
8 bit 16 bit
8 bit 16+8 bit (banking)
16 bit 16 bit
16 bit 16+8 bit (banking)
32 bit 32 bit
Generally, the address bus will always be >= than the data bus. I can't think of any CPU where the data bus would be larger than the address bus.
Here's a somewhat dirty trick that may or may not solve the issue:
typedef union
{
CAN_HANDLE handle;
long value;
} CAN_HANDLE_t;
This should be fairly portable, even though you will likely have to adapt this union to the specific system (far pointers etc).