fwrite() in c writes bytes in a different order - c

#include <stdio.h>
#include <stdlib.h>
int main(void)
{
int *int_pointer = (int *) malloc(sizeof(int));
// open output file
FILE *outptr = fopen("test_output", "w");
if (outptr == NULL)
{
fprintf(stderr, "Could not create %s.\n", "test_output");
return 1;
}
*int_pointer = 0xabcdef;
fwrite(int_pointer, sizeof(int), 1, outptr);
//clean up
fclose(outptr);
free(int_pointer);
return 0;
}
this is my code and when I see the test_output file with xxd it gives following output.
$ xxd -c 12 -g 3 test_output
0000000: efcdab 00 ....
I'm expecting it to print abcdef instead of efcdab.

Which book are you reading? There are a number of issues in this code, casting the return value of malloc for example... Most importantly, consider the cons of using an integer type which might vary in size and representation from system to system.
An int is guaranteed the ability to store values between the range of -32767 and 32767. Your implementation might allow more values, but to be portable and friendly with people using ancient compilers such as Turbo C (there are a lot of them), you shouldn't use int to store values larger than 32767 (0x7fff) such as 0xabcdef. When such out-of-range conversions are performed, the result is implementation-defined; it could involve saturation, wrapping, trap representations or raising a signal corresponding to computational error, for example, the latter of two which could cause undefined behaviour later on.
You need to translate to an agreed-upon field format. When sending data over the write, or writing data to a file to be transferred to other systems, it's important that the protocol for communication be agreed upon. This includes using the same size and representation for integer fields. Both output and input should be followed by a translation function (serialisation and deserialisation, respectively).
Your fields are binary, and so your file should be opened in binary mode. For example, use fopen(..., "wb") rather than "w". In some situations, '\n' characters might be translated to pairs of \r\n characters, otherwise; Windows systems are notorious for this. Can you imagine what kind of havoc and confusion this could wreak? I can, because I've answered a question about this problem.
Perhaps uint32_t might be a better choice, but I'd choose unsigned long as uint32_t isn't guaranteed to exist. On that note, for systems which don't have htonl (which returns uint32_t according to POSIX), that function could be implemented like so:
uint32_t htonl(uint32_t x) {
return (x & 0x000000ff) << 24
| (x & 0x0000ff00) << 8
| (x & 0x00ff0000) >> 8
| (x & 0xff000000) >> 24;
}
As an example inspired by the above htonl function, consider these macros:
typedef unsigned long ulong;
#define serialised_long(x) serialised_ulong((ulong) x)
#define serialised_ulong(x) (x & 0xFF000000) / 0x1000000 \
, (x & 0xFF0000) / 0x10000 \
, (x & 0xFF00) / 0x100 \
, (x & 0xFF)
typedef unsigned char uchar;
#define deserialised_long(x) (x[3] <= 0x7f \
? deserialised_ulong(x) \
: -(long)deserialised_ulong((uchar[]) { 0x100 - x[0] \
, 0xFF - x[1] \
, 0xFF - x[2] \
, 0xFF - x[3] })
#define deserialised_ulong(x) ( x[0] * 0x1000000UL \
+ x[1] * 0x10000UL \
+ x[2] * 0x100UL \
+ x[3] )
#include <stdio.h>
#include <stdlib.h>
int main(void)
{
FILE *f = fopen("test_output", "wb+");
if (f == NULL)
{
fprintf(stderr, "Could not create %s.\n", "test_output");
return 1;
}
ulong value = 0xABCDEF;
unsigned char datagram[] = { serialised_ulong(value) };
fwrite(datagram, sizeof datagram, 1, f);
printf("%08lX serialised to %02X%02X%02X%02X\n", value, datagram[0], datagram[1], datagram[2], datagram[3]);
rewind(f);
fread(datagram, sizeof datagram, 1, f);
value = deserialised_ulong(datagram);
printf("%02X%02X%02X%02X deserialised to %08lX\n", datagram[0], datagram[1], datagram[2], datagram[3], value);
fclose(f);
return 0;
}

Use htonl()
It converts from whatever the host-byte-order is (endianness of your machine) to network byte order. So whatever machine you're running on you will get the the same byte order. These calls are used so that regardless of the host you're running on the bytes are sent over the network in the right order, but it works for you too.
See the man pages of htonl and byteorder. There are various conversion functions available, also for different integer sizes, 16-bit, 32-bit, 64-bit ...
#include <stdio.h>
#include <stdlib.h>
#include <arpa/inet.h>
int main(void) {
int *int_pointer = (int *) malloc(sizeof(int));
// open output file
FILE *outptr = fopen("test_output", "w");
if (outptr == NULL) {
fprintf(stderr, "Could not create %s.\n", "test_output");
return 1;
}
*int_pointer = htonl(0xabcdef); // <====== This ensures correct byte order
fwrite(int_pointer, sizeof(int), 1, outptr);
//clean up
fclose(outptr);
free(int_pointer);
return 0;
}

Related

Linux device driver: invalid operands to binary << (have char * and int)

I am trying to write a driver to read the rtc time. In the main code I have written the following. The driver side has the same definitions and ioctl handler returns the condition of flag.
On compiling the main code I get the error of "invalid operands to binary << (have 'char *' and 'int')" for the RD_RTC_TIME_UPDTD definition.
What is the reason for the error?
main.c:
#define RTC_MAGIC_NO "p"
#define RTC_TIME_UPDTD_CMD 0x1F
#define RD_RTC_TIME_UPDTD _IOR(RTC_MAGIC_NO, RTC_TIME_UPDTD_CMD, int*)
int rtc_time_updtd_sts = 0;
rtcDev = open("dev/rtc0", O_RDWR);
ret = ioctl(rtcDev, RD_RTC_TIME_UPDTD, (int*)&rtc_time_updtd_sts);
driver: rtc.c
#define RTC_MAGIC_NO "p"
#define RTC_TIME_UPDTD_CMD 0x1F
#define RD_RTC_TIME_UPDTD _IOR(RTC_MAGIC_NO, RTC_TIME_UPDTD_CMD, int*)
int rtc_do_ioctl(unsigned int cmd, unsigned long arg, int kernel){
switch(cmd) {
case RD_RTC_TIME_UPDTD: /* Read the flag to check if the RTC time is set by system or not */
{
copy_to_user((int *)arg, &rtc_time_updtd, sizeof(rtc_time_updtd));
break;
}
}
RTC_MAGIC_NO implies it should be a number. The first argument to _IOR should be an 8-bit number. Instead you defined it as "p" which is a string literal which will give you a char *. Perhaps you wanted to use 'p' instead, which would be treated as an int (and should fit the 8-bit criteria).
_IOR(RTC_MAGIC_NO, RTC_TIME_UPDTD_CMD, int*)
expands to
// _IOC(dir,type,nr,size)
_IOC(2U,("p"),(0x1F),((sizeof(int*))))
which expands to
(((2U) << _IOC_DIRSHIFT) | \
(("p") << _IOC_TYPESHIFT) | \
((0x1F) << _IOC_NRSHIFT) | \
((sizeof(int*)) << _IOC_SIZESHIFT))
Here you try to leftshift a char* (the "p") _IOC_TYPESHIFT steps. Shifting a pointer is not a supported operation in C so change "p" to 'p' like the other Real Time Clock ioctl macros in linux/rtc.h.

How to send data in little endian order C

I am communicating with a board that requires I send it 2 signed byte.
explaination of data type
what I need to send
Would I need to bitwise manipulation or can I just send 16bit integer as the following?
int16_t rc_min_angle = -90;
int16_t rc_max_angle = 120;
write(fd, &rc_min_angle, 2);
write(fd, &rc_max_angle, 2);
int16_t has the correct size but may or may not be the correct endianness. To ensure little endian order use macros such as the ones from endian.h:
#define _BSD_SOURCE
#include <endian.h>
...
uint16_t ec_min_angle_le = htole16(ec_min_angle);
uint16_t ec_max_angle_le = htole16(ec_max_angle);
write(fd, &ec_min_angle_le, 2);
write(fd, &ec_max_angle_le, 2);
Here htole16 stands for "host to little endian 16-bit". It converts from the host machine's native endianness to little endian: if the machine is big endian it swaps the bytes; if it's little endian it's a no-op.
Also note that you have you pass the address of the values to write(), not the values themselves. Sadly, we cannot inline the calls and write write(fd, htole16(ec_min_angle_le), 2).
If endian functions are not available, simply write the bytes in little endian order.
Perhaps with a compound literal.
// v------------- compound literal ---------------v
write(fd, &(uint8_t[2]){rc_min_angle%256, ec_min_angle/256}, 2);
write(fd, &(uint8_t[2]){rc_max_angle%256, ec_max_angle/256}, 2);
// ^-- LS byte ---^ ^-- MS byte ---^
// &
I added the & assuming the write() is a like write(2) - Linux.
If you don't need to have it type-generic, you can simply do:
#include <stdint.h>
#include <unistd.h>
/*most optimizers will turn this into `return 1;`*/
_Bool little_endian_eh() { uint16_t x = 1; return *(char *)&x; }
void swap2bytes(void *X) { char *x=X,t; t=x[0]; x[0]=x[1]; x[1]=t; }
int main()
{
int16_t rc_min_angle = -90;
int16_t rc_max_angle = 120;
//this'll very likely be a noop since most machines
//are little-endian
if(!little_endian_eh()){
swap2bytes(&rc_min_angle);
swap2bytes(&rc_max_angle);
}
//TODO error checking on write calls
int fd =1;
write(fd, &rc_min_angle, 2);
write(fd, &rc_max_angle, 2);
}
To send little-endian data, you can just generate the bytes manually:
int write_le(int fd, int16_t val) {
unsigned char val_le[2] = {
val & 0xff, (uint16_t) val >> 8
};
int nwritten = 0, total = 2;
while (nwritten < total) {
int n = write(fd, val_le + nwritten, total - nwritten);
if (n == -1)
return nwritten > 0 ? nwritten : -1;
nwritten += n;
}
return nwritten;
}
A good compiler will recognize that the code does nothing and compile the bit manipulation to no-op on a little-endian platform. (See e.g. gcc generating the same code for the variant with and without the bit-twiddling.)
Note also that you shouldn't ignore the return value of write() - not only can it encounter an error, it can also write less than you gave it to, in which case you must repeat the write.

How to fprintf an int to binary in C?

I'm trying to write the binary number of 16-bit signed integer to a file. I searched a lot and ofcourse I found many examples which converts integer variables to binary. But in my case these functions will not be efficient, because I need to convert 50e6 samples/s. Calling a function to convert each sample will need a lot of computing time.
So what I want to do is:
int array[] = {233, 431, 1024, ...}
for (i = 0; i < sizeof(array); i++){
fprintf(outfile, "%any_binary_format \n", array[i]);
}
result in the file should be:
0000000011101001
0000000110101111
0000010000000000
fprintf is intended for formatted output - the formatting being "human readable" text, it is therefore not the appropriate function to use if you want binary output. For that you should use fwrite():
for (i = 0; i < sizeof(array) / sizeof(*array); i++ )
{
fwrite (&array[i], sizeof(*array), 1, outfile ) ;
}
Note I have also fixed your loop termination to correctly iterate the number of elements in the array. But in fact the loop is unnecessary - the output is binary, the array is binary - you can just output the entire array thus:
fwrite( array, sizeof(array), 1, outfile ) ;
Your performance requirement of 50Msps will require write performance of around 95Mb/s sustained - that is a lot to ask, and unlikely to be achieved by writing one sample at a time. You may be better off using a memory mapped file, but unless you are using a real-time OS, there are no guarantees that you will sustain that output rate indefinitely - it only takes some other process to access the drive, and it may introduce an unacceptable delay.
Also note that the file must have been opened for binary output - especially on Windows to prevent translation of CR to CR+LF which will be disastrous for your sample data.
If you want to use printf you can use something like this:
#define BYTE_TO_BINARY_PATTERN "%c%c%c%c%c%c%c%c\n"
#define BYTE_TO_BINARY(byte) \
(byte & 0x80 ? '1' : '0'), \
(byte & 0x40 ? '1' : '0'), \
(byte & 0x20 ? '1' : '0'), \
(byte & 0x10 ? '1' : '0'), \
(byte & 0x08 ? '1' : '0'), \
(byte & 0x04 ? '1' : '0'), \
(byte & 0x02 ? '1' : '0'), \
(byte & 0x01 ? '1' : '0')
int main()
{
uint8_t value = 5;
printf(BYTE_TO_BINARY_PATTERN, BYTE_TO_BINARY(value));
return 0;
}
Should print 00000101. I use this sometimes in embedded code when debugging to check register values. Just replace printf with fprintf if you want to write the ascii binary strings to file.
If your compiler supports inline you don't need to worry about the overhead of a small function, take a look at this.
Anyway you can simply implement the function as a macro.
If you want a faster approach you can use a larger buffer (the size for the faster runtime is machine-dependent) for example char str[1 << 16], writing the results to the buffer and using fwrite/write to the out stream.
Another approach is to map the process via mmap/msync.
Anyway you don't need to look at a faster function, but rather a deeper knowledge of the system you're working on.
#define SHORT_WIDTH 16
#define TEST 1
#define PADDING 1 /* set to 0 if you don't need the leading 0s */
char *ShortToBin(unsigned short x, char *buffer) {
#if PADDING
int i;
for(i = 0; i < SHORT_WIDTH; ++i)
buffer[SHORT_WIDTH - i - 1] = '0' + ((x >> i) & 1);
return buffer;
#else
char *ptr = buffer + SHORT_WIDTH;
do {
*(--ptr) = '0' + (x & 1);
x >>= 1;
} while(x);
return ptr;
#endif
}
#if TEST
#include <stdio.h>
int main() {
short n;
char str[SHORT_WIDTH+1]; str[SHORT_WIDTH]='\0';
while(scanf("%hd", &n) == 1)
puts(ShortToBin(n, str));
return 0;
}
#endif

Store C structs for multiple platform use - would this approach work?

Compiler: GNU GCC
Application type: console application
Language: C
Platforms: Win7 and Linux Mint
I wrote a program that I want to run under Win7 and Linux. The program writes C structs to a file and I want to be able to create the file under Win7 and read it back in Linux and vice versa.
By now, I have learned that writing complete structs with fwrite() will give almost 100% assurance that it won't be read back correctly by the other platform. This due to padding and maybe other causes.
I defined all structs myself and they (now, after my previous question on this forum) all have members of type int32_t, int64_t and char. I am thinking about writing a WriteStructname() function for each struct that will write the individual members as int32_t, int64_t and char to the outputfile. Likewise, a ReadStructname() function to read the individual struct members from the file and copy them to an empty struct again.
Would this approach work? I prefer to have maximum control over my sourcecode, so I'm not looking for libraries or other dependencies to achieve this unless I really have to.
Thanks for reading
Element-wise writing of data to a file is your best approach, since structs will differ due to alignment and packing differences between compilers.
However, even with the approach you're planning on using, there are still potential pitfalls, such as different endianness between systems, or different encoding schemes (ie: two's complement versus one's complement encoding of signed numbers).
If you're going to do this, you should consider something like a JSON parser to encode and decode your data so you don't corrupt it due to the issues mentioned above.
Good luck!
If you use GCC or any other compiler that supports "packed" structs, as long you avoid yourself from using anything but [u]intX_t types in the struct, and execute endianness fix in any field where type is bigger than 8 bits, you are platform safe :)
This is an example code where you get portability between platforms, do not forget to manually edit the endianness UIP_BYTE_ORDER.
#include <stdint.h>
#include <stdio.h>
/* These macro are set manually, you should use some automated detection methodology */
#define UIP_BIG_ENDIAN 1
#define UIP_LITTLE_ENDIAN 2
#define UIP_BYTE_ORDER UIP_LITTLE_ENDIAN
/* Borrowed from uIP */
#ifndef UIP_HTONS
# if UIP_BYTE_ORDER == UIP_BIG_ENDIAN
# define UIP_HTONS(n) (n)
# define UIP_HTONL(n) (n)
# define UIP_HTONLL(n) (n)
# else /* UIP_BYTE_ORDER == UIP_BIG_ENDIAN */
# define UIP_HTONS(n) (uint16_t)((((uint16_t) (n)) << 8) | (((uint16_t) (n)) >> 8))
# define UIP_HTONL(n) (((uint32_t)UIP_HTONS(n) << 16) | UIP_HTONS((uint32_t)(n) >> 16))
# define UIP_HTONLL(n) (((uint64_t)UIP_HTONL(n) << 32) | UIP_HTONL((uint64_t)(n) >> 32))
# endif /* UIP_BYTE_ORDER == UIP_BIG_ENDIAN */
#else
#error "UIP_HTONS already defined!"
#endif /* UIP_HTONS */
struct __attribute__((__packed__)) s_test
{
uint32_t a;
uint8_t b;
uint64_t c;
uint16_t d;
int8_t string[13];
};
struct s_test my_data =
{
.a = 0xABCDEF09,
.b = 0xFF,
.c = 0xDEADBEEFDEADBEEF,
.d = 0x9876,
.string = "bla bla bla"
};
void save()
{
FILE * f;
f = fopen("test.bin", "w+");
/* Fix endianness */
my_data.a = UIP_HTONL(my_data.a);
my_data.c = UIP_HTONLL(my_data.c);
my_data.d = UIP_HTONS(my_data.d);
fwrite(&my_data, sizeof(my_data), 1, f);
fclose(f);
}
void read()
{
FILE * f;
f = fopen("test.bin", "r");
fread(&my_data, sizeof(my_data), 1, f);
fclose(f);
/* Fix endianness */
my_data.a = UIP_HTONL(my_data.a);
my_data.c = UIP_HTONLL(my_data.c);
my_data.d = UIP_HTONS(my_data.d);
}
int main(int argc, char ** argv)
{
save();
return 0;
}
Thats the saved file dump:
fanl#fanl-ultrabook:~/workspace-tmp/test3$ hexdump -v -C test.bin
00000000 ab cd ef 09 ff de ad be ef de ad be ef 98 76 62 |..............vb|
00000010 6c 61 20 62 6c 61 20 62 6c 61 00 00 |la bla bla..|
0000001c
This is a good approach. If all fields are integer types of a specific size such as int32_t, int64_t, or char, and you read/write the appropriate number of them to/from arrays, you should be fine.
The one thing you need to watch out for is endianness. Any integer type should be written in a known byte order and read back in the proper byte order for the system in question. The simplest way to do this is with the ntohs and htons functions for 16-bit ints and the ntohl and htonl functions for 32-bit ints. There's no corresponding standard functions for 64-bit ints, but that shouldn't be to difficult to write.
Here's a sample of how you could write these functions for 64 bit:
uint64_t htonll(uint64_t val)
{
uint8_t v[8];
uint64_t *result = (uint64_t *)v;
int i;
for (i=0; i<8; i++) {
v[i] = (uint8_t)(val >> ((7-i) * 8));
}
return *result;
}
uint64_t ntohll(uint64_t val)
{
uint8_t *v = (uint8_t *)&val;
uint64_t result = 0;
int i;
for (i=0; i<8; i++) {
result |= (uint64_t)v[i] << ((7-i) * 8);
}
return result;
}

unionize 32 bit struct with uint32_t and GCC atomic instructions

In a multithreaded program that was written I have some performance problems with very high lock contention.
I have solved this issue by having a few flags within an 32 bit unsigned integer.
currently I just bit shift the values in a temporary variable and then atomically write it.
But I don`t really like to remember the exact amount of bit shifts or where exactly what flag resides.
So I have been wondering if I could just make a union with a uint32_t and the struct with my bitflags with the same size, couldn`t I acces the bitflags by the struct and atomically write it as a uint32_t?
below is the code on how I`d like it to work. It does work but I am unsure on whether this is allowed
#include <stdio.h>
#include <stdlib.h>
#include <inttypes.h>
typedef struct atomic_flags {
unsigned int flags1 : 2;
unsigned int flags2 : 2;
unsigned int flags3 : 2;
unsigned int flags4 : 2;
unsigned int flags5 : 8;
unsigned int reserved : 16;
}atomic_flags;
union data {
atomic_flags i;
uint32_t q;
} data;
int main() {
union data test1;
union data test2;
test1.i.flags1 = 1;
test1.i.flags2 = 2;
test1.i.flags3 = 3;
test1.i.flags4 = 2;
test1.i.flags5 = 241;
test1.i.reserved = 1337;
printf("%u\n", test1.q);
__atomic_store_n(&test2.q, test1.q, __ATOMIC_SEQ_CST);
printf("test1 flags1: %u\n", test1.i.flags1);
printf("test1 flags2: %u\n", test1.i.flags2);
printf("test1 flags3: %u\n", test1.i.flags3);
printf("test1 flags4: %u\n", test1.i.flags4);
printf("test1 flags5: %u\n", test1.i.flags5);
printf("test1 reserved: %u\n", test1.i.reserved);
printf("test2 flags1: %u\n", test2.i.flags1);
printf("test2 flags2: %u\n", test2.i.flags2);
printf("test2 flags3: %u\n", test2.i.flags3);
printf("test2 flags4: %u\n", test2.i.flags4);
printf("test2 flags5: %u\n", test2.i.flags5);
printf("test2 reserved: %u\n", test2.i.reserved);
}
or maybe this is even possible?
__atomic_store_n(&test2.i.flags1, 2, __ATOMIC_SEQ_CST);
It is implementation defined.
If you want to make all the masking and shifting easier and to reduce the likelihood of mistakes, then a sturdier (but uglier) way would be enlist the preprocessor to help you out:
/*
* widths of the bitfields; these values can be changed independently of anything
* else, provided that the total number of bits does not exceed 32.
*/
#define FLAG_flag1_BITS 2
#define FLAG_flag2_BITS 2
#define FLAG_flag3_BITS 2
#define FLAG_flag4_BITS 2
#define FLAG_flag5_BITS 8
/* Macro evaluating to the number of bits in the named flag */
#define FLAG_BITS(flagname) (FLAG_ ## flagname ## _BITS)
/*
* Positions of the flags in the overall bitmask; these adapt to the flag widths
* above, but a new macro (with the same pattern) will be needed if a bitfield
* is added.
*/
#define FLAG_flag1_SHIFT 0
#define FLAG_flag2_SHIFT (FLAG_flag1_SHIFT + FLAG_flag1_BITS)
#define FLAG_flag3_SHIFT (FLAG_flag2_SHIFT + FLAG_flag2_BITS)
#define FLAG_flag4_SHIFT (FLAG_flag3_SHIFT + FLAG_flag3_BITS)
#define FLAG_flag5_SHIFT (FLAG_flag4_SHIFT + FLAG_flag4_BITS)
/* Macro evaluating to the position of the named flag in the overall bitfield */
#define FLAG_SHIFT(flagname) (FLAG_ ## flagname ## _SHIFT)
/* evaluates to a bitmask for selecting the named flag's bits from a bitfield */
#define FLAG_MASK(flagname) \
((~(((uint32_t) 0xffffffff) << FLAG_BITS(flagname))) << FLAG_SHIFT(flagname))
/* evaluates to a bitfield having the specified flag set to the specified value */
#define FLAG(flagname, v) ((v << FLAG_SHIFT(flagname)) & FLAG_MASK(flagname))
/* macro to set the specified flag in the specified bitfield to the specified value */
#define SET_FLAG(flagname, i, v) \
do { i = (i & ~FLAG_MASK(flagname)) | FLAG(flagname, v); } while (0)
/* macro to retrieve the value of the specified flag from the specified bitfield */
#define GET_FLAG(flagname, i) (((i) & FLAG_MASK(flagname)) >> FLAG_SHIFT(flagname))
/* usage example */
int function(uint32_t bitfield) {
uint32_t v;
SET_FLAG(flag2, bitfield, 1);
v = GET_FLAG(flag5, bitfield);
}
Though that involves a prodigous stack of macros, it's mostly driven by the first set, that give the bitfield widths. Substantially all of that will compile down to the same shift and mask operations that you would use anyway, as the computations will be performed mostly by the preprocessor and/or compiler. Actual usage is very simple.

Resources