Working with a program in c that reads in a file and then I have to do both 8 bit and 16 bit checksum for a program.I only have 8 bit checksum done so far.
This is what I understand
I read the file and store the information in an array of characters and at end it takes the newline feed. so for example to calculate 8 bit check sum this is what happens essentially
File has 3 letters total ( 3 a's and a newline feed)
so array holds 4 chars aaa+(newline) (97+97+97+10)
To my understanding I add all the bytes in the array then do % 256 and that is my checksum.
97 * 3 = //3 a's (small a ) pulled from ascii table from what I understand
291 + 10 = 301 // + newline
301 % 256 = cc in hex //
however I am getting confused on how to calculate the 16 bit checksum because I can't add 2 characters at a time if its a single character array?
any help would be greatly appreciated
To calculate a 16-bit checksum, you process the array in increments of 2, and put one byte into the low-order byte of the value that you're adding, and the other byte into the high-order byte.
uint8_t array[MAX]; // The data gets copied into here
size_t length; // This is the length of the data
uint16_t checksum = 0;
size_t even_length = length - length%2; // Round down to multiple of 2
for (int i = 0; i < even_length; i += 2) {
uint16_t val = array[i] + 256 * array[i+1];
checksum += val;
}
if (i < length) { // Last byte if it's odd length
checksum += array[i];
}
There's no need to use modulus, since unsigned integers implement modular arithmetic automatically.
Related
I have an array with integer values from 0-511 (9 bits max). I am trying to write this to a file with fwrite.
For Example, with the array:
[257, 258, 259]
Which is 100000001, 100000010, 100000011
I am trying to write
100000001100000010100000011 + extra padding of 0s to the file
But since fwrite limits writes to 1 byte at a time, I am not sure how to go about doing this. I am new to bitwise operations and am not how to separate out the individual bytes.
You need a bit buffer.
Since you are writing 8 bits at the time, you must
have data type that can hold at least 9+7 bits at minimum. uint16_t would do,
but I recommend using size that would at least as big as your native int. Make sure you use unsigned types to avoid shifting issues.
uint32_t bitBuffer = 0; // Our temporary bit storage
uint32_t count = 0; // Number of bits in buffer
Let's assume we have single data:
uint32_t data9b = 257; // 1 0000 0001
Adding bits to buffer is simple; just shift bits at the end of the buffer,
and combine with OR.
bitBuffer |= (data9b << count); // At first iteration, shift does nothing
count += 9; // Update counter
After 9 bits are added, we can flush 8 bits to file.
while(count >= 8) {
writeToFile(bitBuffer & 0xFF); // Mask out lowest bits with AND
bitBuffer >>= 8; // Remove written bits
count -= 8; // Fix counter
}
After each cycle you have 0 - 7 bits left over in the buffer. At the end of all data, if you finish with non-multiple of 8 bits, just write remaining contents of bitBuffer to file.
if(count > 0)
writeToFile(bitBuffer);
Ok, so did it using bit shifting, oring (can also do with *, '+', % and /) but shift is more appropriate / readable, imo.
// Your data, n is the number of 9-bits values
uint16_t dat[] = { 257, 258, 259 };
int i,n = sizeof(dat)/sizeof(*dat);
// out file
FILE *fp = fopen("f8.bin","w");
uint16_t one = 0;
int shift = 0;
uint8_t b;
// main loop
for(i=0 ; i<n ; i++) {
b = (dat[i] << shift) | one; // one is remainder from previous value
one = dat[i]>>(8-shift); // Move the remaining MSb to the right
shift = (shift+9) % 8; // next shift
fwrite(&b, 1, 1, fp); // write our b byte
}
// remainder, always have a remainder
fwrite(&one, 1, 1, fp);
fclose(fp);
Had fun :-)
If I have a number string (char array), one digit is one char, resulting in that the space for a four digit number is 5 bytes, including the null termination.
unsigned char num[] ="1024";
printf("%d", sizeof(num)); // 5
However, 1024 can be written as
unsigned char binaryNum[2];
binaryNum[0] = 0b00000100;
binaryNum[1] = 0b00000000;
How can the conversion from string to binary be made effectively?
In my program i would work with ≈30 digit numbers, so the space gain would be big.
My goal is to create datapackets to be sent over UDP/TCP.
I would prefer not to use libraries for this task, since the available space the code can take up is small.
EDIT:
Thanks for quick response.
char num = 0b0000 0100 // "4"
--------------------------
char num = 0b0001 1000 // "24"
-----------------------------
char num[2];
num[0] = 0b00000100;
num[1] = 0b00000000;
// num now contains 1024
I would need ≈ 10 bytes to contain my number in binary form. So, if I as suggested parse the digits one by one, starting from the back, how would that build up to the final big binary number?
In general, converting a number in string representation to decimal is easy because each character can be parsed separately. E.g. to convert "1024" to 1024 you can just look at the '4', convert it to 4, multiply by 10, then convert the 2 and add it, multiply by 10, and so on until you have parsed the whole string.
For binary it is not so easy, e.g. you can convert 4 to 100 and 2 to 010 but 42 is not 100 010 or 110 or something like that. So, your best bet is to convert the whole thing to a number and then convert that number to binary using mathematical operations (bit shifts and such). This will work fine for numbers that fit in one of the C++ number types, but if you want to handle arbitrarily large numbers you will need a BigInteger class which seems to be a problem for you since the code has to be small.
From your question I gather that you want to compress the string representation in order to transmit the number over a network, so I am offering a solution that does not strictly convert to binary but will still use fewer bytes than the string representation and is easy to use. It is based on the fact that you can store a number 0..9 in 4 bits, and so you can fit two of those numbers in a byte. Hence you can store an n-digit number in n/2 bytes. The algorithm could be as follows:
Take the last character, '4'
Subtract '0' to get 4 (i.e. an int with value 4).
Strip the last character.
Repeat to get 0
Concatenate into a single byte: digits[0] = (4 << 4) + 0.
Do the same for the next two numbers: digits[1] = (2 << 4) + 1.
Your representation in memory will now look like
4 0 2 1
0100 0000 0010 0001
digits[0] digits[1]
i.e.
digits = { 64, 33 }
This is not quite the binary representation of 1024, but it is shorter and it allows you to easily recover the original number by reversing the algorithm.
You even have 5 values left that you don't use for storing digits (i.e. everything larger than 1010) which you can use for other things like storing the sign, decimal point, byte order or end-of-number delimiter).
I trust that you will be able to implement this, should you choose to use it.
If I understand your question correctly, you would want to do this:
Convert your string representation into an integer.
Convert the integer into binary representation.
For step 1:
You could loop through the string
Subtract '0' from the char
Multiply by 10^n (depending on the position) and add to a sum.
For step 2 (for int x), in general:
x%2 gives you the least-significant-bit (LSB).
x /= 2 "removes" the LSB.
For example, take x = 6.
x%2 = 0 (LSB), x /= 2 -> x becomes 3
x%2 = 1, x /= 2 -> x becomes 1
x%2 = 1 (MSB), x /= 2 -> x becomes 0.
So we we see that (6)decimal == (110)bin.
On to the implementation (for N=2, where N is maximum number of bytes):
int x = 1024;
int n=-1, p=0, p_=0, i=0, ex=1; //you can use smaller types of int for this if you are strict on memory usage
unsigned char num[N] = {0};
for (p=0; p<(N*8); p++,p_++) {
if (p%8 == 0) { n++; p_=0; } //for every 8bits, 1) store the new result in the next element in the array. 2) reset the placing (start at 2^0 again).
for (i=0; i<p_; i++) ex *= 2; //ex = pow(2,p_); without using math.h library
num[n] += ex * (x%2); //add (2^p_ x LSB) to num[n]
x /= 2; // "remove" the last bit to check for the next.
ex = 1; // reset the exponent
}
We can check the result for x = 1024:
for (i=0; i<N; i++)
printf("num[%d] = %d\n", i, num[i]); //num[0] = 0 (0b00000000), num[1] = 4 (0b00000100)
To convert a up-to 30 digit decimal number, represented as a string, into a serious of bytes, effectively a base-256 representation, takes up to 13 bytes. (ceiling of 30/log10(256))
Simple algorithm
dest = 0
for each digit of the string (starting with most significant)
dest *= 10
dest += digit
As C code
#define STR_DEC_TO_BIN_N 13
unsigned char *str_dec_to_bin(unsigned char dest[STR_DEC_TO_BIN_N], const char *src) {
// dest[] = 0
memset(dest, 0, STR_DEC_TO_BIN_N);
// for each digit ...
while (isdigit((unsigned char) *src)) {
// dest[] = 10*dest[] + *src
// with dest[0] as the most significant digit
int sum = *src - '0';
for (int i = STR_DEC_TO_BIN_N - 1; i >= 0; i--) {
sum += dest[i]*10;
dest[i] = sum % 256;
sum /= 256;
}
// If sum is non-zero, it means dest[] overflowed
if (sum) {
return NULL;
}
}
// If stopped on something other than the null character ....
if (*src) {
return NULL;
}
return dest;
}
I'm writing an algorithm that compresses data (LZSS) and it requires me to have two 13-bit values which I'll have to later merge together.
In some cases, however, I don't need 13 bits; 8 are enough.
For this purpose I have a structure like this:
typedef struct pattern
{
char is_compressed:1; //flag
short index :13; //first value
short length :13; //second value
unsigned char c; //is 8 bits are enough, use this instead
} Pattern;
I therefore have an array of these structures, and each structure can either contain the two 13-bit values or an 8-bit value.
I am now looping over this array, and my objective is to merge all these bits together.
I easily calculated the total number of bits used and the number of arrays of unsigned chars (8 bits) needed in order to store all the values:
int compressed = 0, plain = 0;
//count is the amount of patterns i have and p is the array of patterns (the structures)
for (int i = 0; i < count; i++)
{
if (p[i]->is_compressed)
compressed++;
else
plain++;
}
//this stores the number of bits used in the pattern (13 for length and 13 for the index or 8 for the plain uchar)
int tot_bits = compressed * 26 + plain * 8;
//since we can only write a minimum of 8 bits, we calculate how many arrays are needed to store the bits
int nr_of_arrays = (tot_bits % 8 == 0) ? tot_bits / 8 : (tot_bits / 8) + 1;
//we allocate the needed memory for the array of unsigned chars that will contain, concatenated, all the bits
unsigned char* uc = (unsigned char*) malloc(nr_of_arrays * sizeof(unsigned char));
After allocating the memory for the array I'm going to fill, I simply loop through the array of structures and recognize whether the structure I'm looking at contains the two 13-bit values or just the 8-bit one
for (int i = 0; i < count; i++)
{
if (p->is_compressed)
{
//The structure contains the two 13 bits value
}
else
{
//The structure only contains the 8 bits value
}
}
Here I'm stuck and can't seem to figure out a proper way of getting the job done.
Does anybody of you know how to implement that part there?
A practical example would be:
pattern 1 contains the 2 13-bit values:
1111 1111 1111 1
0000 0000 0000 0
pattern 2 contains the 8-bit value
1010 1010
total bits: 34
number of arrays required: 5 (that will waste 6 bits)
resulting array is:
[0] 1111 1111
[1] 1111 1000
[2] 0000 0000
[3] 0010 1010
[4] 1000 0000 (the remaining 6 bits are set to 0)
One way to do that is to write bytes one by one and keep track of partial bytes as you write.
You need a pointer to your char array, and an integer to keep track of how many bits you wrote to the last byte. Every time you write bits, you check how many bits you can write to the last byte, and you write these bits accordingly (ex: if there is 5 bits free, you shift your next value by 3 and add it to the last byte). Every time a byte is complete, you increment your array pointer and reset your bit tracker.
A clean way to implement this would be to write functions like :
void BitWriter_init( char *myArray );
void BitWriter_write( int theBitsToWrite, int howManyBits );
Now you just have to figure out how to implement these functions, or use any other method of your choice.
The problem intrigued me. Here's a possible implementation of "by using a lot of bitwise operations":
/* A writable bit string, with an indicator of the next available bit */
struct bitbuffer {
uint8_t *bytes;
size_t next_bit;
};
/*
* writes the bits represented by the given pattern to the next available
* positions in the specified bit buffer
*/
void write_bits(struct bitbuffer *buffer, Pattern *pattern) {
/* The index of the byte containing the next available bit */
size_t next_byte = buffer->next_bit / 8;
/* the number of bits already used in the next available byte */
unsigned bits_used = buffer->next_bit % 8;
if (pattern->is_compressed) {
/* assemble the bits to write in a 32-bit block */
uint32_t bits = pattern->index << 13 + pattern->length;
if (bits_used == 7) {
/* special case: the bits to write will span 5 bytes */
/* the first bit written will be the last in the current byte */
uint8_t first_bit = bits >> 25;
buffer->bytes[next_byte] |= first_bit;
/* write the next 8 bits to the next byte */
buffer->bytes[++next_byte] = (bits >> 17) & 0xFF;
/* align the tail of the bit block with the buffer*/
bits <<= 7;
} else {
/* the first bits written will fill out the current byte */
uint8_t first_bits = (bits >> (18 + bits_used)) & 0xFF;
buffer->bytes[next_byte] |= first_bits;
/* align the tail of the bit block with the buffer*/
bits <<= (6 - bits_used);
}
/*
* Write the remainder of the bit block to the buffer,
* most-significant bits first. Three (more) bytes will be modified.
*/
buffer->bytes[++next_byte] = (bits >> 16) & 0xFF;
buffer->bytes[++next_byte] = (bits >> 8) & 0xFF;
buffer->bytes[++next_byte] = bits & 0xFF;
/* update the buffer's index of the next available bit */
buffer->next_bit += 26;
} else { /* the pattern is not compressed */
if (bits_used) {
/* the bits to write will span two bytes in the buffer */
buffer->bytes[next_byte] |= (pattern->c >> bits_used);
buffer[++next_byte] = (pattern->c << bits_used) & 0xFF;
} else {
/* the bits to write exactly fill the next buffer byte */
buffer->bytes[next_byte] = pattern->c;
}
/* update the buffer's index of the next available bit */
buffer->next_bit += 8;
}
}
I'm writing some code for an old 8-bit microprocessor (the Hu6280 - a WDC 65C02 derivative in the old NEC PC-Engine console) with 32kb of ram and up to 2.5mbytes of data/code rom. The language is a variant of Small-C but is limited to just the two following basic types:
char (1 byte)
int (2 byte)
It has no struct support and no long int support.
I'm writing a FAT filesystem library to interface with a SD card reader that was primarily developed for loading game ROM images, however an enterprising hacker has written some assembly to allow raw sector reading from the console side. He achieves this by stuffing the 4 8bit values of a 32bit sector address into 4 consecutive memory addresses (char address[4];).
My C code leverages his work to read (for the moment) the dos MBR boot sector and partition type information off the SD card. I've got MBR checksum verifcation and FAT partition detection working.
However, as I need to support FAT32 (which is what the FPGA on the SD card device supports), most of the sector and cluster arithmetic to look up directory entries and files will be based on 32bit LBA sector values.
What easy mechanisms do I have to do add/subtract/multiply 8/16/32bit integers, based on the above limitations? Does anyone have any ready made C routines to handle this? Maybe something along the lines of:
char int1[4], int2[4], int3[4];
int1[0] = 1;
int1[1] = 2;
int1[2] = 3;
int1[3] = 4;
int2[0] = 4;
int2[1] = 3;
int2[2] = 2;
int2[3] = 1;
int3 = mul_32(int1, int2);
int3 = add_32(int1, int2);
int3 = sub_32(int1, int2);`
EDIT: Based on the above replies, this is what I've come up with so far - this is untested as yet and I'll need to do similar for multiplication and subtraction:
char_to_int32(int32_result, int8)
char* int32_result;
char int8;
{
/*
Takes an unsigned 8bit number
and converts to a packed 4 byte array
*/
int32_result[0] = 0x00;
int32_result[1] = 0x00;
int32_result[2] = 0x00;
int32_result[3] = int8;
return 0;
}
int_to_int32(int32_result, int16)
char* int32_result;
int int16;
{
/*
Takes an unsigned 16bit number
and converts to a packed 4 byte array
*/
int32_result[0] = 0x00;
int32_result[1] = 0x00;
int32_result[2] = (int16 >> 8);
int32_result[3] = (int16 & 0xff);
return 0;
}
int32_is_zero(int32)
char* int32;
{
/*
Is a packed 4 byte array == 0
returns 1 if true, otherwise 0
*/
if ((int32[0] == 0) & (int32[1] == 0) & (int32[2] == 0) & (int32[3] == 0)) {
return 1;
} else {
return 0;
}
}
add_32(int32_result, int32_a, int32_b)
char* int32_result;
char* int32_a;
char* int32_b;
{
/*
Takes two 32bit values, stored as 4 bytes each -
adds and stores the result.
Returns 0 on success, 1 on error or overflow.
*/
int sum;
char i;
char carry;
carry = 0x00;
/* loop over each byte of the 4byte array */
for (i = 4; i != 0; i--) {
/* sum the two 1 byte numbers as a 2 byte int */
sum = int32_a[i-1] + int32_b[i-1] + carry;
/* would integer overflow occur with this sum? */
if (sum > 0x00ff) {
/* store the most significant byte for next loop */
carry = (sum >> 8);
} else {
/* no carry needed */
carry = 0x00
}
/* store the least significant byte */
int32_result[i+1] = (sum & 0xff);
}
/* Has overflow occured (ie number > 32bit) */
if (carry != 0) {
return 1;
} else {
return 0;
}
}
EDIT 2: Here's an updated and tested version of the emulated 32bit + 32bit integer add code. It works with all values I've tried so far. Overflow for values bigger than a 32bit unsigned integer is not handled (will not be required for my purposes):
add_int32(int32_result, int32_a, int32_b)
char* int32_result;
char* int32_a;
char* int32_b;
{
/*
Takes two 32bit values, stored as 4 bytes each -
adds and stores the result.
Returns 0 on success, 1 on error or overflow.
*/
int sum;
char i, pos;
char carry;
zero_int32(int32_result);
carry = 0x00;
/* loop over each byte of the 4byte array from lsb to msb */
for (i = 1; i < 5; i++) {
pos = 4 - i;
/* sum the two 1 byte numbers as a 2 byte int */
sum = int32_a[pos] + int32_b[pos] + carry;
/* would integer overflow occur with this sum? */
if (sum > 0x00ff) {
/* store the most significant byte for next loop */
carry = (sum >> 8);
} else {
/* no carry needed */
carry = 0x00;
}
/* store the least significant byte */
int32_result[pos] = (sum & 0x00ff);
}
/* Has overflow occured (ie number > 32bit) */
if (carry != 0) {
return 1;
} else {
return 0;
}
}
I also found some references to 32bit arithmetic on some PIC controllers after searching SO a bit more:
http://web.media.mit.edu/~stefanm/yano/picc_Math32.html
Although there is some PIC assembly inline in their add/subtract code, there are some useful platform agnostic char-based C functions there that have already implemented shifts, comparisons, increment/decrement etc, which will be very useful. I will look into subtract and multiply next - thanks for the info; I guess I was looking at things and thinking they were much harder than they needed to be.
I know you know how to do this. go back to your grade school math...
When you multiply to numbers, base 10
12
x34
====
You do four multiplications right and then add four numbers together right?
4x2 = 8
4x1 = 4
3x2 = 6
3x1 = 3
then
12
x34
====
0008
0040
0060
+0300
======
Now what about addition
12
+34
===
We learned to break that down into two additions
2+4 = 6 carry a zero
1+3+carryin of 0 = 4
With that knowledge that you already have from childhood, you then simply apply that. remember that basic math works whether we have 2 digits operated on 2 digits or 2 million digits operated on 2 million digits.
The above uses single decimal numbers, but the math works if it were single base 16 numbers or single bits or octal or bytes, etc.
Your C compiler should already be handling these things for you but if you need to synthesize them you can, multiplication at the easiest form for digital is to use bits.
addition is easier with bytes using assembly because the carry out is right there, C does not have a carry out so you have to do the exercise of figuring out the carry out using 8 bit math (it can be determined) without needing a 9th bit. or you can just do something less than 8 bit math, 7 or 4 or whatever.
As Joachim pointed out, this topic hsa been beat to death decades/centuries ago. At the same time it is so simple that it often doesnt warrent a lot of discussion. StackOverflow certainly has this topic covered several times over.
I have some code below that is supposed to be converting a C (Arduino) 8-bit byte array to a 16-bit int array, but it only seems to partially work. I'm not sure what I'm doing wrong.
The byte array is in little endian byte order. How do I convert it to an int (two bytes per enty) array?
In layman's terms, I want to merge every two bytes.
Currently it is outputting for an input BYTE ARRAY of: {0x10, 0x00, 0x00, 0x00, 0x30, 0x00}. The output INT ARRAY is: {1,0,0}. The output should be an INT ARRAY is: {1,0,3}.
The code below is what I currently have:
I wrote this function based on a solution in Stack Overflow question Convert bytes in a C array as longs.
I also have this solution based off the same code which works fine for byte array to long (32-bits) array http://pastebin.com/TQzyTU2j.
/**
* Convert the retrieved bytes into a set of 16 bit ints
**/
int * byteA2IntA(byte * byte_slice, int sizeOfB, int * ret_array){
//Variable that stores the addressed int to be stored in SRAM
int currentInt;
int sizeOfI = sizeOfB / 2;
if(sizeOfB % 2 != 0) ++sizeOfI;
for(int i = 0; i < sizeOfB; i+=2){
currentInt = 0;
if(byte_slice[i]=='\0') {
break;
}
if(i + 1 < sizeOfB)
currentInt = (currentInt << 8) + byte_slice[i+1];
currentInt = (currentInt << 8) + byte_slice[i+0];
*ret_array = currentInt;
ret_array++;
}
//Pointer to the return array in the parent scope.
return ret_array;
}
What is the meaning of this line of code?
if(i + 1 < sizeOfB) currentInt = (currentInt << 8) + byte_slice[i+1];
Here currentInt is always 0 and 0 << 8 = 0.
Also what you do is, for each couple of bytes (let me call them uint8_t from now on), you pack an int (let me call it uint16_t from now on) by doing the following:
You take the rightmost uint8_t
You shift it 8 positions to the left
You add the leftmost uint8_t
Is this really what you want?
Supposing you have byte_slice[] = {1, 2}, you pack a 16 bit integer with the value 513 (2<<8 + 1)!
Also, you don't need to return the pointer to the array of uint16_t as the caller has already provided it to the function.
If you use the return of your function, as Joachim said, you get a pointer starting from a position of the uint16_t array which is not position [0].
Vincenzo has a point (or two), you need to be clear what you're trying to do;
Combine two bytes to one 16-bit int, one byte being the MSB and one byte being the LSB
int16 result = (byteMSB << 8) | byteLSB;
Convert an array of bytes into 16-bit
for(i = 0; i < num_of_bytes; i++)
{
myint16array[i] = mybytearray[i];
}
Copy an array of data into another one
memcpy(dest, src, num_bytes);
That will (probably, platform/compiler dependent) have the same effect as my 1st example.
Also, beware of using ints as that suggests signed values, use uints, safer and probably faster.
The problem is most likely that you increase ret_array and then return it. When you return it, it will point to one place beyond the destination array.
Save the pointer at the start of the function, and use that pointer instead.
Consider using a struct. This is kind of a hack, though.
Off the top of my head it would look like this.
struct customINT16 {
byte ByteHigh;
byte ByteLow;
}
So in your case you would write:
struct customINT16 myINT16;
myINT16.ByteHigh = BYTEARRAY[0];
myINT16.ByteLow = BYTEARRAY[1];
You'll have to go through a pointer to cast it, though:
intpointer = (int*)(&myINT16);
INTARRAY[0] = *intpointer;