C / STM32 - Read and copy a data from .wav file - c

I am trying to copy a .wav file from my flash RAM memory.
#define AUDIO_BUFFER_SIZE (1024 * 8) /* Size (in bytes) of the buffer containing the PCM samples */
uint8_t Buffer[AUDIO_BUFFER_SIZE]; // Buffer containig the PCM samples to play
...
/* Fill in the buffer with new data */
if (f_read(&File, (uint8_t *)Buffer, AUDIO_BUFFER_SIZE, &bytesRead) != FR_OK)
{
Error_Handler();
}
if (counter==1){
HAL_GPIO_WritePin(LED_RED_GPIO_Port, LED_RED_Pin, GPIO_PIN_SET);
//uint8_t string[20]="Hello, world!";
//f_write(&OutFile, Buffer, (UINT)sizeof(Buffer),&bytesRead);
for(int i = 0; i <= sizeof(Buffer); i++){
f_printf(&OutFile, "%d\n",Buffer[i]);
osDelay(10);
}
counter++;
}
else{
HAL_GPIO_WritePin(LED_RED_GPIO_Port, LED_RED_Pin, GPIO_PIN_RESET);
f_close(&OutFile);
}
When i do this, I get a file with some values like that (right part of this screenshot)
Output file
How can i get the correct values as we can see them on the left part of my screenshot ?
Regards

Based on the values, your samples looks like being encoded in signed 16-bit little endian format.
To decode the format, you can do like this (assuming that the format specifier of f_printf is just like the standard printf):
// 2 bytes per sample, also use < instead of <=
for(int i = 0; i < sizeof(Buffer); i += 2){
int value = Buffer[i] | (Buffer[i + 1] << 8); // merge the 2 bytes into one integer
if (value >= 0x8000) value -= 0x10000; // because the samples are signed
f_printf(&OutFile, "%.4f\n", value / (double)0x8000); // divide with the maximum value
osDelay(10);
}
If you cannot print floating-point number via f_printf, you can print with rounding by doing:
Multiply 10000 because there are 4 digits after the decimal point
Multiply 2 and add or subtract 0x8000 (the value used for division) based of the sign of the value
Divide by 0x8000 * 2
Print the value before and after the decimal point
// 2 bytes per sample, also use < instead of <=
for(int i = 0; i < sizeof(Buffer); i += 2){
int v;
int value = Buffer[i] | (Buffer[i + 1] << 8); // merge the 2 bytes into one integer
if (value >= 0x8000) value -= 0x10000; // because the samples are signed
// divide with the maximum value
v = ((value * 10000) * 2 + (value >= 0 ? 0x8000 : -0x8000)) / (0x8000 * 2);
f_printf(&OutFile, "%s%d.%04d\n",
v < 0 && v / 10000 == 0 ? "-" : "", // sign (because typical integers don't have -0)
v / 10000, // value before the decimal point
(v < 0 ? -v : v) % 10000); // value after the decimal point
osDelay(10);
}

Related

Converting binary data in bytes to sextets and the reverse in C

I want to convert a buffer of binary data in bytes into a buffer of sextets, where a sextet is a byte with the two most significant bits set to zero. I also want to do the reverse, i.e. convert a buffer of sextets back to bytes. As a test I am generating a buffer in bytes using a pseudo-random number generator that creates numbers between 0 and 255 using the built in version available in C. This is in order to simulate binary data. The details of the pseudo-random number generator and how good it is is of little importance, just that a stream of byte with various values is generated. Eventually a binary file will be read.
I've modified the functions in the link:
How do I base64 encode (decode) in C?
so that instead of encoding bytes to base64 characters, then decoding them back to bytes, sextets are used instead of base64. My encoding functions is as follows:
int bytesToSextets(int inx, int iny, int numBytes, CBYTE* byteData, BYTE* sextetData) {
static int modTable[] = { 0, 2, 1 };
int numSextets = 4 * ((numBytes + 2) / 3);
int i, j;
for (i = inx, j = iny; i < numBytes;) {
BYTE byteA = i < numBytes ? byteData[i++] : 0;
BYTE byteB = i < numBytes ? byteData[i++] : 0;
BYTE byteC = i < numBytes ? byteData[i++] : 0;
UINT triple = (byteA << 0x10) + (byteB << 0x08) + byteC;
sextetData[j++] = (triple >> 18) & 0x3F;
sextetData[j++] = (triple >> 12) & 0x3F;
sextetData[j++] = (triple >> 6) & 0x3F;
sextetData[j++] = triple & 0x3F;
}
for (int i = 0; i < modTable[numBytes % 3]; i++) {
sextetData[numSextets - 1 - i] = 0;
}
return j - iny;
}
where inx is the index in the input byte buffer where I want to start encoding, iny is the index in the output sextet buffer where the beginning of the sextets are written to, numBytes is the number of bytes to be encoded, and *byteData, *sextetData are the respective buffers to read from and write to. The last for-loop sets elements of sextetData to zero, not to '=' as given in the original code when there is padding. Although zero bytes can be valid data, as the length of the buffers are known in advance, I presume this is not a problem. The function returns with the number of sextets written, which can be checked against 4 * ((numBytes + 2) / 3). The first few sextets of the output buffer encode the number of bytes of data encodes in the rest of the buffer, with the number of sextets given in the formula.
The code for decoding sextets back to bytes is as follows:
int sextetsToBytes(int inx, int iny, int numBytes, CBYTE* sextetData, BYTE* byteData) {
int numSextets = 4 * ((numBytes + 2) / 3);
int padding = 0;
if (sextetData[numSextets - 1 + inx] == 0) padding++;
if (sextetData[numSextets - 2 + inx] == 0) padding++;
int i, j;
for (i = inx, j = iny; i < numSextets + inx;) {
UINT sextetA = sextetData[i++];
UINT sextetB = sextetData[i++];
UINT sextetC = sextetData[i++];
UINT sextetD = sextetData[i++];
UINT triple = (sextetA << 18) + (sextetB << 12) + (sextetC << 6) + sextetD;
if (j < numBytes) byteData[j++] = (triple >> 16) & 0xFF;
if (j < numBytes) byteData[j++] = (triple >> 8) & 0xFF;
if (j < numBytes) byteData[j++] = triple & 0xFF;
}
return j - iny - padding;
}
where as before inx and iny are the indices to start reading from and writing to a buffer, numBytes is the number of bytes that will be in the output buffer, from which the number of input sextets are calculated. The length of the input buffer is found from the first few sextets written by bytesToSextets(), so inx is the position in the input sextet buffer to start the actual conversion back to bytes. In the original function the number of sextets is given, from which the number of bytes is calculated using numSextets / 4 * 3. As this is already known, this is not done and should not make a difference. The last two arguments *sextetData and *byteData are the respectively input and output buffers.
An input buffer in bytes is created, converted to sextets, then as a test converted back to bytes. A comparison is made between the generated initial buffer of bytes and the output buffer in bytes after converting back from the intermediate sextet buffer. When the length of the input buffer is a multiple of 3, the match is perfect and the final output buffer is exactly the same. However, if the number of bytes in the initial buffer is not a multiple of 3, the last 3 bytes in the final output buffer may not match the original bytes. This has obviously something to do with the padding when the number of bytes is not a multiple of 3, but I am unable to find the source of the problem. Incidentally, the return values from the two functions are always correct, even when the last few bytes do not match.
In a header file I have the following typedefs:
typedef unsigned char BYTE;
typedef const unsigned char CBYTE;
typedef unsigned int UINT;
Although the main function is more complicated, in its simplest version it would have a form like:
// Allocate memory for bufA and bufB.
// Write the data length and other information into sextets 0 to 4 in bufB.
// Convert the bytes in bufA starting at index 0 to sextets in bufB starting at index 5.
int countSextets = bytesToSextets(0, 5, lenBufA, bufA, bufB);
// Allocate memory for bufC.
// Convert the sextets in bufB starting at index 5 back to bytes in bufC starting at index 0.
int countBytes = sextetsToBytes(5, 0, lenBufC, bufB, bufC);
As I said, this all works correctly, except that when the lenBufA is not a multiple of 3, the last 3 recovered bytes in bufC do not match those in bufA, but the calculated buffer lengths are all correct.
Perhaps someone can kindly help throw some light on this.
sextetData[numSextets - 1 - i] = 0; should be sextetData[iny + numSextets - 1 - i] = 0;.
The version of sextetsToBytes() I originally posted had the problem that I tested for padding by using:
if (sextetData[numSextets - 1 + inx] == 0) padding++;
if (sextetData[numSextets - 2 + inx] == 0) padding++;
as of course testing for '=' for base64 cannot be used, however, testing for zero can still cause problems, as zero can be a valid data item. This indeed sometimes caused a difference between the specified number of output bytes and the number found by counting up the bytes in the loop and subtracting the padding bytes. By just removing the padding bytes from the function, then checking the counted number returned against the specified input value numBytes, works. The modified code is as follows:
int sextetsToBytes(int numBytes, CBYTE* sextetData, BYTE* byteData) {
int numSextets = 4 * ((numBytes + 2) / 3);
int i, j;
for (i = 0, j = 0; i < numSextets;) {
UINT sextetA = sextetData[i++];
UINT sextetB = sextetData[i++];
UINT sextetC = sextetData[i++];
UINT sextetD = sextetData[i++];
UINT triple = (sextetA << 18) + (sextetB << 12) + (sextetC << 6) + sextetD;
if (j < numBytes) byteData[j++] = (triple >> 16) & 0xFF;
if (j < numBytes) byteData[j++] = (triple >> 8) & 0xFF;
if (j < numBytes) byteData[j++] = triple & 0xFF;
}
return j;
}

c, obtaining a special random number

I have a algorithm problem that I need to speed up :)
I need a 32bit random number, with exact 10 bits set to 1. But in the same time, patterns like 101 (5 dec) and 11 (3 dec) to be considered illegal.
Now the MCU is a 8051 (8 bit) and I tested all this in Keil uVision. My first attempt completes, giving the solution
0x48891249
1001000100010010001001001001001 // correct, 10 bits 1, no 101 or 11
The problem is that it completes in 97 Seconds or 1165570706 CPU cycles which is ridiculous!!!
Here is my code
// returns 1 if number is not good. ie. contains at leats one 101 bit sequence
bool checkFive(unsigned long num)
{
unsigned char tmp;
do {
tmp = (unsigned char)num;
if(
(tmp & 7) == 5
|| (tmp & 3) == 3
) // illegal pattern 11 or 101
return true; // found
num >>= 1;
}while(num);
return false;
}
void main(void) {
unsigned long v,num; // count the number of bits set in v
unsigned long c; // c accumulates the total bits set in v
do {
num = (unsigned long)rand() << 16 | rand();
v = num;
// count all 1 bits, Kernigen style
for (c = 0; v; c++)
v &= v - 1; // clear the least significant bit set
}while(c != 10 || checkFive(num));
while(1);
}
The big question for a brilliant mind :)
Can be done faster? Seems that my approach is naive.
Thank you in advance,
Wow, I'm impressed, thanks all for suggestions. However, before accept, I need to test them these days.
Now with the first option (look-up) it's just not realistic, will complete blow my 4K RAM of entire 8051 micro controller :) As you can see in image bellow, I tested for all combinations in Code Blocks but there are way more than 300 and it's not finished yet until 5000 index...
The code I use to test
#include <stdio.h>
#include <stdint.h>
#include <stdlib.h>
#include <stdbool.h>
//#define bool bit
//#define true 1
//#define false 0
// returns 1 if number is not good. ie. contains at leats one 101 bit sequence
bool checkFive(uint32_t num)
{
uint8_t tmp;
do {
tmp = (unsigned char)num;
if(
(tmp & 7) == 5
|| (tmp & 3) == 3
) // illegal pattern 11 or 101
return true; // found
num >>= 1;
}while(num);
return false;
}
void main(void) {
uint32_t v,num; // count the number of bits set in v
uint32_t c, count=0; // c accumulates the total bits set in v
//printf("Program started \n");
num = 0;
printf("Program started \n");
for(num=0; num <= 0xFFFFFFFF; num++)
{
//do {
//num = (uint32_t)rand() << 16 | rand();
v = num;
// count all 1 bits, Kernigen style
for (c = 0; v; c++)
v &= v - 1; // clear the least significant bit set
//}while(c != 10 || checkFive(num));
if(c != 10 || checkFive(num))
continue;
count++;
printf("%d: %04X\n", count, num);
}
printf("Complete \n");
while(1);
}
Perhaps I can re-formulate the problem:
I need a number with:
precise (known) amount of 1 bits, 10 in my example
not having 11 or 101 patterns
remaining zeroes can be any
So somehow, shuffle only the 1 bits inside.
Or, take a 0x00000000 and add just 10 of 1 bits in random positions, except the illegal patterns.
Solution
Given a routine r(n) that returns a random integer from 0 (inclusive) to n (exclusive) with uniform distribution, the values described in the question may be generated with a uniform distribution by calls to P(10, 4) where P is:
static uint32_t P(int a, int b)
{
if (a == 0 && b == 0)
return 0;
else
return r(a+b) < a ? P(a-1, b) << 3 | 1 : P(a, b-1) << 1;
}
The required random number generator can be:
static int r(int a)
{
int q;
do
q = rand() / ((RAND_MAX+1u)/a);
while (a <= q);
return q;
}
(The purpose of dividing by (RAND_MAX+1u)/a and the do-while loop is to trim the range of rand to an even multiple of a so that bias due to a non-multiple range is eliminated.)
(The recursion in P may be converted to iteration. This is omitted as it is unnecessary to illustrate the algorithm.)
Discussion
If the number cannot contain consecutive bits 11 or 101, then the closest together two 1 bits can be is three bits apart, as in 1001. Fitting ten 1 bits in 32 bits then requires at least 28 bits, as in 1001001001001001001001001001. Therefore, to satisfy the constraints that there is no 11 or 101 and there are exactly 10 1 bits, the value must be 1001001001001001001001001001 with four 0 bits inserted in some positions (including possibly the beginning or the end).
Selecting such a value is equivalent to placing 10 instances of 001 and 4 instances of 0 in some order.1 There are 14! ways of ordering 14 items, but any of the 10! ways of rearranging the 10 001 instances with each other are identical, and any of the 4! ways of rearranging the 0 instances with each other are identical, so the number of distinct selections is 14! / 10! / 4!, also known as the number of combinations of selecting 10 things from 14. This is 1,001.
To perform such a selection with uniform distribution, we can use a recursive algorithm:
Select the first choice with probability distribution equal to the proportion of the choices in the possible orderings.
Select the remaining choices recursively.
When ordering a instances of one object and b of a second object, a/(a+b) of the potential orderings will start with the first object, and b/(a+b) will start with the second object. Thus, the design of the P routine is:
If there are no objects to put in order, return the empty bit string.
Select a random integer in [0, a+b). If it is less than a (which has probability a/(a+b)), insert the bit string 001 and then recurse to select an order for a-1 instances of 001 and b instances of 0.
Otherwise, insert the bit string 0 and then recurse to select an order for a instances of 001 and b-1 instances of 0.
(Since, once a is zero, only 0 instances are generated, if (a == 0 && b == 0) in P may be changed to if (a == 0). I left it in the former form as that shows the general form of a solution in case other strings are involved.)
Bonus
Here is a program to list all values (although not in ascending order).
#include <stdint.h>
#include <stdio.h>
static void P(uint32_t x, int a, int b)
{
if (a == 0 && b == 0)
printf("0x%x\n", x);
else
{
if (0 < a) P(x << 3 | 1, a-1, b);
if (0 < b) P(x << 1, a, b-1);
}
}
int main(void)
{
P(0, 10, 4);
}
Footnote
1 This formulation means we end up with a string starting 001… rather than 1…, but the resulting value, interpreted as binary, is equivalent, even if there are instances of 0 inserted ahead of it. So the strings with 10 001 and 4 0 are in one-to-one correspondence with the strings with 4 0 inserted into 1001001001001001001001001001.
One way to satisfy your criteria in a limited number of solutions is to utilize the fact that there can be no more that four groups of 000s within the bit population. This also means that there can one be one group of 0000 in the value. Knowing this, you can seed your value with a single 1 in bits 27-31 and then continue adding random bits checking that each bit added satisfies your 3 or 5 constraints.
When adding random bits to your value and satisfying your constraints, there can always be combinations that lead to a solution that can never satisfy all constraints. To protect against those cases, just keep an iteration count and reset/restart the value generation if iterations exceed that value. Here, if a solution is going to be found, it will be found in less than 100 iterations. And is generally found in 1-8 attempts. Meaning for each value you generate, you have on average no more than 800 iterations which will be a far cry less than "97 Seconds or 1165570706 CPU cycles" (I haven't counted cycles, but the return is almost instantaneous)
There are many ways to approach this problem, this is just one that worked in a reasonable amount of time:
#include <stdio.h>
#include <stdlib.h>
#include <time.h>
#include <limits.h>
#define BPOP 10
#define NBITS 32
#define LIMIT 100
/** rand_int for use with shuffle */
static int rand_int (int n)
{
int limit = RAND_MAX - RAND_MAX % n, rnd;
rnd = rand();
for (; rnd >= limit; )
rnd = rand();
return rnd % n;
}
int main (void) {
int pop = 0;
unsigned v = 0, n = NBITS;
size_t its = 1;
srand (time (NULL));
/* one of first 5 bits must be set */
v |= 1u << (NBITS - 1 - rand_int (sizeof v + 1));
pop++; /* increment pop count */
while (pop < BPOP) { /* loop until pop count 10 */
if (++its >= LIMIT) { /* check iterations */
#ifdef DEBUG
fprintf (stderr, "failed solution.\n");
#endif
pop = its = 1; /* reset for next iteration */
v = 0;
v |= 1u << (NBITS - 1 - rand_int (sizeof v + 1));
}
unsigned shift = rand_int (NBITS); /* get random shift */
if (v & (1u << shift)) /* if bit already set */
continue;
/* protect against 5 (101) */
if ((shift + 2) < NBITS && v & (1u << (shift + 2)))
continue;
if ((int)(shift - 2) >= 0 && v & (1u << (shift - 2)))
continue;
/* protect against 3 (11) */
if ((shift + 1) < NBITS && v & (1u << (shift + 1)))
continue;
if ((int)(shift - 1) >= 0 && v & (1u << (shift - 1)))
continue;
v |= 1u << shift; /* add bit at shift */
pop++; /* increment pop count */
}
printf ("\nv : 0x%08x\n", v); /* output value */
while (n--) { /* output binary confirmation */
if (n+1 < NBITS && (n+1) % 4 == 0)
putchar ('-');
putchar ((v >> n & 1) ? '1' : '0');
}
putchar ('\n');
#ifdef DEBUG
printf ("\nits: %zu\n", its);
#endif
return 0;
}
(note: you will probably want a better random source like getrandom() or reading from /dev/urandom if you intend to generate multiple random solutions within a loop -- expecially if you are calling the executable in a loop from your shell)
I have also included a DEBUG define that you can enable by adding the -DDEBUG option to your compiler string to see the number of failed solutions and number of iterations on the final.
Example Use/Output
The results for 8 successive runs:
$ ./bin/randbits
v : 0x49124889
0100-1001-0001-0010-0100-1000-1000-1001
v : 0x49124492
0100-1001-0001-0010-0100-0100-1001-0010
v : 0x48492449
0100-1000-0100-1001-0010-0100-0100-1001
v : 0x91249092
1001-0001-0010-0100-1001-0000-1001-0010
v : 0x92488921
1001-0010-0100-1000-1000-1001-0010-0001
v : 0x89092489
1000-1001-0000-1001-0010-0100-1000-1001
v : 0x82491249
1000-0010-0100-1001-0001-0010-0100-1001
v : 0x92448922
1001-0010-0100-0100-1000-1001-0010-0010
As Eric mentioned in his answer, since each 1 but must be separated by at least two 0 bits, you basically start with the 28-bit pattern 1001001001001001001001001001. It's then a matter of placing the remaining four 0 bits within this bit pattern, and there are 11 distinct places to insert each zero.
This can be accomplished by first selecting a random number from 1 to 11 to determine where to place a bit. Then you left shift all the bits above the target bit by 1. Repeat 3 more times, and you have your value.
This can be done as follows:
#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>
#include <time.h>
void binprint(uint32_t n)
{
int i;
for (i=0;i<32;i++) {
if ( n & (1u << (31 - i))) {
putchar('1');
} else {
putchar('0');
}
}
}
// inserts a 0 bit into val after pos "1" bits are found
uint32_t insert(uint32_t val, int pos)
{
int cnt = 0;
uint32_t mask = 1u << 31;
uint32_t upper, lower;
while (cnt < pos) {
if (val & mask) { // look for a set bit and count if you find one
cnt++;
}
mask >>= 1;
}
if (mask == (1u << 31)) {
return val; // insert at the start: no change
} else if (mask == 0) {
return val << 1; // insert at the end: shift the whole thing by 1
} else {
mask = (mask << 1) - 1; // mask has all bits below the target set
lower = val & mask; // extract the lower portion
upper = val & (~mask); // extract the upper portion
return (upper << 1) | lower; // recombine with the upper portion shifted 1 bit
}
}
int main()
{
int i;
uint32_t val = 01111111111; // hey look, a good use of octal!
srand(time(NULL));
for (i=0;i<4;i++) {
int p = rand() % 11;
printf("p=%d\n", p);
val = insert(val, p);
}
binprint(val);
printf("\n");
return 0;
}
Sample output for two runs:
p=3
p=10
p=9
p=0
01001001000100100100100100100010
...
p=3
p=9
p=3
p=1
10001001000010010010010010010001
Run time is negligible.
Since you don't want a lookup table here is the way:
Basically you have this number with 28 bits set to 0 and 1 in which you need to insert 4x 0 :
0b1001001001001001001001001001
Hence you can use the following algorithm:
int special_rng_nolookup(void)
{
int secret = 0b1001001001001001001001001001;
int low_secret;
int high_secret;
unsigned int i = 28; // len of secret
unsigned int rng;
int mask = 0xffff // equivalent to all bits set in integer
while (i < 32)
{
rng = __asm__ volatile(. // Pseudo code
"rdrand"
);
rng %= (i + 1); // will generate a number between 0 and 28 where you will add a 0. Then between 0 and 29, 30, 31 for the 3 next loop.
low_secret = secret & (mask >> (i - rng)); // locate where you will add your 0 and save the lower part of your number.
high_secret = (secret ^ low_secret) << (!(!rng)); // remove the lower part to your int and shift to insert a 0 between the higher part and the lower part. edit : if rng was 0 you want to add it at the very beginning (left part) so no shift.
secret = high_secret | low_secret; // put them together.
++i;
}
return secret;
}

Efficient algorithm for finding a byte in a bit array

Given a bytearray uint8_t data[N] what is an efficient method to find a byte uint8_t search within it even if search is not octet aligned? i.e. the first three bits of search could be in data[i] and the next 5 bits in data[i+1].
My current method involves creating a bool get_bit(const uint8_t* src, struct internal_state* state) function (struct internal_state contains a mask that is bitshifted right, &ed with src and returned, maintaining size_t src_index < size_t src_len) , leftshifting the returned bits into a uint8_t my_register and comparing it with search every time, and using state->src_index and state->src_mask to get the position of the matched byte.
Is there a better method for this?
If you're searching an eight bit pattern within a large array you can implement a sliding window over 16 bit values to check if the searched pattern is part of the two bytes forming that 16 bit value.
To be portable you have to take care of endianness issues which is done by my implementation by building the 16 bit value to search for the pattern manually. The high byte is always the currently iterated byte and the low byte is the following byte. If you do a simple conversion like value = *((unsigned short *)pData) you will run into trouble on x86 processors...
Once value, cmp and mask are setup cmp and mask are shifted. If the pattern was not found within hi high byte the loop continues by checking the next byte as start byte.
Here is my implementation including some debug printouts (the function returns the bit position or -1 if pattern was not found):
int findPattern(unsigned char *data, int size, unsigned char pattern)
{
int result = -1;
unsigned char *pData;
unsigned char *pEnd;
unsigned short value;
unsigned short mask;
unsigned short cmp;
int tmpResult;
if ((data != NULL) && (size > 0))
{
pData = data;
pEnd = data + size;
while ((pData < pEnd) && (result == -1))
{
printf("\n\npData = {%02x, %02x, ...};\n", pData[0], pData[1]);
if ((pData + 1) < pEnd) /* still at least two bytes to check? */
{
tmpResult = (int)(pData - data) * 8; /* calculate bit offset according to current byte */
/* avoid endianness troubles by "manually" building value! */
value = *pData << 8;
pData++;
value += *pData;
/* create a sliding window to check if search patter is within value */
cmp = pattern << 8;
mask = 0xFF00;
while (mask > 0x00FF) /* the low byte is checked within next iteration! */
{
printf("cmp = %04x, mask = %04x, tmpResult = %d\n", cmp, mask, tmpResult);
if ((value & mask) == cmp)
{
result = tmpResult;
break;
}
tmpResult++; /* count bits! */
mask >>= 1;
cmp >>= 1;
}
}
else
{
/* only one chance left if there is only one byte left to check! */
if (*pData == pattern)
{
result = (int)(pData - data) * 8;
}
pData++;
}
}
}
return (result);
}
I don't think you can do much better than this in C:
/*
* Searches for the 8-bit pattern represented by 'needle' in the bit array
* represented by 'haystack'.
*
* Returns the index *in bits* of the first appearance of 'needle', or
* -1 if 'needle' is not found.
*/
int search(uint8_t needle, int num_bytes, uint8_t haystack[num_bytes]) {
if (num_bytes > 0) {
uint16_t window = haystack[0];
if (window == needle) return 0;
for (int i = 1; i < num_bytes; i += 1) {
window = window << 8 + haystack[i];
/* Candidate for unrolling: */
for (int j = 7; j >= 0; j -= 1) {
if ((window >> j) & 0xff == needle) {
return 8 * i - j;
}
}
}
}
return -1;
}
The main idea is to handle the 87.5% of cases that cross the boundary between consecutive bytes by pairing bytes in a wider data type (uint16_t in this case). You could adjust it to use an even wider data type, but I'm not sure that would gain anything.
What you cannot safely or easily do is anything involving casting part or all of your array to a wider integer type via a pointer (i.e. (uint16_t *)&haystack[i]). You cannot be ensured of proper alignment for such a cast, nor of the byte order with which the result might be interpreted.
I don't know if it would be better, but i would use sliding window.
uint counter = 0, feeder = 8;
uint window = data[0];
while (search ^ (window & 0xff)){
window >>= 1;
feeder--;
if (feeder < 8){
counter++;
if (counter >= data.length) {
feeder = 0;
break;
}
window |= data[counter] << feeder;
feeder += 8;
}
}
//Returns index of first bit of first sequence occurrence or -1 if sequence is not found
return (feeder > 0) ? (counter+1)*8-feeder : -1;
Also with some alterations you can use this method to search for arbitrary length (1 to 64-array_element_size_in_bits) bits sequence.
If AVX2 is acceptable (with earlier versions it didn't work out so well, but you can still do something there), you can search in a lot of places at the same time. I couldn't test this on my machine (only compile) so the following is more to give to you an idea of how it could be approached than copy&paste code, so I'll try to explain it rather than just code-dump.
The main idea is to read an uint64_t, shift it right by all values that make sense (0 through 7), then for each of those 8 new uint64_t's, test whether the byte is in there. Small complication: for the uint64_t's shifted by more than 0, the highest position should not be counted since it has zeroes shifted into it that might not be in the actual data. Once this is done, the next uint64_t should be read at an offset of 7 from the current one, otherwise there is a boundary that is not checked across. That's fine though, unaligned loads aren't so bad anymore, especially if they're not wide.
So now for some (untested, and incomplete, see below) code,
__m256i needle = _mm256_set1_epi8(find);
size_t i;
for (i = 0; i < n - 6; i += 7) {
// unaligned load here, but that's OK
uint64_t d = *(uint64_t*)(data + i);
__m256i x = _mm256_set1_epi64x(d);
__m256i low = _mm256_srlv_epi64(x, _mm256_set_epi64x(3, 2, 1, 0));
__m256i high = _mm256_srlv_epi64(x, _mm256_set_epi64x(7, 6, 5, 4));
low = _mm256_cmpeq_epi8(low, needle);
high = _mm256_cmpeq_epi8(high, needle);
// in the qword right-shifted by 0, all positions are valid
// otherwise, the top position corresponds to an incomplete byte
uint32_t lowmask = 0x7f7f7fffu & _mm256_movemask_epi8(low);
uint32_t highmask = 0x7f7f7f7fu & _mm256_movemask_epi8(high);
uint64_t mask = lowmask | ((uint64_t)highmask << 32);
if (mask) {
int bitindex = __builtin_ffsl(mask);
// the bit-index and byte-index are swapped
return 8 * (i + (bitindex & 7)) + (bitindex >> 3);
}
}
The funny "bit-index and byte-index are swapped" thing is because searching within a qword is done byte by byte and the results of those comparisons end up in 8 adjacent bits, while the search for "shifted by 1" ends up in the next 8 bits and so on. So in the resulting masks, the index of the byte that contains the 1 is a bit-offset, but the bit-index within that byte is actually the byte-offset, for example 0x8000 would correspond to finding the byte at the 7th byte of the qword that was right-shifted by 1, so the actual index is 8*7+1.
There is also the issue of the "tail", the part of the data left over when all blocks of 7 bytes have been processed. It can be done much the same way, but now more positions contain bogus bytes. Now n - i bytes are left over, so the mask has to have n - i bits set in the lowest byte, and one fewer for all other bytes (for the same reason as earlier, the other positions have zeroes shifted in). Also, if there is exactly 1 byte "left", it isn't really left because it would have been tested already, but that doesn't really matter. I'll assume the data is sufficiently padded that accessing out of bounds doesn't matter. Here it is, untested:
if (i < n - 1) {
// make n-i-1 bits, then copy them to every byte
uint32_t validh = ((1u << (n - i - 1)) - 1) * 0x01010101;
// the lowest position has an extra valid bit, set lowest zero
uint32_t validl = (validh + 1) | validh;
uint64_t d = *(uint64_t*)(data + i);
__m256i x = _mm256_set1_epi64x(d);
__m256i low = _mm256_srlv_epi64(x, _mm256_set_epi64x(3, 2, 1, 0));
__m256i high = _mm256_srlv_epi64(x, _mm256_set_epi64x(7, 6, 5, 4));
low = _mm256_cmpeq_epi8(low, needle);
high = _mm256_cmpeq_epi8(high, needle);
uint32_t lowmask = validl & _mm256_movemask_epi8(low);
uint32_t highmask = validh & _mm256_movemask_epi8(high);
uint64_t mask = lowmask | ((uint64_t)highmask << 32);
if (mask) {
int bitindex = __builtin_ffsl(mask);
return 8 * (i + (bitindex & 7)) + (bitindex >> 3);
}
}
If you are searching a large amount of memory and can afford an expensive setup, another approach is to use a 64K lookup table. For each possible 16-bit value, the table stores a byte containing the bit shift offset at which the matching octet occurs (+1, so 0 can indicate no match). You can initialize it like this:
uint8_t* g_pLookupTable = malloc(65536);
void initLUT(uint8_t octet)
{
memset(g_pLookupTable, 0, 65536); // zero out
for(int i = 0; i < 65536; i++)
{
for(int j = 7; j >= 0; j--)
{
if(((i >> j) & 255) == octet)
{
g_pLookupTable[i] = j + 1;
break;
}
}
}
}
Note that the case where the value is shifted 8 bits is not included (the reason will be obvious in a minute).
Then you can scan through your array of bytes like this:
int findByteMatch(uint8_t* pArray, uint8_t octet, int length)
{
if(length >= 0)
{
uint16_t index = (uint16_t)pArray[0];
if(index == octet)
return 0;
for(int bit, i = 1; i < length; i++)
{
index = (index << 8) | pArray[i];
if(bit = g_pLookupTable[index])
return (i * 8) - (bit - 1);
}
}
return -1;
}
Further optimization:
Read 32 or however many bits at a time from pArray into a uint32_t and then shift and AND each to get byte one at a time, OR with index and test, before reading another 4.
Pack the LUT into 32K by storing a nybble for each index. This might help it squeeze into the cache on some systems.
It will depend on your memory architecture whether this is faster than an unrolled loop that doesn't use a lookup table.

How to represent binary data in 8 bits in C

#include<stdio.h>
int main()
{
long int decimalNumber,remainder,quotient;
int binaryNumber[100],i=1,j;
printf("Enter any decimal number: ");
scanf("%ld",&decimalNumber);
quotient = decimalNumber;
while(quotient!=0)
{
binaryNumber[i++]= quotient % 2;
quotient = quotient / 2;
}
printf("Equivalent binary value of decimal number %d: ",decimalNumber);
for(j = i -1 ;j> 0;j--)
printf("%d",binaryNumber[j]);
return 0;
}
I want the output in 8 bit binary form, but the result as shown below, is there any operator in C which can convert 7 bit data to its equivalent 8 bit data? thank you
Sample output:
Enter any decimal number: 50
Equivalent binary value of decimal number 50: 110010
Required output is 00110010 which is 8 bit, how to append a zero in MSB position?
A very convenient way it so have a function return a binary representation in the form of a string. This allows the binary representation to be used within a normal printf format string rather than having the bits spit out at the current cursor position. To specify the exact number of digits, you must pad the binary string to the required number of places (e.g. 8, 16, 32...). The following makes use of a static variable to allow the return of the buffer, but the same can easily be implemented by allocating space for the buffer with dynamically. The preprocessor checks are not required as you can simply hardwire the length of the buffer to 64 + 1, but for the sake of completeness a check for x86/x86_64 is included and BITS_PER_LONG is set accordingly.
#include <stdio.h>
#if defined(__LP64__) || defined(_LP64)
# define BUILD_64 1
#endif
#ifdef BUILD_64
# define BITS_PER_LONG 64
#else
# define BITS_PER_LONG 32
#endif
char *binstr (unsigned long n, size_t sz);
int main (void) {
printf ("\n 50 (decimal) : %s (binary)\n\n", binstr (50, 8));
return 0;
}
/* returns pointer to binary representation of 'n' zero padded to 'sz'. */
char *binstr (unsigned long n, size_t sz)
{
static char s[BITS_PER_LONG + 1] = {0};
char *p = s + BITS_PER_LONG;
register size_t i;
if (!n) {
*s = '0';
return s;
}
for (i = 0; i < sz; i++)
*(--p) = (n>>i & 1) ? '1' : '0';
return p;
}
Output
$ ./bin/bincnv
50 (decimal) : 00110010 (binary)
Note: you cannot make repeated calls in the same printf statement due to the static buffer. If you allocate dynamically, you can call the function as many times as you like in the same printf statement.
Also, note, if you do not care about padding the binary return to any specific length and just want the binary representation to start with the most significant bit, the following simpler version can be used:
/* simple return of binary string */
char *binstr (unsigned long n)
{
static char s[BITS_PER_LONG + 1] = {0};
char *p = s + BITS_PER_LONG;
if (!n) {
*s = '0';
return s;
}
while (n) {
*(--p) = (n & 1) ? '1' : '0';
n >>= 1;
}
return p;
}
Modify your code as shown below:
quotient = quotient / 2;
}
/* ---- Add the following code ---- */
{
int group_size = 8; /* Or CHAR_BIT */
int padding = group_size - ((i-1) % group_size); /* i was inited with 1 */
if(padding != group_size) {
/* Add padding */
while(padding-- != 0) binaryNumber[i++] = 0;
}
}
/* ------- Modification ends -------- */
printf("Equivalent binary value of decimal number %d: ",decimalNumber);
This code calculates the number of padding bits required to print the number and fills the padding bits with 0.
Live demo here
If you want 7 bit answer, change group_size to 7.
Use this for printing your result:
for(j = 7; j>0; j--)
printf("%d", binaryNumber[j]);
This always prints 8 binary digits.
Edit
The int array binaryNumber must be initialized with zeros to make this work:
for(int i=0; i<8; i++) binaryNumber[i] = 0;

C - Serialization of the floating point numbers (floats, doubles)

How to convert a floating point number into a sequence of bytes so that it can be persisted in a file? Such algorithm must be fast and highly portable. It must allow also the opposite operation, deserialization. It would be nice if only very tiny excess of bits per value (persistent space) is required.
Assuming you're using mainstream compilers, floating point values in C and C++ obey the IEEE standard and when written in binary form to a file can be recovered in any other platform, provided that you write and read using the same byte endianess. So my suggestion is: pick an endianess of choice, and before writing or after reading, check if that endianess is the same as in the current platform; if not, just swap the bytes.
This might give you a good start - it packs a floating point value into an int and long long pair, which you can then serialise in the usual way.
#define FRAC_MAX 9223372036854775807LL /* 2**63 - 1 */
struct dbl_packed
{
int exp;
long long frac;
};
void pack(double x, struct dbl_packed *r)
{
double xf = fabs(frexp(x, &r->exp)) - 0.5;
if (xf < 0.0)
{
r->frac = 0;
return;
}
r->frac = 1 + (long long)(xf * 2.0 * (FRAC_MAX - 1));
if (x < 0.0)
r->frac = -r->frac;
}
double unpack(const struct dbl_packed *p)
{
double xf, x;
if (p->frac == 0)
return 0.0;
xf = ((double)(llabs(p->frac) - 1) / (FRAC_MAX - 1)) / 2.0;
x = ldexp(xf + 0.5, p->exp);
if (p->frac < 0)
x = -x;
return x;
}
You could always convert to IEEE-754 format in a fixed byte order (either little endian or big endian). For most machines, that would require either nothing at all or a simple byte swap to serialize and deserialize. A machine that doesn't support IEEE-754 natively will need a converter written, but doing that with ldexp and frexp (standard C library functions)and bit shuffling is not too tough.
What do you mean, "portable"?
For portability, remember to keep the numbers within the limits defined in the Standard: use a single number outside these limits, and there goes all portability down the drain.
double planck_time = 5.39124E-44; /* second */
5.2.4.2.2 Characteristics of floating types <float.h>
[...]
10 The values given in the following list shall be replaced by constant
expressions with implementation-defined values [...]
11 The values given in the following list shall be replaced by constant
expressions with implementation-defined values [...]
12 The values given in the following list shall be replaced by constant
expressions with implementation-defined (positive) values [...]
[...]
Note the implementation-defined in all these clauses.
Converting to an ascii representation would be the simplest, but if you need to deal with a colossal number of floats, then of course you should go binary. But this can be a tricky issue if you care about portability. Floating point numbers are represented differently in different machines.
If you don't want to use a canned library, then your float-binary serializer/deserializer will simply have to have "a contract" on where each bit lands and what it represents.
Here's a fun website to help with that: link.
sprintf, fprintf ? you don't get any more portable than that.
What level of portability do you require? If the file is to be read on a computer with the same OS that it was generated on, than you using a binary file and just saving and restoring the bit pattern should work. Otherwise as boytheo said, ASCII is your friend.
This version has excess of only one byte per one floating point value to indicate the endianness. But I think, it is still not very portable however.
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <ctype.h>
#define LITEND 'L'
#define BIGEND 'B'
typedef short INT16;
typedef int INT32;
typedef double vec1_t;
typedef struct {
FILE *fp;
} WFILE, RFILE;
#define w_byte(c, p) putc((c), (p)->fp)
#define r_byte(p) getc((p)->fp)
static void w_vec1(vec1_t v1_Val, WFILE *p)
{
INT32 i;
char *pc_Val;
pc_Val = (char *)&v1_Val;
w_byte(LITEND, p);
for (i = 0; i<sizeof(vec1_t); i++)
{
w_byte(pc_Val[i], p);
}
}
static vec1_t r_vec1(RFILE *p)
{
INT32 i;
vec1_t v1_Val;
char c_Type,
*pc_Val;
pc_Val = (char *)&v1_Val;
c_Type = r_byte(p);
if (c_Type==LITEND)
{
for (i = 0; i<sizeof(vec1_t); i++)
{
pc_Val[i] = r_byte(p);
}
}
return v1_Val;
}
int main(void)
{
WFILE x_FileW,
*px_FileW = &x_FileW;
RFILE x_FileR,
*px_FileR = &x_FileR;
vec1_t v1_Val;
INT32 l_Val;
char *pc_Val = (char *)&v1_Val;
INT32 i;
px_FileW->fp = fopen("test.bin", "w");
v1_Val = 1234567890.0987654321;
printf("v1_Val before write = %.20f \n", v1_Val);
w_vec1(v1_Val, px_FileW);
fclose(px_FileW->fp);
px_FileR->fp = fopen("test.bin", "r");
v1_Val = r_vec1(px_FileR);
printf("v1_Val after read = %.20f \n", v1_Val);
fclose(px_FileR->fp);
return 0;
}
Here we go.
Portable IEEE 754 serialisation / deserialisation that should
work regardless of the machine's internal floating point
representation.
https://github.com/MalcolmMcLean/ieee754
/*
* read a double from a stream in ieee754 format regardless of host
* encoding.
* fp - the stream
* bigendian - set to if big bytes first, clear for little bytes
* first
*
*/
double freadieee754(FILE *fp, int bigendian)
{
unsigned char buff[8];
int i;
double fnorm = 0.0;
unsigned char temp;
int sign;
int exponent;
double bitval;
int maski, mask;
int expbits = 11;
int significandbits = 52;
int shift;
double answer;
/* read the data */
for (i = 0; i < 8; i++)
buff[i] = fgetc(fp);
/* just reverse if not big-endian*/
if (!bigendian)
{
for (i = 0; i < 4; i++)
{
temp = buff[i];
buff[i] = buff[8 - i - 1];
buff[8 - i - 1] = temp;
}
}
sign = buff[0] & 0x80 ? -1 : 1;
/* exponet in raw format*/
exponent = ((buff[0] & 0x7F) << 4) | ((buff[1] & 0xF0) >> 4);
/* read inthe mantissa. Top bit is 0.5, the successive bits half*/
bitval = 0.5;
maski = 1;
mask = 0x08;
for (i = 0; i < significandbits; i++)
{
if (buff[maski] & mask)
fnorm += bitval;
bitval /= 2.0;
mask >>= 1;
if (mask == 0)
{
mask = 0x80;
maski++;
}
}
/* handle zero specially */
if (exponent == 0 && fnorm == 0)
return 0.0;
shift = exponent - ((1 << (expbits - 1)) - 1); /* exponent = shift + bias */
/* nans have exp 1024 and non-zero mantissa */
if (shift == 1024 && fnorm != 0)
return sqrt(-1.0);
/*infinity*/
if (shift == 1024 && fnorm == 0)
{
#ifdef INFINITY
return sign == 1 ? INFINITY : -INFINITY;
#endif
return (sign * 1.0) / 0.0;
}
if (shift > -1023)
{
answer = ldexp(fnorm + 1.0, shift);
return answer * sign;
}
else
{
/* denormalised numbers */
if (fnorm == 0.0)
return 0.0;
shift = -1022;
while (fnorm < 1.0)
{
fnorm *= 2;
shift--;
}
answer = ldexp(fnorm, shift);
return answer * sign;
}
}
/*
* write a double to a stream in ieee754 format regardless of host
* encoding.
* x - number to write
* fp - the stream
* bigendian - set to write big bytes first, elee write litle bytes
* first
* Returns: 0 or EOF on error
* Notes: different NaN types and negative zero not preserved.
* if the number is too big to represent it will become infinity
* if it is too small to represent it will become zero.
*/
int fwriteieee754(double x, FILE *fp, int bigendian)
{
int shift;
unsigned long sign, exp, hibits, hilong, lowlong;
double fnorm, significand;
int expbits = 11;
int significandbits = 52;
/* zero (can't handle signed zero) */
if (x == 0)
{
hilong = 0;
lowlong = 0;
goto writedata;
}
/* infinity */
if (x > DBL_MAX)
{
hilong = 1024 + ((1 << (expbits - 1)) - 1);
hilong <<= (31 - expbits);
lowlong = 0;
goto writedata;
}
/* -infinity */
if (x < -DBL_MAX)
{
hilong = 1024 + ((1 << (expbits - 1)) - 1);
hilong <<= (31 - expbits);
hilong |= (1 << 31);
lowlong = 0;
goto writedata;
}
/* NaN - dodgy because many compilers optimise out this test, but
*there is no portable isnan() */
if (x != x)
{
hilong = 1024 + ((1 << (expbits - 1)) - 1);
hilong <<= (31 - expbits);
lowlong = 1234;
goto writedata;
}
/* get the sign */
if (x < 0) { sign = 1; fnorm = -x; }
else { sign = 0; fnorm = x; }
/* get the normalized form of f and track the exponent */
shift = 0;
while (fnorm >= 2.0) { fnorm /= 2.0; shift++; }
while (fnorm < 1.0) { fnorm *= 2.0; shift--; }
/* check for denormalized numbers */
if (shift < -1022)
{
while (shift < -1022) { fnorm /= 2.0; shift++; }
shift = -1023;
}
/* out of range. Set to infinity */
else if (shift > 1023)
{
hilong = 1024 + ((1 << (expbits - 1)) - 1);
hilong <<= (31 - expbits);
hilong |= (sign << 31);
lowlong = 0;
goto writedata;
}
else
fnorm = fnorm - 1.0; /* take the significant bit off mantissa */
/* calculate the integer form of the significand */
/* hold it in a double for now */
significand = fnorm * ((1LL << significandbits) + 0.5f);
/* get the biased exponent */
exp = shift + ((1 << (expbits - 1)) - 1); /* shift + bias */
/* put the data into two longs (for convenience) */
hibits = (long)(significand / 4294967296);
hilong = (sign << 31) | (exp << (31 - expbits)) | hibits;
x = significand - hibits * 4294967296;
lowlong = (unsigned long)(significand - hibits * 4294967296);
writedata:
/* write the bytes out to the stream */
if (bigendian)
{
fputc((hilong >> 24) & 0xFF, fp);
fputc((hilong >> 16) & 0xFF, fp);
fputc((hilong >> 8) & 0xFF, fp);
fputc(hilong & 0xFF, fp);
fputc((lowlong >> 24) & 0xFF, fp);
fputc((lowlong >> 16) & 0xFF, fp);
fputc((lowlong >> 8) & 0xFF, fp);
fputc(lowlong & 0xFF, fp);
}
else
{
fputc(lowlong & 0xFF, fp);
fputc((lowlong >> 8) & 0xFF, fp);
fputc((lowlong >> 16) & 0xFF, fp);
fputc((lowlong >> 24) & 0xFF, fp);
fputc(hilong & 0xFF, fp);
fputc((hilong >> 8) & 0xFF, fp);
fputc((hilong >> 16) & 0xFF, fp);
fputc((hilong >> 24) & 0xFF, fp);
}
return ferror(fp);
}
fwrite(), fread()? You will likely want binary, and you cannot pack the bytes any tighter unless you want to sacrifice precision which you would do in the program and then fwrite() fread() anyway; float a; double b; a=(float)b; fwrite(&a,1,sizeof(a),fp);
If you are carrying different floating point formats around they may not convert in a straight binary sense, so you may have to pick apart the bits and perform the math, this to the power that plus this, etc. IEEE 754 is a dreadful standard to use but widespread so it would minimize the effort.

Resources