Concatenating arrays in place - c

I ran into an issue while implementing a circular buffer that must occasionally be aligned.
Say I have two arrays, leftArr and rightArr. I want to move the right array to byteArr and the left array to byteArr + the length of the right array. Both leftArr and rightArr are greater than byteArr, and rightArr is greater than leftArr. (this is not quite the same as rotating a circular buffer because the left array does not need to start at byteArr) Although the left and right arrays do not overlap, the combined array stored at byteArr may overlap with the current arrays, stored at leftArr and rightArr. All memory from byteArr to rightArr + rightArrLen can be safely written to. One possible implementation is:
void align(char* byteArr, char* leftArr, int leftArrLen, char* rightArr, int rightArrLen) {
char *t = malloc(rightArrLen + leftArrLen);
// form concatenated data
memcpy(t, right, rightArrLen);
memcpy(t + rightArrLen, left, leftArrLen);
// now replace
memcpy(byteArr, t, rightArrLen + leftArrLen);
free(t);
}
However, I must accomplish this with constant memory complexity.
What I have so far looks like this:
void align(char* byteArr, char* leftArr, int leftArrLen, char* rightArr, int rightArrLen)
{
// first I check to see if some combination of memmove and memcpy will suffice, if not:
unsigned int lStart = leftArr - byteArr;
unsigned int lEnd = lStart + leftArrLen;
unsigned int rStart = rightArr - byteArr;
unsigned int rEnd = rStart + rightArrLen;
unsigned int lShift = rEnd - rStart - lStart;
unsigned int rShift = -rStart;
char temp1;
char temp2;
unsigned int nextIndex;
bool alreadyMoved;
// move the right array
for( unsigned int i = 0; i < rEnd - rStart; i++ )
{
alreadyMoved = false;
for( unsigned int j = i; j < rEnd - rStart; j-= rShift )
{
if( lStart <= j + rStart - lShift
&& j + rStart - lShift < lEnd
&& lStart <= (j + rStart) % lShift
&& (j + rStart) % lShift < lEnd
&& (j + rStart) % lShift < i )
{
alreadyMoved = true;
}
}
if(alreadyMoved)
{
// byte has already been moved
continue;
}
nextIndex = i - rShift;
temp1 = byteArr[nextIndex];
while( rStart <= nextIndex && nextIndex < rEnd )
{
nextIndex += rShift;
temp2 = byteArr[nextIndex];
byteArr[nextIndex] = temp1;
temp1 = temp2;
while( lStart <= nextIndex && nextIndex < lEnd )
{
nextIndex += lShift;
temp2 = byteArr[nextIndex];
byteArr[nextIndex] = temp1;
temp1 = temp2;
}
if( nextIndex <= i - rShift )
{
// byte has already been moved
break;
}
}
}
// move the left array
for( unsigned int i = lStart; i < lShift && i < lEnd; i++ )
{
if( i >= rEnd - rStart )
{
nextIndex = i + lShift;
temp1 = byteArr[nextIndex];
byteArr[nextIndex] = byteArr[i];
while( nextIndex < lEnd )
{
nextIndex += lShift;
temp2 = byteArr[nextIndex];
byteArr[nextIndex] = temp1;
temp1 = temp2;
}
}
}
}
This code works in the case lStart = 0, lLength = 11, rStart = 26, rLength = 70 but fails in the case lStart = 0, lLength = 46, rStart = 47, rLength = 53. The solution that I can see is to add logic to determine when a byte from the right array has already been moved. While this would be possible for me to do, I was wondering if there's a simpler solution to this problem that runs with constant memory complexity and without extra reads and writes?
Here's a program to test an implementation:
bool testAlign(int lStart, int lLength, int rStart, int rLength)
{
char* byteArr = (char*) malloc(100 * sizeof(char));
char* leftArr = byteArr + lStart;
char* rightArr = byteArr + rStart;
for(int i = 0; i < rLength; i++)
{
rightArr[i] = i;
}
for(int i = 0; i < lLength; i++)
{
leftArr[i] = i + rLength;
}
align(byteArr, leftArr, lLength, rightArr, rLength);
for(int i = 0; i < lLength + rLength; i++)
{
if(byteArr[i] != i) return false;
}
return true;
}

Imagine dividing byteArr into regions (not necessarily to scale):
X1 Left X2 Right
|---|--------|---|------|
The X1 and X2 are gaps in byteArr before the start of the left array and between the two arrays. In the general case, any or all of those four regions may have zero length.
You can then proceed like this:
Start by partially or wholly filling in the leading space in byteArr
If Left has zero length then move Right to the front (if necessary) via memmove(). Done.
Else if X1 is the same length as the Right array or larger then move the right array into that space via memcpy() and, possibly, move up the left array to abut it via memmove(). Done.
Else, move the lead portion of the Right array into that space, producing the below layout. If X1 had zero length then R1 also has zero length, X2' == X2, and R2 == Right.
R1 Left X2' R2
|---|--------|------|---|
There are now two alternatives
If R2 is the same length as Left or longer, then swap Left with the initial portion of R2 to produce (still not to scale):
R1' X2'' Left R2'
|------|-----|-------|--|
Otherwise, swap the initial portion of Left with all of R2 to produce (still not to scale):
R1' L2 X2'' L1
|------|---|-------|----|
Now recognize that in either case, you have a strictly smaller problem of the same form as the original, where the new byteArr is the tail of the original starting immediately after region R1'. In the first case the new leftArr is the (final) Left region and the new rightArr is region R2'. In the other case, the new leftArr is region L2, and the new rightArr is region L1. Reset parameters to reflect this new problem, and loop back to step (1).
Note that I say to loop back to step 1. You could, of course, implement this algorithm (tail-)recursively, but then to achieve constant space usage you would need to rely on your compiler to optimize out the tail recursion, which otherwise consumes auxiliary space proportional to the length ratio of the larger of the two sub-arrays to the smaller.

Related

Cunit test invalid read/write of size8

Invalid read and write of size 8 happening in modify_tab_size().
what am I doing wrong? Ive tried almost everything, I dont understand it.
// Function being tested.
int erase_repeated(int *nb_words, char **words) {
for (int i = 0; i < *nb_words; ++i) {
if (words[i] != 0) {
for (int b = 0; b < *nb_words; ++b) {
if (strcmp(words[i], words[b]) == 0 && b != i)
modify_tab_size(&b, nb_words, words);
}
}
}
return *nb_mots;
}
void modify_tab_size(int *b, int *nb_words_update, char **words) {
free(words[*b]);
for (int k = *b; k < *nb_words_update; k++) {
words[k] = words[k + 1]; <--------------------------read error
words[*nb_words_update + 1] = 0; <--------------------------write error
}
(*nb_words_update)--;
(*b)--;
}
The problem is k+1 and *nb_words_update + 1 can walk off the array, and it is. Add printf("k:%d, k+1:%d, *nb_words_update + 1: %d\n", k, k+1, *nb_words_update + 1); into the loop to see.
k:1, k+1:2, *nb_words_update + 1: 4
k:2, k+1:3, *nb_words_update + 1: 4
You've only allocated three slots, 3 and 4 walk off the end of the array.
Since nb_words_update starts as the length of the array, words[*nb_words_update + 1] = 0; is always going to be too large. words[*nb_words_update] = 0; is also too large.
What you seem to be trying to do is deleting an element from an array by shifting everything after it to the left.
void delete_element(char **words, int *b, int *size) {
// Free the string to be deleted.
free(words[*b]);
// Only go up to the second to last element to avoid walking off the array.
for (int i = *b; i < *size-1; i++) {
// Shift everything to the left.
words[i] = words[i+1];
}
// Null out the last element.
// Don't use 0 for NULL, it's confusing.
words[*size-1] = NULL;
// Decrement the size of the array.
(*size)--;
// Redo the check with the newly shifted element.
(*b)--;
}
This sort of thing is better done with a linked list.
Note that your code has a bug. The result is an array of two elements, but one of them is blank. In addition to the return value of erase_repeated, also test its side effect which is to modify words. Test that words contains what you think it does.

How to rotate 2D array of characters

I'm trying to rotate this clockwise 90 degrees, but it works not as expected. Must be something to do with pointers which I'm not that familiar.
OUTPUT Before Rotation:
hiivp
nxhxd
tszeg
xdlqo
kwpae
void rotate_right(char **m, int n) {
char **temp = m;
for (int i = 0; i < n; i++) {
for (int j = 0; j < n; j++) {
m[i][j] = temp[n-1-j][i];
}
}
}
OUTPUT After Rotation:
kxtnk
wdsdx
plzst
aqsdn
entxk
You're partially overwriting your matrix.
To perform a 90° rotation, let us consider the four corners of the square (in caps):
HiivP
nxhxd
tszeg
xdlqo
WwpaE
You need to put the H in the P place, save the P, and put the P in the E place, then the E in the W place, and finally the W in the H place. These four shift must be done each time saving the overwritten value, that becomes the new running value for the next stage.
Then you can see that each cell is part of such a four-cell shift: the corners are numbered 1, and 1 is the first shift. Then you work cells in position 2, then those in 3, and so on.
12341
45652
36-63
25654
14321
Having an odd side, the central square does not rotate and there is no cycle number 7. After six cycles, each involving four cells, the square is rotated (6*4 = 24, and 24+1 = 25). With a larger square, you can see the "law" linking the x,y coordinates for each cycle.
Building on LSerni's answer, adding an implementation for completeness sake.
Assuming the following matrix:
#define STRING_SIZE (5)
char m[][STRING_SIZE+1] = { "hiivp",
"nxhxd",
"tszeg",
"xdlqo",
"kwpae" };
You can define the rotate function like this:
char override_and_backup(char* dest, char val)
{
char tmp = *dest;
*dest = val;
return tmp;
}
void rotate_right(char m[][STRING_SIZE+1], int size)
{
// rotate layer by layer
for (int layer = 0; layer < (size / 2); ++layer)
{
int current_size = size - (2*layer); // side size of current layer
int ncycles = current_size - 1; // number of cycles to complete rotation
// perform the rotating cycles
for (int cycle = 0; cycle < ncycles; ++cycle)
{
char* first = &m[layer][layer+cycle];
char* second = &m[layer+cycle][layer+current_size-1];
char* third = &m[layer+current_size-1][layer+current_size-cycle-1];
char* fourth = &m[layer+current_size-cycle-1][layer];
char tmp = *fourth;
tmp = override_and_backup(first, tmp);
tmp = override_and_backup(second, tmp);
tmp = override_and_backup(third, tmp);
override_and_backup(fourth, tmp);
}
}
}

Segfault after refactoring nested loops

I have some MATLAB code from a digital audio course that I've ported to C. Given an array of numeric data (for example, PCM audio encoded as double-precision floating-point), produce an array of data segments of a specified width and which overlap each other by a specified amount. Here's the relevant code.
typedef struct AudioFramesDouble {
const size_t n, // number of elements in each frame
num_frames;
double* frames[];
} AudioFramesDouble;
/*
* Produce a doubly-indexed array of overlapping substrings (a.k.a windows, frames,
* segments ...) from a given array of data.
*
* x: array of (i.e., pointer to) data
* sz: number of data elements to consider
* n: number of elements in each frame
* overlap: each frame overlaps the next by a factor of 1 - 1/overlap.
*/
AudioFramesDouble* audio_frames_double(register const double x[], const size_t sz, const unsigned n, const unsigned overlap) {
// Graceful exit on nullptr
if (!x) return (void*) x;
const double hop_d = ((double) n) / ((double) overlap); // Lets us "hop" to the start of the next frame.
const unsigned hop = (unsigned) ceil(hop_d);
const unsigned remainder = (unsigned) sz % hop;
const double num_frames_d = ((double) sz) / hop_d;
const size_t num_frames = (size_t) (remainder == 0
? floor(num_frames_d) // paranoia about floating point errors
: ceil(num_frames_d)); // room for zero-padding
const size_t total_samples = (size_t) n * num_frames;
AudioFramesDouble af = {.n = n, .num_frames = num_frames};
// We want afp->frames to appear as (double*)[num_frames].
AudioFramesDouble* afp = malloc((sizeof *afp) + (sizeof (double*) * num_frames));
if (!afp) return afp;
memcpy(afp, &af, sizeof af);
for (size_t i = 0; i < num_frames; ++i) {
/* Allocate zero-initialized space at the start of each frame. If this
fails, free up the memory and vomit a null pointer. */
afp->frames[i] = calloc(n, sizeof(double));
if (!afp->frames[i]) {
double* p = afp->frames[i];
for (long ii = ((long)i) - 1; 0 <= ii; ii--) {
free(afp->frames[--i]);
}
free(afp);
return (void*) p;
}
for (size_t j = 0, k; j < n; ++j) {
if (sz <= (k = (i*hop) + j)) break;
afp->frames[i][j] = x[k];
}
}
return afp;
}
This performs as expected. I wanted to optimize the nested FOR to the following
for (size_t i = 0, j = 0, k; i < num_frames; (j == n - 1) ? (j = 0,i++) : ++j) {
// If we've reached the end of the frame, reset j to zero.
// Then allocate the next frame and check for null.
if (j == 0 && !!(afp->frames[i] = calloc(n, sizeof(double)))) {
double* p = afp->frames[i];
for (long ii = ((long)i) - 1; 0 <= ii; ii--) {
free(afp->frames[--i]);
}
free(afp);
return (void*) p;
}
if (sz <= (k = (i*hop) + j)) break;
afp->frames[i][j] = x[k];
}
This actually compiles and runs just fine; but in my testing, when I try to access the last frame as in
xFrames->frames[xFrames->num_frames-1],
I get a segmentation fault. What's going on here? Am I neglecting an edge case in my loop? I've been looking over the code for awhile, but I might need a second set of eyes. Sorry if the answer is glaringly obvious; I'm a bit of a C novice.
P.S. I'm a fan of branchless programming, so if anyone has tips for cutting out those IFs, I'm all ears. I was using ternary operators before, but reverted to IFs for readability in debugging.
Remember that the logical operator && and || does short-circuit evaluation.
That means if j != 0 then you won't actually call calloc, and you'll have an invalid pointer in afp->frames[i].

why is this else block executed twice?

I have the following code which correctly calculates the jaccard similarity between an input char array and an existing array of char arrays. jacc_sim_rec[] is used to record the similarities which satisfy a minimum threshold value. The for loop is used to iterate through the multidimensional array and the loop is supposed to continue checking similarity if minimum threshold is not satisfied at if (jacc_sim < SIM_THRESHOLD); else record the result at
else
{
jacc_sim_rec[j] = jacc_sim;//keep record of similarity
++j;//record number of highly similar elements
}
my problem is, the whole statements in the else block is executed twice every time the threshold value is satisfied.
int j=0;
void calc_jac_sim( char*INCOMING, int grp)
{
unsigned long i, j11 = 0, j01 = 0, j10 = 0,m=0;
char *m11, *m01, *m10;
float jacc_sim = 0.0;
char r1[SBF_LEN] = { NULL };
char r2[SBF_LEN] = { NULL };
char r3[SBF_LEN] = { NULL };
int cnt = SBF_LEN - 1;
clear_jacc_sim_info();
for (int i = 0; i <= SBF_REC[grp]; ++i)
{
while (cnt >= 0)
{
r1[cnt] = SBF[grp][i][cnt] & INCOMING[cnt];
r2[cnt] = ~SBF[grp][i][cnt] & INCOMING[cnt];
r3[cnt] = SBF[grp][i][cnt] & ~INCOMING[cnt];
cnt--;
}
m11 = ( char*)r1;
m01 = ( char*)r2;
m10 = ( char*)r3;
for (m = SBF_LEN * sizeof( char); m--;
j11 += NumberOfSetBits(*m11++),
j01 += NumberOfSetBits(*m01++),
j10 += NumberOfSetBits(*m10++));
jacc_sim = j11 / (float)(j11 + j01 + j10);
if (jacc_sim < SIM_THRESHOLD);
//continue;//do nothing
else
{
jacc_sim_rec[j] = jacc_sim;//keep record of similarity
++j;//record number of highly similar elements
}
}
}
I don't understand the code, but I'll bet the problem is that you're not reinitializing cnt each time through the for loop, so you only fill in r1, r2, and r3 when i = 0.
Change that loop to:
for (int cnt = SBF_LEN - 1; cnt >= 0; cnt--)
{
r1[cnt] = SBF[grp][i][cnt] & INCOMING[cnt];
r2[cnt] = ~SBF[grp][i][cnt] & INCOMING[cnt];
r3[cnt] = SBF[grp][i][cnt] & ~INCOMING[cnt];
}
I'm also not sure why this needs to count down instead of up, like a typical loop, but it shouldn't make a difference.

Optimization O(n^2) to O(n) (Unsorted String)

I'm having an optimization problem here. I would like to have this code running in O(n), which I tried for several hours now.
Byte-arrays c contains a string, e contains the same string, but sorted. Int-arrays nc and ne contain the indexes within the string, eg
c:
s l e e p i n g
nc:
0 0 0 1 0 0 0 0
e:
e e g i l n p s
ne:
0 1 0 0 0 0 0 0
The problem now is that get_next_index is linear - is there a way to solve this?
void decode_block(int p) {
BYTE xj = c[p];
int nxj = nc[p];
for (int i = 0; i < block_size; i++) {
result[i] = xj;
int q = get_next_index(xj, nxj, c, nc);
xj = e[q];
nxj = ne[q];
}
fwrite(result, sizeof (BYTE), block_size, stdout);
fflush(stdout);
}
int get_next_index(BYTE xj, int nxj, BYTE* c, int* nc) {
int i = 0;
while ( ( xj != c[i] ) || ( nxj != nc[i] ) ) {
i++;
}
return i;
}
This is part of an Burrows-Wheeler implementation
It starts with
xj = c[p]
nxj = nc[p]
Next I have to block_size (= length c = length nc = length e = length ne) times
store the result xj in result
find the number index for which c[i] == xj
xj is now e[i]
ne and nc are only used to make sure that every character in e and c is unique (e_0 != e_1).
Since your universe (i.e. a char) is small, I think you can get away with linear time. You need a linked list and any sequence container a lookup table for this.
First your go through your sorted string and populate a lookup table that allows you to find the first list element for a given character. For instance, your lookup table could look like std::array<std::list<size_t>,(1<<sizeof(char))> lookup. If you don't want a list, you can also use an std::deque or even an std::pair<std::vector,size_t> while the second item represents the index of the first valid entry of the vector (that way you don't need to pop the element later on, but just increment the index).
So for each element c in your sorted string you append that to you container in lookup[c].
Now, when you iterate over your unsorted array, for each element, you can lookup the corresponding index in your lookup table. Once you're done, you pop the front element in the lookup table.
All in all this is linear time and space.
To clarify; When initialising the lookup table:
// Instead of a list, a deque will likely perform better,
// but you have to test this yourself in your particular case.
std::array<std::list<size_t>,(1<<sizeof(char))> lookup;
for (size_t i = 0; i < sortedLength; i++) {
lookup[sorted[i]].push_back(i);
}
When finding the "first index" for the index i in the unsorted array:
size_t const j = lookup[unsorted[i]].front();
lookup[unsorted[i]].pop_front();
return j;
Scan xj and nxj once and build a lookup table. This is a two O(n) operations.
The most sensible way would be to have a binary tree, sorted on the value of xj or nxj. The node would contain your sought index. This would reduce your lookup to O(lg n).
Here is my complete implementation of the Burrowes-Wheeler transform:
u8* bwtCompareBuf;
u32 bwtCompareLen;
s32 bwtCompare( const void* v1, const void* v2 )
{
u8* c1 = bwtCompareBuf + ((u32*)v1)[0];
u8* c2 = bwtCompareBuf + ((u32*)v2)[0];
for ( u32 i = 0; i < bwtCompareLen; i++ )
{
if ( c1[i] < c2[i] ) return -1;
if ( c1[i] > c2[i] ) return +1;
}
return 0;
}
void bwtEncode( u8* inputBuffer, u32 len, u32& first )
{
s8* tmpBuf = alloca( len * 2 );
u32* indices = new u32[len];
for ( u32 i = 0; i < len; i++ ) indices[i] = i;
bwtCompareBuf = tmpBuf;
bwtCompareLen = len;
qsort( indices.data(), len, sizeof( u32 ), bwtCompare );
u8* tbuf = (u8*)tmpBuf + ( len - 1 );
for ( u32 i = 0; i < len; i++ )
{
u32 idx = indices[i];
if ( idx == 0 ) idx = len;
inputBuffer[i] = tbuf[idx];
if ( indices[i] == 1 ) first = i;
}
delete[] indices;
}
void bwtDecode( u8* inputBuffer, u32 len, u32 first )
{
// To determine a character's position in the output string given
// its position in the input string, we can use the knowledge about
// the fact that the output string is sorted. Each character 'c' will
// show up in the output stream in in position i, where i is the sum
// total of all characters in the input buffer that precede c in the
// alphabet, plus the count of all occurences of 'c' previously in the
// input stream.
// compute the frequency of each character in the input buffer
u32 freq[256] = { 0 };
u32 count[256] = { 0 };
for ( u32 i = 0; i < len; i++ )
freq[inputBuffer[i]]++;
// freq now holds a running total of all the characters less than i
// in the input stream
u32 sum = 0;
for ( u32 i = 0; i < 256; i++ )
{
u32 tmp = sum;
sum += freq[i];
freq[i] = tmp;
}
// Now that the freq[] array is filled in, I have half the
// information needed to position each 'c' in the input buffer. The
// next piece of information is simply the number of characters 'c'
// that appear before this 'c' in the input stream. I keep track of
// that information in the count[] array as I go. By adding those
// two numbers together, I get the destination of each character in
// the input buffer, and I just write it directly to the destination.
u32* trans = new u32[len];
for ( u32 i = 0; i < len; i++ )
{
u32 ch = inputBuffer[i];
trans[count[ch] + freq[ch]] = i;
count[ch]++;
}
u32 idx = first;
s8* tbuf = alloca( len );
memcpy( tbuf, inputBuffer, len );
u8* srcBuf = (u8*)tbuf;
for ( u32 i = 0; i < len; i++ )
{
inputBuffer[i] = srcBuf[idx];
idx = trans[idx];
}
delete[] trans;
}
The decode in O(n).

Resources