Segmentation fault on recursive merge sort - arrays

Question

I'm trying to write a fully recursive version of merge sort
I get segmentation fault on merge function when I try to order more than 100k records, I read online that it would be a stack overflow issue, related to VLAs, but I can't figure it out.
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
typedef struct _MBArray MBArray;
struct _MBArray {
void ** array;
int length;
};
void mergeSort(MBArray *mbarray, int start, int end, int (*compare)(void *, void *)) {
if (start == end) {
return;
} else {
int middle = (end + start) / 2;
mergeSort(mbarray, start, middle, compare);
mergeSort(mbarray, middle + 1, end, compare);
merge(mbarray, start, middle + 1, end, compare, (mbarray->array)[middle + 1]);
return;
}
}
void merge(MBArray *mbarray, int startA, int startB, int length, int (*compare)(void *, void *),void *elem) {
if (startA > startB - 1) {
return;
} else
if (startB > length) {
return;
} else
if ((*(compare))((mbarray->array)[startA], (mbarray->array)[startB])) {
merge(mbarray, startA + 1, startB, length, compare, (mbarray->array)[startB]);
return;
} else {
memcpy(mbarray->array + (startA + 1), mbarray->array + startA, (startB - startA) * sizeof(void *));
(mbarray->array)[startA] = elem;
merge(mbarray, startA + 1, startB + 1, length, compare, (mbarray->array)[startB + 1]);
return;
}
}

Answer 1

Your merge and mergesort functions are incorrect:
the mergesort function is given index values into the array such that start is included and end is excluded. Here is how to fix it:
void mergeSort(MBArray *mbarray, int start, int end,
int (*compare)(void *, void *))
{
if (end - start > 1) {
int middle = start + (end - start) / 2;
mergeSort(mbarray, start, middle, compare);
mergeSort(mbarray, middle, end, compare);
merge(mbarray, start, middle, end, compare);
}
}
The merge function should be simplified. If you really intend for this function to recurse instead of using a temporary array, you might recurse as many as length/2 times hence are likely to cause a stack overflow even faster than by allocating a VLA in the merge function.
Here is a VLA based merge function:
void merge(MBArray *mbarray, int low, int middle, int end,
int (*compare)(void *, void *))
{
int templen = middle - low;
void *temp[templen];
memcpy(temp, mbarray->array + low, sizeof(temp));
int i = 0, j = middle, k = low;
while (i < templen) {
if (j >= end || compare(temp[i], mbarray->array[j])) {
mbarray->array[k++] = temp[i++];
else
mbarray->array[k++] = mbarray->array[j++];
}
}
A fully recursive version should save the element and recurse before setting it into its final position. This is much more costly in stack space than allocating a VLA so stack overflow is more likely. Allocating a temporary array once with malloc() and passing it to mergesort is much safer and also more efficient.
Here is a modified fully recursive merge function:
void mergeRecur(MBArray *mbarray, int dest, int a0, int a1, int b0, int b1,
int (*compare)(void *, void *))
{
if (a0 < a1) {
if (b0 >= b1 || compare(mbarray->array[a0], mbarray->array[b0])) {
void *temp = mbarray->array[a0];
mergeRecur(mbarray, dest + 1, a0 + 1, a1, b0, b1, compare);
mbarray->array[dest] = temp;
} else {
void *temp = mbarray->array[b0];
mergeRecur(mbarray, dest + 1, a0, a1, b0 + 1, b1, compare);
mbarray->array[dest] = temp;
}
}
}
void merge(MBArray *mbarray, int low, int middle, int end,
int (*compare)(void *, void *))
{
mergeRecur(mbarray, low, low, middle, middle, end, compare);
}
For large arrays, I would recommend allocating a temporary array once and calling a recursive version of mergesort that uses the temporary array instead of allocating a VLA:
#include <stdlib.h>
#include <string.h>
struct _MBArray {
void ** array;
int length;
};
void mergeTemp(MBArray *mbarray, int low, int middle, int end,
int (*compare)(void *, void *), void **temp)
{
int templen = middle - low;
int i = 0, j = middle, k = low;
memcpy(temp, mbarray->array + low, sizeof(*temp) * templen);
while (i < templen) {
if (j >= end || compare(temp[i], mbarray->array[j])) {
mbarray->array[k++] = temp[i++];
else
mbarray->array[k++] = mbarray->array[j++];
}
}
void mergeSortTemp(MBArray *mbarray, int start, int end,
int (*compare)(void *, void *), void **temp)
{
if (end - start > 1) {
int middle = start + (end - start) / 2;
mergeSortTemp(mbarray, start, middle, compare, temp);
mergeSortTemp(mbarray, middle, end, compare, temp);
mergeTemp(mbarray, start, middle, end, compare, temp);
}
}
void mergeSort(MBArray *mbarray, int (*compare)(void *, void *)) {
if (mbarray->length > 1) {
void **temp = malloc(sizeof(*temp) * (mbarray->length / 2));
mergeSortTemp(mbarray, 0, mbarray->length, compare, temp);
free(temp);
}
}

Answer 2

The problem is most likely due to merge() calling itself for each element merged. It would be better to have an entry function for mergeSort() that does a one time allocation of a second array (of pointers), then pass the second array as a parameter to a recursive MergeSortR(), and merge back and forth between the two arrays.