I am trying my luck with pthreads for a problem -
I have two questions about that and will add some excerpts from the code:
I have a struct, which contains an input array and one as output (or result) array. Where do I have to allocate the memory for this result, within the main-function or within the thread-function?
These results shall be merged into one big result array using realloc and memcpy. I think I am using memcpy and realloc not correctly in this case, or do I?
thread-Function:
struct thread_input {
unsigned char *buf;
unsigned char *result;
int start;
int end;
int tid;
};
void* thread_function(void* arg){
struct thread_input *param = (struct thread_input*)arg;
param->result = (unsigned char*)malloc((1+(param->end-param->start))*sizeof(unsigned char));
for(int i = param->start; i <= param->end; i++){
param->result[i] = some_function(param->buf[i]);
printf("%d %d\n", param->tid, param->result[i]);
}
return NULL;
}
So the some_function is working with every unsigned char and returns it to the param->result array.
This shall be merged here (where n_elemts is the number calculated in every thread):
unsigned char* arr1 = (unsigned char*)malloc(n_elements*sizeof(unsigned char));
for (int t = 0; t < n_threads; t++){
pthread_join(threads[t], NULL);
memcpy(arr1, thread_args[t].result, n_elements);
if(t<n_threads-1){
arr1 = (unsigned char*)realloc(arr1, (t+1)*n_elements*sizeof(unsigned char));
}
}
print_function(arr1, fileSize);
The problem given by the compiler is located at the realloc and called "invalid read of size 1", I suppose it is due to a wrong file_size, or array length, after the realloc. Plus the allocation of the param->result that I could not figure out completely.
Thanks for lookin at these excerpts and for any comment! Hope this question was asked in concise enough way, if not bombard me with critic.
Related
I am writing a small program which prints the time it took to allocate the memory.
I want to free the memory later so I want to save it in a array, but since I can loop it as many times as I want I want to make a Dynamic array to store all the addresses from the allocated Memory. This is my
init code:
static __init int init_kmalloc(void)
{
int size = sizeof(char*);
char *buffer = kmalloc_array(loop_cnt, size, GFP_KERNEL);
unsigned int i = 0;
printk(KERN_DEBUG "Allocating %d times the memory size of %d\n", loop_cnt, alloc_size);
while(i < loop_cnt)
{
unsigned long long start;
unsigned long long stop;
start = get_rdtsc();
buffer[i] = kmalloc(alloc_size, GFP_KERNEL);
stop = get_rdtsc();
printk(KERN_DEBUG "%i: It took %lld ticks to allocate the memory\n", i, stop - start);
i++;
}
while(i > 0)
{
kfree(buffer[i]);
printk(KERN_DEBUG "Cleared\n");
i--;
}
return 0;
}
I Always get these errors:
What is wrong is that you choose char as the element of the array by selecting char* for the type of buffer. The elements of the array should pointers, so buffer should be a pointer to pointers like this (for example):
char **buffer = kmalloc_array(loop_cnt, size, GFP_KERNEL);
Use two *s, not one.
I have a dynamically allocated array of structures, 'buff'. Each element is a structure that has a few integer variables and a pointer 'buffer_ptr' which points to another dynamically allocated array of structures. The size of both arrays is given as command line input.
int buffer_size;
int user_num;
struct tuple
{
char userID[5];
char topic[16];
int weight;
};
struct buff_ctrl
{
struct tuple* buffer_ptr;
int in;
int out;
int numItems;
int done;
};
The arrays are created and initialized in main() as follows:
int main(int argc, char* argv[])
{
void *status;
pthread_t mapThd;
if(argc != 4)
{
printf("Input format: ./combiner <buffer_size> <number_of_users> <input_file>\n");
return -1;
}
buffer_size = atoi(argv[1]);
user_num = atoi(argv[2]);
struct buff_ctrl *buff = (struct buff_ctrl*)malloc(user_num * sizeof(struct buff_ctrl));
for(int i=0; i<user_num; i++)
{
struct buff_ctrl* curr_buff = (buff + (i*sizeof(struct buff_ctrl)));
struct tuple *ptr = (struct tuple*)malloc(buffer_size * sizeof(struct tuple));
curr_buff->buffer_ptr = ptr;//points to another array
curr_buff->in = 8;
curr_buff->out = 4;
curr_buff->numItems = 7;
curr_buff->done = 0;
printf("%p\n",curr_buff);
}
Then, I need to pass the 'buff' pointer as an argument when creating thread using pthread_create:
pthread_create(&mapThd, NULL, mapper, (void*)buff);
pthread_join(mapThd, &status);
free(buff);
/*end of main*/
My function pointer is as follows:
void* mapper(void *buff)
{
struct buff_ctrl* arr = (struct buff_ctrl *)buff;
struct buff_ctrl* temp_ptr;
printf("######################################################\n");
for(int k=0; k<user_num; k++)
{
/*Printing just to check values */
temp_ptr = arr + (k*sizeof(struct buff_ctrl));
printf("buffer ptr = %p\n", temp_ptr->buffer_ptr);
printf("in = %d\n", temp_ptr->in);
printf("out = %d\n", temp_ptr->out);
printf("numItems = %d\n", temp_ptr->numItems);
}
printf("######################################################\n");
pthread_exit((void*)buff);
}
But, when I print the values of 'buffer_ptr' from the created thread (only one), for ODD number of user_num, there is always ONE element of the array 'buff' which gives garbage value after pthread_create statement! When the values are checked in main itself after removing calls to pthread, it runs fine.
This line
struct buff_ctrl* curr_buff = (buff + (i*sizeof(struct buff_ctrl)));
should be
struct buff_ctrl* curr_buff = buff + i;
buff + i is pointer arithmetic and the compiler already takes the size of the
object pointed to by buff into consideration. By doing i*sizeof(struct buff_ctrl) you are assigning
a pointer that may be after the allocated memory.
As general suggestion:
Don't cast malloc. And instead of using sizeof(<type>), use sizeof *variable, this is more safe, because
it's easier to make mistakes when writing sizeof(<type>).
So:
struct buff_ctrl *buff = malloc(user_num * sizeof *buff);
...
struct tuple *ptr = malloc(buffer_size * sizeof *ptr);
And you don't need to declare a separate pointer, you can do:
for(int i=0; i<user_num; i++)
{
buff[i].buffer_ptr = malloc(buffer_size * sizeof *buff[i].buffer_ptr);
buff[i].in = 8;
buff[i].out = 4;
buff[i].numItems = 7;
buff[i].done = 0;
}
Also you should always check for the return value of malloc. If it returns
NULL, you cannot access that memory.
This is wrong:
struct buff_ctrl* curr_buff = (buff + (i*sizeof(struct buff_ctrl)));
When you do pointer arithmetic, it operates in units of the size of what the pointer points to, so you don't need to multiply by sizeof. As a result, you're effectively multiplying twice and accessing outside the array bounds.
Just treat buff as an array, rather than dealing with pointers.
for(int i=0; i<user_num; i++)
{
struct tuple *ptr = malloc(buffer_size * sizeof(struct tuple));
buff[i].buffer_ptr = ptr;//points to another array
buff[i].in = 8;
buff[i].out = 4;
buff[i].numItems = 7;
buff[i].done = 0;
}
Also, see Do I cast the result of malloc?
You have a fundamental error.
Pointer arithmetics works by adding the offset in multiples of the pointer type, so adding the offset yourself will not work as you apparently expect it to.
If it was a char * pointer then you would need to add the offset manually, increments would be multiplied by one. But in your case increments by n are multiplied by the size of the pointer base type.
There are times when doing pointer arithmetics with the addition notation makes sense, but most of the time it's much clearer to write index notation instead.
My code uses two structures, block and layout (which is a collection of an arbitrary number of blocks).
struct block{
char type;
unsigned short int loc;
unsigned short int size[2];
};
struct layout{
unsigned short int no;
struct block *blocks;
short int **moves;
};
I am using this function to quickly initialize (and partly fill) the structure layout, based a set of blocks:
struct layout init_layout(int block_no, struct block *blocks){
struct layout new_layout;
int i, j;
new_layout.no = (unsigned short int)block_no;
// the following two lines cause an memory corruption error
new_layout.blocks = (struct block *)malloc(block_no);
new_layout.moves = (short int **)malloc(block_no);
for(i = 0; i < block_no; i++){
new_layout.blocks[i] = blocks[i];
new_layout.moves[i] = (short int *)malloc(2);
for(j = 0; j < 2; j++)
new_layout.moves[i][j] = 0;
}
return new_layout;
}
So far, I do not see, that there is something wrong with it. However, when I call function like this
int main(int argc, char** argv){
// just some arbitrary values for 10 blocks
int size[2] = {2, 2};
struct block *blocks = (struct block *)malloc(10);
for(length = 0; length < 10; length++){
blocks[length] = init_block('R', 1, size);
}
struct layout puzzle;
puzzle = init_layout(10, blocks);
return 0;
}
I end up with an memory corruption error, as marked by the comment in init_layout().
What do I miss in my implementation?
When you are allocating memory for anything, you need to analyze, closely -- "What is it that I'm allocating memory for?"
Below, you incorrectly assume a cast of an arbitrary number block_no will adequately size the memory needed for both new_layout.blocks and new_layout.moves -- it won't:
new_layout.blocks = (struct block *)malloc(block_no);
new_layout.moves = (short int **)malloc(block_no);
What you are allocating for new_layout.blocks is actually space for struct block *blocks; (a pointer-to-struct-block), while you can malloc (block_no * sizeof (struct block)); to allocate space for block_no struct block, it is far better to allocate based upon what you are creating (i.e. space for an array new_layout.blocks (again a pointer-to-struct-block) which needs block_no * sizeof *new_layout.blocks bytes of memory to hold block_no of type struct block, e.g.:
new_layout.blocks = malloc(sizeof *new_layout.blocks * block_no);
new_layout.moves = malloc(sizeof *new_layout.moves * block_no);
(simply dereferencing the object you are allocating an array of, will accurate allow you to use sizeof to get the object (element) size for the array. (e.g. sizeof *new_layout.blocks) which you multiply by how many you need (e.g. sizeof *new_layout.blocks * block_no)
The same applies to:
new_layout.moves[i] = malloc(**new_layout.moves * 2);
(note: here you are allocating for 2 shorts, so you will need to dereference you pointer-to-pointer-to-short twice to be allocating for sizeof (short))
See Also: Do I cast the result of malloc? for thorough explanation.
For starters, this
new_layout.blocks = (struct block *)malloc(block_no);
should be
new_layout.blocks = malloc(block_no * sizeof *new_layout.blocks);
For the moves this is a bit more complicated.
Assuming short int **moves; should reference a certain number of int[2] the declaration is not optimal and better should be:
short int (*moves)[2]; /* Define a pointer to
an array with two elements of type short int. */
And allocation then should look like this:
new_layout.moves = malloc(block_no * sizeof *new_layout.moves);
Finally initialisation goes like this:
for(i = 0; i < block_no; i++){
new_layout.blocks[i] = blocks[i];
for(j = 0; j < sizeof new_layout.moves[0]/sizeof new_layout.moves[0][0]; j++)
new_layout.moves[i][j] = 0;
}
You might have noticed:
No memory allocation in the loop any more.
The magic number 2 only appears once.
:-)
I need to know if this StrBuff struct is supposed to operate like an array. I've looked and looked, and honestly can't tell just due to the pointer syntax - it seems like as an array it could work, and as an array it could not work.
I see that in the second method, malloc() is used, so I'm guessing that the buf-str uChar is supposed to be an array.
Teh codez:
typedef struct {
unsigned char *str;
unsigned int len;
} StrBuf;
static StrBuf *
strbuf_new ()
{
StrBuf *buf;
buf = (StrBuf *) calloc (sizeof (StrBuf), 1);
buf->str = (unsigned char *) strdup ("");
return buf;
}
static void
strbuf_append (StrBuf *buf, unsigned char *data, int len)
{
int offset;
if (len <= -1)
len = strlen ((char *) data);
offset = buf->len;
buf->len += len;
buf->str = (unsigned char *) realloc (buf->str, buf->len + 1);
memcpy (buf->str + offset, data, len);
buf->str[buf->len] = '\0';
}
So, judging from these methods I'm guessing for any C/C++ veterans out there this should be a piece of cake.
Edit:
My goal has been to convert an app (which uses this code here) into a Java port, but I've been quite confused as to how I should do it. I've gotten fairly far doing (for the most part) the same thing in Java, only this time using a byte[] array, seeing as how unsigned chars are supposed to be equivalent to bytes in Java.
It's not an array. It's a structure to hold values (probably strings) using dymamic memory allocation. If you use an array to allocate some datas, then array size is determined at compile time.
For example:
char buf[10];
With a structure like StrBuf you can allocate the required memory when the string buf of the given length is supplied :
buf->str = (unsigned char *) realloc (buf->str, buf->len + 1);
yesterday I had posted a question: How should I pass a pointer to a function and allocate memory for the passed pointer from inside the called function?
From the answers I got, I was able to understand what mistake I was doing.
I'm facing a new problem now, can anyone help out with this?
I want to dynamically allocate a 2D array, so I'm passing a Pointer-to-Pointer from my main() to another function called alloc_2D_pixels(...), where I use malloc(...) and for(...) loop to allocate memory for the 2D array.
Well, after returning from the alloc_2D_pixels(...) function, the pointer-to-pointer still remains NULL, so naturally, when I try accessing or try to free(...) the Pointer-to-Pointer, the program hangs.
Can anyone suggest me what mistakes I'm doing here?
Help!!!
Vikram
SOURCE:
main()
{
unsigned char **ptr;
unsigned int rows, cols;
if(alloc_2D_pixels(&ptr, rows, cols)==ERROR) // Satisfies this condition
printf("Memory for the 2D array not allocated"); // NO ERROR is returned
if(ptr == NULL) // ptr is NULL so no memory was allocated
printf("Yes its NULL!");
// Because ptr is NULL, with any of these 3 statements below the program HANGS
ptr[0][0] = 10;
printf("Element: %d",ptr[0][0]);
free_2D_alloc(&ptr);
}
signed char alloc_2D_pixels(unsigned char ***memory, unsigned int rows, unsigned int cols)
{
signed char status = NO_ERROR;
memory = malloc(rows * sizeof(unsigned char** ));
if(memory == NULL)
{
status = ERROR;
printf("ERROR: Memory allocation failed!");
}
else
{
int i;
for(i = 0; i< cols; i++)
{
memory[i] = malloc(cols * sizeof(unsigned char));
if(memory[i]==NULL)
{
status = ERROR;
printf("ERROR: Memory allocation failed!");
}
}
}
// Inserted the statements below for debug purpose only
memory[0][0] = (unsigned char)10; // I'm able to access the array from
printf("\nElement %d",memory[0][0]); // here with no problems
return status;
}
void free_2D_pixels(unsigned char ***ptr, unsigned int rows)
{
int i;
for(i = 0; i < rows; i++)
{
free(ptr[i]);
}
free(ptr);
}
One mistake is posting code that won't compile :). Below is corrected code with my comments in
/* this style */:
/* Next four lines get your code to compile */
#include <stdio.h>
#include <stdlib.h>
#define NO_ERROR 0
#define ERROR 1
/* prototypes for functions used by main but declared after main
(or move main to the end of the file */
signed char alloc_2D_pixels(unsigned char*** memory, unsigned int rows, unsigned int cols);
void free_2D_pixels(unsigned char** ptr, unsigned int rows);
/* main should return int */
int main()
{
unsigned char** ptr;
/* need to define rows and cols with an actual value */
unsigned int rows = 5, cols = 5;
if(alloc_2D_pixels(&ptr, rows, cols) == ERROR) // Satisfies this condition
printf("Memory for the 2D array not allocated"); // ERROR is returned
if(ptr == NULL) // ptr is NULL so no memory was allocated
printf("Yes its NULL!");
else
{
/* Added else clause so below code only runs if allocation worked. */
/* Added code to write to every element as a test. */
unsigned int row,col;
for(row = 0; row < rows; row++)
for(col = 0; col < cols; col++)
ptr[0][0] = (unsigned char)(row + col);
/* no need for &ptr here, not returning anything so no need to pass
by reference */
free_2D_pixels(ptr, rows);
}
return 0;
}
signed char alloc_2D_pixels(unsigned char*** memory, unsigned int rows, unsigned int cols)
{
signed char status = NO_ERROR;
/* In case we fail the returned memory ptr will be initialized */
*memory = NULL;
/* defining a temp ptr, otherwise would have to use (*memory) everywhere
ptr is used (yuck) */
unsigned char** ptr;
/* Each row should only contain an unsigned char*, not an unsigned
char**, because each row will be an array of unsigned char */
ptr = malloc(rows * sizeof(unsigned char*));
if(ptr == NULL)
{
status = ERROR;
printf("ERROR: Memory allocation failed!");
}
else
{
/* rows/cols are unsigned, so this should be too */
unsigned int i;
/* had an error here. alloced rows above so iterate through rows
not cols here */
for(i = 0; i < rows; i++)
{
ptr[i] = malloc(cols * sizeof(unsigned char));
if(ptr[i] == NULL)
{
status = ERROR;
printf("ERROR: Memory allocation failed!");
/* still a problem here, if exiting with error,
should free any column mallocs that were
successful. */
}
}
}
/* it worked so return ptr */
*memory = ptr;
return status;
}
/* no need for *** here. Not modifying and returning ptr */
/* it also was a bug...would've needed (*ptr) everywhere below */
void free_2D_pixels(unsigned char** ptr, unsigned int rows)
{
/* should be unsigned like rows */
unsigned int i;
for(i = 0; i < rows; i++)
{
free(ptr[i]);
}
free(ptr);
}
In your alloc_2D_pixels function, you need another level of indirection when accessing memory. As it is now, you only modify the parameter, not the pointer pointed to by the parameter. For example,
memory = malloc(rows * sizeof(unsigned char** ));
// becomes
*memory = malloc(rows * sizeof(unsigned char** ));
// and later...
memory[i] = malloc(cols * sizeof(unsigned char));
// becomes
(*memory)[i] = malloc(cols * sizeof(unsigned char));
(basically, anywhere you are using memory, you need to use (*memory); the parentheses are only needed when you are using subscripts to ensure that the operators are applied in the correct order)
It also looks like, You are using uninitialized rows and cols variables
Using multidimensional arrays in this way in C is "suboptimal" for performance.
In no unclear words: Please do not use - and definitely not initialize - multidimensional arrays in the way you've illustrated. Multiple calls to malloc() will create you a batch of disjoint memory locations that doesn't map well to how actual graphics (as contiguous, single buffers) are stored anywhere. Also, if you have to do it hundreds or thousands of times, malloc() can be hideously expensive.
Also, due to the fact that you're using malloc() very often, it's also a nightmare (and bug to bite you eventually) for cleaning up. You've even mentioned that in the comments in your code, and yet ... why ?
If you absolutely must have this ptr[rows][cols] thing, create it better like this:
signed char alloc_2D_pixels(unsigned char*** memory,
unsigned int rows,
unsigned int cols)
{
int colspan = cols * sizeof(char);
int rowspan = rows * sizeof(char*);
unsigned char **rowptrs = *memory = malloc(rowspan + rows * colspan));
/* malloc failure handling left to the reader */
unsigned char *payload = ((unsigned char *)rowptrs) + rowspan;
int i;
for (i = 0; i < rows; payload += colspan, i++)
rowptrs[i] = payload;
}
that way you're allocating only a single block of memory and the whole thing can be freed in one go - ditch free_2D_pixels().