I write a C code that reads a file and do some works on it, using multi-threads functions. I read file in the fun1 so I expect that file read linearly, but some tests I do on this code show me that it seems that the file does not read in the right order. What is wrong about my code?!
#include <pthread.h>
#define BUFSIZE 1024*10
#define NUM_THREADS 4
typedef struct _thread_data_t {
unsigned char id;
char *msg;
unsigned int msg_len;
} thread_data_t;
/* thread function */
void *thr_func(void *arg) {
thread_data_t *data = (thread_data_t *)arg;
fun2(data->msg, data->msg_len);
pthread_exit(NULL);
}
void fun1(FILE *file) {
unsigned char i, j, buf[BUFSIZE];
pthread_t thr[NUM_THREADS];
thread_data_t thr_data[NUM_THREADS];
int rc, fr, fd = fileno(file);
for (;;) {
for (i = 0; i < NUM_THREADS; i++) {
fr = read(fd, buf, BUFSIZE);
if (fr <= 0) break;
thr_data[i].id = i;
thr_data[i].msg = buf;
thr_data[i].msg_len = fr;
if ((rc = pthread_create(&thr[i], NULL, thr_func, &thr_data[i]))) {
fprintf(stderr, "error: pthread_create, rc: %d\n", rc);
fr = -1;
break;
}
}
for (j = 0; j < i; j++) {
pthread_join(thr[j], NULL);
}
if (fr <= 0) break;
}
}
Edit:
I think that until all threads finish their works nothing new read from the file. Is it true?
I think your problem is the single buffer:
buf[BUFSIZE];
In each loop you read data into that buffer and then prepare data for the thread
thr_data[i].msg = buf;
which I assume doesn't include a copy of the buffer itself. I assume msg is just a pointer.
So in the next iteration, you'll overwrite the buf with new data from the file and consequently change data for already created threads.
I guess you'll need
buf[NUM_THREADS][BUFSIZE];
so that each thread gets its own data area to work on.
Quote:
I think that until all threads finish their works nothing new read from the file. Is it true?
Correct, that's what pthread_join does for you
Related
I have 2 threads and they should use the same memory. Main method should start both threads. Trå A must read the contents of a file and share it with Trå B. Trå B must also receive the data that Trå A has shared and loop through and count the number of bytes in the file. Both Threads run but on the last step before the program terminates before I memory segment fault. I use Semaphore to communicate between the Threads. here i my code:
#include <stdio.h>
#include <pthread.h>
#include <stdlib.h>
#include <unistd.h>
#include <semaphore.h>
#define BUFFER_SIZE 4096
typedef struct _Buffer
{
int size;
char data[BUFFER_SIZE];
} Buffer;
sem_t task1, task2;
void *thread_A(void *arg);
void *thread_B(void *arg);
int main(int argc, char *argv[])
{
Buffer *memory = malloc(sizeof(Buffer));
sem_init(&task1, 0, 0);
sem_init(&task2, 0, 0);
pthread_t thread_A_id;
pthread_t thread_B_id;
pthread_create(&thread_A_id, NULL, &thread_A, &memory);
pthread_create(&thread_B_id, NULL, &thread_B, &memory);
if (pthread_join(thread_A_id, NULL) != 0)
{
perror("Error joining thread A");
exit(1);
}
if (pthread_join(thread_B_id, NULL) != 0)
{
perror("Error joining thread B");
exit(1);
}
free(memory);
return 0;
}
void *thread_A(void *arg)
{
Buffer *buffer = (Buffer*) arg;
FILE *pdf_file = fopen("file.pdf", "rb");
if (pdf_file == NULL)
{
perror("Can not open the file");
}
printf("size of struct %ld\n", sizeof(Buffer));
buffer->size = fread(&buffer->data, sizeof(char), BUFFER_SIZE, pdf_file);
fclose(pdf_file);
sem_post(&task1);
sem_wait(&task2);
printf("A is out\n");
return NULL;
}
void *thread_B(void *arg)
{
printf("IAM IN TREAD B");
Buffer *buffer = (Buffer*) arg;
sem_wait(&task1);
int i=0;;
int byte_counts[256] = {0};
while (buffer->size != i) {
unsigned char byte = buffer->data[i];
byte_counts[byte]++;
i++;
}
for (int i = 0; i < 256; i++)
{
printf("Byte-value %02X: %d\n", i, byte_counts[i]);
}
sem_post(&task2);
printf("threadB is done 2\n");
return NULL;
}
memory is a pointer to a Buffer (Buffer *), and by taking its address, you get a pointer to a pointer to a buffer (Buffer **):
Buffer *memory = malloc(sizeof(Buffer));
...
pthread_create(&thread_A_id, NULL, &thread_A, &memory);
pthread_create(&thread_B_id, NULL, &thread_B, &memory);
But in the thread functions, you're assuming that arg is a Buffer *:
Buffer *buffer = (Buffer*) arg;
This causes undefined behaviour.
Clearly there's one indirection too many; memory is already a pointer so we don't need to take its address:
pthread_create(&thread_A_id, NULL, &thread_A, memory);
pthread_create(&thread_B_id, NULL, &thread_B, memory);
If file fails to open, fread will return -1 and it's not checked. So the loop in thread_B will read first garbage from buffer->data and then will continue out of limit (because of comparison with -1).
So, at first, there is missing handling of error from fopen() - thread_a continues after perror, second - missing error check after fread().
By the way, the check for
if (buffer->size == i)
after while (buffer->size != i) is superfluous :)
I am learning how to use threads in C and have run into a problem when creating the threads. I am making a program that takes in 2 or more file names as command line arguments, counts the number of bytes in each file in their own thread, and then outputs the name of the largest file. When I use pthread_join() directly after creating a thread, the program runs as intended. However, I know this isn't how threads should be used because it defeats the purpose. When I use pthread_join() in a for loop after creating all the threads, then the program does not work correctly. Could anyone tell me what I am doing wrong? All help is appreciated. Here is my main function.
pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER; //mutex for changing max_bytes and max_name
int max_bytes = 0;
char max_name[100];
struct arg_struct{ //struct to hold args to pass the threads
int fd;
char name[100];
};
int main(int argc, char* argv[])
{
if(argc < 3){ //checks for correct number of arguments passed
perror("Wrong number of arguments");
return EXIT_FAILURE;
}
int arg_num = argc - 1; //holds number of arguments passed
pthread_t threadid[arg_num]; //array of thread IDs
struct arg_struct args;
for(int i = 0; i < arg_num; i++){
args.fd = open(argv[i+1], O_RDONLY);
memcpy(args.name, argv[i+1], sizeof(args.name)); //copies file name into arg_struct
int thread_err = pthread_create(&threadid[i], NULL, count_bytes, (void*)&args); //create thread by calling count_bytes and passing it a struct of args
//pthread_join(threadid[i], NULL);
if(thread_err != 0){
perror("pthread_create failed");
return EXIT_FAILURE;
}
}
for(int i = 0; i < arg_num; i++){
pthread_join(threadid[i], NULL);
}
printf("%s is the largest of the submitted files\n", max_name);
return 0;
}
This is the function that the threads are running.
void *count_bytes(void* arguments)
{
struct arg_struct *args = (struct arg_struct*)arguments; //casting arguments back to struct from void*
int fd = args -> fd;
char name[100];
memcpy(name, args -> name, sizeof(name)); //copies file name into name from args.name
int bytes = 0;
int size = 10;
char* buffer = (char*) malloc(size);
if(buffer == NULL){
perror("malloc failed");
exit(EXIT_FAILURE);
}
int buffer_count = 0;
for(int i = 0; i < size; i++){
buffer[i] = '\0'; //sets all elements to '\0' to determine end of file later
}
int read_return = read(fd, &buffer[buffer_count], 1);
if(read_return == -1){
perror("reading failed");
exit(EXIT_FAILURE);
}
while(buffer[buffer_count] != '\0'){
bytes++;
buffer_count++;
buffer[buffer_count] = '\0'; //sets previous element to '\0' to determine end of file later
if(buffer_count >= size){
buffer_count = 0; //buffer will hold up to 10 elements and then go back to the beginning
}
read_return = read(fd, &buffer[buffer_count], 1);
if(read_return == -1){
perror("reading failed");
exit(EXIT_FAILURE);
}
}
printf("%s has %d bytes\n", name, bytes);
pthread_mutex_lock(&mutex);
if(bytes > max_bytes){
max_bytes = bytes;
memcpy(max_name, name, sizeof(max_name));
}
//locks mutex to avoid race condition
//then sets bytes to max_bytes if it is later than max_bytes
//then locks mutex to allow another thread to have access
pthread_mutex_unlock(&mutex);
return NULL;
}
If it is of any use, these are the two outputs produced when it is running correctly
./a.out another buffered_readword.c
another has 8 bytes
buffered_readword.c has 3747 bytes
buffered_readword.c is the largest of the submitted files
And not correctly
./a.out another buffered_readword.c
buffered_readword.c has 1867 bytes
buffered_readword.c has 1881 bytes
buffered_readword.c is the largest of the submitted files
The problem is that there is only one args structure. After pthread_create is called the new thread may not run immediately. By the time the threads run it is likely that they will both see the same args values. Calling pthread_join inside the thread creation loop "fixes" that because it ensures each thread finishes before args is updated to the next value.
To fix properly pass a different args to each thread. Illustrative code to do that:
struct arg_struct args[arg_num];
for(int i = 0; i < arg_num; i++){
args[i].fd = open(argv[i+1], O_RDONLY);
memcpy(args[i].name, argv[i+1], sizeof(args[i].name));
int thread_err = pthread_create(&threadid[i], NULL, count_bytes, &args[i]);
....
The problem I'm running into is that when the file tries to copy, it only copies part of the file and the other part is a bunch of unreadable characters. This is for an academic assignment that wants us to use barrier synchronization so we need to use open, write, and read.
I've reworked the thread function many times but if it's the problem I can change it again, I haven't changed the for loop in main at all so even that might be the problem but I don't know what it could be. Lastly, I don't really know what to do with the barrier; my professor was very vague and I can't really ask him questions, maybe the barrier is the part that I'm truly missing.
#include <fcntl.h>
#include <signal.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <semaphore.h>
#include <pthread.h>
#include <unistd.h>
typedef struct args {
int fd;
int copy;
long int start;
long int size;
}threadarg;
int barrier = 0;
int main(int argc, char *argv[])
{
void usage(char *progname);
int chkdst(char **argv);
void die(char *why);
long int filesize(char *srcpath);
void buildpath(char *src, char *dst, char **dstpath);
int isvalid(char *path, char *dst);
void *dowork(void *arg);
if (argc < 4) usage("a8");
int workers, i;
char *check;
workers = strtol(argv[3], &check, 10);
if (!check) usage("a8");
else if (!chkdst(&argv[2])) die ("DST not valid!");
long int size = filesize(argv[1]);
if (size == -1) die ("Could not find file size");
char *dstpath; buildpath(argv[1], argv[2], &dstpath);
if (!isvalid(argv[1], dstpath)) die ("scr not valid!");
long int chunksize = size / workers;
long int remainder = size % workers;
int fd = open(argv[1], O_RDONLY);
int copy = open(dstpath, O_CREAT | O_RDWR, 0644);
if (fd < 0 || copy < 0) die("Fail to access or create files");
barrier = workers;
threadarg threadargs[workers];
pthread_t threads[workers];
for (i = 0; i < workers; i++)
{
threadargs[i].fd = fd;
threadargs[i].copy = copy;
threadargs[i].start = i * chunksize;
if (i == workers - 1)
threadargs[i].size = chunksize + remainder;
else
threadargs[i].size = chunksize;
if (pthread_create(&threads[i], NULL, dowork, (void *) &threadargs[i]))
die("Thread Creation Failure");
}
for (i = 0; i < workers; i++)
pthread_join(threads[i], NULL);
}
void usage(char *progname)
{
fprintf(stderr, "./%s srcpath dstpath workercount\n", progname);
exit(0);
}
void die(char *why)
{
fprintf(stderr, "Program Killed...\nReason: %s\n", why);
exit(1);
}
long int filesize(char *srcpath)
{
struct stat st;
if(stat(srcpath, &st) != 0) return 0;
return st.st_size;
}
/*
void domd5(char *path)
{
}
*/
void *dowork(void *arg)
{
threadarg *args = (threadarg *)arg;
int fd = args->fd,
copy = args->copy, rd;
long int start = args->start,
size = args->size;
char bufs[2048], *remains;
lseek(fd, start, SEEK_SET);
lseek(copy, start, SEEK_SET);
printf("%d thread with offset %ldKB, reached barrier\n", (int) pthread_self(), start);
barrier--;
while (barrier > 0);
long int count = 0, remainder = 0, i;
for (i = 0; i < size; i += 2048)
{
if (i + 2048 > size)
{
remainder = size - count;
remains = malloc(remainder * sizeof(char));
rd = read (fd, remains, sizeof(remains));
if (write(copy, remains, rd) != rd)
die("Error accessing files during copy");
count += remainder;
}
else
{
rd = read(fd, bufs, sizeof(bufs));
if (write(copy, bufs, rd) != rd)
die("Error accessing files during copy");
count += 2048;
}
}
pthread_exit(NULL);
}
/* Takes a single pointer, *argv, and passes it to isdir()
to check if the directory exists. If isdir returns a 1 a
1 is returned from this module. Otherwise, an error message
is printed and a 0 is returned.
Calls isdir().
Called by main().
*/
int chkdst(char **argv)
{
int isdir(char *path);
if (isdir(*argv)) return 1;
return 0;
}
/* Uses the Stat struct to construct a struct, sbuf,
and uses stat() to obtain information from the file and
write it to sbuf. Uses S_ISDIR() on sbuf.st_mode to see
the mode of the file. A 1 is returned if the file is a
directory otherwise a 0 is returned.
Called by isvalid().
*/
int isdir(char *path)
{
struct stat sbuf;
if (stat(path, &sbuf)) return 0;
return S_ISDIR(sbuf.st_mode);
}
/* Uses the Stat struct to construct a struct, sbuf,
and uses stat() to obtain information from the file and
write it to sbuf. Uses S_ISREG on sbuf.st_mode to see if
the file is regular. A 1 is returned if the S_ISREG is true
otherwise a 0 is returned.
Called by isvalid().
*/
int isregular(char *path)
{
struct stat sbuf;
if (stat(path, &sbuf)) return 0;
return S_ISREG(sbuf.st_mode);
}
/* Checks if the source path is a directory first, then if its
a regular file return 0 if it is dir and if it isn't a regular
file, then checks if the destionation path was created or if
the file exist at the destination if either return 0, if none
of these return 1.
Calls isdir() and isregular().
Called by copyfiles().
*/
int isvalid(char *path, char *dst)
{
if (isdir(path))
{
return 0;
}
else if (!isregular(path))
{
return 0;
}
else if (dst == NULL)
{
return 0;
}
else if (isregular(dst))
{
return 0;
}
return 1;
}
/* Builds destination-path using strrchr() function from library,
dstpath is null on error and defined otherwise. The src file has
its original destination removed and replaced with the new one if
it has a original destination on it otherwise it is just added to
the end of the existing name of the file.
Called by copyfiles().
*/
void buildpath(char *src, char *dst, char **dstpath)
{
char *ptr;
int n;
ptr = strrchr(src, '/');
if (ptr) n = strlen(dst) + strlen(ptr) + 2;
else n = strlen(dst) + strlen(src) + 2;
*dstpath = malloc(n);
if (!dstpath) return;
if (ptr)
{
strcpy(*dstpath, dst);
strcat(*dstpath, ptr);
}
else
{
strcpy(*dstpath, dst);
strcat(*dstpath, "/");
strcat(*dstpath, src);
}
}
I am trying to read a file with aio.h byte by byte using aio_read with a number of threads. But I don't know if I am on the right track since there are not so many stuff to read on the Internet.
I have just created a worker function to pass it to my threads. And also as an argument to pass to the thread, I created a struct called thread_arguments and I pass a few necessary arguments in it, which will be needed for aiocb such as offset, file_path to open, buffer size, and priority.
I can read a file with one thread from start to end successfully. But when it comes to reading a file as chunks from within a few threads, I couldn't make it. And I am not even sure if I can do that with aio->reqprio without using semaphores or mutexes. (Trying to open a file from within a few threads at the same time?)
How can I read a few number of bytes from within a few threads asynchronously?
Let's say the file contains "foobarquax" and we have three threads using aio library.
Then first one should read "foo",
the second should read "bar" and
the last one should read "quax" asynchronously.
You can see screenshots of issues regarding running it with multiple threads on here
#include <stdio.h>
#include <pthread.h>
#include <stdlib.h>
#include <aio.h>
#include <string.h>
#include <fcntl.h> // open -> file descriptor O_RDONLY, O_WRONLY, O_RDWR
#include <errno.h>
#include <unistd.h>
typedef struct thread_args {
char *source_path;
char *destination_path;
long int buffer_size;
long int buffer_size_last; // buffer size for the last thread in case there is a remainder.
long int offset;
int priority;
} t_args;
void *worker(void *thread_args) {
t_args *arguments = (t_args *) thread_args;
struct aiocb *aiocbRead;
aiocbRead = calloc(1, sizeof(struct aiocb));
aiocbRead->aio_fildes = open(arguments->source_path, O_RDONLY);
if (aiocbRead->aio_fildes == -1) {
printf("Error");
}
printf("opened on descriptor %d\n", aiocbRead->aio_fildes);
aiocbRead->aio_buf = malloc(sizeof(arguments->buffer_size));
aiocbRead->aio_offset = arguments->offset;
aiocbRead->aio_nbytes = (size_t) arguments->buffer_size;
aiocbRead->aio_reqprio = arguments->priority;
int s = aio_read(aiocbRead);
if (s == -1) {
printf("There was an error");
}
while (aio_error(aiocbRead) == EINPROGRESS) {}
printf("Bytes read %ld", aio_return(aiocbRead));
close(aiocbRead->aio_fildes);
for (int i = 0; i < arguments->buffer_size ; ++i) {
printf("%c\n", (*((char *) aiocbRead->aio_buf + i)));
}
}
// Returns a random alphabetic character
char getrandomChar() {
int letterTypeFlag = (rand() % 2);
if (letterTypeFlag)
return (char) ('a' + (rand() % 26));
else
return (char) ('A' + (rand() % 26));
}
void createRandomFile(char *source, int numberofBytes) {
FILE *fp = fopen(source, "w");
for (int i = 0; i < numberofBytes; i++) {
fprintf(fp, "%c", getrandomChar());
}
fclose(fp);
}
int main(int argc, char *argv[]) {
char *source_path = argv[1];
char *destination_path = argv[2];
long int nthreads = strtol(argv[3], NULL, 10);
// Set the seed.
srand(time(NULL));
// Get random number of bytes to write to create the random file.
int numberofBytes = 10 + (rand() % 100000001);
// Create a random filled file at the source path.
createRandomFile(source_path, 100);
// Calculate the payload for each thread.
long int payload = 100 / nthreads;
long int payloadLast = payload + 100 % nthreads;
// Create a thread argument to pass to pthread.
t_args *thread_arguments = (t_args *) malloc(nthreads * sizeof(t_args));
for (int l = 0; l < nthreads; ++l) {
// Set arguments in the struct.
(&thread_arguments)[l]->source_path = source_path;
(&thread_arguments)[l]->destination_path = destination_path;
(&thread_arguments)[l]->buffer_size = payload;
(&thread_arguments)[l]->buffer_size_last = payloadLast;
(&thread_arguments)[l]->offset = l * payload;
(&thread_arguments)[l]->priority = l;
}
pthread_t tID[nthreads];
// Create pthreads.
for (int i = 0; i < nthreads; ++i) {
pthread_create(&tID[i], NULL, worker, (void *) &thread_arguments[i]);
}
// Wait for pthreads to be done.
for (int j = 0; j < nthreads; ++j) {
pthread_join(tID[j], NULL);
}
free(thread_arguments);
return 0;
}
This code is reading succesfully if I just call it from one thread but doesn't work if I use it for more than one threads which is what I want.
I am testing kernel asynchronous io functions (not posix aio) and am trying to figure out how it works. The code below is a complete program where I simply write an array repeatedly to a file opened using O_DIRECT. I get an error in the callback function "write missed bytes expect 1024 got 0" (see the fprintf statement in work_done()).
For those not familiar with kernel aio, the code below does the following:
Init some structs
Prepare aio (io_prep_pwrite)
Submit io requests (io_submit)
Check for event completion (io_getevents)
Call a callback function to see if everything went ok.
I get an error at step 5. If I do not open the file using O_DIRECT, things work fine, but it beats the purpose of having async writes.
Can someone tell me what I am doing wrong? Is this the correct usage of kernel aio, for example, is my use of callbacks correct? Are there any restrictions on the usage of O_DIRECT?
I compile using 'gcc -Wall test.c -laio'
Thanks in advance.
/*
* File: myaiocp.c
* Author: kmehta
*
* Created on July 11, 2011, 12:50 PM
*
*
* Testing kernel aio.
* Program creates a 2D matrix and writes it multiple times to create a file of desired size.
* Writes are performed using kernel aio functions (io_prep_pwrite, io_submit, etc.)
*/
#define _GNU_SOURCE
#define _XOPEN_SOURCE 600
#include <stdio.h>
#include <stdlib.h>
#include <getopt.h>
#include <pthread.h>
#include <fcntl.h>
#include <string.h>
#include <sys/uio.h>
#include <sys/time.h>
#include <omp.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <errno.h>
#include <libaio.h>
char ** buf;
long seg_size;
int seg_rows;
double total_size;
char * filename;
static int wait_count = 0;
void io_task();
void cleanup();
void allocate_2D_matrix(int[]);
int file_open(char *);
void wr_done(io_context_t ctx, struct iocb* iocb, long res, long res2);
int main(int argc, char **argv) {
total_size = 1048576; //1MB
seg_size = 1024; //1kB
seg_rows = 1024;
filename = "aio.out";
int dims[] = {seg_rows, seg_size};
allocate_2D_matrix(dims); //Creates 2D matrix
io_task();
cleanup();
return 0;
}
/*
* Create a 2D matrix
*/
void allocate_2D_matrix(int dims[2]) {
int i;
char *data;
//create the matrix
data = (char *) calloc(1, dims[0] * dims[1] * sizeof (char));
if (data == NULL) {
printf("\nCould not allocate memory for matrix.\n");
exit(1);
}
buf = (char **) malloc(dims[0] * sizeof (char *));
if (buf == NULL) {
printf("\nCould not allocate memory for matrix.\n");
exit(1);
}
for (i = 0; i < dims[0]; i++) {
buf[i] = &(data[i * dims[1]]);
}
}
static void io_error(const char *func, int rc)
{
if (rc == -ENOSYS)
fprintf(stderr, "AIO not in this kernel\n");
else if (rc < 0)
fprintf(stderr, "%s: %s\n", func, strerror(-rc));
else
fprintf(stderr, "%s: error %d\n", func, rc);
exit(1);
}
/*
* Callback function
*/
static void work_done(io_context_t ctx, struct iocb *iocb, long res, long res2)
{
if (res2 != 0) {
io_error("aio write", res2);
}
if (res != iocb->u.c.nbytes) {
fprintf(stderr, "write missed bytes expect %lu got %ld\n",
iocb->u.c.nbytes, res2);
exit(1);
}
wait_count --;
printf("%d ", wait_count);
}
/*
* Wait routine. Get events and call the callback function work_done()
*/
int io_wait_run(io_context_t ctx, long iter)
{
struct io_event events[iter];
struct io_event *ep;
int ret, n;
/*
* get up to aio_maxio events at a time.
*/
ret = n = io_getevents(ctx, iter, iter, events, NULL);
printf("got %d events\n", n);
/*
* Call the callback functions for each event.
*/
for (ep = events ; n-- > 0 ; ep++) {
io_callback_t cb = (io_callback_t)ep->data ; struct iocb *iocb = ep->obj ; cb(ctx, iocb, ep->res, ep->res2);
}
return ret;
}
void io_task() {
long offset = 0;
int bufIndex = 0;
//Open file
int fd = file_open(filename);
//Initialize structures
long i;
long iter = total_size / seg_size; //No. of iterations to reach desired file size (total_size)
io_context_t myctx;
if(0 != io_queue_init(iter, &myctx))
{
perror("Could not initialize io queue");
exit(EXIT_FAILURE);
}
struct iocb * ioq[iter];
//loop through iter times to reach desired file size
for (i = 0; i < iter; i++) {
struct iocb *io = (struct iocb*) malloc(sizeof (struct iocb));
io_prep_pwrite(io, fd, buf[bufIndex], seg_size, offset);
io_set_callback(io, work_done);
ioq[i] = io;
offset += seg_size;
bufIndex ++;
if (bufIndex > seg_rows - 1) //If entire matrix written, start again from index 0
bufIndex = 0;
}
printf("done preparing. Now submitting..\n");
if(iter != io_submit(myctx, iter, ioq))
{
perror("Failure on submit");
exit(EXIT_FAILURE);
}
printf("now awaiting completion..\n");
wait_count = iter;
int res;
while (wait_count) {
res = io_wait_run(myctx, iter);
if (res < 0)
io_error("io_wait_run", res);
}
close(fd);
}
void cleanup() {
free(buf[0]);
free(buf);
}
int file_open(char *filename) {
int fd;
if (-1 == (fd = open(filename, O_DIRECT | O_CREAT | O_WRONLY | O_TRUNC, 0666))) {
printf("\nError opening file. \n");
exit(-1);
}
return fd;
}
First of all, good job using libaio instead of POSIX aio.
Are there any restrictions on the usage of O_DIRECT ?
I'm not 100% sure this is the real problem, but O_DIRECT has some requirements (quoting mostly from TLPI):
The data buffer being transferred must be aligned on a memory boundary that is a multiple of the block size (use posix_memalign)
The file or device offset at which data transfer commences must be a multiple of the block size
The length of the data to be transferred must be a multiple of the block size
At a glance, I can see you are not taking aby precautions to align memory in allocate_2D_matrix.
If I do not open the file using O_DIRECT, things work fine, but it
beats the purpose of having async writes.
This happens not to be the case. Asynchronous I/O works well without O_DIRECT (for instance think of the number of system calls slashed).