I'm hoping that someone can help me out. I have not written much in C code in over a decade and just picked this back up 2 days ago so bear with me please as I am rusty. THANK YOU!
What:
I'm working on creating a very simple thread pool for an application. This code is written in C on CodeBlocks using GNU GCC for the compiler. It is built as a command line application. No additional files are linked or included.
The code should create X threads (in this case I have it set to 10) each of which sits and waits while watching an array entry (identified by the threads thread index or count) for any incoming data it might need to process. Once a given child has processed the data coming in via the array there is no need to pass the data back to the main thread; rather the child should simply reset that array entry to 0 to indicate that it is ready to process another input. The main thread will receive requests and will dole them out to whatever thread is available. If none are available then it will refuse to handle that input.
For simplicity sake the code below is a complete and working but trimmed and gutted version that DOES exhibit the stack overflow I am trying to track down. This compiles fine and initially runs fine but after a few passes the threadIndex value in the child thread process (workerThread) becomes corrupt and jumps to weird values - generally becoming the number of milliseconds I have put in for the 'Sleep' function.
What I have checked:
The threadIndex variable is not a global or shared variable.
All arrays are plenty big enough to handle the max number of threads I am creating.
All loops have the loopvariable reset to 0 before running.
I have not named multiple variables with the same name.
I use atomic_load to make sure I don't write to the same global array variable with two different threads at once please note I am rusty... I may be misunderstanding how this part works
I have placed test cases all over to see where the variable goes nuts and I am stumped.
Best Guess
All of my research confirms what I recall from years back; I likely am going out of bounds somewhere and causing stack corruption. I have looked at numerous other problems like this on google as well as on stack overflow and while all point me to the same conclusion I have been unable to figure out what specifically is wrong in my code.
#include<stdio.h>
//#include<string.h>
#include<pthread.h>
#include<stdlib.h>
#include<conio.h>
//#include<unistd.h>
#define ESCAPE 27
int maxThreads = 10;
pthread_t tid[21];
int ret[21];
int threadIncoming[21];
int threadRunning[21];
struct arg_struct {
char* arg1;
int arg2;
};
//sick of the stupid upper/lowercase nonsense... boom... fixed
void* sleep(int time){Sleep(time);}
void* workerThread(void *arguments)
{
//get the stuff passed in to us
struct arg_struct *args = (struct arg_struct *)arguments;
char *address = args -> arg1;
int threadIndex = args -> arg2;
//hold how many we have processed - we are unlikely to ever hit the max so no need to round robin this number at this point
unsigned long processedCount = 0;
//this never triggers so it IS coming in correctly
if(threadIndex > 20){
printf("INIT ERROR! ThreadIndex = %d", threadIndex);
sleep(1000);
}
unsigned long x = 0;
pthread_t id = pthread_self();
//as long as we should be running
while(__atomic_load_n (&threadRunning[threadIndex], __ATOMIC_ACQUIRE)){
//if and only if we have something to do...
if(__atomic_load_n (&threadIncoming[threadIndex], __ATOMIC_ACQUIRE)){
//simulate us doing something
//for(x=0; x<(0xFFFFFFF);x++);
sleep(2001);
//the value going into sleep is CLEARLY somehow ending up in index because you can change that to any number you want
//and next thing you know the next line says "First thread processing done on (the value given to sleep)
printf("\n First thread processing done on %d\n", threadIndex);
//all done doing something so clear the incoming so we can reuse it for our next one
//this error should not EVER be able to get thrown but it is.... something is corrupting our stack and going into memory that it shouldn't
if(threadIndex > 20){ printf("ERROR! ThreadIndex = %d", threadIndex); }
else{ __atomic_store_n (&threadIncoming[threadIndex], 0, __ATOMIC_RELEASE); }
//increment the processed count
++processedCount;
}
else{Sleep(10);}
}
//no need to do atomocity I don't think for this as it is only set on the exit and not read till after everything is done
ret[threadIndex] = processedCount;
pthread_exit(&ret[threadIndex]);
return NULL;
}
int main(void)
{
int i = 0;
int err;
int *ptr[21];
int doLoop = 1;
//initialize these all to set the threads to running and the status on incoming to NOT be processing
for(i=0;i < maxThreads;i++){
threadIncoming[i] = 0;
threadRunning[i] = 1;
}
//create our threads
for(i=0;i < maxThreads;i++)
{
struct arg_struct args;
args.arg1 = "here";
args.arg2 = i;
err = pthread_create(&(tid[i]), NULL, &workerThread, (void *)&args);
if (err != 0){ printf("\ncan't create thread :[%s]", strerror(err)); }
}
//loop until we hit escape
while(doLoop){
//see if we were pressed escape
if(kbhit()){ if(getch() == ESCAPE){ doLoop = 0; } }
//just for testing - actual version would load only as needed
for(i=0;i < maxThreads;i++){
//make sure we synchronize so we don't end up pointing into a garbage address or half loading when a thread accesses us or whatever was going on
if(!__atomic_load_n (&threadIncoming[i], __ATOMIC_ACQUIRE)){
__atomic_store_n (&threadIncoming[i], 1, __ATOMIC_RELEASE);
}
}
}
//exiting...
printf("\n'Esc' pressed. Now exiting...\n");
//call to end them all...
for(i=0;i < maxThreads;i++){ __atomic_store_n (&threadRunning[i], 0, __ATOMIC_RELEASE); }
//join them all back up - if we had an actual worthwhile value here we could use it
for(i=0;i < maxThreads;i++){
pthread_join(tid[i], (void**)&(ptr[i]));
printf("\n return value from thread %d is [%d]\n", i, *ptr[i]);
}
return 0;
}
Output
Here is the output I get. Note that how long it takes before it starts going crazy does seem to possibly vary but not much.
Output Screen with Error
I don't trust your handling of args, there seems to be a race condition. What if you create N threads before the first one of them gets to run? Then the first thread created will probably see the args for the N:th thread, rather than for the first, and so on.
I don't believe there's a guarantee that automatic variables used in a loop like that are created in non-overlapping areas; after all they go out of scope with each iteration of the loop.
Related
I have a program that communicate trough a TCP socket, with a server and a client.
Besides other things I have a buffer with pendent requests from the client, and I have also one thread that prints the requests that are being placed in the buffer by the main-thread.
So, for example, I have 3 requests for print 3 files, and the printer_thread have to print the 3 files one after the other. For doing this, I have a function "get", that get the the file to print and a function "put" that put the files in the buffer. When I do the get of something of the buffer it works pretty well and the printing of the file works too.
The problem arises when the client want to know how many files are in the buffer to be printed. I need to have a counter that any time that a put a thing in the buffer it increment, and any time that I get something it decrement, something easy.
But it doesn't work, my program only increment the variable and doesn't make any decrement.
int count = 0;
struct prodcons buffer;
/* some other code that is not important for now and works well */
void main_thread(int port_number){
/* more code */
put(&buffer, f_open);
count++; ------> it increment every time that I do a put
nw = myWriteSocket(sc, "File has been Queued.", ARGVMAX);
/* more code */
void *printing(void *arg){
/* variables and other code that works */
file_desc = get(&buffer);
count--; ---> now it never decrement, but the get is working because the files are printed
int main (int argc, char *argv[]) {
/* more code */
pthread_create(&printer_thread,NULL,printing, (void *)terminal);
main_thread(port_number);
What can be the problem? Why the get is working and all is working too and the count-- doesn't???
Sorry if the question is not well structured.
I am a newbie to CGI programming in C. I was trying to generate a random number and keep displaying on the web-page. I made a program in C compiled it and have place in the path of fedora core /var/www/cgi-bin/rand_mumber.cgi. I can access this cgi page via local host and the results are fine. But, when I try to access the web page from another machine on the network, a problem is seen. The webpage keeps loading the data and finally times out. What my analysis says is that the web-browser will not upload the page until the program has terminated. In the example below, the program is never-terminating. Also, one example of this kind is the local time after Google -search, the local time doesn't keep updating by its own. It needs a refresh click. But my
requirement is that the page should keep updating constantly not waiting for the completion or a refresh click. As the random number is generated it should be displayed on web-page and then again should read for another random number. This process should keep repeating rather than the web-page waiting for a program termination.
Here is the program, which I compiled and renamed as rand.cgi
Please advice me a method to keep updating and displaying the web-page.
It should constantly keep updating the webpage when the random number is generated.
It's a never ending program.
#include"stdio.h"
#include"pthread.h"
#include"unistd.h"
pthread_t tid;
void *print_series_forever()
{
int n;
int k = 0;
while( 1 )
{
n = rand()%100 + 1;
printf("%d",n);
k++;
if ( k == 110 )
{
printf("\n"); /** Enter next line **/
k = 0;
}
usleep(100);
}
}
int main(int argc, char **argv)
{
int err;
err = pthread_create(&tid,NULL,&print_series_forever,NULL);
pthread_join(tid,NULL);
sleep(2);
return 0;
}
Any help is welcome.
I am studying mutexes and I am stuck in an exercise. For each file in a given directory, I have to create a thread to read it and display its contents (no problem if order is not correct).
So far, the threads are running this function:
void * reader_thread (void * arg)
{
char * file_path = (char*)arg;
FILE * f;
char temp[20];
int value;
f=fopen(file_path, "r");
printf("Opened %s.\n",file_path);
while (fscanf(f, "%s",temp)!=EOF)
if (!get_number (temp, &value)) /*Gets int value from given string (if numeric)*/
printf("Thread %lu -> %s: %d\n", pthread_self(), file_path, value );
fclose(f);
pthread_exit(NULL);
}
Being called by a function that receives a DIR pointer, previously created by opendir().
(I have omitted some error checking here to make it cleaner, but I get no error at all.)
int readfiles (DIR * dir, char * path)
{
struct dirent * temp = NULL;
char * file_path;
pthread_t thList [MAX_THREADS];
int nThreads=0, i;
memset(thList, 0, sizeof(pthread_t)*MAX_THREADS);
file_path=malloc((257+strlen(path))*sizeof(char));
while((temp = readdir (dir))!=NULL && nThreads<MAX_THREADS) /*Reads files from dir*/
{
if (temp->d_name[0] != '.') /*Ignores the ones beggining with '.'*/
{
get_file_path(path, temp->d_name, file_path); /*Computes rute (overwritten every iteration)*/
printf("Got %s.\n", file_path);
pthread_create(&thList[nThreads], NULL, reader_thread, (void * )file_path)
nThreads++;
}
}
printf("readdir: %s\n", strerror (errno )); /*Just in case*/
for (i=0; i<nThreads ; i++)
pthread_join(thList[i], NULL)
if (file_path)
free(file_path);
return 0;
}
My problem here is that, although paths are computed perfectly, the threads don't seem to receive the correct argument. They all read the same file. This is the output I get:
Got test/testB.
Got test/testA.
readdir: Success
Opened test/testA.
Thread 139976911939328 -> test/testA: 3536
Thread 139976911939328 -> test/testA: 37
Thread 139976911939328 -> test/testA: -38
Thread 139976911939328 -> test/testA: -985
Opened test/testA.
Thread 139976903546624 -> test/testA: 3536
Thread 139976903546624 -> test/testA: 37
Thread 139976903546624 -> test/testA: -38
Thread 139976903546624 -> test/testA: -985
If I join the threads before the next one begins, it works OK. So I assume there is a critical section somewhere, but I don't really know how to find it. I have tried mutexing the whole thread function:
void * reader_thread (void * arg)
{
pthread_mutex_lock(&mutex_file);
/*...*/
pthread_mutex_unlock(&mutex_file);
}
And also, mutexing the while loop in the second function. Even both at the same time. But it won't work in any way. By the way, mutex_file is a global variable, which is init'd by pthread_mutex_init() in main().
I would really appreciate a piece of advice with this, as I don't really know what I'm doing wrong. I would also appreciate some good reference or book, as mutexes and System V semaphores are feeling a bit difficult to me.
Thank you very much.
Well, you are passing exactly the same pointer as file path to both threads. As a result, they read file name from the same string and end up reading the same file. Actually, you get a little bit lucky here because in reality you have a race condition — you update the contents of the string pointer by file_path while firing up threads that read from that pointer, so you may end up with a thread reading that memory while it is being changed. What you have to do is allocate an argument for each thread separately (i.e. call malloc and related logic in your while loop), and then free those arguments once thread is exited.
Looks like you're using the same file_path buffer for all threads, just loading it over and over again with the next name. You need to allocate a new string for each thread, and have each thread delete the string after using it.
edit
Since you already have an array of threads, you could just make a parallel array of char[], each holding the filename for the corresponding thread. This would avoid malloc/free.
I'm working on a class project in which I must write a command line shell with the following requirements:
The shell must able to read buffered input
Buffer should be 64 characters
Error conditions should be handled
Exceeded buffer size
Interruptions (when a signal arrives) – see the man page for read()
Invalid input (unparsable characters, blank lines, etc)
Any other error that may be encountered.
Shell must have a history of at least 20 items, and the history must not be of a static size. When the history buffer is full, the oldest item is removed and the newest item added.
Programs should be able to run in the foreground or background. (using &)
Ctrl-D will exit the shell
Ctrl-C will print the complete history
The Command ‘history’ will also print the complete history. Newest items will be at the bottom of the list.
All other signals will be trapped and displayed to the user in the shell
Program will use the read() command to read in input, unless the arrow keys are supported
I have opted to implement arrow keys for history cycling, so I'm using ncurses for input, rather than read(). I think I'm doing all right using strtok() to parse input, and fork() and execvp() to run the processes, but I'm not doing all right implementing ncurses correctly. All I've gotten it to do so far is init a new screen, display the prompt, then segfault upon any key press. Not good.
I reckon the problem must be in my design. I'm not wrapping my head around ncurses too well. What sort of data structures should I be using for this project? How should I handle the ncurses setup, teardown, and everything in between? What's the deal with windows and screens, and should I have a single globally accessible window/screen that I work with? Also, I've been trying to use a char* for the input buffer, and a char** for the command history, but I have no experience in C, so despite reading up on malloc, calloc, and realloc, I'm not sure of the best way to store commands in the buffer and the history. Any tips on managing these char arrays?
tl;dr: How do I use ncurses correctly to make a command line shell, and how do I handle the command memory management with C?
I realize this is a pretty hefty question. :(
edit: I have already seen http://www.gnu.org/software/libc/manual/html_node/Implementing-a-Shell.html and http://www.linuxinfor.com/english/NCURSES-Programming/ but the ncurses documentation has actually too much overhead. I just want to use its ability to recognize arrow keys.
Here's some sample code which:
Performs dynamic memory allocation.
Reads from the console in non-blocking mode.
Uses VT100 codes to print a frame buffer to the console.
It compiles on Linux using GCC without warnings or errors. It's far from bug free, but it should give you some ideas of what's possible. Compile and run it, pressing [up] and [down] will print messages, typing characters and hitting [enter] will "execute" the command.
#include <poll.h>
#include <signal.h>
#include <stdio.h>
#include <termios.h>
#include <unistd.h>
#include <stdlib.h>
#include <string.h>
#include <time.h>
/** VT100 command to clear the screen. Use puts(VT100_CLEAR_SCREEN) to clear
* the screen. */
#define VT100_CLEAR_SCREEN "\033[2J"
/** VT100 command to reset the cursor to the top left hand corner of the
* screen. */
#define VT100_CURSOR_TO_ORIGIN "\033[H"
struct frame_s
{
int x;
int y;
char *data;
};
static int draw_frame(struct frame_s *frame)
{
int row;
char *data;
int attrib;
puts(VT100_CLEAR_SCREEN);
puts(VT100_CURSOR_TO_ORIGIN);
for (row = 0, data = frame->data; row < frame->y; row++, data += frame->x)
{
/* 0 for normal, 1 for bold, 7 for reverse. */
attrib = 0;
/* The VT100 commands to move the cursor, set the attribute, and the
* actual frame line. */
fprintf(stdout, "\033[%d;%dH\033[0m\033[%dm%.*s", row + 1, 0, attrib, frame->x, data);
fflush(stdout);
}
return (0);
}
int main(void)
{
const struct timespec timeout = { .tv_sec = 1, .tv_nsec = 0 };
struct frame_s frame;
struct termios tty_old;
struct termios tty_new;
unsigned char line[128];
unsigned int count = 0;
int ret;
struct pollfd fds[1];
sigset_t sigmask;
struct tm *tp;
time_t current_time;
/* Set up a little frame. */
frame.x = 80;
frame.y = 5;
frame.data = malloc(frame.x * frame.y);
if (frame.data == NULL)
{
fprintf(stderr, "No memory\n");
exit (1);
}
memset(frame.data, ' ', frame.x * frame.y);
/* Get the terminal state. */
tcgetattr(STDIN_FILENO, &tty_old);
tty_new = tty_old;
/* Turn off "cooked" mode (line buffering) and set minimum characters
* to zero (i.e. non-blocking). */
tty_new.c_lflag &= ~ICANON;
tty_new.c_cc[VMIN] = 0;
/* Set the terminal attributes. */
tcsetattr(STDIN_FILENO, TCSANOW, &tty_new);
/* Un-mask all signals while in ppoll() so any signal will cause
* ppoll() to return prematurely. */
sigemptyset(&sigmask);
fds[0].events = POLLIN;
fds[0].fd = STDIN_FILENO;
/* Loop forever waiting for key presses. Update the output on every key
* press and every 1.0s (when ppoll() times out). */
do
{
fds[0].revents = 0;
ret = ppoll(fds, sizeof(fds) / sizeof(struct pollfd), &timeout, &sigmask);
if (fds[0].revents & POLLIN)
{
ret = read(STDIN_FILENO, &line[count], sizeof(line) - count);
if (ret > 0)
{
line[count + ret] = '\0';
if (strcmp(&line[count], "\033[A") == 0)
{
snprintf(frame.data, frame.x, "up");
count = 0;
}
else if (strcmp(&line[count], "\033[B") == 0)
{
snprintf(frame.data, frame.x, "down");
count = 0;
}
else if (line[count] == 127) // backspace
{
if (count != 0) { count -= ret;}
}
else if (line[count] == '\n')
{
snprintf(frame.data, frame.x, "entered: %s", line);
count = 0;
}
else
{
count += ret;
}
}
}
/* Print the current time to the output buffer. */
current_time = time(NULL);
tp = localtime(¤t_time);
strftime(&frame.data[1 * frame.x], frame.x, "%Y/%m/%d %H:%M:%S", tp);
/* Print the command line. */
line[count] = '\0';
snprintf(&frame.data[(frame.y - 1) * frame.x], frame.x, "$ %s", line);
draw_frame(&frame);
}
while (1);
/* Restore terminal and free resources. */
tcsetattr(STDIN_FILENO, TCSANOW, &tty_old);
free(frame.data);
return (0);
}
If your input buffer is defined to be 64 characters, then I would recommend using a char array instead of a char*. Something like char input_buffer[65]; should serve your purposes (add an extra character for the trailing '\0').
As far as command history goes, you can use a two-dimensional array for that. Something like char command_history[20][65]; should let you store 20 old commands of 64 characters each.
Allocating these buffers statically should make things a bit easier for you, as you won't have to worry about malloc and friends.
It's hard to give you too much specific advice without seeing your code. I have a feeling that you are making the same type of mistakes that are typical to people first learning C. Can you post the part of your code that is giving you problems so that we can learn more about what you are doing?
Update after posted provided code:
One problem I'm seeing is that the function takeInput doesn't have a return statement. When you use input = takeInput(); inside your main function, the value of input isn't being set to what you think it is. It's probably not a valid pointer, which is causing your line that says input[j] to segfault.
Your usage of cmdHistory also needs revisiting. You allocate it with cmdHistory = (char**)calloc(21,sizeof(int));, which gives you enough space to store 21 integers. In the function printHistory, you pass elements of cmdHistory to printw as if they were strings (they're only integers). This is most definitely not doing what you want it to do. Instead, your allocation logic for cmdHistory needs to look more like your de-allocation logic (except backwards). Allocate an array of char**, then iterate through the array, assigning each pointer to a newly-allocated buffer. Just like you have one free statement for each element in the array plus a free for the array as a whole, you should have one malloc for each element plus one malloc for the array as a whole.
Even if you can't use a statically-allocated stack, try writing your program using one anyway. This will let you work the kinks out of your key detection logic, etc without having to worry about the dynamic memory part of the program. Once the rest of it is working, go back in and swap out the static memory for dynamic memory allocation. That way, you're only having to debug a little bit at a time.
Have you looked at the Readline library? It's ideal for use in your project.
http://cnswww.cns.cwru.edu/php/chet/readline/rltop.html
I have a bit of queue code that I was working on. I was trying to use a global int to keep track of the queue's size.
#define MAX 100
int size=0;
int gEnqueue=gDequeue=0;
int enqueue()
{
gEnqueue++;
if( size == MAX )
return QUEUE_FULL;
/* snip the actual queue handling */
size++;
return 0;
}
int dequeue()
{
gDequeue++;
if(!size)
return QUEUE_EMPTY;
/* snip actual queue handling */
if(size)
size--;
return 0;
}
there is of course much more code then that, but too much to post.
What is happening is the size gets stuck at the max I have set. Both functions get called an even number of times. If I dump the queue I can see that there are only 3 items in it.
What would cause this problem?
edit #1: made the code example match what I actually coded
This is not threaded.
edit #2: I am an idiot and should have done this instead of assuming.
I was wrong about the calls being even to the enqueue() and dequeue().
Note to self, use real metrics not guesses.
If you can't use a debugger I would suggest adding print statements inside both functions showing what size equals and then after running the program examine the output. Usually when looking at the print log the problem is pretty obvious.
The easiest solution is not to call "enqueue" if size==MAX.
But if that's not possible try this:
int size=0;
int overflow=0;
int enqueue()
{
if( size < MAX )
size++;
else
overflow++;
return 0;
}
int dequeue()
{
if(overflow)
overflow--;
else if(size)
size--;
return 0;
}
There's nothing obviously wrong with the code you posted, so this suggests there's something wrong with the code you snipped, or in the way you're calling the code. You'll have to debug this for yourself. There are two main debugging techniques that would help you at this point:
As #KPexEA suggested, debugging using printf() or other logging statements. Put a printf() at the beginning and end of both functions, printing out as much state as you think might possibly be useful.
int enqueue()
{
printf("enqueue(): Enter: size=%d\n", size);
if( size == MAX ) {
printf("enqueue(): Exit: QUEUE_FULL\n");
return QUEUE_FULL;
}
/* snip the actual queue handling */
size++;
printf("enqueue(): Exit: size=%d\n", size);
return 0;
}
int dequeue()
{
printf("dequeue(): Enter: size=%d\n", size);
if(!size) {
printf("dequeue(): QUEUE_EMPTY\n");
return QUEUE_EMPTY;
}
/* snip actual queue handling */
if(size)
size--;
printf("dequeue(): Exit: size=%d\n", size);
return 0;
}
By examining the output, it should become apparent what's happening with the size of your queue. (You could also count the actual number of elements in your queue and print that when you enter and exit your functions.)
The other technique is interactive debugging. This is especially useful to determine exactly how your code is flowing, but you have to sit there every time you run your program to watch how it's running. (If your bug occurs every time, that's easy; if it occurs every once and a while, it's hard to go back and recreate your program's flow after the fact.) Set a breakpoint at the beginning of each of your functions and use the debugger to display the value size. Set another breakpoint at the end of each function and make sure (1) the breakpoint actually gets hit, and (2) your expectations of any changes made to size are met.