I observed something in a log file that I cannot explain:
All code in project is ANSI C, 32bit exe running on Windows 7 64bit
I have a worker function similar to this one, running in a single threaded program, using no recursion. During debugging logging was included as shown:
//This function is called from an event handler
//triggered by a UI timer similar in concept to
//C# `Timer.OnTick` or C++ Timer::OnTick
//with tick period set to a shorter duration
//than this worker function sometimes requires
int LoadState(int state)
{
WriteToLog("Entering ->"); //first call in
//...
//Some additional code - varies in execution time, but typically ~100ms.
//...
WriteToLog("Leaving <-");//second to last call out
return 0;
}
The function above is simplified from our actual code but is sufficient for illustrating the issue.
Occasionally we have seen log entries such as this:
Where the time/date stamp is on the left, then message, the last field is duration in clock() ticks between calls to logging function. This logging indicates that the function was entered twice in a row before exiting.
Without recursion, and in a single threaded program, how is it (or is it) possible that execution flow can enter a function twice before the first call was completed?
EDIT: (to show top call of logging function)
int WriteToLog(char* str)
{
FILE* log;
char *tmStr;
ssize_t size;
char pn[MAX_PATHNAME_LEN];
char path[MAX_PATHNAME_LEN], base[50], ext[5];
char LocationKeep[MAX_PATHNAME_LEN];
static unsigned long long index = 0;
if(str)
{
if(FileExists(LOGFILE, &size))
{
strcpy(pn,LOGFILE);
ManageLogs(pn, LOGSIZE);
tmStr = calloc(25, sizeof(char));
log = fopen(LOGFILE, "a+");
if (log == NULL)
{
free(tmStr);
return -1;
}
//fprintf(log, "%10llu %s: %s - %d\n", index++, GetTimeString(tmStr), str, GetClockCycles());
fprintf(log, "%s: %s - %d\n", GetTimeString(tmStr), str, GetClockCycles());
//fprintf(log, "%s: %s\n", GetTimeString(tmStr), str);
fclose(log);
free(tmStr);
}
else
{
strcpy(LocationKeep, LOGFILE);
GetFileParts(LocationKeep, path, base, ext);
CheckAndOrCreateDirectories(path);
tmStr = calloc(25, sizeof(char));
log = fopen(LOGFILE, "a+");
if (log == NULL)
{
free(tmStr);
return -1;
}
fprintf(log, "%s: %s - %d\n", GetTimeString(tmStr), str, GetClockCycles());
//fprintf(log, "%s: %s\n", GetTimeString(tmStr), str);
fclose(log);
free(tmStr);
}
}
return 0;
}
I asked the question wondering at the time if there was some obscure part of the C standard that allowed execution flow to enter a function more than once without first exiting (given that multi-threading or recursion was not present)
Your comments, I believe, have clearly answered the question. Borrowing from what #Oli Charlesworth said in one comment, he summarizes it up pretty well:
If the code is truly single-threaded, and the log-function is truly sane, and there's truly no other piece of code that can be outputting to the log, then obviously this can't happen (UB notwithstanding).
But since the actual log files (which I could not post for proprietary reasons) on several occasions have demonstrated this pattern, one of the conditions #Oli Charlesworth listed is not actually true for our software. My best guess at this point, given that the logging function is sane and is the only input to the file, is to consider the alternate context/Fiber possibility suggested by #jxh:
"Primary thread only" can mean multiple things. The library could still possibly use <ucontext.h> on POSIX, or Fibers on Windows.
So, I will post this same question to the supplier of my environment, specifically if their UI Timers are run in such a way as to allow parallel calls due to a fiber or thread.
If any are interested, I will also update this answer with their response.
Edit to show conclusion:
As it turns out, the cause of the double entry of execution flow into a function was implicit recursion. That is, while the worker function did not reference itself explicitly, it was designated as the event handler for two separate event generators. That, coupled with a call to Process System Events (a function available in our environment forcing events in the queue to be processed now) can (and did) result in recursive execution flow into the event handler function. Here is a quote from a person who has expertise in the relationship between UI timers and system events in our environment:
"Timer events being nested" does equate to execution flow entering a function twice before leaving. Basically, it's the same thing as basic recursion: while you're inside one function, you call that same function. The only difference between this case and basic recursion is that the recursion call is implicit (via ProcessSystemEvents) and not explicit. But the end result is the same."
Related
I am not even sure if threads are a way to go with what I am trying to accomplish, but my intuition is telling me so.
I am implemeting a simple input in a while loop, character by character. If the time between the character input is greater than 2 seconds, the timeout should occur. The timeout is currently a simple printf in the main function.
This is my code:
typedef struct {
clock_t startTime;
} timerStruct;
void *TimerThread(void *arg) {
timerStruct timerThreadArgument = *((timerStruct *) arg);
clock_t differenceTime;
while(1) {
differenceTime = clock() - timerThreadArgument.startTime;
int millis = differenceTime * 1000 / CLOCKS_PER_SEC;
if (millis >= TIMEOUT_TIME) {
return (void *) 2;
}
}
}
int main() {
pthread_t timerThreadId;
void *threadReturn;
char inputChar;
printf("Input characters one by one or paste the string:\n");
while (1) {
timerStruct *timerThreadArgument = malloc(sizeof(*timerThreadArgument));
timerThreadArgument->startTime = clock();
pthread_create(&timerThreadId, NULL, TimerThread, timerThreadArgument);
pthread_join(timerThreadId, &threadReturn);
if ((int) threadReturn == 2) {
printf("Timeout!\n");
}
scanf(" %c", &inputChar);
}
}
The problem is, since I am using the pthread_join function, it blocks the main thread from executing and asking for input. I am aware why this is happening. If I don't use the pthread_join function, I am not going to be able to return the data from the thread, which is important because I wish to break the while loop if the timeout has occurred.
If anyone has an idea on how I could approach this, please share it.
Thank you in advance for your time.
There are a number of problems with what you are attempting. One of them is that stdio (ie. scanf) involves buffering; so you won't get a single character until a whole line is available. The other is that what you have implemented is more akin to sleep() than a timeout.
In order to have direct control over tty input, you need to inform the device driver in the kernel of your desire. The standard interface to this is tcgetattr, tcsetattr; which enables rather fine grained control of the tty. The n?curses library is a little coarser, but much, much easier to use.
There are a lot of advantages to designing your system as a set of asynchronous services which provide events to one another. In this, your keyboard handling might occupy a thread sending "Key Events" to some sort of dispatcher; and maybe another sending "Alarm Events" around to track timeouts. This isn't a particularly easy pattern though. Its value only comes to the fore in relatively large systems.
Some newer programming languages, Go for instance, have constructs specifically geared to this sort of architecture. You might find it worthwhile to look at it, at least to get a flavour of how tidily it can be accomplished.
I have a question concerning scope and how memory is handled in C. Example:
int main(){
int some,
instance,
variables;
{ // scoped block to do some work
int more,
data,
just,
forThisBlock;
}
// ...
}
Essentially my question is this.... Does it make sense to use a block as above to section off areas to instantiate temporarily used data? Memory-wise, what happens to the variables allocated in the block? Are they de-allocated after the block exits, ostensibly saving memory further on in the function? I realize a proper function could also serve the purpose, but in some cases adding another function is not as neat if the work is only a couple lines long.
Edit: my particular use case:
Many functions return an int to show if an error occurred or not. I have noticed from looking around that a style convention is to 'pile' variable declarations at the beginning of their scope block. If you have a few of these function uses, then you could potentially cloud up the declarations with the variables to store these return values (or really any variable that is only used temporarily, and the processing of which requires many of the other more 'permanent' variables that would make a for a long function declaration). Essentially the purpose of the question is determining the ramifications of scoping off these usages to attempt a cleaner look and declaration section. I realize for some cases one could reuse this return value, but the purpose of this question is to better understand my options.
For example:
int infoRtn;
char *serverHost;
...
infoRtn = getaddrinfo(
serverHost,
port,
&addrCriteria,
&addrList
);
if(infoRtn != 0){
fprintf(stderr, "%s # %d: Failed to get address info, error returned: %d\n", __FILE__, __LINE__, infoRtn);
exit(1);
}
...
vs
char *serverHost;
...
{
int infoRtn = getaddrinfo(
serverHost,
port,
&addrCriteria,
&addrList
);
if(infoRtn != 0){
fprintf(stderr, "%s # %d: Failed to get address info, error returned: %d\n", __FILE__, __LINE__, infoRtn);
exit(1);
}
}
...
Does it make sense to use a block as above to section off areas to instantiate temporarily used data?
Personally I haven't seen many use cases of having anonymous blocks to do something.
Functions are designed for that and you should use it, Using functions also gives visibility that you are trying to do something (but hides its implementation) and is better for readability eg
int averagreMarks = getAvgMarks(students);
Here when I read I see average marks of students are requested. If I want to see its implementation I go and see else I just skip over to main logic (Better readability)
Memory-wise, what happens to the variables allocated in the block
Yes they are destroyed (at least in most C compilers) so memory wise it can be analogous to a function but I would strongly urge to use functions for better readability and maintainability
EDIT (After details got added to question)
I propose a 3rd solution to this
{
char *serverHost;
validateConnection() //Add params that you want to send
...
}
validateConnection() {
int infoRtn;
infoRtn = getaddrinfo(
serverHost,
port,
&addrCriteria,
&addrList
);
if(infoRtn != 0){
fprintf(stderr, "%s # %d: Failed to get address info, error returned: %d\n", __FILE__, __LINE__, infoRtn);
exit(1);
}
The above solution has advantage that user knows ok now we do validate and continue. Plus in future if your validate logic is modified (example you add retries) you dont clutter the original function.
I was working on a project for a course on Operating Systems. The task was to implement a library for dealing with threads, similar to pthreads, but much more simpler. The purpose of it is to practice scheduling algorithms. The final product is a .a file. The course is over and everything worked just fine (in terms of functionality).
Though, I got curious about an issue I faced. On three different functions of my source file, if I add the following line, for instance:
fprintf(stderr, "My lucky number is %d\n", 4);
I get a segmentation fault. The same doesn't happen if stdout is used instead, or if the formatting doesn't contain any variables.
That leaves me with two main questions:
Why does it only happen in three functions of my code, and not the others?
Could the creation of contexts using getcontext() and makecontext(), or the changing of contexts using setcontext() or swapcontext() mess up with the standard file descriptors?
My intuition says those functions could be responsible for that. Even more when given the fact that the three functions of my code in which this happens are functions that have contexts which other parts of the code switch to. Usually by setcontext(), though swapcontext() is used to go to the scheduler, for choosing another thread to execute.
Additionally, if that is the case, then:
What is the proper way to create threads using those functions?
I'm currently doing the following:
/*------------------------------------------------------------------------------
Funct: Creates an execution context for the function and arguments passed.
Input: uc -> Pointer where the context will be created.
funct -> Function to be executed in the context.
arg -> Argument to the function.
Return: If the function succeeds, 0 will be returned. Otherwise -1.
------------------------------------------------------------------------------*/
static int create_context(ucontext_t *uc, void *funct, void *arg)
{
if(getcontext(uc) != 0) // Gets a context "model"
{
return -1;
}
stack_t *sp = (stack_t*)malloc(STACK_SIZE); // Stack area for the execution context
if(!sp) // A stack area is mandatory
{
return -1;
}
uc->uc_stack.ss_sp = sp; // Sets stack pointer
uc->uc_stack.ss_size = STACK_SIZE; // Sets stack size
uc->uc_link = &context_end; // Sets the context to go after execution
makecontext(uc, funct, 1, arg); // "Makes everything work" (can't fail)
return 0;
}
This code is probably a little modified, but it is originally an online example on how to use u_context.
Assuming glibc, the explanation is that fprintf with an unbuffered stream (such as stderr by default) internally creates an on-stack buffer which as a size of BUFSIZE bytes. See the function buffered_vfprintf in stdio-common/vfprintf.c. BUFSIZ is 8192, so you end up with a stack overflow because the stack you create is too small.
I have a C application whose one of the jobs is to call an executable file. That file has performance measurement routines inserted during compilation, at the level of intermediate code. It can measure time or L1/L2/L3 cache misses. In other words, I have modified the LLVM compiler to insert a call to that function and print the result to stdout for any compiled program.
Now, like I mentioned at the beginning, I would like to execute the program (with this result returned to stdout) from a separate C application and save that result. The way I'm doing it right now is:
void executeProgram(const char* filename, char* time) {
printf("Executing selected program %s...\n", filename);
char filePath[100] = "/home/michal/thesis/Drafts/output/";
strcat(filePath, filename);
FILE *fp;
fp = popen(filePath, "r");
char str[30];
if (fp == NULL) {
printf("Failed to run command\n" );
exit(1);
}
while (fgets(str, sizeof(str) - 1, fp) != NULL) {
strcat(time, str);
}
pclose(fp);
}
where filename is the name of the compiled executable to run. The result is saved to time string.
The problem is, that the results I'm getting are pretty different and unstable compared to those that are returned by simply running the executable 'by hand' from the command line (./test16). They look like:
231425
229958
230450
228534
230033
230566
231059
232016
230733
236017
213179
90515
229775
213351
229316
231642
230875
So they're mostly around 230000 us, with some occasional drops. The same executable, run from within the other application, produces:
97097
88706
91418
97970
97972
94597
95846
95139
91070
95918
107006
89988
90882
91986
90997
88824
129136
94976
102191
94400
95215
95061
92115
96319
114091
95230
114500
95533
102294
108473
105730
Note that it is the same executable that's being called. Yet the measured time it returns is different. The program that is being measured consists of a function call to a simple nested loop, accessing array elements. Here is the code:
#include "test.h"
#include <stdio.h>
float data[1000][1000] = {0};
void test(void)
{
int i0, i1;
int N = 80;
float mean[1000];
for (i0 = 0; i0 < N; i0++)
{
mean[i0] = 0.0;
for (i1 = 0; i1 < N; i1++) {
mean[i0] += data[i0][i1];
}
mean[i0] /= 1000;
}
}
I'm suspecting that there is something wrong in the way the program is invoked in the code, maybe the process should be forked or something? Any ideas?
You didnt specify where exactly your time measuring subroutines are inserted, so all I can really offer is guesswork.
The results seem to hint to the exact opposite - running the application from shell is slower, so I wouldn't worry about the way you're starting the process from the C code. My guess would be - when you run your program from shell, it's the terminal that's slowing you down. When you're running the process from your C code, you pipe the output back to your 'starter' application, which is already waiting for input on the pipe.
As a side note, consider switching from strcat to something safer, like strncat.
I am developing a kernel module using kernel version 3.x.
I have a function that is responsible to determine if a running process has a given file opened.
Here is my code (see my comments after):
struct task_struct * process = NULL;
struct files_struct * task_files = NULL;
struct fdtable * fdt = NULL;
int fd_i;
char tmpbuf[256];
char * process_path = "";
for_each_process(process)
{
// Ignore processes without files
if (process->files == NULL)
continue;
printk(KERN_INFO "task_lock()...\n");
task_lock(process);
printk(KERN_INFO "task_lock() DONE\n");
task_files = process->files;
printk(KERN_INFO "task_unlock()...\n");
task_unlock(process);
printk(KERN_INFO "task_unlock() DONE\n");
printk(KERN_INFO "files_fdtable()...\n");
fdt = files_fdtable(task_files);
printk(KERN_INFO "files_fdtable() DONE\n");
printk(KERN_INFO "Iterating files...\n");
for (fd_i = 0; fd_i < fdt->max_fds; fd_i++)
{
if (fcheck_files(task_files, fd_i) == my_file)
{
if (process->mm)
{
if (process->mm->exe_file)
{
process_path = d_path(&process->mm->exe_file->f_path, tmpbuf, sizeof(tmpbuf));
break;
} else {
printk(KERN_INFO "process->mm->exe_file is NULL\n");
}
} else {
printk(KERN_INFO "process->mm is NULL\n");
}
}
}
printk(KERN_INFO "Files iteration finished\n");
}
This code is working and the variable process_path contains the path of the process where the given file is opened.
But when there is huge load on the machine (so going through this code really often), the machine freeze (after a certain amount of time) and the latest printed debug is:
task_unlock() DONE
Then I don't see what I'm doing wrong.
for_each_process isn't calling spin_lock and spin_unlock on processes so I'm using task_lock and task_unlock.
files_fdtable is calling spin_lock and spin_unlock so I don't.
fcheck_files is also calling spin_lock and spin_unlock so I don't.
d_path is taking care to lock so I don't do it.
Could you please explain me why my code is freezing the machine and how to fix it ?
The way you designed your module is responsible for system being freeze. Note that, you've used for_each_process(), that means, it'll traverse every process of the system. So, when you put load on the system the number of process gets bigger. Moreover, inside your for_each_process() loop you're calling task_lock/unlock() and tried various operations on processes, all these operations are expensive, cause they all have their own lock to take. When system load is low they're not noticeable, but the complexity of your modules runtime increases as system gets more and more loaded, but remains less noticeable on low load. I'd suggest to use feature like ftrace to instrument your module also avoid over use of printk (cause printk also needs to be scheduled, klogd is used on that purpose). And with low load inspect how your module runs inside kernel. Measure how much time it spends on every loop, you get to know yourself. Kernel is a big beast, a lot of things happens inside...
Try the following:
read_lock(&tasklist_lock);
do_each_thread(g, p) {
task_lock(p);
if (check_for_file(p, file)) {
task_unlock(p);
goto next;
}
task_unlock(p);
} while_each_thread(g, p);
next:
read_unlock(&tasklist_lock);
I have finally fixed my code.
I have first implemented a kind of caching of the relation between a process and an opened file in a linked list, and re-factored my code in multiples methods (and after having fixed memory leaks) it is now working.
Thank you all for your help.