I have a main proccess which sends signal to the children, like this:
void handleInputs(query_data *q_data)
{
char input[DEFAULT_INPUT_SIZE];
while(1)
{
if(fgets(input,DEFAULT_INPUT_SIZE,stdin)==NULL) ERR("FGETS");
input[strlen(input)-1] = '\0';
if(!strcmp(input,"exit"))
{
kill(0,SIGTERM);
exit(EXIT_SUCCESS);
}
else if(!strcmp(input,"index"))
kill(0,SIGUSR2);
else if(!strcmp(input,"status"))
kill(0,SIGUSR1);
else
{
char* token = strtok(input, " ");
if(!token) printf("Wrong input\n");
else
{
if(!strcmp(token,"query"))
{
initializeQueryNumbers(q_data,token);
createQueryThread(q_data);
}
else printf("Wrong input\n");
}
}
}
}
Then I have a child processes which wait for signals using sigwait in a loop like this:
void waitForSignals(data* ch_data)
{
char* pid_directory = strConcat(ch_data->dir, "/.numf_pid");
int signo;
for(;;)
{
//how to make this block independent to the sigwait
printf("time elapse: %ld interval: %d\n",time(NULL)-ch_data->end, ch_data->i);
if((time(NULL)- ch_data->end)>ch_data->i && ch_data->status ==0)
{
printf("Time elapsed from the last indexing > %d\n",ch_data->i);
createIndexingThread(ch_data);
}
//end here
if(sigwait(&(ch_data->mask), &signo)) ERR("SIGWAIT");
switch(signo)
{
case SIGUSR1:
if(ch_data->status == 1)
{
printf("Indexing in progress for directory: %s\n",ch_data->dir);
printf("Time elapsed: %ld\n", time(NULL)- ch_data->start);
}
else printf("No indexing in progress for directory: %s\n",ch_data->dir);
break;
case SIGUSR2:
if(ch_data->status ==0)
createIndexingThread(ch_data);
else printf("Indexing already in progess for directory: %s\n",ch_data->dir);
break;
case SIGTERM:
if(remove(pid_directory)<0) ERR("REMOVE");
free(pid_directory);
if(pthread_cancel(ch_data->tid)==0)
printf("Stopped the indexing of directory %s\n",ch_data->dir);
exit(EXIT_SUCCESS);
}
}
if(pthread_join(ch_data->tid,NULL)) ERR("PTHREAD_JOIN");
}
My child proccess creates thread which does some indexing. I want to create such thread when signaled SIGUSR2 from main proccess (which is working fine) or when time elapsed from last indexing is bigger than some user-given time (stored in ch_data->i). However right now the block between the comments only comes alive after I send some signal from the main proccess. What is the best way to make it work in parallel?
If I understand correctly, you are asking how to schedule a threaded operation performed either upon a signal or upon time elapsed since the most recent operation, whichever comes first.
You might use sigtimedwait(interesting_sigs, NULL, timeout) instead of sigwait, recomputing your timeout as needed.
You might arm a POSIX timer to launch the operation (or, perhaps, to signal yourself). You'll need to check that the operation hasn't been performed too recently, and you'll need to re-arm (reschedule) the next timer appropriately, perhaps at the end of every operation.
You might simply have an idle-looping background thread that does what a timer would do. Sometimes these are easier to reason about than sigevent timers.
You might incorporate an event-driven framework like libevent that knows how to handle timeouts and signals and not worry about the implementation. You'll likely still need to recompute the expiry, however.
Related
I want to have multiple threads execute sequentially.
Because I need to do some work on CPU cache. So I need two threads to run on different CPU cores. Two physical CPU cores share the Last level cache. My job is to operate Last level cache
This can be achieved with pthread_attr_setaffinity_np.
This can be achieved by locking.
But when implementing it, I found it easy to have two threads execute sequentially.
void trasmitter_func(void *addr){
while(1){
pthread_mutex_lock(&lock1);
if (sequence == 0) {
pthread_cond_wait(&cond1, &lock1);
}
if(sequence == -1){
pthread_cond_signal(&cond1);
pthread_mutex_unlock(&lock1);
break;
}
printf("trasmitter start\n");
sequence = 0;
pthread_cond_signal(&cond1);
pthread_mutex_unlock(&lock1);
}
}
void receiver_func(void *addr) {
for(int i=0;i<SAMPLE_NUMBER;i++){
pthread_mutex_lock(&lock1);
if (sequence == 1) {
pthread_cond_wait(&cond1, &lock1);
}
// printf("receiver start\n");
sequence = 1;
pthread_cond_signal(&cond1);
pthread_mutex_unlock(&lock1);
}
sequence = -1;
}
But when implementing three threads, I need to use two locks and two pthread_cond_signal. And the controls are complicated. I'm curious, is there some simple implementation here that would allow three or more than three threads to execute sequentially?
I can simply pass some tags and functions to ensure their execution.
execute_func_sequence(0, func_1);
execute_func_sequence(1, func_2);
execute_func_sequence(2, func_3);
execute_func_sequence(3, func_4);
Then fun_1 to fun_4 will be executed sequentially.
This seems to be difficult to implement using locks and signals.
What I want to emphasize is: each is not executed once.
receiver start
trasmitter start
receiver start
trasmitter start
receiver start
trasmitter start
receiver start
trasmitter start
receiver start
trasmitter start
receiver start
trasmitter start
receiver start
trasmitter start
receiver start
trasmitter start
receiver start
trasmitter start
The execution result of the above sample code is as follows. I need to execute two threads non-stop sequentially.
I want to run multiple threads in this form.
func_1(){
for(int i=0;i <SAMPLE_NUMBER; i++){
do work1;
// block and enter func_2;
}
end = true;
}
func_1(){
while(1){
do work2;
if(end)
break
// block and enter func_3;
}
}
func_1(){
while(1){
do work3;
if(end)
break
// block and enter func_1;
}
}
Finally got this working.
work1,
work2,
work3,
work1,
work2,
work3,
work1,
work2,
work3,
...
work1,
work2,
work3,
A possible way of implementing what you want is:
void func_1() {
// do work...
sem_post(&thread2_sem);
}
void func_2() {
sem_wait(&thread2_sem);
// do work...
sem_post(&thread3_sem);
}
void func_3() {
sem_wait(&thread3_sem);
// do work...
sem_post(&thread4_sem);
}
void func_4() {
sem_wait(&thread4_sem);
// do work...
}
int main() {
sem_init(&thread2_sem, 0, 0);
sem_init(&thread3_sem, 0, 0);
sem_init(&thread4_se,, 0, 0);
NEW_THRAD(func_1)
NEW_THRAD(func_2)
NEW_THRAD(func_3)
NEW_THRAD(func_4)
}
As you can see, every thread is locked in some semaphore, and the previous thread is responsible for unlocking it.
The program starts with all the semaphores initialized to zero and, on each step, one semaphore gets incremented in order to activate the next thread, which will immediately decrement it.
All semaphores must go back to zero in case there's "another batch". i.e.: the threads are ready to process another order (whatever that means).
Note the example functions don't have a loop, y assume you would want something like:
void func_3() {
while (g_running) {
sem_wait(&thread3_sem);
// do work...
sem_post(&thread4_sem);
}
}
In this case, you may need something similar to wake up func_1.
I have a main thread which create child threads to do various task. There is a child thread which is tasked to report on the status every 100s
My current mechanism of stopping the thread is to observe a global boolean. Somewhat like this
Child thread
void* ReportThread(bool* operation)
{
while(*operation)
{
// do its reporting task
// ........
int counter = 0;
while( counter < 100 && operation )
{
// let it sleep for 1 seconds and wake up to check
sleep(1);
sleepCounter += 1;
}
}
}
Parent (Main) Thread:
bool operation = false;
int main(){
pthread_t tid;
err = pthread_create(&tid), NULL, &ReportThread, &operation);
printf("Please input esc to end operation \n");
while ((ch = getchar()) != 27);
operation =true;
pthread_join(tid,NULL);
return 0;
}
The problem:
It seem that using sleep(n). The number of seconds seem very inconsistent. When the program is stopped, this thread takes a while maybe 10 second to actually stop
Is there a way to interrupt a thread to sleep? I heard you could use signal. I am coding in linux
Can I just simply just use a pthread_cancel(tid) instead of pthread_join(tid)?
Regards
This part
while( counter < 100 || operation )
{
// let it sleep for 1 seconds and wake up to check
sleep(1);
sleepCounter += 1;
}
is wrong.
First I assume that sleepCounter += 1; is really a typo and that it should be:
while( counter < 100 || operation )
{
// let it sleep for 1 seconds and wake up to check
sleep(1);
counter += 1;
}
Then the problem is that even if operation is set to false by some other thread, the while will not finish until counter reach 100.
The code should be
while( counter < 100 && operation )
{
// let it sleep for 1 seconds and wake up to check
sleep(1);
counter += 1;
}
Further, in main you never set operation to false. Another typo?
You don't need two while loops. And if you want to set a timer, use time functions for it, because sleep is a cancellation point and it is not guaranteed that sleep actually sleeps that amount of time.
Example:
void* ReportThread(void *args)
{
time_t start = time(NULL);
time_t now;
bool *operation = (bool*) args;
while (*operation) { //while active
now = time(NULL); //get current time
if (now - start >= 100) { //if the threshold is exceeded
start = now; //reset timer
//and probably do other stuff
}
sleep(1); //sleep for one second
}
return NULL;
}
The example above has a max lag of one second, that means if you set operation to false right at that moment when the thread entered the sleep state, you have to wait until sleep returns, only then it will recognize the modified state. The example also has the advantage, that you can easily modify the threshold value (since it depends on the 'real' time, instead of a counter and a non accurate sleep time).
Btw. the variable operation should be either an atomic boolean or protected by a mutex (since it is accessed from different threads).
To answer the questions of your problem:
should be answered by the example above
since i mentioned it before, sleep is a cancellation point, that means it gets interrupted if the process handles a signal (see man pthreads - section Cancellation points).
see man pthread_cancel - section Notes
On Linux, cancellation is implemented using signals. Under the NPTL threading implementation, the first real-time signal (i.e., signal 32) is used for this purpose. On LinuxThreads, the second real-time signal is used, if real-time signals are available, otherwise SIGUSR2 is used.
You cannot use pthread_cancel over pthread_join! You have to use pthread_join in either case (described in detail in the man page).
I don't know if this will fix all your problems, but it's a bit too much for a comment. One problem, your ReportThread function signature is wrong. It should be:
void* ReportThread(void* args);
And then in that function you need to do something like:
void* ReportThread(void* args)
{
bool* operation = (bool*)args;
while(*operation)
{
...
}
}
I'm not sure how it's working right now, but your compiler should at least be issuing a warning trying to convert a bool* type to a bool.
Also be aware of race conditions on operation
There are 3 series' to compute using multithread.
To access a series you need to wait for semaphore, and after that you need to know if you are builder thread or cleaning thread.
Everything works great when I have num of threads which all of them has what to do and there are no 'floating' thread that cannot access semaphore or mutex.
If I do have too many threads and the dont have what to do the program get stuck unless after WaitSemRes/WaitMutexRes I kill that thread but I dont want to do that bc I loosing to much computing time.
I just want them to keep waiting until they have something to do.
Am I missing something?
/* Thread is trying to enter specific series, if couldnt - try the next one*/
WaitSemRes = WaitForSingleObject(main_series[series_idx].semaphore, 0L);
if (WaitSemRes == WAIT_OBJECT_0) {
/* Check if i'm the cleaning thread */
if (main_series[series_idx].ready_to_clean) {
/* Catching series mutex to clean it alone */
WaitMutexRes = WaitForSingleObject(main_series[series_idx].mutex, INFINITE);
if (WaitMutexRes == WAIT_OBJECT_0)
// do something //
}
else {
ExitThread(0);
}
// do stuff
else {
ExitThread(0);
}
You could use a shutdown event to signalize the threads when there's no more work to be done, this way they can keep waiting indefinitely. It can be created using CreateEvent, then all you need to do is to make your worker threads wait upon it. When you all work is done you can make all threads exit almost simultaneously by using SetEvent, and then wait upon a HANDLE array that contains all the thread handles
For details regarding how to wait on multiple objects you can refer to WaitForMultipleObjects Example
Worker thread
int thread_function(...) {
[...]
while(WaitForSingleObject(thread_shutdown_event, 0) != WAIT_OBJECT_0) {
// Keep trying to process data until the event is signalized
}
// Exit thread
return 0;
}
Pool handling functions
void end_thread_pool(...) {
HANDLE wait_result;
if(thread_shutdown_event == NULL)
return;
// Activate shutdown event
SetEvent(thread_shutdown_event);
// Wait upon thread handle list until all threads can return safely
wait_result = (HANDLE)WaitForMultipleObjects(...);
if(wait_result != WAIT_OBJECT_0) {
// Failed to wait
}
CloseHandle(thread_shutdown_event);
[...]
return;
}
void init_thread_pool(...) {
[...]
thread_shutdown_event = CreateEvent(NULL, // Default security attributes
TRUE, // Manual reset?
FALSE,// Starting value
TEXT("Thread Shutdown event"));// Event name
if(thread_shutdown_event == NULL) {
// Failed to create event
return;
}
// Initiates threads and also puts their HANDLEs in a array
// that is going to be used in end_thread_pool to wait
// See 'WaitForMultipleObjects Example'
thread_create(thread_function, number_of_threads);
[...]
return;
}
I have two threads
xThread : Continuously Prints X on the console
inputThread: Gets input from the stdin
The continuous printing stops when the user enters 'C' or 'c'
#include<stdio.h>
#include<sys/select.h>
#include<pthread.h>
#define S sleep(0)
int read_c = 0;
pthread_mutex_t read_c_mutex = PTHREAD_MUTEX_INITIALIZER;
void* inputThread_fn(void* arg)
{
char inputChar;
while(1)
{
S;
printf("\nChecking input");
scanf("%c",&inputChar);
if(inputChar=='C' || inputChar == 'c')
{
pthread_mutex_trylock(&read_c_mutex); /*<--This must be _lock ?
because with the use of trylock even If i don't aquire a lock I go ahead and modify
the variable?*/
read_c = 1;
pthread_mutex_unlock(&read_c_mutex);
pthread_exit(NULL);
}
}
}
void* xThread_fn(void* arg)
{
while(1)
{
S;
pthread_mutex_trylock(&read_c_mutex);
if(!read_c)
printf(" X");
else
pthread_exit(NULL);
pthread_mutex_unlock(&read_c_mutex);
}
}
void* yThread_fn(void* arg)
{
while(1)
{
S;
pthread_mutex_trylock(&read_c_mutex);
if(!read_c)
printf(" Y");
else
pthread_exit(NULL);
pthread_mutex_unlock(&read_c_mutex);
}
}
int main()
{
pthread_t xThread,yThread,inputThread;
pthread_create(&xThread,NULL,xThread_fn,NULL);
pthread_create(&inputThread,NULL,inputThread_fn,NULL);
pthread_join(xThread,NULL);
pthread_join(inputThread,NULL);
return 0;
}
When I use sleep(1) the threads are spawned and [irrespective of which thread is started first] when the program reaches scanf in inputThread it halts for the user input and the code does not proceed until I enter an input.
When I execute the code with sleep(0), scanf does not halt for the input, it keeps printing 'X' until I enter 'C' or 'c'
Does sleep() interfere with scanf in someway?
Note: I am aware of select being used for non-blocking input. I have tried the same too and the code runs fine. I just want to know in the above case why inconsistent behaviour arises?
Update (Using trylock)
void* inputThread_fn(void* arg)
{
char inputChar;
while(1)
{
S;
scanf("%c",&inputChar);
if(inputChar=='C' || inputChar == 'c')
{
pthread_mutex_trylock(&read_c_mutex);
read_c = 1;
pthread_mutex_unlock(&read_c_mutex);
pthread_exit(NULL);
}
}
}
void* xThread_fn(void* arg)
{
while(1)
{
S;
pthread_mutex_trylock(&read_c_mutex);
if(!read_c)
{
pthread_mutex_unlock(&read_c_mutex);
printf(" X");
}
else
{
pthread_mutex_unlock(&read_c_mutex);
pthread_exit(NULL);
}
fflush(stdout);
}
}
void* yThread_fn(void* arg)
{
while(1)
{
S;
pthread_mutex_trylock(&read_c_mutex);
if(!read_c)
{
pthread_mutex_unlock(&read_c_mutex);
printf(" Z");
fflush(stdout);
}
else
{
pthread_mutex_unlock(&read_c_mutex);
pthread_exit(NULL);
}
}
}
The reason you don't see output is because you're not flushing the buffer.
The reason you don't need to flush the buffer with sleep(0) is because the writer thread writes so much data that the buffer fills up and is automatically flushed.
#define SLEEP_TIME 1
void* xThread_fn(void* arg)
{
while (1) {
sleep(SLEEP_TIME);
pthread_mutex_lock(&read_c_mutex);
if (read_c) {
pthread_mutex_unlock(&read_c_mutex);
return NULL;
}
pthread_mutex_unlock(&read_c_mutex);
printf(" X");
fflush(stdout); // <-- necessary
}
}
Don't use pthread_mutex_trylock()
Don't use pthread_mutex_trylock() here. It's wrong.
The difference between lock() and trylock() is that lock() will always succeed1 but trylock() will sometimes fail. That's why it's called "try".
Since trylock() sometimes fails, you have to handle the case where it failed. Your code doesn't handle the case: it simply plows forward, pretending it acquired the lock. So, suppose trylock() doesn't lock the mutex. What happens?
pthread_mutex_trylock(&read_c_mutex); // Might fail (i.e., not lock the mutex)
read_c = 1; // Modifying shared state (Wrong!)
pthread_mutex_unlock(&read_c_mutex); // Unlocking a mutex (Wrong!)
Then there's the question of how the code should handle trylock() failing. If you can't answer this question, then the default answer is "use lock()".
In the reader thread, you can't use trylock() because you have to lock the mutex:
int r = pthread_mutex_trylock(&read_c_mutex);
if (r != 0) {
// Uh... what are we supposed to do here? Try again?
} else {
read_c = 1;
pthread_mutex_unlock(&read_c_mutex);
}
In the writer thread, there's no point in using trylock():
int r = pthread_mutex_trylock(&read_c_mutex);
if (r != 0) {
// Okay, just try again next loop...
} else {
if (read_c) {
pthread_mutex_unlock(&read_c_mutex);
pthread_exit(NULL);
} else {
pthread_mutex_unlock(&read_c_mutex);
}
}
However, this is entirely pointless. The only reason trylock() will fail in the writer thread is if the reader thread owns the lock, which only happens if it is currently in the process of setting read_c = 1;. So you might as well wait for it to finish, since you know you're going to exit anyway (why write more output after you know that the user has signaled your program to stop?)
Just use lock(). You'll use lock() 99% of the time, and trylock() is for the other 1%.
1: The lock() function can fail, but this usually means you've misused the mutex.
Misconceptions about lock() and trylock()
You said this about trylock():
If i have another thread accessing the variable read_input then will it be appropriate to use it?
I think there is a very fundamental misunderstanding here about the nature of mutexes. If another thread weren't accessing the variable at the same time, then you wouldn't need a mutex at all.
Suppose you're doing important work at the office, and you need to use the photocopier. Only one person can use the photocopier at a time. You go to the photocopier and someone's already using it.
If you wait in line until it's your turn, then that's lock().
If you give up and go back to your desk, then that's trylock(). (Your program actually ignores the return code for trylock(), so you basically start mashing buttons on the photocopier even if someone else is using it.)
Now imagine that it takes one minute to use the photocopier, only two people ever use the photocopier, and they only use the photocopier once every twenty years.
If you use lock(), then you wait in line for at most one minute before using the photocopier.
If you use trylock(), then you give up and go back to your desk and wait twenty years before trying the photocopier again.
It doesn't make any sense to use trylock(), does it? Are your threads so impatient that they can't spend even one minute in line once every twenty years?
Now your boss comes down and said, "Where is that report I asked you to photocopy?" And you say, "Well, I went to the photocopier six years ago but someone was using it."
The numbers (one minute every twenty years) are based on Latency Numbers Every Programmer Should Know, where it notes that locking/unlocking a mutex is about 25ns. So if we pretend that it takes one minute to lock and then unlock a mutex, then sleep(1) causes the thread to wait for twenty years.
I'm having a problem in the combined use of execl() and pthread.
My idea is quite simple: write a daemon that in certain situation starts an external process (a separate executable with respect to the daemon itself) and wait for the return value of that process. Moreover I want to have the possibility to start multiple instances of the same process at the same time.
The part of my code to handle multiple threads:
...
for (c_thread=0,i=0;i<N;i++)
{
/* Start actions before start threads */
for (j=c_thread;j<c_thread+config.max_threads;j++)
Before_Process(act[act_index[j]].measID);
/* Now create threads */
for (c=0,j=c_thread;j<c_thread+config.max_threads;j++)
{
Print_Log(LOG_DEBUG,"Create tread n. %d, measurementID=%s",c,act[act_index[j]].measID);
if ((ret=pthread_create(&pth[c],NULL,Start_Process_Thread,(void *) &act[act_index[j]].measID)))
{
Print_Log(LOG_ERR,"Error in creating thread (errorcode: %d)",ret);
exit(EXIT_FAILURE);
}
c++;
}
/* Joint threads */
for (j=0;j<config.max_threads;j++)
{
if ((ret=pthread_join(pth[j], (void**) &r_value[j])))
{
Print_Log(LOG_ERR,"Error in joint thread (errorcode: %d)",ret);
exit(EXIT_FAILURE);
}
}
/* Perform actions after the thread */
for (j=0;j<config.max_threads;j++)
{
status=*(int*) r_value[j];
Print_Log(LOG_DEBUG,"Joint tread n. %d. Return value=%d",j,status);
After_Process(act[act_index[c_thread+j]].measID,status);
}
c_thread += config.max_threads;
}
...
And the function Start_Process_Thread:
void *Start_Process_Thread(void *arg)
{
int *ret;
char *measID;
measID=(char*)arg;
if (!(ret=malloc(sizeof(int))))
{
Print_Log(LOG_ERR, "allocation memory failed, code=%d (%s)",
errno, strerror(errno) );
exit(EXIT_FAILURE);
}
*ret=Start_Process(measID);
pthread_exit(ret);
}
int Start_Process(char *measID)
{
...
pipe(pfd);
pid=fork();
if (!pid)
{
signal(SIGALRM,Timeout);
alarm(config.timeout_process);
flag=0;
/*
Start the Process.
*/
ret=execl(config.pre_processor,buff_list[TokCount-1],config.db_name,measID,(char *) 0);
if (ret==-1)
{
alarm(0);
flag=1;
Print_Log(LOG_ERR,"Cannot run script %s, code=%d (%s)",config.process, errno, strerror(errno));
}
alarm(0);
close(1);
close(pfd[0]);
dup2(pfd[1],1);
write(1,&flag,sizeof(int));
}
else
{
wait(&status);
close(pfd[1]);
read(pfd[0],&flag,sizeof(int));
close(pfd[0]);
if (!flag)
{
if (WIFEXITED(status))
{
if (!(return_value=WEXITSTATUS(status)))
{
/*
Process gives no errors.
*/
Print_Log(LOG_INFO, "Processing of measurementID=%s ended succesfully!",measID);
}
else
{
/*
Process gives errors.
*/
Print_Log(LOG_WARNING,"Processor failed for measurementID=%s, code=%d",measID, return_value);
}
}
else
{
/*
Timeout for Process
*/
Print_Log( LOG_WARNING,"Timeout occurred in processing measurementID=%s",measID);
return_value=255;
}
}
}
}
The above code works fine from technical point of view but I have a problem somewhere in handling the return values of the different instances of the called external process. In particular it happens that the return value associated to a certain instance is attributed to a different one randomly.
For example suppose 4 different instances of the external process are called with the arguments meas1, meas2, meas3 and meas4 respectively and suppose that meas1, meas2 and meas3 are successfully processed and that for meas4 the process fails. In situation like that my code mix up the return vales giving success for meas1, meas3, and meas4 and failure for meas2 or success for meas1, meas2, meas4 and failure for meas3.
Any idea on why this can happens?
Any help is really welcome.
Thank you in advance for your attention.
When any thread in a process executes wait(), it gets the information about any of the process's dead children — not necessarily about the last child started by the thread that is doing the waiting.
You are going to need to think about:
Capturing the PID of the process that died (it is returned by wait(), but you ignore that).
Having a single thread designated as the 'disposer of corpses' (a thread that does nothing but wait() and record and report on deaths in the family of child processes).
A data structure that allows the threads that start processes to record that they are interested in the status of the child when it dies. Presumably, the child should wait on a suitable condition once a child starts so that it is not consuming CPU time doing nothing useful.
The 'disposer of corpses' thread handles notifications of the appropriate other thread whenever it collects a corpse.
Worry about timeouts on the processes, and killing children who run wild for too long.
It's a morbid business at times...