I have problem creating thousands of threads and closing it.
Look at this code:
HANDLE threadHandles[i];
for(int i = 0; i < 1000; i++)
{
CreateThread(0, 0, &func, 0, CREATE_SUSPENDED, threadHandles[i]);
printf("%i - %i\n", i, threadHandles[i]);
CloseHandle(threadHandles[i])
}
printf("Last Error is %i", GetLastError());
It should proceed this output:
0 - 236423
1 - 23456236
2 - 2373547
3 - 73521346
4 - 23456775
5 - 78543683465
...
2998 - 754752
2999 - 23462346
Last Error is 0
like this.
But actually it will print nothing. Why? Because one of the created threads had a conflict with stdout handle.
I realized it using this code:
HANDLE stdout = GetStdHandle(STD_OUTPUT_HANDLE);
HANDLE threadHandles[i];
for(int i = 0; i < 1000; i++)
{
CreateThread(0, 0, &func, 0, CREATE_SUSPENDED, threadHandles[i]);
printf("%i - %i\n", i, threadHandles[i]);
if(threadHandles[i] != stdout) // I have stdout == 12 on my machine
CloseHandle(threadHandles[i])
}
and it worked like this:
0 - 236423
1 - 23456236
2 - 2373547
3 - 0
4 - 23456775
5 - 78543683465
...
2998 - 0
2999 - 23462346
Last Error is 6
Where is the problem? Why there is a conflict between created handles and existing standard handles?
The thread handle is returned by the function! The last parameter receives the thread id, not the handle.
HANDLE threadHandles[1000] = { 0 };
UINT i;
for (i = 0; i < 1000; ++i)
{
DWORD threadId;
threadHandles[i] = CreateThread(0, 0, &func, 0, /*CREATE_SUSPENDED*/0, &threadId);
if (!threadHandles[i])
{
printf("Failed to create thread, error %u\n", GetLastError());
break;
}
printf("Thread #%u: handle=%p id=%u\n", i, threadHandles[i], threadId);
// Not closing the handle here so the example will show unique thread handles and ids: CloseHandle(threadHandles[i]), threadHandles[i] = NULL;
}
// TODO: Wait for threads or do some other work?
for (i = 0; i < 1000; ++i)
if (threadHandles[i])
CloseHandle(threadHandles[i]);
On my machine this prints
Thread #0: handle=0000002C id=16424
Thread #1: handle=00000030 id=21192
Thread #2: handle=00000034 id=21180
Thread #3: handle=00000038 id=17336
Thread #4: handle=0000003C id=21184
Thread #5: handle=00000040 id=4460
...
Thread #991: handle=00000FE8 id=12280
Thread #992: handle=00000FEC id=20360
Thread #993: handle=00000FF0 id=20328
Thread #994: handle=00000FF4 id=16060
Thread #995: handle=00000FF8 id=4556
Thread #996: handle=00000FFC id=20296
Thread #997: handle=00001004 id=10316
Thread #998: handle=00001008 id=20604
Thread #999: handle=0000100C id=20264
Related
I found a similar post pthread_create and passing an integer as the last argument however when implementing it I still receive the wrong ID values (as shown in the output log at the bottom of the post).
I have this portion of code which creates the threads. Its pretty standard. I have an array of threads which allocate a thread id which is passed into the function.
int nthreads = 7;
pthread_t tid[nthreads];
fprintf(stdout, "Creating threads.\n");
for (int i =0; i < nthreads; i++){
tid[i] = i;
if(pthread_create(&tid[i], NULL, threadRequestFile, &tid[i]) != 0){
fprintf(stdout, "Error pthread_create().\n");
steque_destroy(workerQueue);
free(workerQueue);
workerQueue = NULL;
exit(EXIT_FAILURE);
}
}
fprintf(stdout, "Created %d threads.\n", nthreads);
for(int i = 0; i < nthreads; i++){
fprintf(stdout, "Awaiting Thread %d.\n", i);
pthread_join(tid[i], NULL);
}
Finally my pointer function with has the ids stored in the address of tid[i] are passed into to print to the console:
void *threadRequestFile(void *nRequests){
int totalRequests = * ((int *)nRequests);
fprintf(stdout, "Total Request: %d\n", totalRequests);
}
Unfortunately closely following the code in the post above and from other sources I found online, my console is still printing weird numbers rather than 0-6. Can anyone help me out as to why this is occurring?
Creating threads.
Total Request: -1210059008 // Should be 0
Total Request: -1218451712 // ..
Total Request: -1226844416 // ..
Total Request: -1235237120 // ..
Total Request: -1243629824 // ..
Total Request: -1252022528 // ..
Total Request: -1260415232 // 6
Created 7 threads.
Awaiting Thread 0.
Awaiting Thread 1.
Awaiting Thread 2.
Awaiting Thread 3.
Awaiting Thread 4.
Awaiting Thread 5.
Awaiting Thread 6.
I met a strange problem while studying the Peterson Algorithm. When I finish my program, I was told that I should restrict that two particular threads (thread1 and thread2) to a single CPU core (I don't exactly know why I should do this, but my teacher told me that if I want to run two mutual exclusion processes with single resources without conflict, multi-core CPU may cause bug so I should restrict them to a single core) so I add the SetThreadAffinityMask() to do this.
But after I added this function, my code doesn't work, it was stuck into a infinite loop in thread1, not like the result before I add this function, which is work as well as I expected.
Here's my code
#include "stdafx.h"
#include <stdlib.h>
#include <time.h>
#include <Windows.h>
int nAccount1 = 0, nAccount2 = 0;
DWORD start, end;
int flag = 0;
int Process[2] = { 0 };
int Turn = 0;
DWORD WINAPI ThreadExecutive_1(LPVOID lpParameter) {
int nLoop = 0;
int nTemp1, nTemp2, nRandom;
int* pID = (int*)lpParameter;
//printf("Thread: %d Built successfully!\n", *pID);
do {
Process[0] = 1;
Turn = 1;
while (Process[1] == 1 && Turn == 1);
nTemp1 = nAccount1;
nTemp2 = nAccount2;
nRandom = rand();
nAccount1 = nTemp1 + nRandom;
nAccount2 = nTemp2 - nRandom;
Process[0] = 0;
nLoop++;
if (nLoop % 100000 == 0) {
// printf("here thread %d\n", *pID);
}
} while (nLoop < 1000000);
//printf("Thread%d was terminated Result:nAccount1 = %d, nAccount2 = %d\n", *pID, nAccount1, nAccount2);
flag++;
return 0;
}
DWORD WINAPI ThreadExecutive_2(LPVOID lpParameter) {
int nLoop = 0;
int nTemp1, nTemp2, nRandom;
int* pID = (int*)lpParameter;
printf("Thread: %d Built successfully!\n", *pID);
do {
Process[1] = 1;
Turn = 0;
while (Process[0] == 1 && Turn == 0);
nTemp1 = nAccount1;
nTemp2 = nAccount2;
nRandom = rand();
nAccount1 = nTemp1 + nRandom;
nAccount2 = nTemp2 - nRandom;
Process[1] = 0;
nLoop++;
if (nLoop % 100000 == 0) {
printf("here thread %d\n", *pID);
}
} while (nLoop < 1000000);
printf("Thread%d was terminated Result:nAccount1 = %d, nAccount2 = %d\n", *pID, nAccount1, nAccount2);
flag++;
return 0;
}
int main()
{
start = GetTickCount();
printf("===============Peterson Algorithm================\n");
HANDLE handle1, handle2;
int Thread1 = 1;
int Thread2 = 2;
//===========This is where the problem occourred===================
handle1 = CreateThread(NULL, 0, ThreadExecutive_1, &Thread1, 0, NULL);
SetThreadAffinityMask(handle1, 0x00000001);
handle2 = CreateThread(NULL, 0, ThreadExecutive_2, &Thread2, 0, NULL);
SetThreadAffinityMask(handle2, 0x00000001);
//===========This is where the problem occourred===================
if (handle1 == NULL) {
printf("Handle1 build failed\n");
exit(0);
}
if (handle2 == NULL) {
printf("Handle2 build failed\n");
exit(0);
}
CloseHandle(handle1);
CloseHandle(handle2);
while (1) {
if (flag == 2) {
break;
}
}
end = GetTickCount();
printf("Execution time : %d\n", (end - start));
system("pause");
return 0;
}
Here are the results. Without SetThreadAffinityMask():
===============Peterson Algorithm================
Thread: 1 Built successfully!
Thread: 2 Built successfully!
here thread 1
here thread 2
here thread 1
here thread 2
here thread 1
here thread 2
here thread 1
here thread 2
here thread 1
here thread 2
here thread 1
here thread 2
here thread 1
here thread 2
here thread 1
here thread 2
here thread 1
here thread 2
here thread 1
Thread1 was terminated Result:nAccount1 = -1621397855, nAccount2 = 1621397855
here thread 2
Thread2 was terminated Result:nAccount1 = -1583839139, nAccount2 = 1583839139
Execution time : 312
With SetThreadAffinityMask():(An infinite loop occurred)
===============Peterson Algorithm================
Thread: 2 Built successfully!
here thread 2
here thread 2
here thread 2
here thread 2
Thread: 1 Built successfully!
This is for an Operating Systems programming assignment. I'm attempting to read n number of files, use threads to search each file for a number of occurrences for a specific character.
./mycount j new.txt some.txt here.txt hello.txt
The output for my test code as is should be:
argumentCount: 6
threadCount: 4
pthread_create() for thread 0 returns: 0
Thread 1
pthread_create() for thread 1 returns: 0
Thread 2
pthread_create() for thread 2 returns: 0
Thread 3
pthread_create() for thread 3 returns: 0
Thread 4
However each execution of mycount is different, with the last thread usually not executing/printing. Either that or they'll print sporadically, in tandem, etc.
I'm attempting to utilize a mutex to ensure the integrity of my data but I'm not sure what's really happening behind the scenes.
How do I ensure that everything finishes the same way each time? The last thread always returns 0, but it sometimes won't execute the function I give it completely.
Code:
//GLOBALS
int occurrences = 0;
//PROTOTYPES
void *scanFile( void *filePtr );
//Initialize space for mutex.
pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER;
//Receive arguments from .exe call
void main ( int argumentCount, char *argumentVariables[] )
{
//Exit if argumentCount is > 50.
if (argumentCount > 50)
{
perror("Too many arguments. Enter less than 50.\n");
exit(EXIT_FAILURE);
}
printf("argumentCount: %d \n", argumentCount);
//Instantiate variables.
//i - iterator
//*newCommand - Used to hold string value of first command entered.
//*newVector - Used to hold string of the rest of the commands. Is a vector.
int i;
char *searchCharacter;
char *newVector[argumentCount];
//Iterate through command line arguments and split them.
for (i = 0; i < argumentCount; i++)
{
searchCharacter = argumentVariables[1];
if (i < argumentCount - 1)
{
newVector[i] = argumentVariables[1 + i];
}
else
{
newVector[i] = NULL;
}
}
//Exit if newest command is NULL.
if (searchCharacter == NULL)
{
perror("No character entered!\n");
exit(EXIT_FAILURE);
}
int threads = argumentCount - 2;
printf("threadCount: %d \n", threads);
pthread_t * thread = malloc(sizeof(pthread_t)*threads);
for (int i = 0; i < threads; i++)
{
int ret;
char *message = "Thread";
ret = pthread_create(&thread[i], NULL, scanFile, (void*) message);
if (ret != 0)
{
printf("Error - pthread_create() return code: %d\n", ret);
exit(EXIT_FAILURE);
}
printf("pthread_create() for thread %d returns: %d\n", i, ret);
}
exit(EXIT_SUCCESS);
}
void *scanFile( void *filePtr )
{
pthread_mutex_lock( &mutex );
char *message;
message = (char *) filePtr;
occurrences += 1;
printf("%s %d\n", message, occurrences);
pthread_mutex_unlock( &mutex );
}
Found the solution thanks to user2864740 and Ken Thomases.
Added:
for (int j = 0; j < threads; j++)
{
//Join the threads so all the data is good to go.
pthread_join(thread[j], NULL);
}
Correction:
for (int i = 0; i < threads; i++)
{
request[i].file = argumentVariables[i + 2];
request[i].character = searchCharacter;
//Create the thread. Any output from the integer newThread is an error code.
newThread = pthread_create(&thread[i], NULL, *scanFile, &request[i]);
if (newThread != 0)
{
printf("Error - pthread_create() return code: %d\n", newThread);
exit(EXIT_FAILURE);
}
}
for (int j = 0; j < threads; j++)
{
//Join the threads so all the data is good to go.
pthread_join(thread[j], NULL);
}
This is my first time dealing with threads.
When I run the program without the GetCurrentThreadId() function it executes without any issue.
When I add that line of code it still executes but crashes once it reaches the end. Why is this?
#include <Windows.h>
#include <stdio.h>
#include <conio.h>
static int tix[500];
static int done = 0;
HANDLE ghSemaphore;
DWORD WINAPI ThreadFunction();
int main(void)
{
DWORD threadID1, threadID2, threadID3, threadID4;
HANDLE hThread1, hThread2, hThread3, hThread4;
for (int i = 0; i < 500; i++) //initialize array
{
tix[i] = 0;
}
ghSemaphore = CreateSemaphore(NULL, 1, 10, NULL);
hThread1 = CreateThread(NULL, 0, ThreadFunction, NULL, 0, &threadID1);
hThread2 = CreateThread(NULL, 0, ThreadFunction, NULL, 0, &threadID2);
hThread3 = CreateThread(NULL, 0, ThreadFunction, NULL, 0, &threadID3);
hThread4 = CreateThread(NULL, 0, ThreadFunction, NULL, 0, &threadID4);
//printf("The thread ID: %d.\n", threadID1);
//printf("The thread ID: %d.\n", threadID2);
//printf("The thread ID: %d.\n", threadID3);
//printf("The thread ID: %d.\n", threadID4);
if (done = 1)
{
CloseHandle(hThread1);
CloseHandle(hThread2);
CloseHandle(hThread3);
CloseHandle(hThread4);
}
for (int j = 0; j < 500; j++)
{
if (tix[j] = 0)
{
printf("not sold");
}
else if (tix[j] = 1)
{
printf("sold");
}
}
return 0;
}
DWORD WINAPI ThreadFunction()
{
WaitForSingleObject(ghSemaphore, 0);
printf("current thread running : %d\n", GetCurrentThreadId());
int i = 0;
if (done != 0) // if loop to test wether or not the array is full
{
while (tix[i] = 1) //traverse the array to find a open spot
{
i++;
}
tix[i] = 1;
}
if (i == 499) //if i is 499, set test variable to 1
{
done = 1;
return 0;
}
ReleaseSemaphore(ghSemaphore, 1, NULL);
}
Your thread function has incorrect signature. Thread takes one PVOID context argument:
DWORD WINAPI ThreadProc(
_In_ LPVOID lpParameter
);
Your threads could exit without releasing a semaphore. Also since you had initialized it with a value, that is greater than thread amount and never check WaitForSingleObject result, no synchronization is provided and multiple threads will modify a shared buffers in inconsistent manner. Even worse - nothing stops your program main thread exiting earlier, than ThreadFunction.
There is no return statement in the end of your thread function, so this is an undefined behavior. In fact it is a great wonder that your code even compiles. This entire approach to multithreading is incorrect and has to be remade from the scratch.
I had this code:
int main(int argc, char** argv)
{
pthread_t thread[thr_num];
pthread_attr_t attr;
pthread_attr_init(&attr);
pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_JOINABLE);
// just for debugging //
struct rlimit rlim;
getrlimit(RLIMIT_NPROC, &rlim);
printf ("soft = %d \n", rlim.rlim_cur);
printf ("hard = %d \n", rlim.rlim_max);
////
for ( i = 1 ; i <= thr_num ; i++) {
if(pthread_create( &thread[i], &attr, loggerThread, (void*)argv ) ) {
printf("pthread_create failure, i = %d, errno = %d \n", i, errno);
exit(1);
}
}
pthread_attr_destroy(&attr);
for ( i = 1 ; i <= thr_num ; i++) {
if( pthread_join(thread[i], (void**)&status ) ) {
exit(1);
}
}
return 0;
}
void* loggerThread(void* data)
{
char** sthg = ((char**)data);
pthread_exit(NULL);
}
I don't understand why when I run this code with thr_num=291, I got an error:
pthread_create failure, i = 291, errno = 11 (EAGAIN)
with thr_num=290 worked fine. I run this code on a Linux 2.6.27.54-0.2-default (SLES 11)
The rlim.rlim_cur has value 6906 the rlim.rlim_max also. The same I saw with 'ulimit -a' for 'max user processes'.
I checked also /proc/sys/kernel/threads-max (it was 13813) guided by pthread_create man page.
Did not find any parameters with value 290 for 'sysctl -a' output either.
Ocassionally I found out from this link:
pthread_create and EAGAIN
that: "Even if pthread_exit or pthread_cancel is called, the parent process still need to call pthread_join to release the pthread ID, which will then become recyclable"
so just as a try I modified my code to this:
for ( i = 1 ; i <= thr_num ; i++) {
if(pthread_create( &thread[i], &attr, loggerThread, (void*)argv ) ) {
printf("pthread_create failure, i = %d, errno = %d \n", i, errno);
exit(1);
}
if( pthread_join(thread[i], (void**)&status ) ) {
printf("pthread_join failure, i = %d, errno = %d \n", i, errno);
exit(1);
}
}
pthread_attr_destroy(&attr);
and then everything worked: I didn't get the error at 291 cycle.
I would like to understand why with my original code I got the error:
1. because of a wrong programing with threads
2. or I hit some system limit what I couldn't identify
Also would like to know if my correction is good for this problem or what hidden things, pitfalls I eventually introduced with this solution ?
Thanks !
I would like to understand why with my original code I got the error: 1. because of a wrong
programing with threads 2. or I hit some system limit what I couldn't identify
You likely hit a system limit. Likely you ran out of address space. Default, each thread gets 8-10Mb of stack space on linux. If you create 290 threads, that's using nearly 3Gb of address space - the max for a 32 bit process.
You get EAGAIN in such a case, since there arn't enough resources to create the thread just now (since there isn't enough address space available at the time).
When a thread exits, not all resources of the thread is released (on linux, the entire stack of the thread is kept around).
If the thread is in a detached state, e.g. you called pthread_detach() or specified a detached state when it was created as an attribute to pthread_create(), all resources are release when the thread exits - but you can't pthread_join() a detached thread.
If the thread is not detached, you need to call pthread_join() on it to release the resources.
Note that the modified code of yours where you call pthread_join() inside the loop will:
spawn a thread
Wait for that thread to finish
go to 1
i.e. only one other thread is running at a time - which seems a bit pointless.
You can certainly spawn more than one thread that run concurrently - but there's a limit. On your machine, you seem to have found the limit to be around 290.
I initially wrote this as a comment, but just in case...
Your code:
for ( i = 1 ; i <= thr_num ; i++) {
if(pthread_create( &thread[i], &attr, loggerThread, (void*)argv ) ) {
printf("pthread_create failure, i = %d, errno = %d \n", i, errno);
exit(1);
}
}
...
for ( i = 1 ; i <= thr_num ; i++) {
if( pthread_join(thread[i], (void**)&status ) ) {
exit(1);
}
}
In both the for() loops you check from 1 - thr_num. This means you are out of bounds in your array thread[thr_num] since arrays start at index 0. You should thus iterate from 0 to one less than thr_num:
for ( i = 0 ; i < thr_num ; i++)
I'm actually surprised you didn't get a segmentation fault before hitting 291 as thr_num.