I have a task in which I have to write a program in C language that manages access and reading/writing to a file.
When the program starts it should create two processes(using fork()).
-The first process will be responsible for the initial write to the file(The file is a text file with 2000 random characters from a to z).
-The second process will be responsible for reading from the file ,after the first process has finished writing.
My question is :
How can I synchronize the execution order by using semaphores(sem() call system) in order to ensure that the first process always starts first and the second process starts only after the first process has finished writing?
I can recommend using binarysemaphores:
//
https://www.freertos.org/xSemaphoreCreateBinary.html
https://controllerstech.com/how-to-use-binary-semaphore-in-stm32/
If you are working in an embedded context i would recommend using Tasknotification since they are less ram hungry and therefore may be more fitting in a less powerfull system.
https://www.freertos.org/RTOS-task-notifications.html
Related
So I want to pass a variable from one c program to another c program.
For example:
main()
{
char str[]="Hello,there!";
system("program2.exe");
}
And I want to use str[] in program2.exe. Is there a way to pass a variable to another program?
I used files to write data from first program and read data from second program but I want to know is there any other way to do this?
Is it good to use files for passing data from program to another?
You can't literally pass a variable between two processes because each process on a system will generally have it's own memory space - each variable belongs to a process and therefore can't be accessed from another process (or so I believe). But you can pass data between processes using pipes.
Pipes are buffers that are implemented by the OS and are a much more effective method of sharing data between processes than files (yes, you can use files for inter-process communication). This is because files must be written to a disk before being accessed which makes them slow for inter-process communication. You'd also have to implement some kind of method for ensuring the two processes don't corrupt a file when reading and writing to it.
Also, pipes can be used to ensure continuous communication between two processes, making them useful in many situations. When using half duplex pipes (linked above), you can have a pipe for each process to establish a communication channel between them (i.e. a one way communication channel for each).
you can:
1) pass arguments to a program.
2) use sockets to communicate between processes.
I am working on an implementation where multiple processes read a regular file A. While this is happening, a new process P starts up and copies the contents from A to regular file B. All processes starting up after this, should now read file B.
To make the switch from A to B, process P creates a temporary file T once B is written. All the processes check whether T exists to decide where to read data from (i.e., read from A if T does NOT exist, and read from B if T exists).
Since T file is just an indicator here, is it better to use a memory mapped file rather than regular file for faster performance?
Using a tmp-file for synchronization is not safe. The check, if file exists and reading that file is not atomic. One process could switch the files just after another process has completed the check and is about to read.
If you are developing in C and allowed to use IPC API, you could set the flag in shared memory and guard it by a semaphore.
Also, the processes should signal, that they have finished the reading.
I try to find out a way to read from a txt file while I amin different processes. For example, I am in process A and I read the first 10 records of the file ( lets say that there are 100 records). In process B I want to read the next ten records. The problem is that only in process A I take the right records and when I am in process B i take only 0. Can someone help? Thanks in advance!
If you dig deeper especially in linux environment, you would find out that threads are lighter than processes when it is about achieving something which needs multiple strands of execution.
I would do this in following manner:
Create a process that has 10 threads. Each thread will read 10 records from the TXT file.
( I will need to use pthread_create() instead of fork() ). I will also create a mutex which each thread will lock while it reads the file.
I will create Th1 (thread_1) using the above call, lock the mutex, open the file, read is using read() call, the buffer will hold all the 100 records, filter out which 10 I need at that point of time, unlock the mutex when it is done, end it using pthread_join()
Repeat the step 2 for 10 times in total so that I have all the records.
This can be done using Inter-process Communication which mainly involves shared-memory, semaphores or message-queues. For a basic knowledge regarding this you can read my blog
One another way of doing it is by passing the file-descriptors between the processes. One process opens the file, reads 10 records, then passes it to the second process. This does the same thing and sends it to the first one. And the entire process repeats until the end of file is reached.
The passing of file-descriptors is mainly done using UNIX Domain Sockets and you can find the code related to this in this answer
Hope this helps.
On linux, when writing to a pipe, if the data is equal or less than the memory page size (4k atleast on 64bit rhel), the OS provides the guarantee that the whole write will either succeed or fail, but there would be no corruption of data, when multiple process are doing write at the same time. This applies to writing to regular files also.
My question is that is this atomicity a feature of virtual memory of linux? If yes, consider a shared memory scenario between two process, where one process is swapped out in middle of the write by the scheduler. Does Virtual memory subsytem ensures that the memory page to which the process was writing, is also locked, so that the second process cannot write to the same page?
Is this atomicity at page level only applicable across process , or also between threads of the same process?
No. If two processes are using shared memory, there is no implicit lock between the processes from this. You will have to arrange such a lock yourself (and if the owner of the lock is swapped out, then your other process will have to darn well wait until the owner gets swapped in and releases the lock after finishing whatever it was doing whilst holding the lock).
I don't believe there is any implicit (or explicit) rule that pages are different from other memory overall. The specific rules apply to writing to pipes and files, that if all the data fits in one page, it can be written as one block by the OS - I think you'll find that the OS holds a lock on the resource that it is writing to for one page at a time. If the data is bigger than a page, when the lock is relesed, the other process [or thread] may well be ready to run and thus "steal" the lock from the first process. Less than a page, it does the whole write in one locked run.
But to be clear, there is no implicit locks on writes (or reads) of memory pages in general. It applies strictly to CERTAIN functions. Typically, a particular function will also have a lock of some sort that prevent other processes from running in the same function [at least with a given resource - e.g. a file descriptor or similar - it's perfectly possible that some other process can read from another file simultaneously to your process reading from or writing to your file, but YOUR file is atomic per some size of block that the lock is held for, but not for your "write the entire Shakespeares works at once" system call, as that could potentially block some other important process.
I want to create a program, using POSIX threads, having n threads running at different priorities.
There are files (say m files) which are shared among these n threads. If one thread is using the file (assuming that it writing onto the file), no other thread will be allowed to use it. The code should maintain a Table that tells: which file it has acquired and for which file its requests are pending.
Also, we need a Monitor Thread to check for deadlocks ; any implementations hints/ideas?
You don't need to check for deadlocks. You have to write a nice code that makes it impossible to run into deadlock scenario. For that reason, I'd recommend you use try-lock approach to lock down a chain of files and unlock them back shall any of the lock acquisition fail.
Also, if you are using C buffered I/O, I'd recommend you stick with ftrylockfile and funlockfile APIs. Otherwise use a synchronization mechanism that is most appropriate for your case, be that futex API or locks implemented using atomic instructions.
The standard unix way to accomplish this is: spooldirectories.
file operations, such as rename / link / unlink are atomic
have one central input spool-dir, where input files can be placed
a process / thread that wants to process a file, starts by moving it to another name, or better: to another (work) directory (using the thread_id or process number as directory name is obvious.)
(since this move is atomic there is no possible race condition!)
after processing, the finished files can be moved to an output directory
the scoreboard function is simply a readdir(+stat), maybe even inotify, on the work directories
process starvation will always be a problem. Incompletely processed files will live forever in de workdirs. Having a stamp/ pid file in the workdirectories could help cleanup / restart.
if designed well, this structure could work even after machine failure. The workers would have to maintain their own backup / log /stamp-file mechanism.
if you haven't noticed yet: no locking will be needed.
I hate C. I have to try and think of a way to do this without classes:(
OK, a 'Sfile' struct to represent each file. Has name, path, file fd/handle, everything to do with one file, plus an 'inUse' boolean.
A 'waitingThreads' array for those threads waiting for a set of files.
A 'Sfiles' struct with an array of *Sfile to hold all the files, a waitingThreads array and a lock, (mutex/futex/criticalSection).
Each thread should have an event/semaphore/something that it can wait on until its files all become available and some way to access to the set of files that it needs and somewhere to store the fds/handles/whatever for the files.
OK, off we go:
Any thread that wants files locks up the Sfiles and iterates the *Sfile array, checking if every file it needs is free to use. If they all are, it sets the 'inUse' boolean, loads itself up with the fd/handles, unlocks and runs on - it has all its files. If any file it needs is in use, it pushes itself onto the waitingThreads array and waits on its event/sema.
Whne a thread is done with its files, it locks the Sfiles and clears the 'inUse' boolean for the files it was using. It then iterates the waitingThreads array - if the array is empty, it just unlocks and exits. If the array is not empty, it tries to find threads that can now run with the files that are now free. If it finds none, it just unlocks and returns. If it does find one, it loads that thread up with the fd/handles, sets the inUse boolean and signals its event/sema - that thread will then run with its desired set of files. The thread continues to iterate the waitingThreads array to the end, looking for mre threads that it can load up and signal with the remaining free files. When it reaches the end of the array, it returns.
That, or something like it, will ensure that the threads always run with their complete set of files, prevent any deadlocks due to threads locking partial sets of files and does not require any polling.
If you really, really need that table thingy, you can build it inside the lock every time a thread enters or leaves the lock. I would suggest mallocing a suitable struct, loading it up with all the details of the free files and waiting threads, and queueing it off to another thread. You could just have some 'monitoring' thread that periodically locks up the Sfiles, dumps all the info and unlocks, but that keeps the Sfiles locked for the entire 'dump' time - you may not want that overhead - it's up to you.
Edit:
OH - forgot the priority thingy. The OS thread priority is probably useless for your purpose. Have each thread expose a priority enum/int and keep the 'waitingThreads' array sorted by that priority, so giving the higher priority threads the first bite at whatever files are returned.
Is that good enough for your homework assignment?