(correct me if im wrong on my terms) So i need to read from a file descriptor, but the read method takes in a int for byte size to read that much OR i can use O_NONBLOCK, but i still have to setup up a buffer size of an unknown size. making it difficult. heres what i have so far
this is my method that handles all the polling and mkfifo. and N is already predefined in main
struct pollfd pfd[N];
int i;
for(i = 0; i < N; i++)
{
char fileName[32];
snprintf (fileName, sizeof(fileName), "%d_%di", pid, i);
mkfifo(fileName, 0666);
pfd[i].fd = open(fileName, O_RDONLY | O_NDELAY);
pfd[i].events = POLLIN;
pfd[i].revents = 0;
snprintf (fileName, sizeof(fileName), "%d_%do", pid, i);
mkfifo(fileName, 0666);
i++;
pfd[i].fd = open(fileName, O_WRONLY | O_NDELAY);
pfd[i].events = POLLOUT;
pfd[i].revents = 0;
i--;
}
while(1)
{
int len, n;
n = poll(pfd, N, 2000);
if( n < 0 )
{
printf("ERROR on poll");
continue;
}
if(n == 0)
{
printf("waiting....\n");
continue;
}
for(i = 0; i < N; i++)
{
char buff[1024]; <---i dont want to do this
if (pfd[i].revents & POLLIN)
{
printf("Processing input....\n");
read(pfd[i].fd, buff, O_NONBLOCK);
readBattlefield(buff);
print_battleField_stats();
pfd[i].fd = 0;
}
}
}
i also read somewhere that once read() reads all the data coming, it empties the pipe, meaning i can use the same again for another incoming data. but it doesnt empty the pipe because i cant use the same pipe again. I asked my professor but all he says was to use something like scanf, but how do use scanf if scanf takes a FILE stream, and the poll.fd is an int? essentially my ultimate question is, how to read the incoming data through the file descriptor using scan or of other sort? using scan will help me more with handling the data.
EDIT:
in another terminal i have to put cat file > (named_file)
and my main program will read the input data. heres what the input data looks like
3 3
1 2 0
0 2 0
3 0 0
first 2 numbers are grid information and player number, and after that is grid, but this a simplified version, ill be dealing with sizes over 100's of players and grids of over 1000's
char buff[1024]; <---i dont want to do this
What would you like to do then? This is how it works. This is not how it works:
read(pfd[i].fd, buff, O_NONBLOCK);
This will compile because O_NONBLOCK is an integer #define, but it is absolutely and unequivocally incorrect. The third argument to read() is a number of bytes to read. Not a flag. Period. It may be zero, but what you've done here is pass an arbitrary number -- whatever the value of O_NONBLOCK is, which could easily be more than 1024, the size of your buffer. This does not set the read non-block. recv() is similar to read() and does take such flags as a forth argument, but you can't use that with a file descriptor. If you want to set non-block on a file descriptor, you must do it with open() or fcntl().
how to read the incoming data through the file descriptor using scan or of other sort?
You can create a FILE* stream from an open descriptor with fdopen().
i also read somewhere that once read() reads all the data coming, it empties the pipe, meaning i can use the same again for another incoming data. but it doesnt empty the pipe because i cant use the same pipe again.
Once you reach EOF (because the writer closed the connection), read() will return 0, and continue to return 0 immediately until someone opens the pipe again.
If you set the descriptor non-block, read() will always return immediately; if there is someone connected and nothing to read, it will return -1 but errno will == EAGAIN. See man 2 read.
man fifo is definitely something you should read; if there's anything you aren't sure about, ask a specific question based on that.
And don't forget: Fix that read() call. It's wrong. W R O N G. Your prof/TA/whoever will not miss that.
Related
In the code below, I am trying to read from a socket and store the results in a file.
What actually happens, is that my client sends a GET request to my server for a file.html. My server finds the file and writes the contents of it to the socket. Lastly my client reads the content from thread_fd and recreates the file.
For some reason the recreated file has less content than the original. I have located the problem to be some lines in the end, that are missing. When I use printf("%s", buffer) inside the while loop everything seems fine in STDOUT but my fprintf misses somewhat 3.000 bytes for a file of 81.000 bytes size.
#define MAXSIZE 1000
int bytes_read, thread_fd;
char buffer[MAXSIZE];
FILE* new_file;
memset(buffer, 0, MAXSIZE);
if((new_file = fopen(path, "wb+")) == NULL)
{
printf("can not open file \n");
exit(EXIT_FAILURE);
}
while ((bytes_read = read(thread_fd, buffer, MAXSIZE)) > 0)
{
fprintf(new_file, "%s", buffer);
if(bytes_read < MAXSIZE)
break;
memset(buffer, 0, MAXSIZE);
}
You read binary data from the socket that may or may not contain a \0 byte. When you then fprintf that data the fprintf will stop at the first \0 it encounters. In your case that is 3000 bytes short of the full file. If your file contains no \0 byte the fprintf will simply continue printing the ram contents until it segfaults.
Use write() to write the data back to the file and check for errors. Don't forget to close() the file and check that for errors too.
Your code should/could look like:
int readfile(int thread_fd, char *path)
{
unsigned int bytes_read;
char buffer[MAXSIZE];
int new_file;
if ((new_file = open(path, _O_CREAT|_O_BINARY,_S_IWRITE)) == -1) return -1;
while ((bytes_read = read(thread_fd, buffer, MAXSIZE)) > 0)
{
if (write(new_file, buffer, bytes_read)!= bytes_read) {
close(new_file);
return -2;
}
}
close(new_file);
return 0;
}
There are a few issues with your code that can cause this.
The most likely cause is this :
if(bytes_read < MAXSIZE)
break;
This ends the loop when read returns less than the requested amount of bytes. This is however perfectly normal behavior, and can happen eg. when not enough bytes are available at the time of the read call (it's reading from a network socket after all). Just let the loop continue as long as read returns a value > 0 (assuming the socket is a blocking socket - if not, you'll also have to check for EAGAIN and EWOULDBLOCK).
Additionally, if the file you're receiving contains binary data, then it's not a good idea to use fprintf with "%s" to write to the target file. This will stop writing as soon as it finds a '\0' byte (which is not uncommon in binary data). Use fwrite instead.
Even if you're receiving text (suggested by the html file extension), it's still not a good idea to use fprintf with "%s", since the received data won't be '\0' terminated.
This worked!
ps: I don't know if I should be doing this, since I am new here, but really there is no reason for negativity. Any question is a good question. Just answer it if you know it. Do not judge it.
#define MAXSIZE 1000
int bytes_read, thread_fd, new_file;
char buffer[MAXSIZE];
memset(buffer, 0, MAXSIZE);
if((new_file = open(path, O_RDONLY | O_WRONLY | O_CREAT)) < 0)
{
printf("can not open file \n");
exit(EXIT_FAILURE);
}
while ((bytes_read = read(thread_fd, buffer, MAXSIZE)) > 0)
write(new_file, buffer, bytes_read);
close(new_file);
I am trying to wrap my head around the read() system call.
How can I read an actual file byte by byte using read()?
The first parameter is the file descriptor which is of type int.
How can I pass a file to the read() call?
You open the file with open(); you pass the file descriptor returned by open() to read().
int fd;
if ((fd = open(filename, O_RDWR)) >= 0)
{
char c;
while (read(fd, &c, 1) == 1)
putchar(c);
}
There are other functions that return file descriptors: creat(), pipe(), socket(), accept(), etc.
Note that while this would work, it is inefficient because it makes a lot system calls. Normally, you read large numbers of bytes at a time so as to cut down on the number of system calls. The standard I/O libraries (in <stdio.h>) handle this automatically. If you use the low-level open(), read(), write(), close() system calls, you have to worry about buffering etc for yourself.
The last argument to read() is the number of bytes to read from the file, so passing 1 to it would do it. Before that, you use open() to get a file handle, something like this (untested code):
int fh = open("filename", O_RDONLY);
char buffer[1];
read(fh, buffer, 1);
However, it's usually not recommended to read files byte by byte, as it affects performance significantly. Instead, you should buffer your input and process it in chunks, like so:
int fh = open("filename", O_RDONLY);
char buffer[BUFFER_SIZE];
read(fh, buffer, BUFFER_SIZE);
for (int i=0 ; i < BUFFER_SIZE ; ++i) {
// process bytes at buffer[i]
}
You would finally wrap your reads in a loop until EOF is reached.
The concept of read() system call to Kernel is this (In simple english)
read (from this file (file descriptor), into this buffer in the memory, of this size )
Example: Read a character by character from a file which is in the disk into this buffer BUFF
int fd // initialize the File Descriptor
fd = open ("file_name", O_RDONLY); //open a file with file name in read only mode.
char BUFF;
read (fd,&BUFF,sizeof(char)); // read file with file descriptor into the address of the BUFF buffer in the memory of a character of size CHAR data type.
I have a program that creates a number of input named pipes for which I must use poll() in order to watch over those pipes and get the information that has been written in them at the time that something has been written in them. I'm very new to polling and I couldn't find many examples that would clearly show how to use poll for multiple file descriptors.
Here is how I wrote the code:
char buffer [1024];
size_t count = 0;
ssize_t = bytes;
while(1)
{
int n = poll(pollFd, number_of_pipes, 3000);
if(n != 0)
{
if (n == -1)
{
perror("poll");
exit(1);
}
for(j = 0; j < number_of_pipes; j++)
{
if(pollFd[j].revents & POLLIN)
{
//read the written pipe
if((bytes = read(fd[j], buffer, sizeof(buffer))) > 0)
count += (size_t) bytes;
}
}
}
}
However, I'm not sure if this the correct way to handle the multiple input pipes while using poll(); since I'm also not sure how to know when the read function have reached the end of the file.
The code looks ok, if incomplete (you don't show how you set up the pollFd and fd arrays). It does ignore the actual data read, just counting the total amount; for a real program you probably want to do something with the data.
A couple of comments
If you change it to read from pollFd[j].fd instead of fd[j], you don't need the redundant fd array -- the descriptors are necessarily all in the pollFd array
You don't check for EOF or errors on read -- if read returns 0 or -1, you should remove that entry from the pollFd array and reduce number_of_pipes.
would install valgrind to tell me what the problem is, but unfortunately can't any new programs on this computer... Could anyone tell me if there's an obvious problem with this "echo" program? Doing this for a friend, so not sure what the layout of the client is on the other side, but I know that both reads and writes are valid socket descriptors, and I've tested that n = write(writes,"I got your message \n",20); and n = write(reads,"I got your message \n",20); both work so can confirm that it's not a case of an invalid fd. Thanks!
int
main( int argc, char** argv ) {
int reads = atoi(argv[1]) ;
int writes = atoi(argv[3]) ;
int n ;
char buffer[MAX_LINE];
memset(buffer, 0, sizeof(buffer));
int i = 0 ;
while (1) {
read(reads, buffer, sizeof(buffer));
n = write(writes,buffer,sizeof(buffer));
if (n < 0) perror("ERROR reading from socket");
}
There are a few problems, the most pressing of which is that you're likely pushing garbage data down the the write socket by using sizeof(buffer) when writing. Lets say you read data from the reads socket and it's less than MAX_LINES. When you go to write that data, you'll be writing whatever you read plus the garbage at the end of the buffer (even though you memset at the very beginning, continual use of the same buffer without reacting to different read sizes will probably generate some garbage.
Try getting the return value from read and using it in your write. If the read return indicates an error, clean up and either exit or try again, depending on how you want your program to behave.
int n, size;
while (1) {
size = read(reads, buffer, sizeof(buffer));
if (size > 0) {
n = write(writes, buffer, size);
if (n != size) {
// write error, do something
}
} else {
// Read error, do something
}
}
This, of course, assumes your writes and reads are valid file descriptors.
These two lines look very suspicious:
int reads = atoi(argv[1]) ;
int writes = atoi(argv[3]) ;
Do you really get file/socket descriptor numbers on the command line? From where?
Check the return value of your read(2) and write(2), and then the value of errno(3) - they probably tell you that your file descriptors are invalid (EBADF).
One point not made thus far: Although you know that the file descriptors are valid, you should include some sanity checking of the command line.
if (argc < 3) {
printf("usage: foo: input output\n");
exit(0);
}
Even with this sanity checking passing parameters like this on a command line can be dangerous.
The memset() is not needed, provided you change the following (which you should do nevertheless).
read() has a result, telling you how much it has actually read. This you should give to write() in order to write only what you actually have, removing the need for zeroing.
MAX_LINE should be at least 512, if not more.
There probably are some more issues, but I think I have the most important ones.
I have got a small program that prints the contents of files using the system call - read.
unsigned char buffer[8];
size_t offset=0;
size_t bytes_read;
int i;
int fd = open(argv[1], O_RDONLY);
do{
bytes_read = read(fd, buffer, sizeof(buffer));
printf("0x%06x : ", offset);
for(i=0; i<bytes_read; ++i)
{
printf("%c ", buffer[i]);
}
printf("\n");
offset = offset + bytes_read;
}while(bytes_read == sizeof(buffer));
Now while running I give a file name that doesn't exist.
It prints some kind of data mixed with environment variables and a segmentation fault at the end.
How is this possible? What is the program printing?
Thanks,
John
It's printing rubbish because fd will invariably be set to -1 which is not a good thing to pass to read since it will, in turn do nothing other than return -1 as well. It will leave your buffer untouched meaning that it's holding whatever rubbish you had in there when you started.
You could probably put the entire do loop inside something like:
if (fd == -1) {
printf ("error here");
} else {
// do loop here
}
read is returning -1 because fd is invalid, you store that in bytes_read which is of type size_t which is unsigned, so your loop prints (size_t)-1 chars, which is a very large number, much larger than the size of buffer. So, you're printing a big chunk of your address space and then getting a segfault when you eventually reach the end and access an invalid address.
As others have mentioned (without answering your actual question), you should be checking the results of open for an error. e.g.,
int fd = open(argv[1], O_RDONLY);
if( fd < 0 ){
fprintf(stderr, "error opening %s: %s\n", argv[1], strerror(errno));
exit(1);
}
A caveat: if you do another system call, or call any routine that might do a system call (e.g., printf) before calling strerror, you must save errno and then pass the saved copy to strerror.
Another note about your program:
while(bytes_read == sizeof(buffer))
This is not a good test, because read can return less than the amount you ask for. Your loop should continue until read returns <= 0.
You should probably check that the file descriptor returned by open is valid before using it. As per these docs, you should get a non-negative response for a valid file. Reading from an invalid descriptor is likely the source of your problem.
Upon successful completion, open function shall open the file and return a non-negative integer representing the file descriptor. Otherwise, -1 shall be returned and errno set to indicate the error. So please check fd before entering the loop to perform the read.