UPDATE: i updated the code and problem description to reflect my changes.
I know now that i'm trying a Socket operation on nonsocket. or that my fd_set is not valid since:
select returns -1 and
WSAGetLastError()returns 10038.
But i can't seem to figure out what it is. Platform is Windows. I have not posted the WSAStartup part.
int loop = 0;
FILE *output
int main()
{
fd_set fd;
output = _popen("tail -f test.txt","r");
while(forceExit == 0)
{
FD_ZERO(&fd);
FD_SET(_fileno(output),&fd);
int returncode = select(_fileno(output)+1,&fd,NULL,NULL,NULL);
if(returncode == 0)
{
printf("timed out");
}
else if (returncode < 0)
{
printf("returncode: %d\n",returncode);
printf("Last Error: %d\n",WSAGetLastError());
}
else
{
if(FD_ISSET(_fileno(output),&fd))
{
if(fgets(buff, sizeof(buff), output) != NULL )
{
printf("Output: %s\n", buff);
}
}
else
{
printf(".");
}
}
Sleep(500);
}
return 0;
}
The new outcome now is of course the print out of the returncode and the last error.
You have some data ready to be read, but you are not actually reading anything. When you poll the descriptor next time, the data will still be there. Drain the pipe before you continue to poll.
As far as I can tell, Windows anonymous pipes cannot be used with non-blocking calls like select. So, while your _popen and select code looks good independently, you can't join the two together.
Here's a similar thread elsewhere.
It's possible that calling SetNamedPipeHandleState with the PIPE_NOWAIT flag might work for you, but MSDN is more than a little cryptic on the subject.
So, I think you need to look at other ways of achieving this. I'd suggest having the reading in a separate thread, and use normal blocking I/O.
First of all, as yourself and others have pointed out, select() is only valid for sockets under Windows. select() does not work on streams which is what _popen() returns. Error 10038 clearly identifies this.
I don't get what the purpose of your example is. If you simply want to spawn a process and collect it's stdout, just do this (which comes directly from the MSDN _popen page):
int main( void )
{
char psBuffer[128];
FILE *pPipe;
if( (pPipe = _popen("tail -f test.txt", "rt" )) == NULL )
exit( 1 );
/* Read pipe until end of file, or an error occurs. */
while(fgets(psBuffer, 128, pPipe))
{
printf(psBuffer);
}
/* Close pipe and print return value of pPipe. */
if (feof( pPipe))
{
printf( "\nProcess returned %d\n", _pclose( pPipe ) );
}
else
{
printf( "Error: Failed to read the pipe to the end.\n");
}
}
That's it. No select required.
And I'm not sure how threads will help you here, this will just complicate your problem.
The first thing that I notice is wrong is that you are calling FD_ISSET on your exceptfds in each conditional. I think that you want something like this:
if (FD_ISSET(filePointer,&fd))
{
printf("i have data\n");
}
else ....
The except field in the select is typically used to report errors or out-of-band data on a socket. When one of the descriptors of your exception is set, it doesn't mean an error necessarily, but rather some "message" (i.e. out-of-band data). I suspect that for your application, you can probably get by without putting your file descriptor inside of an exception set. If you truly want to check for errors, you need to be checking the return value of select and doing something if it returns -1 (or SOCKET_ERROR on Windows). I'm not sure of your platform so I can't be more specific about the return code.
select() first argument is the highest number file descriptor in your set, plus 1. (i.e. output+1)
select(output+1, &fd, NULL, &exceptfds, NULL);
The first FD_ISSET(...) should be on the fd_set fd.
if (FD_ISSET(filePointer, &fd))
Your data stream has data, then you need to read that data stream. Use fgets(...) or similar to read from the data source.
char buf[1024];
...
fgets(buf, sizeof(buf) * sizeof(char), output);
The first argument to select needs to be the highest-numbered file descriptor in any of the three sets, plus 1:
int select(int nfds, fd_set *readfds, fd_set *writefds,
fd_set *exceptfds, struct timeval *timeout);
Also:
if(FD_ISSET(filePointer,&exceptfds))
{
printf("i have data\n");
}
Should be:
if(FD_ISSET(filePointer,&fd))
{
printf("i have data\n");
}
You should check the return code from select().
You also need to reset the fdsets each time you call select().
You don't need timeout since you're not using it.
Edit:
Apparently on Windows, nfds is ignored, but should probably be set correctly, just so the code is more portable.
If you want to use a timeout, you need to pass it into the select call as the last argument:
// Reset fd, exceptfds, and timeout before each select()...
int result = select(maxFDPlusOne, &fd, NULL, &exceptfds, &timeout);
if (result == 0)
{
// timeout
}
else if (result < 0)
{
// error
}
else
{
// something happened
if (FD_ISSET(filePointer,&fd))
{
// Need to read the data, otherwise you'll get notified each time.
}
}
since select doesn't work i used threads, specifically _beginthread , _beginthreadex.
Related
I'm writing a program to read from a pipe and I want to know what is the correct way of handling the return values. According to the read man page,
On success, the number of bytes read is returned (zero indicates end of file), and the file position is advanced by this number. It is not an error if this number is smaller than the number of bytes requested; this may happen for example because fewer bytes are actually available right now (maybe because we were close to end-of-file, or because we are reading from a pipe, or from a terminal), or because read() was interrupted by a signal.
I'm worried about the case where it may read only half of the data. Also, what is the correct way to handle the case when the return value is zero?
Here is my sample code.
struct day
{
int date;
int month;
};
while(1)
{
ret = select(maxfd+1, &read_fd, NULL, &exc_fd,NULL);
if(ret < 0)
{
perror("select");
continue;
}
if(FD_ISSET(pipefd[0], &read_fd))
{
struct day new_data;
if((ret = read(pipefd[0], &new_data, sizeof(struct day)))!= sizeof(struct day))
{
if(ret < 0)
{
perror("read from pipe");
continue;
}
else if(ret == 0)
{
/*how to handle?*/
}
else
/* truncated read. How to handle?*/
}
}
...
}
I believe read() cannot read more data than the size specified. please correct me if I'm wrong.
Please help me with the handling of return value of read.
When you read you request for a given amount of data, but nothing can guarantee you that there is as much available data to read as you requested. For example, you may encounter the end of file, or the writer part didn't write too much data in your pipe. So read returns you what was effectively read, aka the number of bytes read is returned (zero indicates end of file).
If read returns a strictly positive number, it's clear.
If read returns 0, then that means end of file. For a regular file that means that you are currently at the end of the file. For a pipe this means that the pipe is empty and that no single byte will ever be written to. For pipes that means that you already read all data and that there is no more writer on the other end (so that no more byte will be written), so you can then close the now unuseful pipe.
If read returns -1 this means that an error happened, and you must consult errno variable to determine the cause of the trouble.
So, a general schema could be something like:
n = read(descriptor,buffer,size);
if (n==0) { // EOF
close(descriptor);
} else if (n==-1) { // error
switch(errno) { // consult documentations for possible errors
case EAGAIN: // blahblah
}
} else { // available data
// exploit data from buffer[0] to buffer[n-1] (included)
}
If read returns 0, then your process has read all of the data that will come from that file descriptor. Take it out of read_fd and if it was the maxfd reset maxfd to the newest max. Depending on what your process is doing, you might have other cleanup to do as well. If you get a short read, then either process the data you've received or discard it or store it until you get all the data and can process it.
It's hard to give more specific answers to a very general question.
For an assignment we have to create C program that functions similar to the cat command. The first hand-in requires it to mimic very minimal operations of cat....i.e print to output, redirect. The issue I'm having is that one requirement is to print an error in the case that an output file residing on a usb drive is lost, i.e usb pulled out whilst redirecting stdout to it.
How do I catch such an error, also how can perform a test-case for that particular error ??
Many Thanks....really have no idea
UPDATE CODE TEMP
int main(){
char c;
while((c = getchar()) != EOF){
putchar(c);
// Ensure newly created file exists
}
return EXIT_SUCCESS;
}
Assuming you are using fprintf(), from the man pages:
On success, the total number of characters written is returned.
So:
store the size of the char array you will write into a variable x
if fprintf() is less than x, the writing was interrupted.
exit gracefully
EDIT:
There are 2 things I'm thinking of:
1: When putchar() fails, it indicates an error when writing to the file. Since writing one byte doesn't take very long, this should be unlikely since it will be in a safe state once the byte is written (or you assume).
You can do this like so
if(putchar(c) == EOF){
//write error
}
2: If you're asked to quit the instant you detect a file removal, then you need to monitor the directory. Luckily, you're only looking at one directory. However that while loop gets in the way of things because getchar() is a blocking function (cannot return until something happens). You should use inotify to monitor the directory, then probably poll to poll the file descriptor of inotify(). When I did this I used select because we were forced to.
Some kind of an idea how to monitor a directory with inotify()
int length, i = 0;
char buffer[EVENT_BUF_LEN];
memset(buffer, 0, EVENT_BUF_LEN*sizeof(char));
//init inotify
fd = inotify_init();
if(fd < 0){
perror("inotify init");
}
//add directory to watch list
wd = inotify_add_watch(fd, path , IN_DELETE |
IN_DELETE_SELF | IN_MODIFY | IN_MOVE_SELF | IN_MOVED_FROM | IN_MOVED_TO);
fd_set fds;
FD_ZERO(&fds);
FD_SET(fd, &fds);
//wait for event, since read() blocks
length = read( fd, buffer, EVENT_BUF_LEN );
if ( length < 0 ) {
perror("zero event length");
}
struct inotify_event *event;
while (i < length){
//cast the event to a char buffer
event = (struct inotify_event*) &buffer[i];
if (event->len){
//this was a custom function of mine
storeEvent(event);
}
i += EVENT_SIZE + event->len;
}
You'll have to check which attributes to use when adding a directory (like IN_DELETE or IN_MODIFY) since they will determine what triggers an inotify() event. Note this code will only detect one event, and blocks at the read() statement.
I read the man pages, and my understanding is that if write() fails and sets the errno to EAGAIN or EINTR, I may perform the write() again, so I came up with the following code:
ret = 0;
while(ret != count) {
write_count = write(connFD, (char *)buf + ret, count);
while (write_count < 0) {
switch(errno) {
case EINTR:
case EAGAIN:
write_count = write(connFD, (char *)buf + ret, count -ret);
break;
default:
printf("\n The value of ret is : %d\n", ret);
printf("\n The error number is : %d\n", errno);
ASSERT(0);
}
}
ret += write_count;
}
I am performing read() and write() on sockets and handling the read() similarly as above. I am using Linux, with gcc compiler.
You have a bit of a "don't repeat yourself" problem there - there's no need for two separate calls to write, nor for two nested loops.
My normal loop would look something like this:
for (int n = 0; n < count; ) {
int ret = write(fd, (char *)buf + n, count - n);
if (ret < 0) {
if (errno == EINTR || errno == EAGAIN) continue; // try again
perror("write");
break;
} else {
n += ret;
}
}
// if (n < count) here some error occurred
EINTR and EAGAIN handling should often be slightly different. EAGAIN is always some kind of transient error representing the state of the socket buffer (or perhaps, more precisely, that your operation may block).
Once you've hit an EAGAIN you'd likely want to sleep a bit or return control to an event loop (assuming you're using one).
With EINTR the situation is a bit different. If your application is receiving signals non-stop, then it may be an issue in your application or environment, and for that reason I tend to have some kind of internal eintr_max counter so I am not stuck in the theoretical situation where I just continue infinitely looping on EINTR.
Alnitak's answer (sufficient for most cases) should also be saving errno somewhere, as it may be clobbered by perror() (although it may have been omitted for brevity).
I would prefer to poll the descriptor in case of EAGAIN instead of just busy looping and burning up CPU for no good reason. This is kind of a "blocking wrapper" for a non-blocking write I use:
ssize_t written = 0;
while (written < to_write) {
ssize_t result;
if ((result = write(fd, buffer, to_write - written)) < 0) {
if (errno == EAGAIN) {
struct pollfd pfd = { .fd = fd, .events = POLLOUT };
if (poll(&pfd, 1, -1) <= 0 && errno != EAGAIN) {
break;
}
continue;
}
return written ? written : result;
}
written += result;
buffer += result;
}
return written;
Note that I'm not actually checking the results of poll other than the return value; I figure the following write will fail if there is a permanent error on the descriptor.
You may wish to include EINTR as a retryable error as well by simply adding it to the conditions with EAGAIN, but I prefer it to actually interrupt I/O.
Yes, there are cleaner ways to use write(): the class of write functions taking a FILE* as an argument. That is, most importantly, fprintf() and fwrite(). Internally, these library functions use the write() syscall to do their job, and they handle stuff like EAGAIN and EINTR.
If you only have a file descriptor, you can always wrap it into a FILE* by means of fdopen(), so you can use it with the functions above.
However, there is one pitfall: FILE* streams are usually buffered. This can be a problem if you are communicating with some other program and are waiting for its response. This may deadlock both programs even though there is no logical error, simply because fprintf() decided to defer the corresponding write() a bit. You can switch the buffering off, or fflush() output streams whenever you actually need the write() calls to be performed.
I'm currently writing a server application on Linux x86_64 using <sys/socket.h>.
After accepting a connection via accept(), I use fdopen() to wrap the retrieved socket into a FILE* stream.
Writing to, and reading from, that FILE* stream usually works quite well, but the socket becomes unsusable as soon as I write to it while it has a non-empty read buffer.
For demonstration purposes, I've written some code that listens for a connection, then reads the input, line by line, into a read buffer using fgetc(). If the line is too long to fit into the buffer, it's not completely read, but instead read during the next iteration.
#include <unistd.h>
#include <stdio.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
FILE* listen_on_port(unsigned short port) {
int sock = socket(AF_INET, SOCK_STREAM, 0);
struct sockaddr_in name;
name.sin_family = AF_INET;
name.sin_port = htons(port);
name.sin_addr.s_addr = htonl(INADDR_ANY);
if(bind(sock, (struct sockaddr*) &name, sizeof(name)) < 0)
perror("bind failed");
listen(sock, 5);
int newsock = accept(sock, 0, 0);
return fdopen(newsock, "r+");
}
int main(int argc, char** argv) {
int bufsize = 8;
char buf[9];
buf[8] = 0; //ensure null termination
int data;
int size;
//listen on the port specified in argv[1]
FILE* sock = listen_on_port(atoi(argv[1]));
puts("New connection incoming");
while(1) {
//read a single line
for(size = 0; size < bufsize; size++) {
data = fgetc(sock);
if(data == EOF)
break;
if(data == '\n') {
buf[size] = 0;
break;
}
buf[size] = (char) data;
}
//check if the read failed due to an EOF
if(data == EOF) {
perror("EOF: Connection reset by peer");
break;
} else {
printf("Input line: '%s'\n", buf);
}
//try to write ack
if(fputs("ack\n", sock) == EOF)
perror("sending 'ack' failed");
//try to flush
if(fflush(sock) == EOF)
perror("fflush failed");
}
puts("Connection closed");
}
The code should compile in gcc without any special parameters. Run it with the port number as argument and use netcat to connect to it locally.
Now, if you try sending strings that are shorter than 8 characters, this will run flawlessly.
But if you send a string containing more than 10 characters, the program will fail.
This sample input:
ab
cd
abcdefghij
Will create this output:
New connection incoming
Input line: 'ab'
Input line: 'cd'
Input line: 'abcdefgh'
fflush failed: Illegal seek
EOF: Connection reset by peer: Illegal seek
Connection closed
As you see, (rightly) only the first 8 characters of abcdefgh are read, but when the program tries to send the 'ack' string (which the client never receves), and then flush the output buffer, we receive an Illegal seek error, and the next call to fgetc() returns EOF.
If the fflush() part is commented out, the same error still occurs, but the
fflush failed: Illegal seek
line is missing from the server output.
If the fputs(ack) part is commented out, everything seems to work as intended, but a perror() manually called from gdb still reports an 'Illegal seek' error.
If both fputs(ack) and fflush() are commented out, everything does work as intended.
Unfortunately, I've not been able to find any good documentation, nor any Internet discussions on this problem, so your help would be greatly appreciated.
edit
The solution i finally settled for is to not use fdopen() and FILE*, since there seems to be no clean way of converting a socket fd into a FILE* that can reliably used in r+ mode.
Instead, I directly worked on the socket fd, writing my own replacement code for fputs and fprintf.
If anyone needs it, here is the code.
Clearly "r+" (read/write) mode does not work on sockets in this implementation, no doubt because the underlying code assumes that it must do a seek to switch between reading and writing. This is the general case with stdio streams (that you must do some kind of synchronizing operation), because back in the Dim Time, actual stdio implementations had only a single counter per stream, and it was either a counter of "number of characters left to read from stream buffer via getc macro" (in read mode) or "number of characters that can safely be written to stream buffer via putc macro (in write mode). To that that single counter re-set, you had to do a seek-type operation.
Seeks are not allowed on pipes and sockets (since "file offset" is not meaningful there).
One solution is not to wrap a socket with stdio at all. Another, probably easier / better for your purposes, is to wrap it with, not one, but two stdio streams:
FILE *in = fdopen(newsock, "r");
FILE *out = fdopen(newsock, "w");
There's another flaw here though, because when you go to fclose one stream, that closes the other's file descriptor. To work around that, you need to dup the socket descriptor once (in either of the two calls above, it does not matter which one).
If you intend to use select or poll or similar on the socket at some point, you should generally go for the "don't wrap with stdio" solution, since there's no nice clean portable way to track stdio buffering. (There are implementation-specific ways).
Don't use fflush() on network sockets. They are unbuffered streams.
Also, this code:
//read a single line
for(size = 0; size < bufsize; size++) {
data = fgetc(sock);
if(data == EOF)
break;
if(data == '\n') {
buf[size] = 0;
break;
}
buf[size] = (char) data;
}
does not read a single line. It only reads up to the buffer size, which you defined as 8. sock will still have data for you to receive which you should receive before writing to the stream with fputs. BTW you can replace that whole block with
fgets(buf, bufsize, sock);
Yes, you can use one file stream to handle your socket, at least on Linux.
But you should be careful with it: you must only use ferror() to test for errors. I have some code that use this and run flawlessly in production on a major French site.
If you use errno or perror() you'll catch any internal error that the stream will encounter, even if it wants to hide it to you. And "Illegal seek" is one of them.
Also, to test for real EOF conditions, you should use feof(), since when returning true it's mutually exclusive with ferror() returning a non-zero value. It's because, when using fgetc() you don't have any mean to differentiate error from real EOF conditions. So you should probably better use fgets() as another user pointed out.
So, your test:
if(data == EOF) {
perror("EOF: Connection reset by peer");
break;
} else {
printf("Input line: '%s'\n", buf);
}
Should be written as:
int sock_error = ferror(sock);
if (sock_error) {
fprintf(stderr, "Error while reading: %s", strerror(sock_error));
} else {
printf("Input line: '%s'\n", buf);
}
Try this :
#define BUFSIZE 88
FILE* listen_on_port(unsigned short port) {
...
}
int main(int argc, char** argv) {
int bufsize = BUFSIZE;
char buf[ BUFSIZE ];
I'm writing a ncurses based chat program. At first, I wrote just networking stuff (without ncurses) and everything worked fine, but after adding graphics I can't get the client app to work properly.
The main problem is reading from stdin and socket at the same time. In ncurses-less version I've used pthread and it worked like charm. Alas, it seems that pthread and ncurses don't go together very well, so I had to find another solution.
I thought that select() would do, but it still only reads from stdin and completely ignores the socket.
Here is the whole code: code
The interesting part is:
char message[1024];
fd_set master;
fd_set read_fds;
FD_ZERO(&master);
FD_ZERO(&read_fds);
FD_SET(0,&master);
FD_SET(s,&master); // s is a socket descriptor
while(true){
read_fds = master;
if (select(2,&read_fds,NULL,NULL,NULL) == -1){
perror("select:");
exit(1);
}
// if there are any data ready to read from the socket
if (FD_ISSET(s, &read_fds)){
n = read(s,buf,max);
buf[n]=0;
if(n<0)
{
printf("Blad odczytu z gniazdka");
exit(1);
}
mvwprintw(output_window,1,1,"%s\n",buf);
}
// if there is something in stdin
if (FD_ISSET(0, &read_fds)){
getstr(message);
move(CURS_Y++,CURS_X);
if (CURS_Y == LINES-2){
CURS_Y = 1;
}
n = write(s,message,strlen(message));
if (n < 0){
perror("writeThread:");
exit(1);
}
}
}
It's possible that I don't fully understand how select() works, or maybe I shouldn't have connect()ed the socket.. I'm lost here. I would appreciate any help! Thanks.
Your problem is in the select().
The first parameter is not the number of file descriptors you are passing in read_fds, but it's the highest socket ID + 1.
From the man page:
The first nfds descriptors are checked in each set; i.e., the
descriptors from 0 through nfds-1 in the descriptor sets are examined. (Example: If you have set two file descriptors "4" and "17", nfds should not be "2", but rather "17 + 1" or "18".)
So in your code, instead of '2', try passing 's+1'.
You need to specify the highest file descriptor to select:
if (select(s + 1,&read_fds,NULL,NULL,NULL) == -1){
select() needs to know the number of file descriptors that it is supposed to watch.