This question already has answers here:
Why does printf not flush after the call unless a newline is in the format string?
(10 answers)
Closed 5 years ago.
Preface: I am just engaging C, so forgive my incompetence. The cause of this problem is probably basic.
Problem: I am trying to read a file and serve it over http with a socket. For some reason, the printf output of a previously read file varies depending on how much of the subsequent code is included.
Here is my code.
#include <stdio.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/socket.h>
#include <sys/types.h>
#include <unistd.h>
#include <netinet/in.h>
int main()
{
// open a file to serve
FILE *file;
file = fopen("index.html", "r");
if (file == NULL)
{
printf("Failed to open file.");
exit(EXIT_FAILURE);
}
// Get file content
char file_content[1024];
if (fgets(file_content, 1024, file) != NULL)
{
fclose(file);
// Add header to file content
char http_header[2048] = "HTTP/1.1 200 OK\r\n\n";
strncat(http_header, file_content, 1028);
// This output varies depending on inclusion of proceeding code.
printf("%s", http_header);
// create a socket
int server_socket;
server_socket = socket(AF_INET, SOCK_STREAM, 0);
// define the address
struct sockaddr_in server_address;
server_address.sin_family = AF_INET;
server_address.sin_port = htons(8001);
server_address.sin_addr.s_addr = INADDR_ANY;
bind(server_socket, (struct sockaddr *) &server_address, sizeof(server_address));
listen(server_socket, 1);
int client_socket;
while (1)
{
client_socket = accept(server_socket, NULL, NULL);
send(client_socket, http_header, sizeof(http_header), 0);
close(client_socket);
}
return 0;
}
}
If I comment out everything past the printf statement, I get this intended output..
HTTP/1.1 200 OK
<html><body>Hi</body></html>
But if I run all the code, I get this instead..
HTTP/1.1 200 OK
If I can improve my question at all, please let me know how. Thank you.
Mark Benningfield and Yano helped me understand the problem, thanks so much to both of you.
When you execute printf(), instead of the string going directly into the terminal window, each line goes into a buffer.
Each line in the buffer must be flushed for them to show in the terminal. The buffer can be automatically flushed by these events
Strings ending in \n are automatically flushed, this is why the first line HTTP/1.1 200 OK \r\n\n displays fine. The second line <html><body>hi</body></html> does not end with a \n, or trigger any other automatic flush events, so it does not display.
Usually this is not a problem because when the program terminates, the buffer is automatically flushed, and all lines that entered the buffer through printf show up in the terminal. Since my program at the bottom enters an infinite while loop, the program never terminates, and the buffer is never flushed.
Calling fflush(stdout) after printf forces a write of stdout's entire buffer.
More info on fflush
Related
So, I can use FD_ZERO, FD_SET and FD_ISSET to mess around with the file descriptor set. The select() function may now be used to observe if a certain file object becomes readable (hope my wording is right here). What I want to do is: Read in a sequence of characters from stdin and then send this sequence to a server. I leave out some error messages of the socket stuff here, however, because my question is about FD_ISSET.
#include <stdio.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet.in.h>
#include <arpa/inet.h>
int main(int argc, char *argv[])
{
static char buffer[256];
int sock_fd, length, port;
struct sockaddr_in server_addr;
fd_set input_fdset
sock_fd = socket(PF_INET, SOCK_STREAM, 0);
server_addr.sin_family = AF_INET;
server_addr.sin_port = htons(port);
connect(sock_fd, (struct sockaddr *) &server_addr, sizeof(struct sockaddr_in));
while (1)
{
FD_ZERO(&input_fdset);
FD_SET(STDIN_FILENO, &input_fdset);
FD_SET(sock_fd, &input_fdset);
if (select(sock_fd+1, &input_fdset, NULL, NULL, NULL) == -1)
perror("some error");
if (FD_ISSET(STDIN_FILENO, &input_fdset))
{
if(fgets(buffer, 256, stdin) == NULL)
{
printf("connect: Closing socket.");
break;
}
length = strlen(buffer);
send(sock_fd, buffer, length, 0);
}
else
{
length = recv(sock_fd, buffer, 256, 0);
if(length == 0)
{
printf( "Connection closed by remote host.");
break;
}
write(STDOUT_FILENO, buffer, length);
}
}
close(sock_fd);
return(0);
}
You can probably guess I came across this code.
I have troubles to understand the following: Within the while-loop, I add both the socket object and stdin to the file descriptor set. In the next step select checks whether there is anything to read from either of these two objects. Then the if(FD_ISSET(STDIN_FILENO, &input_fdset)) should ALWAYS be executed, right? I mean the condition is always fulfilled, because we set the STDIN_FILENO using FD_SET(STDIN_FILENO, &input_fdset). Now, I am within this if-statement. fgets reads in my terminal input line by line. When I end this input with ENTER, a newline-character is appended to my buffer and reading from stdin stops. But only until the while-loop is executed again, right? Because in that case fgets gets called again as well. Btw I think it is kind of weird to have fgets only within this if-statement. Shouldn't it be called before? How can fgets become NULL? Only if EOF is reached without reading in any character. This won't happen though, will it? I cannot reach EOF when I am reading from stdin. I think. And now the last question: When would I execute the else-block? For this to happen, the previous if-statement would need to fail. But as I said, if(FD_ISSET(STDIN_FILENO, &input_fdset) will never fail.
Ofc I realize I am missunderstanding this code.
Can somebody help me figure this out?
From the documentation for select (man select):
"On return, select() replaces the given descriptor sets with subsets consisting of those descriptors that are ready for the requested operation."
The purpose of FD_ISSET is to determine which sets are ready to be read. The set is changed by the select call, by design.
This question already has answers here:
Why does printf not flush after the call unless a newline is in the format string?
(10 answers)
Closed 4 years ago.
I'm taking a class in the college that require me to create a server-client application, i came up with this code but apparently it has some strange error when I remove the "\n" at the end of the printf(). I already checked with my teacher and it was him who found out that this is was causing my server to freeze and display latter when we send the message, so he put \n in the end of each print(in the beggining there wasn't). We spend a couple of hours trying to find a reason but we didn't find anything to explain, so i'm reaching for you guys(so as my teacher who is curious about), i don't know if we let something pass by.
To create the error that i'm talking about, simply remove \n from each printf and run the code, he'll run but not work properly (even though the port seems to be listen in the telnet command) the lines who was supposed to appear will not show.
#include <strings.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <stdio.h>
#include <arpa/inet.h>
int main()
{
//criar socket -> bind -> listen -> connection
int socket_name = 0, new_conn = 0;
struct sockaddr_in serv_addr;
char msg_recv[20];
char sendBuff[1025];
socket_name = socket(AF_INET, SOCK_STREAM, 0);
if(socket_name < 0)
{
printf("%s","Failed to create socket, exiting\n");
}else{
printf("%s", "Success\n");
}
bzero(&serv_addr, sizeof(serv_addr));
serv_addr.sin_family = AF_INET;
serv_addr.sin_addr.s_addr = htonl(INADDR_ANY);
serv_addr.sin_port = htons(5000);
bind(socket_name, (struct sockaddr*)&serv_addr, sizeof(serv_addr));
listen(socket_name, 10);
while (1)
{
new_conn = accept(socket_name, (struct sockaddr*)NULL, NULL);
read(new_conn, &msg_recv, sizeof(msg_recv));
printf("New: %s\n", msg_recv);
close(new_conn);
}
return 0;
}
I am testing with nc command, sending a message there, without a client implementation.
I'm pretty sure you guys aren't experiencing any errors, the adding of \n does nothing more than flush your printing buffer and make it, in fact, a visible indication within your stdout or a terminal for this matter.
In a simple program where I'm trying to send command-line inputs from client to server, I keep getting a "Broken Pipe" for the server side. I send a string to the server and the server returns the string as lower-case to the client.
Server:
#include <sys/types.h>
#include <sys/socket.h>
#include <netdb.h>
#include <stdio.h>
#include<string.h>
#include <ctype.h>
#include <unistd.h>
int main()
{
char str[100];
int listen_fd, comm_fd;
struct sockaddr_in servaddr;
listen_fd = socket(AF_INET, SOCK_STREAM, 0);
bzero( &servaddr, sizeof(servaddr));
servaddr.sin_family = AF_INET;
servaddr.sin_addr.s_addr = htons(INADDR_ANY);
servaddr.sin_port = htons(37892);
bind(listen_fd, (struct sockaddr *) &servaddr, sizeof(servaddr));
listen(listen_fd, 10);
comm_fd = accept(listen_fd, (struct sockaddr*) NULL, NULL);
while(1){
bzero( str, 100);
read(comm_fd,str,100);
for(int i = 0; i < strlen(str); i++){
str[i] = tolower(str[i]);
}
printf("Echoing back - %s",str);
write(comm_fd, str, strlen(str)+1);
}
}
Client
#include <sys/types.h>
#include <sys/socket.h>
#include <netdb.h>
#include <stdio.h>
#include<string.h>
#include<ctype.h>
#include <unistd.h>
int main(int argc,char **argv)
{
int sockfd,n;
char sendline[100];
char recvline[100];
struct sockaddr_in servaddr;
sockfd=socket(AF_INET,SOCK_STREAM,0);
bzero(&servaddr,sizeof servaddr);
servaddr.sin_family=AF_INET;
servaddr.sin_port=htons(37892);
inet_pton(AF_INET,"127.0.0.1",&(servaddr.sin_addr));
connect(sockfd,(struct sockaddr *)&servaddr,sizeof(servaddr));
if(argc==1) printf("\nNo arguments");
if (1){
{
bzero( sendline, 100);
bzero( recvline, 100);
strcpy(sendline, argv[1]);
write(sockfd,sendline,strlen(sendline)+1);
read(sockfd,recvline,100);
printf("%s",recvline);
}
}
}
The problem I found was that when the client's side is done sending the string, the command line input does not work like fgets() where the loop will wait for another user input. If I change the if(1) in the client's side to a while(1), it will obviously run an infinite loop as no new inputs are being added.
The dilemma is, how would I be able to keep the server's side running to continuously return the string to the client while processing single requests from the command line on the client's side?
Your program has two problems:
1) read() works differently than you think:
Normally read() will read up to a certain number of bytes from some file or stream (e.g. socket).
Because read() does not distinguish between different types of bytes (e.g. letters, the end-of-line marker or even the NUL byte) read() will not work like fgets() (reading line-wise).
read() is also allowed to "split" the data: If you do a write(..."Hello\n"...) on the client the server may receive "Hel" the first time you call read() and the next time it receives "lo\n".
And of course read() can concatenate data: Call write(..."Hello\n"...) and write(..."World\n"...) on the client and one single read() call may receive "Hello\nWorld\n".
And of course both effects may appear at the same time and you have to call read() three times receiving "Hel", "lo\nWo" and "rld\n".
TTYs (= the console (keyboard) and serial ports) have a special feature (which may be switched off) that makes the read() call behave like fgets(). However only TTYs have such a feature!
In the case of sockets read() will always wait for at least one byte to be received and return the (positive) number of bytes received as long as the connection is alive. As soon as read() returns zero or a negative value the connection has been dropped.
You have to use a while loop that processes data until the connection has been dropped.
You'll have to check the data received by read() if it contains the NUL byte to detect the "end" of the data - if "your" data is terminated by a NUL byte.
2) As soon as the client drops the connection the handle returned by accept() is useless.
You should close that handle to save memory and file descriptors (there is a limit on how many file descriptors you can have open at one time).
Then you have to call accept() again to wait for the client to establish a new connection.
Your client sends one request and reads one response.
It then exits without closing the socket.
Your server runs in a loop reading requests and sending responses.
Your server ignores end of stream.
Little or none of this code is error-checked.
Consider the basic client and server programs below (just bare bones / to illustrate my question). The client initiates a connection with the server, prompts the user to enter a message, which is then sent to the server and printed to screen.
If I abruptly quit the client program in the middle of the loop (e.g. by closing the terminal window), sometimes the client will continue to iterate through the loop for a period of time (i.e. the last message sent to the server / currently residing in the write buffer at the time the client is closed, is repeatedly sent to the server, typically until the loop is exhausted). Other times however, the read() call on the server correctly returns 0, and the connection is closed without issue (the behavior seems to be fairly random).
I don't quite understand what's going on here. First off, why do additional loop iterations occur after the program closes? Is there just a lag time between when the terminal window is closed, and when the actual process itself ends? Even if additional loop iterations do occur, shouldn't the call to fgets() block until a message is entered by the user?
I'm using Fedora 25 Workstation with XFCE desktop.
I tried searching for info on this, but didn't have much luck (I'm not sure how to search for this in a succinct way). Any help is much appreciated.
Thanks
CLIENT:
#include <arpa/inet.h>
#include <netinet/in.h>
#include <stdio.h>
#include <sys/socket.h>
#include <unistd.h>
int main(void) {
int sockfd = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
struct sockaddr_in server;
server.sin_family = AF_INET;
server.sin_port = htons(3000);
inet_pton(AF_INET, "127.0.0.1", &server.sin_addr);
connect(sockfd, (struct sockaddr *)&server, sizeof(server));
int i;
for (i = 0; i < 20; i++) {
char buf[512];
printf("Send a message: ");
fgets(buf, 512, stdin);
write(sockfd, buf, sizeof(buf));
}
close(sockfd);
}
SERVER:
#include <arpa/inet.h>
#include <netinet/in.h>
#include <stdio.h>
#include <sys/socket.h>
#include <unistd.h>
int main(void) {
int listenfd = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
struct sockaddr_in server;
server.sin_family = AF_INET;
server.sin_port = htons(3000);
inet_pton(AF_INET, "127.0.0.1", &server.sin_addr);
bind(listenfd, (struct sockaddr *)&server, sizeof(server));
listen(listenfd, 10);
printf("Listening...\n");
struct sockaddr_in client;
socklen_t client_size = sizeof(client);
int clientfd = accept(listenfd, (struct sockaddr *)&client, &client_size);
for (;;) {
char buf[512];
int i = read(clientfd, buf, sizeof(buf));
if (i == 0) {
close(clientfd);
printf("Connection Closed\n");
break;
} else {
printf("%s", buf);
}
}
close(listenfd);
}
When your terminal (and thus the remote/master side of the pty device connected to your process's stdin/out/err) is closed, fgets will see end-of-file status on stdin, and will return immediately with either an incomplete line (not ending in \n) or no input at all (null return value); in practice it's going to be the latter. If you checked the result, you would see this and be able to act accordingly.
in the server, this line:
printf("%s", buf);
should be replaced with:
*buf[i] = '\n';
printf( "%s", buf );
so there is a valid string to print (read() will not terminate the string)
Note: if a I/O error occurs or a signal occurs, etc then read() will return a value less than 0 and should result in exiting the for(;;;) loop, not continuing in the loop, printing the 'old' contents of buf[]
in this line in the client:
write(sockfd, buf, sizeof(buf));
if the write fails, it will return a value less than 0 if/when such an event occurs, the loop should exit, not continue looping,
It is very important to check all error conditions. such error checking (and the resulting handling of the error) can easily double the size of the code, but it must be done; otherwise such 'odd' events as you are seeing will happen, with no simple explanation of what happened.
When a system function returns an error indication, use the function perror() to have some text you provide displayed on stderr, along with the message from the system as to why it thinks the error occurred.
If I abruptly quit the client program in the middle of the loop (e.g. by closing the terminal window),
Closing the terminal window does not quit the client program -- it continues running, just with no input (so any reads from the now-closed terminal will return EOF). However, you never check the return value of fgets in the client so you you never notice, you just keep looping, sending the same buffer.
In addition, the code:
fgets(buf, 512, stdin);
write(sockfd, buf, sizeof(buf));
reads a line of up to 511 chars from the input, but then sends the entire 512 byte buffer, regardless of how long the actual message is. What you want is something more like:
if (fgets(buf, 512, stdin))
write(sockfd, buf, strlen(buf));
else
break; // there was an error or EOF on reading from stdin
Of course, this still has issues with lines longer than 511 bytes and then there's the issue that TCP does not preserve message boundaries, so on the server you might get more than one or only part of a message in a single read call.
I'm currently writing a server application on Linux x86_64 using <sys/socket.h>.
After accepting a connection via accept(), I use fdopen() to wrap the retrieved socket into a FILE* stream.
Writing to, and reading from, that FILE* stream usually works quite well, but the socket becomes unsusable as soon as I write to it while it has a non-empty read buffer.
For demonstration purposes, I've written some code that listens for a connection, then reads the input, line by line, into a read buffer using fgetc(). If the line is too long to fit into the buffer, it's not completely read, but instead read during the next iteration.
#include <unistd.h>
#include <stdio.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
FILE* listen_on_port(unsigned short port) {
int sock = socket(AF_INET, SOCK_STREAM, 0);
struct sockaddr_in name;
name.sin_family = AF_INET;
name.sin_port = htons(port);
name.sin_addr.s_addr = htonl(INADDR_ANY);
if(bind(sock, (struct sockaddr*) &name, sizeof(name)) < 0)
perror("bind failed");
listen(sock, 5);
int newsock = accept(sock, 0, 0);
return fdopen(newsock, "r+");
}
int main(int argc, char** argv) {
int bufsize = 8;
char buf[9];
buf[8] = 0; //ensure null termination
int data;
int size;
//listen on the port specified in argv[1]
FILE* sock = listen_on_port(atoi(argv[1]));
puts("New connection incoming");
while(1) {
//read a single line
for(size = 0; size < bufsize; size++) {
data = fgetc(sock);
if(data == EOF)
break;
if(data == '\n') {
buf[size] = 0;
break;
}
buf[size] = (char) data;
}
//check if the read failed due to an EOF
if(data == EOF) {
perror("EOF: Connection reset by peer");
break;
} else {
printf("Input line: '%s'\n", buf);
}
//try to write ack
if(fputs("ack\n", sock) == EOF)
perror("sending 'ack' failed");
//try to flush
if(fflush(sock) == EOF)
perror("fflush failed");
}
puts("Connection closed");
}
The code should compile in gcc without any special parameters. Run it with the port number as argument and use netcat to connect to it locally.
Now, if you try sending strings that are shorter than 8 characters, this will run flawlessly.
But if you send a string containing more than 10 characters, the program will fail.
This sample input:
ab
cd
abcdefghij
Will create this output:
New connection incoming
Input line: 'ab'
Input line: 'cd'
Input line: 'abcdefgh'
fflush failed: Illegal seek
EOF: Connection reset by peer: Illegal seek
Connection closed
As you see, (rightly) only the first 8 characters of abcdefgh are read, but when the program tries to send the 'ack' string (which the client never receves), and then flush the output buffer, we receive an Illegal seek error, and the next call to fgetc() returns EOF.
If the fflush() part is commented out, the same error still occurs, but the
fflush failed: Illegal seek
line is missing from the server output.
If the fputs(ack) part is commented out, everything seems to work as intended, but a perror() manually called from gdb still reports an 'Illegal seek' error.
If both fputs(ack) and fflush() are commented out, everything does work as intended.
Unfortunately, I've not been able to find any good documentation, nor any Internet discussions on this problem, so your help would be greatly appreciated.
edit
The solution i finally settled for is to not use fdopen() and FILE*, since there seems to be no clean way of converting a socket fd into a FILE* that can reliably used in r+ mode.
Instead, I directly worked on the socket fd, writing my own replacement code for fputs and fprintf.
If anyone needs it, here is the code.
Clearly "r+" (read/write) mode does not work on sockets in this implementation, no doubt because the underlying code assumes that it must do a seek to switch between reading and writing. This is the general case with stdio streams (that you must do some kind of synchronizing operation), because back in the Dim Time, actual stdio implementations had only a single counter per stream, and it was either a counter of "number of characters left to read from stream buffer via getc macro" (in read mode) or "number of characters that can safely be written to stream buffer via putc macro (in write mode). To that that single counter re-set, you had to do a seek-type operation.
Seeks are not allowed on pipes and sockets (since "file offset" is not meaningful there).
One solution is not to wrap a socket with stdio at all. Another, probably easier / better for your purposes, is to wrap it with, not one, but two stdio streams:
FILE *in = fdopen(newsock, "r");
FILE *out = fdopen(newsock, "w");
There's another flaw here though, because when you go to fclose one stream, that closes the other's file descriptor. To work around that, you need to dup the socket descriptor once (in either of the two calls above, it does not matter which one).
If you intend to use select or poll or similar on the socket at some point, you should generally go for the "don't wrap with stdio" solution, since there's no nice clean portable way to track stdio buffering. (There are implementation-specific ways).
Don't use fflush() on network sockets. They are unbuffered streams.
Also, this code:
//read a single line
for(size = 0; size < bufsize; size++) {
data = fgetc(sock);
if(data == EOF)
break;
if(data == '\n') {
buf[size] = 0;
break;
}
buf[size] = (char) data;
}
does not read a single line. It only reads up to the buffer size, which you defined as 8. sock will still have data for you to receive which you should receive before writing to the stream with fputs. BTW you can replace that whole block with
fgets(buf, bufsize, sock);
Yes, you can use one file stream to handle your socket, at least on Linux.
But you should be careful with it: you must only use ferror() to test for errors. I have some code that use this and run flawlessly in production on a major French site.
If you use errno or perror() you'll catch any internal error that the stream will encounter, even if it wants to hide it to you. And "Illegal seek" is one of them.
Also, to test for real EOF conditions, you should use feof(), since when returning true it's mutually exclusive with ferror() returning a non-zero value. It's because, when using fgetc() you don't have any mean to differentiate error from real EOF conditions. So you should probably better use fgets() as another user pointed out.
So, your test:
if(data == EOF) {
perror("EOF: Connection reset by peer");
break;
} else {
printf("Input line: '%s'\n", buf);
}
Should be written as:
int sock_error = ferror(sock);
if (sock_error) {
fprintf(stderr, "Error while reading: %s", strerror(sock_error));
} else {
printf("Input line: '%s'\n", buf);
}
Try this :
#define BUFSIZE 88
FILE* listen_on_port(unsigned short port) {
...
}
int main(int argc, char** argv) {
int bufsize = BUFSIZE;
char buf[ BUFSIZE ];