I had a function that would receive data from an IRC server in 512-byte chunks and print it to the terminal window, it went like this:
int mainLoop(redchan_t *redchan)
{
int socketDescriptor = redchan->socketDescriptor, i, bytesReceived;
char workingBuffer[RECVBUFF] = {[0 ... RECVBUFF - 2] = '0', '\0'};
puts("Recieving data...");
do
{
if ((bytesReceived = recv(
socketDescriptor,
workingBuffer,
RECVBUFF,
0)) == 0)
exit(EXIT_FAILURE);
for (i = 0; i < bytesReceived; ++i)
printf("%c", workingBuffer[i]);
}
while(1);
return 0;
}
But I wanted to make it more orthogonal so I took out the call to recv and put it in its own routine which looks like this:
int redchanRecv(redchan_t *redchan, int size, char *buffer)
{
int totalBytes = 0, bytesReceived;
while (totalBytes < size)
{
if ((bytesReceived = recv(
redchan->socketDescriptor,
buffer + totalBytes,
size - totalBytes,
0)) <= 0)
cleanup_redchan(redchan, serverInfo);
totalBytes += bytesReceived;
}
return 0;
}
I would then just call redchanRecv() in each iteration of my main loop and then print the buffer.
But when I run it, it prints out almost everything, except for one line.
That last line never makes it to the terminal (does it even make it into the buffer?).
I really feel like I'm making a rookie mistake but hell if I can see it.
How can I solve this?
stdout is line-buffered by default in your system, add a fflush(stdout) call to have your last line printed.
Related
So, I'm writing this simple HTTP client in C and I seem to be stuck on this problem - how do I strip the HTTP headers from the response? After all, if I get a binary file I can't just write the headers out to my output file. I can't seem to go in once the data is already written to a file because linux screams when you try to even view the first few lines of a binary file, even if you know they're just text HTTP headers.
Now, here's the rub (well, I suppose the whole thing is a rub). Sometimes the whole header doesn't even in come in on the first response packet, so I can't even guarantee that we'll have the whole header in our first iteration (that is, iteration of receiving an HTTP response. We're using recv(), here), which means I need to somehow... well, I don't even know. I can't seem to mess with the data once it's already written to disk, so I need to deal with it as it's coming in, but we can't be sure how it's going to come in, and even if we were sure, strtok() is a nightmare to use.
I guess I'm just hoping someone out there has a better idea. Here's the relevant code. This is really stripped down, I'm going for MCVE, of course. Also, you can just assume that socket_file_descriptor is already instantiated and get_request contains the text of our GET request. Here is it:
FILE* fp = fopen("output", "wb"); // Open the file for writing
char buf[MAXDATASIZE]; // The buffer
size_t numbytes; // For the size of the response
/*
* Do all the socket programming stuff to get the socket file descriptor that we need
* ...
* ...
*/
send(socket_file_descriptor, get_request, strlen(get_request), 0); // Send the HTTP GET request
while ((numbytes = recv(socket_file_descriptor, buf, MAXDATASIZE - 1, 0)) > 0) {
/* I either need to do something here, to deal with getting rid of the headers before writing to file */
fwrite(buf, 1, numbytes, fp); // Write to file
memset(buf, 0, MAXDATASIZE); // This just resets the buffer to make room for the next packet
}
close(s);
fclose(fp);
/* Or I need to do something here, to strip the file of its headers after it's been written to disk */
So, I thought about doing something like this. The only thing we know for sure is that the header is going to end in \r\n\r\n (two carriage returns). So we can use that. This doesn't really work, but hopefully you can figure out where I'm trying to go with it (comments from above removed):
FILE* fp = fopen("output", "wb");
char buf[MAXDATASIZE];
size_t numbytes;
int header_found = 0; // Add a flag, here
/* ...
* ...
*/
send(socket_file_descriptor, get_request, strlen(get_request), 0);
while ((numbytes = recv(socket_file_descriptor, buf, MAXDATASIZE - 1, 0)) > 0) {
if (header_found == 1) { // So this won't happen our first pass through
fwrite(buf, 1, numbytes, fp);
memset(buf, 0, MAXDATASIZE);
}
else { // This will happen our first pass through, maybe our second or third, the header doesn't always come in in full on the first packet
/* And this is where I'm stuck.
* I'm thinking about using strtok() to parse through the lines, but....
* well I just can't figure it out. I'm hoping someone can at least point
* me in the right direction.
*
* The point here would be to somehow determine when we've seen two carriage returns
* in a row and then mark header_found as 1. But even if we DID manage to find the
* two carriage returns, we still need to write the remaining data from this packet to
* the file before moving on to the next iteration, but WITHOUT including the
* header information.
*/
}
}
close(s);
fclose(fp);
I've been staring at this code for three days straight and am slowly losing my mind, so I really appreciate any insight anyone is able to provide. To generalize the problem, I guess this really comes down to me just not understanding how to do text parsing in C.
The second self-answer is better than the first one, but it still could be made much simpler:
const char* pattern = "\r\n\r\n";
const char* patp = pattern;
while ((numbytes = recv(socket_file_descriptor, buf, MAXDATASIZE - 1, 0)) > 0) {
for (int i = 0; i < numbytes; i++) {
if (*patp == 0) {
fwrite(buf + i, 1, numbytes - i, fp);
break;
}
else if (buf[i] == *patp) ++patp;
else patp = pattern;
}
/* This memset isn't really necessary */
memset(buf, 0, MAXDATASIZE);
}
That looks like a general solution, but it's not really: there are values for pattern for which it might fail to see a terminator under particular circumstances. But this particular pattern is not problematic. You might want to think about what sort of pattern would cause a problem before taking a look at the more general solution.
So, I know this is not the most elegant way to go about this, but... I did get it. For anyone who finds this question and is curious about at least an answer, here it is:
int count = 0;
int firstr_found = 0;
int firstn_found = 0;
int secondr_found = 0;
int secondn_found = 0;
FILE* fp = fopen("output", "wb");
char buf[MAXDATASIZE];
size_t numbytes;
int header_found = 0;
/* ...
* ...
*/
send(socket_file_descriptor, get_request, strlen(get_request), 0);
while ((numbytes = recv(socket_file_descriptor, buf, MAXDATASIZE - 1, 0)) > 0) {
if (header_found == 1) {
fwrite(buf, 1, numbytes, fp);
}
else {
// These buf[i]'s are going to return as integers (ASCII)
// \r is 13 and \n is 10, so we're looking for 13 10 13 10
// This also needs to be agnostic of which packet # we're on; sometimes the header is split up.
for (int i = 0; i < numbytes; i++) {
if (firstr_found == 1 && firstn_found == 1 && secondr_found == 1 && secondn_found == 1) { // WE FOUND IT!
header_found = 1;
// We want to skip the parts of the buffer we've already looked at, that's header, and our numbytes will be decreased by that many
fwrite(buf + i, 1, numbytes - i, fp);
break;
}
if (buf[i] == 13 && firstr_found == 0) { // We found our first \r, mark it and move on to next iteration
firstr_found = 1;
continue;
}
if (buf[i] == 10 && firstr_found == 1 && firstn_found == 0) { // We found our first \n, mark it and move on
firstn_found = 1;
continue;
}
else if (buf[i] != 13 && buf[i] != 10) { // Think about the second r, it'll ignore the first if, but fail on the second if, but we don't want to jump into this else block
firstr_found = 0;
firstn_found = 0;
continue;
}
if (buf[i] == 13 && firstr_found == 1 && firstn_found == 1 && secondr_found == 0) {
secondr_found = 1;
continue;
}
else if (buf[i] != 10) {
firstr_found = 0;
firstn_found = 0;
secondr_found = 0;
continue;
}
if(buf[i] == 10 && firstr_found == 1 && firstn_found == 1 && secondr_found == 1 && secondn_found == 0) {
secondn_found = 1;
continue;
}
}
}
memset(buf, 0, MAXDATASIZE);
count++;
}
close(s);
fclose(fp);
Adding another answer because, well I suppose I think I'm clever. Thanks to #tadman for the idea of a counter. Look here (I'm going to shave off a lot of the bloat and just do the while loop, if you've looked at my other code blocks you should be able to see what I mean here) ...
/* ...
* ...
*/
int consec_success = 0;
while ((numbytes = recv(socket_file_descriptor, buf, MAXDATASIZE - 1, 0)) > 0) {
if (header_found == 1) {
fwrite(buf, 1, numbytes, fp);
}
else {
for (int i = 0; i < numbytes; i++) {
if (consec_success == 4) {
header_found = 1;
fwrite(buf + i, 1, numbytes - i, fp);
break;
}
if (buf[i] == 13 && consec_success % 2 == 0) {
consec_success++;
}
else if (buf[i] == 10 && consec_success % 2 == 1) {
consec_success++;
}
else {
consec_success = 0;
}
}
}
memset(buf, 0, MAXDATASIZE);
}
/* ...
* ...
*/
I am practicing the read and write system call, the below code is working fine with a while loop and also without them. could you please tell me what is the use of while loop here, is it necessary to add it while using read and write system calls. I am a beginner. Thanks.
#include <unistd.h>
#define BUF_SIZE 256
int main(int argc, char *argv[])
{
char buf[BUF_SIZE];
ssize_t rlen;
int i;
char from;
char to;
from = 'e';
to = 'a';
while (1) {
rlen = read(0, buf, sizeof(buf));
if (rlen == 0)
return 0;
for (i = 0; i < rlen; i++) {
if (buf[i] == from)
buf[i] = to;
}
write(1, buf, rlen);
}
return 0;
}
You usually need to use while loops (or some kind of loop in general) with read and write, because, as you should know from the manual page (man 2 read):
RETURN VALUE
On success, the number of bytes read is returned (zero indicates end
of file), and the file position is advanced by this number. It is
not an error if this number is smaller than the number of bytes
requested; this may happen for example because fewer bytes are
actually available right now (maybe because we were close to end-of-
file, or because we are reading from a pipe, or from a terminal), or
because read() was interrupted by a signal. See also NOTES.
Therefore, if you ever want to read more than 1 byte, you need to do this in a loop, because read can always process less than the requested amount.
Similarly, write can also process less than the requested size (see man 2 write):
RETURN VALUE
On success, the number of bytes written is returned (zero indicates nothing was written). It is not an error if this
number is smaller than the number of bytes requested; this may happen for example because the disk device was filled.
See also NOTES.
On error, -1 is returned, and errno is set appropriately.
The only difference here is that when write returns 0 it's not an error or an end of file indicator, you should just retry writing.
Your code is almost correct, in that it uses a loop to keep reading until there are no more bytes left to read (when read returns 0), but there are two problems:
You should check for errors after read (rlen < 0).
When you use write you should also add a loop there too, because as I just said, even write could process less than the requested amount of bytes.
A correct version of your code would be:
#include <stdio.h>
#include <unistd.h>
#define BUF_SIZE 256
int main(int argc, char *argv[])
{
char buf[BUF_SIZE];
ssize_t rlen, wlen, written;
char from, to;
int i;
from = 'e';
to = 'a';
while (1) {
rlen = read(0, buf, sizeof(buf));
if (rlen < 0) {
perror("read failed");
return 1;
} else if (rlen == 0) {
return 0;
}
for (i = 0; i < rlen; i++) {
if (buf[i] == from)
buf[i] = to;
}
for (written = 0; written < rlen; written += wlen) {
wlen = write(1, buf + written, rlen - written);
if (wlen < 0) {
perror("write failed");
return 1;
}
}
}
return 0;
}
I am writing an application to read data from /dev/ttyUSB0.
I found it necessary to call sleep before calling read in the while loop so that I get the entire line at once. Otherwise, sometimes I get part of the line and the rest in the next iteration.
Do I have to package my data with a header containing the length of the string being sent over? Or is there a better way?
while(1) {
usleep(10000);
unsigned char buf[80];
int rdlen;
ioctl(fd, FIONREAD, &rdlen);
if (rdlen > 0) {
rdlen = read(fd, buf, rdlen);
}
if (rdlen > 0) {
...
}
The better way is to simply deal with receiving partial lines, have your code reading the data figure out when you have a complete line if that is important.
something like (not tested at all):
char buffer[1000];
size_t offset = 0;
while(1 ){
int len = read(fd, buffer+offset,(unsigned)(sizeof(buffer)-offset));
if(!strchr(buffer+offset, '\n')){
/* no end-of-line */
offset +=len;
}
else
{
/* deal with complete line */
offset = 0;
}
}
Creating a tcp client <-> server program in c for my home exam. Been running into some problems with reciev and sending data between the server and client. I'm only able to receive one byte, feks if i send "abcd" i receive "a".
(This is happening both ways since i'm using same methods for server and client)
Dont know if it is the sending part thats the problem or receiving
This is my code:
#define BUFFER_SIZE 1024
char* recived_message;
int send_data(int socket, char* data){
int offset = 0, len = strlen(data);
while (offset != len) {
int nb = send(socket, data + offset, len - offset, 0);
if (nb < 0){
perror("send");
return -1;
}
offset += nb;
}
return 0;
}
int recive(int socket){
int offset = 0;
recived_message = malloc(BUFFER_SIZE);
memset(recived_message, 0, BUFFER_SIZE);
while (offset != BUFFER_SIZE) {
int nb = recv(socket, received_message + offset, BUFFER_SIZE - offset, 0);
if(nb == -1){
perror("read");
return -1;
}else if(nb == 0){
return -1;
}
offset += nb;
}
printf("%d\n", offset);
return 0;
}
char* get_data(){
return recived_message;
}
Server side
int recive_data(int socket){
char* buffer;
if(recive(socket) != 0){
return -1;
}
*buffer = *get_data();
printf("socket %d: %s\nlength: %lu%", fd, buffer, strlen(buffer));
return 0;
}
Part of client
char* test = "abcd";
for(i=0; i<10; i++) {
send_data(sock, test);
sleep(1);
}
The problem is here
*buffer = *get_data();
You dereferences the pointer returned by get_data() to get only the first element pointer to by the pointer.
And it's worse than that, because you dereference the uninitialized variable buffer to write that single character. This will lead to undefined behavior. Also, the later functions calls using this uninitialized variable also leads to undefined behavior.
The simple solution to (almost) all your problems: Assign to the actual variable:
buffer = get_data();
And I say that the above solves almost all your problems, because what if the terminating zero of the string isn't transmitted? That will also lead to UB (Undefined Behavior). If the data you receive is always a string, then you should make sure it's terminated, preferably in the receive function:
int recive(int socket){
...
received_message[offset] = '\0';
return 0;
}
I'm programming in C an IRC chat client. everything it's working well except I can't read the whole answer sent by the server. here's the code:
char buffer[2048];
write_on_screen(current_page(), "LOG COMMAND", command);
write(sockfd, command, strlen(command)); //write to socket
bzero(buffer, sizeof(buffer));
read(sockfd, buffer, sizeof(buffer));
write_on_screen(current_page(), "RESPONSE", buffer);
return buffer;
most of the time buffer will contain just a piece of the response (which is shorter than 2048 bytes) and other times it contains nothing. in both cases if I do another read() after the first one, it returns me the rest of the answer or another small piece (and then I've to do another read() again). if I put a sleep(1) between write() and read() I get the whole answer, but I'm sure this not a good pratice.
Is there some way I can avoid this?
thank you in advance
You're making the usual mistakes. It is impossible to write correct network code without storing the result of read() or recv() into a variable. You have to:
Check it for -1, and if so look at errno to see whether was fatal, which it almost always is except for EAGAIN/EWOULDBLOCK, and if fatal close the socket and abandon the process.
Check it for zero, which means the peer disconnected. Again you must close the socket and abandon the process.
Use it as the count of bytes actually received. These functions are not obliged nor guaranteed to fill the buffer. Their contract in blocking mode is that they block until an error, end of stream, or at least one byte is transferred. If you're expecting more than one byte, you normally have to loop until you get it.
According to RFC-1459, a single line of text in IRC can contain up to 512 characters and is terminated by a CRLF (\r\n) pair. However:
You're not guaranteed to receive exactly 512 bytes each time. For example, you might receive a comparatively short message from someone else one in the channel: Hi!
Related to the above: A group of 512 bytes might represent more than one message. For example, the buffer might contain a whole line, plus part of the next line: PRIVMSG <msgtarget> <message>\r\nPRIVMS
Given that you could have zero-or-more complete lines plus zero-or-one incomplete lines in your buffer[] at any time, you could try doing something along the lines of:
char buffer[2048];
while(keep_going)
{
char **lines;
int i, num_lines;
// Receive data from the internet.
receiveData(buffer);
// Create an array of all COMPLETE lines in the buffer (split on \r\n).
lines = getCompleteLines(buffer, &num_lines);
removeCompleteLinesFromBuffer(buffer);
// Handle each COMPLETE line in the array.
for (i = 0; i < num_lines; ++i) { handle_line(lines[i]); }
freeLines(lines);
}
This would allow you to handle zero or more complete lines in one go, with any incomplete line (i.e anything after the final \r\n pair) being kept around until the next call to receiveData().
You need to loop around read() until a CRLF had been detected.
A possible way to do this would be:
#include <stdio.h>
#include <unistd.h>
#include <errno.h>
ssize_t read_until_crlf(int sd, char * p, size_t s, int break_on_interupt)
{
ssize_t bytes_read = 0;
ssize_t result = 0;
int read_cr = 0;
int read_crlf = 0;
while (bytes_read < s)
{
result = read(sd, p + bytes_read, 1);
if (-1 == result)
{
if ((EAGAIN == errno) || (EWOULDBLOCK == errno))
{
continue;
}
else if (EINTR == errno)
{
if (break_on_interupt)
{
break;
}
continue;
}
else
{
perror("read() failed");
break;
}
}
else if (0 == result)
{
break; /* peer disconnected */
}
if ('\r' == p[bytes_read])
{
read_cr = 1;
}
else if (('\n' == p[bytes_read]) && read_cr)
{
read_crlf = 1;
break; /* CRLF detected */
}
else
{
read_cr = 0;
}
++bytes_read;
}
if (!read_crlf)
{
result = -1; /* Buffer full without having read a CRLF. */
errno = ENOSPC; /* ... or whatever might suite. */
}
return (0 >= result) ?result :bytes_read;
}
Call it like this:
#include <stdio.h>
ssize_t read_until_crlf(int sd, char * p, size_t s, int break_on_interupt);
int main(void)
{
int sd = -1;
/* init sd here */
{
char line[2048] = "";
ssize_t result = read_until_crlf(sd, line, sizeof line, 0);
if (-1 == result)
{
perror("read_until_newline() failed");
}
printf("read '%s'\n", line);
}
return 0;
}