I'm working on writing a IRC bot in C, and have ran into a snag.
In my main function, I create my socket and connect, all that happy stuff. Then I have a (almost) infinite loop to read what's being sent back from the server. I then pass what's read off to a helper function, processLine(char *line) - the problem is, that the following code reads until my buffer is full - I want it to only read text until a newline (\n) or carriage return (\r) occurs (thus ending that line)
while (buffer[0] && buffer[1]) {
for (i=0;i<BUFSIZE;i++) buffer[i]='\0';
if (recv(sock, buffer, BUFSIZE, 0) == SOCKET_ERROR)
processError();
processLine(buffer);
}
What ends up happening is that many lines get jammed all together, and I can't process the lines properly when that happens.
If you're not familiar with IRC protocols, a brief summary would be that when a message is sent, it often looks like this: :YourNickName!YourIdent#YourHostName PRIVMSG #someChannel :The rest on from here is the message sent...
and a login notice, for instance, is something like this: :the.hostname.of.the.server ### bla some text bla with ### being a code(?) used for processing - i.e. 372 is an indicator that the following text is part of the Message Of The Day.
When it's all jammed together, I can't read what number is for what line because I can't find where a line begins or ends!
I'd appreciate help with this very much!
P.S.: This is being compiled/ran on linux, but I eventually want to port it to windows, so I am making as much of it as I can multi-platform.
P.S.S.: Here's my processLine() code:
void processLine(const char *line) {
char *buffer, *words[MAX_WORDS], *aPtr;
char response[100];
int count = 0, i;
buffer = strdup(line);
printf("BLA %s", line);
while((aPtr = strsep(&buffer, " ")) && count < MAX_WORDS)
words[count++] = aPtr;
printf("DEBUG %s\n", words[1]);
if (strcmp(words[0], "PING") == 0) {
strcpy(response, "PONG ");
strcat(response, words[1]);
sendLine(NULL, response); /* This is a custom function, basically it's a send ALL function */
} else if (strcmp(words[1], "376") == 0) { /* We got logged in, send login responses (i.e. channel joins) */
sendLine(NULL, "JOIN #cbot");
}
}
The usual way to deal with this is to recv into a persistent buffer in your application, then pull a single line out and process it. Later you can process the remaining lines in the buffer before calling recv again. Keep in mind that the last line in the buffer may only be partially received; you have to deal with this case by re-entering recv to finish the line.
Here's an example (totally untested! also looks for a \n, not \r\n):
#define BUFFER_SIZE 1024
char inbuf[BUFFER_SIZE];
size_t inbuf_used = 0;
/* Final \n is replaced with \0 before calling process_line */
void process_line(char *lineptr);
void input_pump(int fd) {
size_t inbuf_remain = sizeof(inbuf) - inbuf_used;
if (inbuf_remain == 0) {
fprintf(stderr, "Line exceeded buffer length!\n");
abort();
}
ssize_t rv = recv(fd, (void*)&inbuf[inbuf_used], inbuf_remain, MSG_DONTWAIT);
if (rv == 0) {
fprintf(stderr, "Connection closed.\n");
abort();
}
if (rv < 0 && errno == EAGAIN) {
/* no data for now, call back when the socket is readable */
return;
}
if (rv < 0) {
perror("Connection error");
abort();
}
inbuf_used += rv;
/* Scan for newlines in the line buffer; we're careful here to deal with embedded \0s
* an evil server may send, as well as only processing lines that are complete.
*/
char *line_start = inbuf;
char *line_end;
while ( (line_end = (char*)memchr((void*)line_start, '\n', inbuf_used - (line_start - inbuf))))
{
*line_end = 0;
process_line(line_start);
line_start = line_end + 1;
}
/* Shift buffer down so the unprocessed data is at the start */
inbuf_used -= (line_start - inbuf);
memmove(innbuf, line_start, inbuf_used);
}
TCP doesn't offer any sequencing of that sort. As #bdonlan already said you should implement something like:
Continuously recv from the socket into a buffer
On each recv, check if the bytes received contain an \n
If an \n use everything up to that point from the buffer (and clear it)
I don't have a good feeling about this (I read somewhere that you shouldn't mix low-level I/O with stdio I/O) but you might be able to use fdopen.
All you would need to do is
use fdopen(3) to associate your socket with a FILE *
use setvbuf to tell stdio that you want it line-buffered (_IOLBF) as opposed to the default block-buffered.
At this point you should have effectively moved the work from your hands to stdio. Then you could go on using fgets and the like on the FILE *.
Related
I am rewriting a simple reverse-shell program I made in python, now in C.
The program is supposed to be trying to connect to a host(netcat listening on the specified port) and then take the output from nc, sent over a tcp socket, and make a system call using popen() and then send back the terminal output via the socket connection to nc where it is displayed.
When I try sending commands the program returns what I requested, but also some giberish.
Ex.:
/Users/jacob/Library/Developer/Xcode/DerivedData/backdoorfbhufkccmceisqaozrfitkmfsvge/Build/Products/Debu#Ԓ`? ??????0d?r?
(Output from nc in terminal, this was a 'pwd' command)
I also seem to have some kind of problem where a buffer isn't cleared? When I use the 'say' command(say [sentence] ) MacOS is supposed to use voice the sentence. This happens, but only the 2 first letters of the argument after 'say'(the 2 first characters of the sentence) and then says an earlier string. ('Successfully connected!')
Ex.: (command: 'say hello')
heSuccessfully connected!
I have tried to open the FILE stream in different places and
Code (after socket setup and connection):
const char conMsg[25] = "Successfully connected!\n";
send(netSock, conMsg, sizeof(conMsg), 0);
printf("Sent message...\n");
// variable setup
char command[] = "clear";
char buffer[256];
const char INPUTFIELD[3] = "\n> ";
// requests loop
while (1) {
send(netSock, INPUTFIELD, sizeof(INPUTFIELD), 0);
// recv command
recv(netSock, &command, sizeof(command), 0);
printf("recived command...\n");
printf(command);
// exit check
if ( strncmp(command, ":exit", 5) == 0) {
close(netSock);
exit(0);
} else {
// stream init
FILE *in;
in = popen(command, "r");
// popen output, send to host
while ( fgets(buffer, sizeof(buffer), in) != NULL) {
send(netSock, buffer, sizeof(buffer), 0);
}
pclose(in);
}
}
return 0;
How I use the program:
nc -l [specified port] (MAC or Linux terminal command(maybe Windows aswell))
Start binary (should not matter as I plan to have it try to connect, but that functonality is lacking as of now)
// recv command
recv(netSock, &command, sizeof(command), 0);
printf("recived command...\n");
printf(command);
You ignore the return value of recv, so none of your other code has any idea how many bytes of data you received. Also, you pass command to printf. This has two problems:
What if you didn't receive a zero byte? You could run right off the end of the buffer.
What if the received data includes a %s or other string that is special to printf?
I think your main issue is here:
// popen output, send to host
while ( fgets(buffer, sizeof(buffer), in) != NULL) {
send(netSock, buffer, sizeof(buffer), 0);
}
fgets will read just to an end of line (including the end of line character) and then null terminate. Unless a line exceeds the buffer size, it won't fill the buffer completely. Your send call is sending the entire buffer regardless, including any uninitialized gibberish that may be past what fgets read. This might work better:
// popen output, send to host
while ( fgets(buffer, sizeof(buffer), in) != NULL) {
send(netSock, buffer, strlen(buffer), 0);
}
When I write between my server and my client, I make use of the rio functions (specified in the csapp.c), more specifically:
Rio_writen --> when writing to a socket
Rio_readlineb --> when reading from a socket
My problem is that whenever I send information from e.g. my client to my server, example:
Rio_writen(name_server_socket, "null", 4);
Rio_writen(name_server_socket, "\n", 1);
I expect my server when calling:
Rio_readlineb(&rio, name, MAXLINE);
char* lookup = name;
assert(lookup == "null"); --> fail
assert(strcmp(lookup, "null") == 0); --> fail
To assert that the message is equivalent to "null". However, when I use prints to show my messages on either site, this is what I get:
// client side message send
lookup_name: (null)|
// server side message received
lookup_name: (null)
|
I use the | character to see if anything is added after my message is sent (by printf("x: %s|",something)). In this case every message I sent get a newline character at the end, and I do not understand why.
Using the rio function, in particular Rio_getlineb to read inputs: it reads until it reaches the end of the line (provided a newline character). Thus, I'm expected to provide a newline character at the end of any message I sent (hence why i do (Rio_writen(name_server_socket, "\n", 1);) after each message sent).
Can anyone see what I am missing (or in any case know a solution to the problem)? All my setups are working correctly: sockets, rio_t etc. The messages are sent and received, but somehow add a newline at the end.
EDIT: The assertion is just a placeholder for doing something with the data. So adding a newline to the assertion might solve the assert, but not my problem: which is why a newline is added in the first place :-) My point is that I want to parse the "lookup" to a function that compares that value to a name, given in a struct. So I cannot just add a newline to the struct->name that I am looking at.
For what I see in the documentation:
– rio_readlineb reads a text line of up to maxlen bytes from file fd
and stores the line in usrbuf
Especially useful for reading text lines from network sockets
– Stopping conditions
maxlen bytes read
EOF encountered
Newline (‘ \n ’) encountered
That is, the same as fgets does.
And looking at the source code:
/* $begin rio_readlineb */
ssize_t rio_readlineb(rio_t *rp, void *usrbuf, size_t maxlen)
{
int n, rc;
char c, *bufp = usrbuf;
for (n = 1; n < maxlen; n++) {
if ((rc = rio_read(rp, &c, 1)) == 1) {
*bufp++ = c;
if (c == '\n') {
n++;
break;
}
It includes the trailing newline in the buffer.
This newline is very useful, the fact that the message does not include it is an indication that you have not read the whole line (in your case received the entire package), in which case you can continue requesting data.
What we usually do with fgets to skip the trailing newline is:
char *ptr = strchr(lookup, '\n');
if (ptr != NULL)
{
*ptr = '\0'; // Replace newline with NUL
}
I've been given this code for an assignment, there supposed to be errors in it but I can't actually figure out what this function is supposed to do, never mind figure out if there's any issues with it...
I am guessing that it's supposed to read the buffer line by line, but I've never seen it done this way before
The buffer that is sent to the function is empty.
int read_line(int sock, char *buffer) {
size_t length = 0;
while (1) {
char data;
int result = recv(sock, &data, 1, 0);
if ((result <= 0) || (data == EOF)){
perror("Connection closed");
exit(1);
}
buffer[length] = data;
length++;
if (length >= 2 && buffer[length-2] == '\r' && buffer[length-1] == '\n') {
buffer[length-2] = '\0';
return length;
}
}
}
Thanks in advance!
I'd say the purpose of this function is to read a line that ends with \r\n from socket stream and store it in a char array as a string, therefore the \0 (string termination character) placement.
Ok, so what's wrong with the code?
I'd start with the input parameter char *buffer - inside the function you do not know its size so you cannot check if it exceeds its size limit and it could lead to buffer overflow.
So it would be better to send buffer length as a parameter and check with every received byte if it can be stored.
EOF - it is defined as -1 and in this case actually doesn't make any sense, because nothing will be setting your data variable to EOF. The only thing you need to look out for is the end of socket stream (recv documentation). And here is an example for EOFusage.
Feel free to remove (data == EOF) from condition.
Let's say you are receiving everything regularly and you receive your last input and connection closes, so you enter this case:
if ((result <= 0) || (data == EOF)){
perror("Connection closed");
exit(1);
}
The problem here is that you won't process your last line and the program will just end. Although, I might be wrong here since I don't know when the connection is getting regularly shut down.
And a minor note here, result that equals to 0 isn't considered as an error, but a regular connection shutdown (or a 0-byte datagram was received).
I hope I haven't missed anything.
In the code below, I am trying to read from a socket and store the results in a file.
What actually happens, is that my client sends a GET request to my server for a file.html. My server finds the file and writes the contents of it to the socket. Lastly my client reads the content from thread_fd and recreates the file.
For some reason the recreated file has less content than the original. I have located the problem to be some lines in the end, that are missing. When I use printf("%s", buffer) inside the while loop everything seems fine in STDOUT but my fprintf misses somewhat 3.000 bytes for a file of 81.000 bytes size.
#define MAXSIZE 1000
int bytes_read, thread_fd;
char buffer[MAXSIZE];
FILE* new_file;
memset(buffer, 0, MAXSIZE);
if((new_file = fopen(path, "wb+")) == NULL)
{
printf("can not open file \n");
exit(EXIT_FAILURE);
}
while ((bytes_read = read(thread_fd, buffer, MAXSIZE)) > 0)
{
fprintf(new_file, "%s", buffer);
if(bytes_read < MAXSIZE)
break;
memset(buffer, 0, MAXSIZE);
}
You read binary data from the socket that may or may not contain a \0 byte. When you then fprintf that data the fprintf will stop at the first \0 it encounters. In your case that is 3000 bytes short of the full file. If your file contains no \0 byte the fprintf will simply continue printing the ram contents until it segfaults.
Use write() to write the data back to the file and check for errors. Don't forget to close() the file and check that for errors too.
Your code should/could look like:
int readfile(int thread_fd, char *path)
{
unsigned int bytes_read;
char buffer[MAXSIZE];
int new_file;
if ((new_file = open(path, _O_CREAT|_O_BINARY,_S_IWRITE)) == -1) return -1;
while ((bytes_read = read(thread_fd, buffer, MAXSIZE)) > 0)
{
if (write(new_file, buffer, bytes_read)!= bytes_read) {
close(new_file);
return -2;
}
}
close(new_file);
return 0;
}
There are a few issues with your code that can cause this.
The most likely cause is this :
if(bytes_read < MAXSIZE)
break;
This ends the loop when read returns less than the requested amount of bytes. This is however perfectly normal behavior, and can happen eg. when not enough bytes are available at the time of the read call (it's reading from a network socket after all). Just let the loop continue as long as read returns a value > 0 (assuming the socket is a blocking socket - if not, you'll also have to check for EAGAIN and EWOULDBLOCK).
Additionally, if the file you're receiving contains binary data, then it's not a good idea to use fprintf with "%s" to write to the target file. This will stop writing as soon as it finds a '\0' byte (which is not uncommon in binary data). Use fwrite instead.
Even if you're receiving text (suggested by the html file extension), it's still not a good idea to use fprintf with "%s", since the received data won't be '\0' terminated.
This worked!
ps: I don't know if I should be doing this, since I am new here, but really there is no reason for negativity. Any question is a good question. Just answer it if you know it. Do not judge it.
#define MAXSIZE 1000
int bytes_read, thread_fd, new_file;
char buffer[MAXSIZE];
memset(buffer, 0, MAXSIZE);
if((new_file = open(path, O_RDONLY | O_WRONLY | O_CREAT)) < 0)
{
printf("can not open file \n");
exit(EXIT_FAILURE);
}
while ((bytes_read = read(thread_fd, buffer, MAXSIZE)) > 0)
write(new_file, buffer, bytes_read);
close(new_file);
I'm having a hard time trying to figure out why this piece of code doesn't work as it should. I am learning the basics of I/O operations and I have to come up with a C program that writes on a 'log.txt' file what is given from standard input and as the 'stop' word is entered, the program must halt.
So my code is:
#include "main.h"
#define SIZE 1024
int main(int argc, char *argv[])
{
int fd;
int readBytes;
int writBytes;
char *buffer;
if ((fd = open("log.txt", O_WRONLY|O_APPEND)) < 0)
{
perror("open");
}
buffer = (char *) calloc (SIZE, sizeof(char));
while ((readBytes = read(0, buffer, SIZE) < SIZE)&&(strncmp(buffer, "stop", 4) != 0));
if ((writBytes = write(fd, buffer, SIZE)) < 0)
{
perror("write");
}
if ((close(fd)) < 0)
{
perror("close");
}
}
If I enter:
this is just a text
stop
The output is
stop
is just a text
If I enter more than a sentence:
this is just a text
this is more text
and text again
stop
This is what is logged:
stop
ext again
xt
t
And on top of that if I try to edit the log.txt file from vim or just a text editor I can see '\00's. I guess \00 stands for all the bytes left empty from the 1024 available, right? How can I prevent that from happening?
It looks like you're expecting
readBytes = read(0, buffer, SIZE) < SIZE)
to somehow accumulate things in buffer. It doesn't. Every subsequent read will put whatever it read at the start of the buffer, overwriting what the previous read has read.
You need to put your write in the while block - one write for every read, and only write as much as you read, otherwise you'll write garbage (zeros from the calloc and/or leftovers from the previous read) in your log file.
Also note that while your technique will probably work most of the time for a line-buffered input stream, it will not do what you expect if you redirect from a file or a pipe. You should be using formatted input functions (like getline if you your implementation has that, scanf, or fgets).