I am writing a simple FTP server for a school project. Project is almost done, the only problem I do have is sending a file over socket to a client. I can't write to a socket more than 200kb of data, small files are downloaded succesfully. Can anyone tell me what is the correct way of sending large files over Linux sockets ?
Thanks in advance.
PS I am using C and 32-bit linux, server working in PORT mode, been using low level open,write,read and other functions such as sendfile,send,sendto.
you can mmap the file and write it on the socket from there, you can also get its size using fstat, like this:
fd = open(filename, O_RDONLY);
struct stat s;
fstat(fd, &s); // i get the size
adr = mmap(NULL, s.st_size, PROT_READ, MAP_SHARED, fd, 0); // i get the adress
write(socket, adr, s.st_size); // i send the file from this adress directly
You may want to send just the size of the file before you send it entirely. Your client maybe want to send you that he got the good size and that he can manage to download it.
One idea might be reading the file chunk by chunk, something like:
Pseudo-code
#define CHUNK_SIZE 1000
void send(){
uint8_t buff[CHUNK_SIZE];
int actually_read;
while((actually_read = read(fd, buff, sizeof(buff)) > 0)
sendto(sock_fd, buff, actually_read, 0);
}
You should add some error checking, but the idea is to read a considerable amount of bytes from the file you want to send and send that amount. In the server side you need to do a similar thing, by reading from the socket the arriving chunks and writing them to a file. You might want to prefix some metadata to buff just to tell the server which file you're transmitting if you want to handle multiple file transfers. Since FTP uses TCP you shouldn't worry about losing data.
Again, this is just an idea. There are multiple ways of doing this I suppose.
Related
I'm not sure if this is a common way of interfacing with OpenSSL or not, but I can't really find any examples or references to trying to do it this way.
I have some applications that interfact with a number of things using file descriptors (nothing fancy here). This works great for plain text TCP sockets, for example: you can just read/write using normal standard library functions. I've used SSL_write and SSL_read before to read/write from encrypted connections, but I find this very clunky. The objective is handling I/O for things that could either be encrypted or not, and handling them in the same way. With the above, you have to pass around both SSL *ssl for encrypted connections and then a normal file descriptor for unencrypted connections, and depending on whether it's encrypted or not, use the right I/O functions, not all of which even have an SSL equivalent.
For example, the following macros illustrate this:
#ifdef HAVE_SSL_RECV
#define my_recv(ssl, fd, buf, len, flags) (ssl ? SSL_recv(ssl, buf, len, flags) : recv(fd, buf, len, flags)
#endif
#define my_read(ssl, fd, buf, len) (ssl ? SSL_read(ssl, buf, len) : read(fd, buf, len))
#define my_write(ssl, fd, buf, len) (ssl ? SSL_write(ssl, buf, len) : write(fd, buf, len))
What I would like to do is just have a file descriptor referring to the decrypted side of the SSL connection, such that anything written to or read from that file descriptor is encrypted/decrypted as you would expect with using SSL_read and SSL_write. What I'm on unclear on is if there is any builtin or standard way of doing this.
This related question (kind of but not quite) suggested BIO pairs: OpenSSL: perform en-/decryption without SSL_read() / SSL_write()
However, reading the man page for those (https://www.openssl.org/docs/manmaster/man3/BIO_s_bio.html), I don't think this is quite what I want, since BIO pairs do buffering and that's not what I need at all here.
Most similar types of interfaces that I've used provide a way to get a file descriptor that you can use in your program, and OpenSSL does have SSL_get_wfd and SSL_get_rfd, which I assumed would be what I was looking for, but it seems these are actually the raw socket file descriptor itself, exactly the same thing as the file descriptor passed to SSL_set_fd in the first place! (Confirmed by printing out all 3 file descriptors). So I'm not even sure what the point of these functions are in the first place, they don't provide any new information.
Obviously, I could use do this manually, by creating a pipe and then using one end of the pipe as my file descriptor for reading/writing and then spawn another thread to read/write from it and relay to the appropriate SSL functions, appropriately abstracting away that the connection is using TLS. But, is there a better or more standard way of doing this, and a way to avoid the overhead of additional threads for each SSL connection? (If not, that's fine, but I wanted to be sure before I went with that approach.)
I want to write basic chat program. I don't release that, I just want to learn socket programming. The chat program will be between client and server.
server code:
//bla bla code
new_socket = accept(server_fd, (struct sockaddr*)&address,(socklen_t*)&addrlen);
char server_msg[128];
char client_msg[128];
int running = 1;
while(running){
fgets(server_msg, 64, stdin);
send(new_socket, server_msg, strlen(server_msg), 0);
recv(new_socket, client_msg, 128, 0);
}
client code:
char server_msg[128];
char client_msg[128];
int running = 1;
while(running){
fgets(client_msg, 64, stdin);
send(server_socket, client_msg, strlen(client_msg), 0);
recv(server_socket, server_msg, 128, 0);
}
Questions:
Is the new socket fd used only once? That means; Will I do create a new socket for each sending and receiving. Or can I use this forever?
If first question answer is "FOREVER", Will I do something to new socket fd? I don't know maybe clear.
The above code is not working as expected. As I expected. Actually, The code is working very well :D. How do I perform interrupt operations such as getting input, sending messages, receiving messages?
My English is not good, I'm sorry.
Is the new socket fd used only once? That means; Will I do create a new socket for each sending and receiving. Or can I use this forever?
you open a TCP socket connection, so, once in the connected state (the ESTABLISHED state) it remains there until one side closes it (and the other side will notice it by reading 0 bytes as result, this explanation is oversimplified to make it simpler to your case, normally you should detect the end of file in the socket, so you can close it, and accept(2) another connection)
If first question answer is "FOREVER", Will I do something to new socket fd? I don't know maybe clear.
Your program is not a chat program, as the server uses the SAME socket to write what it has read, so it will return the message sent to the same client that issued it (acting more as an echo server, than a chat)
The above code is not working as expected. As I expected. Actually, The code is working very well :D. How do I perform interrupt operations such as getting input, sending messages, receiving messages?
Normally, a chat server should wait for a message to arrive at several sockets (by means of the select(2) system call) and will determine which socket needs attention. As a result, the server will read(2) from that socket (probably in a non blocking way, to avoid being blocked on a read to a socket, that will not be interrupted if data enters on another socket) The chat server should write what it has read over all the other sockets it has connections from, so every other user of your chat system receives a copy of the message (sending back the message to the sender is optional, but not very elegant, the issuer of the message should do local echo of it, although, to ensure the user sees his messages interspersed between the ones from other users)
I recommend you to read the book "Unix Network Programming" from Richard Stevens. It describes perfectly how to use the sockets API and IMHO it is the best reference you can get to learn how to use the socket interface.
One final note:
Don't use plain numbers hardwired in your code, as it is error prone (you declare a buffer string to be 128 bytes, but then read only 64, making 64 unused bytes in the array, you can use sizeof buffer as to make the compiler to use the amount declared in the variable, or you can #define BUFFSIZE 128 and then declare char buffer[BUFFSIZE]; and later do a fgets(buffer, BUFFSIZE, socket_FILE_descriptor); if you use a constant to size a buffer, use a constant name to name it :) and use it everywhere, in case you decide to give it a different value, all the occurences of it will change, instead of having to reviste all places in your code where you have used the number 64.
I'm trying to write simple kernel module that uses a device connected to the system via serial line, but I'm having trouble actually accessing the UART. I think I know how to communicate with the device with kernel_read and kernel_write:
struct file* serial_file;
char buf[32];
lsoff_t off = 0;
serial_file = filp_open("/dev/ttyS0", O_RDWR, 0);
kernel_write(serial_file, "test\n", 5, &off);
kernel_read(serial_file, buf, 32, &off);
The file seems to be open (IS_ERR(serial_file) is 0), but reads and writes return EIO - probably because of the wrong baudrate. Since there is no kernel_ioctl, there's no obvious way to fix this.
Answer to the problem above would be nice, but since this is more of a hack (from my understanding filesystem shouldn't really be accessed from kernel in this fashion), even nicer would be explanation how to do this properly? I'd guess it would involve calling functions from tty_struct directly, but how to get it from file (or from path, or from whatever it should be generated from)?
I am currently sitting at a university task and am facing a problem that cannot be solved for me. I'm developing a TCP client which connects to a server and gets a message from there.
The client should be able to work with strings of any length and output all received characters until the server closes the connection.
My client works and with a fixed string length, I can also receive messages from e.g. djxmmx.net port 17. However, I have no idea how to map this arbitrary length.
My C knowledge is really poor, which is why I need some suggestions, ideas or tips on how to implement my problem.
Actual this is my code for receiving messages:
// receive data from the server
char server_response[512];
recv(client_socket, &server_response, sizeof(server_response), 0);
If you're going to work with input of essentially unlimited length, you will need to call recv() several times in a loop to get each succeeding section of the input. If you can deal with each section at a time, and then discard it and move onto the next section, that's one approach. If you are going to need to process all the input in one go, you're going to have to find a way of storing arbitrarily large amounts of data, probably using dynamic memory allocation.
With recv() you will probably want to loop reading content until it returns 0 indicating that the socket has performed an orderly shutdown (documentation here). That might look something like this:
char server_response[512];
ssize_t bytes_read;
while ((bytes_read = recv(client_socket, &server_response,
sizeof(server_response), 0)) > 0) {
/* do something with the data of length bytes_read
in server_response[] */
}
I am programming TCP server client. I sending the three string seperately using seperate send system call.
But receiving end i getting only single string that is first string which i send. remaining two string missed.
Below i given the part of my server client program.
client.c
char *info = "infolog";
char *size = "filesize";
char *end = "fileend";
send(client, info, strlen(info)+1, 0);
send(client, size, strlen(size)+1, 0);
send(client, end, strlen(end)+1, 0);
server.c
while ((read_size = recv(client, msg, sizeof(msg), 0))) {
printf("Data: %s\n", msg);
memset(msg, 0, sizeof(msg));
}
Actual output:
Data: infolog
Expected output
Data: infolog
Data: filesize
Data: fileend
Thanks.
Try printing out read_size. You probably have received all the messages already.
Due to Nagle's Algorithm, the sender probably batched up your three send() calls and sent a single packet to the server. While you can disable Nagle's algorithm, I don't think it's a good idea in this case. Your server needs to be able to handle receiving of partial data, and handle receiving more data than it expects.
You might want to look into using an upper-layer protocol for your messages, such as Google Protocol Buffers. Take a look at the techniques page, where they describe how they might do it: build up a protocol buffer, and write its length to the stream before writing the buffer itself. That way the receive side can read the length and then determine how many bytes it needs to read before it has a complete message.
TCP is not a message protocol but a byte stream protocol.
The three send-s could be recv-ed as a single input (or something else, e.g. in two or five recv etc....)
The application should analyze the input and buffer it to be able to splice it in meaningful messages.
the transmission may split or merge the messages, e.g. intermediate routers can and will split or merge the "packets".
In practice you'll better have some good conventions about your messages. Either decide that each message is e.g. newline terminated, or decide that it starts with some header giving its size.
Look at HTTP, SMTP, IMAP, SCGI or ONC/XDR (documented in RFC5531) as concrete examples. And document quite well your protocol (a minima, in long descriptive comments for some homework toy project, and more seriously, in a separate public document).