I've written an event-based networking library in C and now I want to add SSL/TLS support via OpenSSL. Instead of using SSL_read() and SSL_write(), I'd rather like to have OpenSSL only perform the encryption/decryption of outgoing/incoming data, letting me transmit/receive the data myself.
I'm new to SSL/TLS and OpenSSL, so:
Is there a way to have OpenSSL only perform encryption/decryption of char arrays?
Something like size_t SSL_encrypt(const char *buf_in, size_t size_in, char *buf_out) would be great.
Exactly what you've asked for isn't possible, because with TLS there isn't a 1-to-1 correspondence between sending something at the application layer and sending something on the network socket. Events like renegotiations mean that sometimes SSL needs to read data from the network in order to make progress sending data, and vice-versa.
However, you can still use OpenSSL to perform SSL but take care of the reading and writing from the network yourself. You do this by calling SSL_set_bio() on the SSL pointer instead of SSL_set_fd():
Use BIO_new_bio_pair() to create a connected BIO pair. One BIO will be read from and written to by the SSL routines, and the other BIO will be read from and written to by your application (allowing it to pass the data to the other end by whatever method it desires).
Use SSL_set_bio() on a new SSL object to set both the read and write BIO to one of the BIOs in the pair generated.
Use BIO_read() and BIO_write() on the other BIO in the pair to read and write the SSL protocol data.
Use SSL_accept(), SSL_connect(), SSL_read() and SSL_write() as normal on the SSL object.
(It's not clear what advantage this would give in your application, though: in this case you can't really do anything other than read and write exactly what OpenSSL passes you, so you might as well let it do the reading and writing too).
Related
When using a non-encrypted socket I'd use
int num bytes = recvmsg(sock, &msg, 0)
and then get the SO_TIMESTAMP info from the msg (see e.g. Linux recvmsg() not getting a software timestamp from socket and https://linux.die.net/man/2/recvmsg).
However, there doesn't seem to be a corresponding API for an SSL-encrypted socket, at least for OpenSSL. The only one I can see is
int SSL_read(SSL *ssl, void *buf, int num);
which obviously doesn't propagate the timestamp info.
Have people tried this before? I can see a few options,
fork/extend OpenSSL at the point where it reads from the raw socket and carry the data across
do a recvmsg(s, data, flags) explicitly and somehow pass that into an OpenSSL function for subsequent decoding.
use a different library?
I find surprisingly little info about this online.
Thanks!
Followed the comments suggestion by Steffen Ullrich (thank you) to make something work.
I ended up implementing a custom BIO along the lines of https://github.com/openssl/openssl/blob/master/crypto/bio/bio_sock.c but with recvmsg for the read path, and grabbing the SO_TIMESTAMP, which I subsequently stick in custom data.
I attach the BIO (SSL_set_bio) to my SSL object, and can retrieve the timestamp after a succesful SSL_read.
It was quite painful.
I mean, I've read questions about Dart RawSockets and also read the API but it was not clear for me to understand how to use them. Are Dart RawSockets the same as C rawsockets?
Also, what is the difference between Dart RawSockets and normal Sockets?
A Socket is a higher level concept. It implements a Stream of bytes (actually byte arrays) and an IOSink. listen to the stream and data arriving at the socket appears in the stream. When you want to send data down the socket, add it to the sink and away it goes.
A RawSocket is the lower level concept. Now, instead of getting a stream of bytes, you are just told when bytes are available to be read. (You get a stream of events telling you when data is available. You are responsible for then calling read to collect them.) This allows you to work more in the mode of a Unix socket where you use select to know that there is data available so that you don't block when trying to read it.
Dart's streams relieve you of much of the responsibility of select/blocking read/separate reader thread of different languages. When reading from a Socket you don't need to worry, data just arrives in the stream when it's available.
Note how there is only a RawDatagramSocket. This makes sense, since UDP packets are discrete, not a byte stream. A UDP socket just tells you that a packet is available to be read, and you then read it.
I am looking to implement some kind of transmission protocol in C, to use on a custom hardware. I have the ability to send and receive through RF, but I need to rely in some protocol that validates the package integrity sent/received, so I though it would be a good idea to implement some kind of UDP library.
Of course, if there is any way that I can modify the existing implementations for UDP or TCP so it works over my RF device it would be of great help. The only thing that I think it needs to be changed is the way that a single bit is sent, if I could change that on the UDP library (sys/socket.h) it would save me a lot of time.
UDP does not exist in standard C99 or C11.
It is generally part of some Internet Protocol layer. These are very complex software (as soon as you want some performance).
I would suggest to use some existing operating system kernel (e.g. Linux) and to write a network driver (e.g. for the Linux kernel) for your device. Life is too short to write a competitive UDP like layer (that could take you dozens of years).
addenda
Apparently, the mention of UDP in the question is confusing. Per your comments (which should go inside the question) you just want some serial protocol on a small 8 bits PIC 18F4550 microcontroller (32Kbytes ROM + 2Kbytes RAM). Without knowing additional constraints, I would suggest a tiny "textual" like protocol (e.g. in ASCII lines, no more than 128 bytes per line, \n terminated ....) and I would put some simple hex checksum inside it. In the 1980s Hayes modems had such things.
What you should then do is define and document the protocol first (e.g. as BNF syntax of the message lines), then implement it (probably with buffering and finite state automaton techniques). You might invent some message format like e.g. DOFOO?123,456%BE53 followed by a newline, meaning do the command DOFOO with arguments 123 then 456 and hex checksum BE53
I am looking at socket programming again. I get the details (well, I can copy them from various websites, and I know the code is enabling the Unix low-level procedures), but I don't get the POSIX logic and thinking in its API.
Why have they not defined a slightly higher-level interface built on these lower-level socket functions?
Presumably, such code could factor out code that is repeated often (and error-prone) into more convenient FILE like interfaces. Factoring would seem even more appropriate than just convenient when the lower level use is the same in > 90% of its use. Almost all sockets use that I see in application programs open a socket, read and write to it and close the socket. Also, why does one need to bind, when this is really something that the open call always does?
What cases does the current interface even cover that could not easily be covered by an interface that would look almost like the FILE interface?
One explanation is that there are uses where one would not bind to a socket, for example, or where fgets/fputs/fprintf/fscanf like functionality would need something extra (time-outs)?
There must be a reason that I am missing. Otherwise, 20 years later, there would already be one or more standard libraries that facilitate this and that would be in wide use. I couldn't find one on google that mimics all the FILE routines.
The point is strikingly simple:
Because sockets are not files.
Let me elaborate: recv/send works quite like read/write, if you limit yourself to linearly reading a file from the beginning, and to appending at its end.
However, you'll say, send doesn't let me write arbitrary lengths of data trough! If I try to send more data than fits into a protocol's packet buffer, it will throw an error!
And that's actually the beauty of sockets: you actually send the data away. you can't keep it; it's gone once it's sent, and it's not stored once it's received. Sockets give you a whole different set of abilities (like sending smaller packets than the maximum packet size of the network, for example), which on the other hand demand you take some control yourself.
EDIT: send will not "throw" an error. "throwing" is not a C/Posix way of handling errors. Instead it will return an error (from man 2 send):
If the message is too long to pass atomically through the underlying protocol, the error EMSGSIZE is returned, and the message is not transmitted.
The C programming language is and will likely always be a lightweight one. You need to understand that C runs basically anywhere and some things need a long research and work to get standardized.
Also, I have seen that new libraries are added because C++ went ahead and made them standard so it's a kind of C sharing.
Please do note that you can "bind" a socket to a file through fdopen(3) and consider it as a binary file. Of course you will still need to bind it, make it listen, accept and all the actions you can do on a socket that won't work for a file.
Indeed, despite the similar interface, a socket acts only partially as a UNIX file: there's even an errno value, ENOTSOCK which indicates a socket specific operation on a non-socket file descriptor.
Furthermore, consider buffering. You do want a file write to be done in large chunks, thus a bigger buffering, to make it faster; this won't work for a socket as you need to send data immediately, that is, undelayed.
Consider this example:
char one = '1', two = '2', three = '3';
fwrite(&one, 1, 1, socket_file);
fprintf(socket_file, "%c\n", two);
send(fd, &three, 1, 0);
where fd is a connected socket(AF_INET, SOCK_STREAM, 0) and socket_file = fdopen(fd, "w+"). The receiver will read 312 because there's no flush except upon process termination at the FILE layer, unlike with send where three is sent immediately.
I saw codes after codes online, but hardly any clear and direct descriptions on what ablkcipher_request_set_callback() does.
In include/linux/crypto.h, there is zero comments. Can anyone give any insights and how to use this function?
This is a function in Linux crypto framework.
The simple theory is:
Linux crypto framework assume there might be a HW chip in your
machine, which could do cryption / decryption work, like AES, DES.
So it construct a request to this HW chip. And the chip would do its
work in parallel with your CPU.
After HW chip is done with the request, it notify your CPU about the
complete event, ususally by IRQ.
Then the IRQ handler (well, actually bottom half for most of time)
would execute the callback function you specify in the request.
So: ablkcipher_request_set_callback() is to set the callback function
of encrypt / decrypt request.
A real example is like:
Suppose you are running IPsec protocol. It would do the following
logic:
Construct a packet encrypt request, with callback function "when
encryption done, please transmit this packet out".
Submit the encryption request to HW chip driver, and further to HW
chip.
After HW chip done with encryption of this packet, HW chip driver
would run this callback function "transmit this encrypted packet out".
=======================================================================
IPsec uses
aead_givcrypt_set_callback()
not
ablkcipher_request_set_callback()
"ablkcipher" means a "block cipher" (like AES), "a" standards for "asynchronous" ( run in parralel with CPU).
"aead" could be considered as combined algorithm of ( "ablkcipher" + compute a hash value ). In IPsec case, the hash value is the packet CRC.
Add the IPSec code fragment: # in $(kernel)/net/ipv4/esp4.c
static int esp_output(struct xfrm_state *x, struct sk_buff *skb)
/* set the callback esp_output_done(), which would essentially call
* dev_queue_xmit() # transmit the encrypted packet out
*/
aead_givcrypt_set_callback(req, 0, esp_output_done, skb)
/*
* sumbit the request to Linux crypto framework, which internally would forward
* the request to a HW chip driver.
*
* The HW chip driver accept the request, and return EINPROGRESS indicating
* the request is being handled in progress or queued for handling.
*
* After HW chip done with encryption, request callback esp_output_done() is
* called
*/
err = crypto_aead_givencrypt(req);
if (err == -EINPROGRESS)
goto error;
--- Well, to much detail not covered, and like you said, "undocumenting" causes a lot of trouble to us.
This is a kernel header file (as are all the headers in /include/linux so it's not really intended for use in a normal application. The assumption is that if you are working with the kernel (such as writing a device driver), then you should read the source to get a better understanding of what is doing.
Anything that you are supposed to use in applications will not be in the linux directory and will have a man page explaining what it does and how to use it.
There is some documentation here, but it doesn't include the function you are asking about.