RSA encryption/decryption of long files - c

Question

I want to encrypt/decrypt a long file with RSA (I know AES is better, but this is just for a comparison) in openssl/libcrypto. I am splitting the input file into blocks of size numBlocks = inputFileLength/maxlen+1 where maxlen = 200. I can successfully encode and decode in the same loop as follows:
for (int i = 0; i < chunks; i++)
{
int bytesDone = i * maxlen;
int remainingLen = inLength - bytesDone;
int thisLen;
if (remainingLen > maxlen)
{
thisLen = maxlen;
} else
{
thisLen = remainingLen;
}
if((encBytes=RSA_public_encrypt(thisLen, data + bytesDone, encryptdata + bytesDone,
rsa_public, RSA_PKCS1_PADDING)) == -1)
{
printf("error\n");
}
if((decBytes=RSA_private_decrypt(encBytes, encryptdata + bytesDone, decryptdata + bytesDone,
rsa_private, RSA_PKCS1_PADDING)) == -1)
{
printf("error\n");
}
}
However, I want to save the encoded buffer encryptdata in a binary file, reading the binary file back and decryption. I try to do this as follows:
for (int i = 0; i < chunks; i++)
{
int bytesDone = i * maxlen;
int remainingLen = inLength - bytesDone;
int thisLen;
if (remainingLen > maxlen)
{
thisLen = maxlen;
} else
{
thisLen = remainingLen;
}
if((encBytes=RSA_public_encrypt(thisLen, data + bytesDone, encryptdata + bytesDone,
rsa_public, RSA_PKCS1_PADDING)) == -1)
{
printf("error\n");
}
}
writeFile("encoded.bin",encryptdata,strlen(encryptdata));
size_t size;
unsigned char *readData = readFile("encoded.bin", size);
int inputlen = size;
for (int i = 0; (i * keylen) < inputlen; i++) //keylen = 256
{
int bytesDone = i * keylen;
if((decBytes=RSA_private_decrypt(encBytes, readData + bytesDone, decryptdata + bytesDone,
rsa_private, RSA_PKCS1_PADDING)) == -1)
{
printf("error\n");
}
}
printf("Decrypted text: %s",decryptdata);
The readFile and writeFile functions are as follows:
void writeFile(char *filename, unsigned char *file, size_t fileLength
{
FILE *fd = fopen(filename, "wb");
if(fd == NULL) {
fprintf(stderr, "Failed to open file: %s\n", strerror(errno));
exit(1);
}
size_t bytesWritten = fwrite(file, 1, fileLength, fd);
if(bytesWritten != fileLength) {
fprintf(stderr, "Failed to write file\n");
exit(1);
}
fclose(fd);
}
unsigned char* readFile(char *filename, size_t size) {
FILE *fd = fopen(filename, "rb");
if(fd == NULL) {
fprintf(stderr, "Failed to open file: %s\n", strerror(errno));
exit(1);
}
// Determine size of the file
fseek(fd, 0, SEEK_END);
size_t fileLength = ftell(fd);
fseek(fd, 0, SEEK_SET);
size = fileLength;
// Allocate space for the file
unsigned char* buffer = (unsigned char*)malloc(fileLength);
// Read the file into the buffer
size_t bytesRead = fread(buffer, 1, fileLength, fd);
if(bytesRead != fileLength) {
fprintf(stderr, "Error reading file\n");
exit(1);
}
fclose(fd);
return buffer;
}
However, the decryption fails with the error message segmentation fault (core dump) and the decrypt function only returns -1 for every block. Any help will be appreciated.

Answer 1

ReadFile modifies the parameter "size" which is passed by value, thus when the readfile function returns, size is not affected.
I would change readfile proto as follows :
unsigned char* readFile(char *filename, size_t *size)
and then change the call into
unsigned char *readData = readFile("encoded.bin", &size);
and finally modify the readFile size update to
size = fileLength;

Answer 2

Your have various technical errors in your code like doing a strlen(..) on binary data in this statement:
writeFile("encoded.bin",encryptdata,strlen(encryptdata));
encryptdata is binary data that can include 0 that would be interprited as a string termination by strlen(..)
But the main problem is that you try to use RSA as a block cipher. Your encrypted blocks are bigger that what you encrypt, but you don't handle that in your code. You might be able to get your code to handle this, but the right approach is to use ciphers invented for bulk encryption, like AES. When you do that, you automatically get support for 'blocking' out of the box.
In addition to this you get something like a factor 1000 faster encryption.