RSA_private_encrypt always fails - c

I am learning to use OpenSSL library in my program. Here in the code I generate a private key and immediately I am encrypting a message using that key. But always it fails. Kindly help me.
private_key = RSA_generate_key(RSA_KEY_LENGTH, RSA_3, NULL, NULL);
if (RSA_check_key(private_key) < 1) {
printf("generate_key: key generation failed\n");
exit(-1);
}
unsigned char msg[25];
unsigned char cipher[128];
strcpy((char*)msg, "hello");
int ret = RSA_private_encrypt(25, msg, cipher, private_key,
RSA_PKCS1_OAEP_PADDING);
if (ret < 0) {
printf("encryption in key generation failed\n");
printf ("%s\n", ERR_error_string (ERR_get_error (), (char *) cipher));
exit (-1);
}
This always fails and this is the error I am getting with ERR_error_string.
error:04066076:lib(4):func(102):reason(118)

See documentation:
man RSA_private_encrypt
RSA_private_encrypt() signs the flen bytes at from (usually a message digest with an algorithm identifier) using the private key rsa and stores the signature in to. to must point to RSA_size(rsa) bytes of memory.
padding denotes one of the following modes:
RSA_PKCS1_PADDING
PKCS #1 v1.5 padding. This function does not handle the algorithmIdentifier specified in PKCS #1. When generating or verifying PKCS #1 signatures, RSA_sign(3) and RSA_verify(3) should be used.
RSA_NO_PADDING
Raw RSA signature. This mode should only be used to implement cryptographically sound padding modes in the application code. Signing user data directly with RSA is insecure.
I do not know where you got RSA_PKCS1_OAEP_PADDING from, but the only supported paddings are listed above.

error:04066076:lib(4):func(102):reason(118)
You can use OpenSSL's errstr to give you meaningful error messages (in most cases):
$ openssl errstr 0x04066076
error:04066076:rsa routines:RSA_EAY_PRIVATE_ENCRYPT:unknown padding type
Even though you narrowed it down to RSA_PKCS1_OAEP_PADDING/RSA_PKCS1_PADDING, you should still use RSA_PKCS1_OAEP_PADDING with RSA encryption. So your next task is to figure out what's still wrong with your code.
Here's a good blog entry on why you should avoid PKCS 1.5 padding for RSA encryption: A bad couple of years for the cryptographic token industry.

I have found the reason for this problem. Actually the padding method RSA_PKCS1_OAEP_PADDING is not working for me in my both centos and ubuntu machines. Once I changed it to RSA_PKCS1_PADDING, it started working fine. But I am not sure why this is happening.

Related

How is a DES key generated from passphrase in C?

(Not to be confused with the DES algorithm subkey generation)
(edit: more examples)
Explanation of problem:
I'm doing this as part of a school assignment where I'm required to recode parts of OpenSSL in C, specifically those pertaining to PKI cryptosystems. So far I've recoded from scratch the core DES algorithm with ecb, cbc, 3des-ecb, and 3des-cbc modes of operation. Other parts of the project include MD5 and SHA256. This portion of the project focuses on RSA key generation, manipulation and usage.
Part of RSA key manipulation includes encrypting a given key with a passphrase.
(not with the pure key + initial vector alone like I've done before with DES)
This requires converting the user-input passphrase to a DES key (and optional additional IV as needed), and then using that to encrypt a RSA key. I know the general term for the function I'm looking for is PBKDF, or Password-Based Key Derivation Function. However, I have not been able (through searching the man pages of OpenSSL or google) to find what exact function (or functions) are used in OpenSSL for key derivation.
Demonstration of DES key generation encrypting RSA keys:
Running the following command with no passphrase generate an unencrypted RSA key example_plain_key.
ssh-keygen -t rsa -f example_plain_key
Then running the following commands will encrypt example_plain_key with the des cipher in ecb mode. Each command outputs the encrypted version to a new file so it doesn't change the original. Use the same passphrase for both commands (password, for example).
openssl rsa -DES-ECB -in id_rsa -out id_rsa_1
openssl rsa -DES-ECB -in id_rsa -out id_rsa_2
You can use head id_rsa and head id_rsa_1 to see how encrypting a key changes the header. If you compare the two new keys with
diff id_rsa_1 id_rsa_2
they will be identical in the header and formatting, but the key itself will be encrypted differently, even though the same passphrase is used. The difference is because the key generation (I believe) generates a new random salt every time it is ran. I would assume the hashing algorithm and the number of iterations would be the same. Also, unlike /etc/shadow on unix machines, the salt doesn't appear to be stored alongside the key (or at least I don't know how to read it).
Demonstration of DES key generation from password:
A more DES-specific example is:
openssl des -P
Running the above command any number of times with the same password will always result in a different key and iv, probably because the salt is different.
My findings, and deducted assumptions:
Searching "how are rsa keys encrypted?" brings up a lot of results on using RSA keys to encrypt. (sometimes I expect too much from Google's nlp)
Searching "how are DES keys generated from passphrase?" brings up a lot of results on how to generate the 16 round des subkeys.
I've skimmed the source of OpenSSL with no luck. I'll do an exhaustive search if absolutely necessary, but the code isn't the most readable or searchable.
php prototype
perl man page
A link I thought would be more helpful than it was
(Note: I don't have an account with OpenSSL but don't think it'd be required to view)
The most helpful findings led me to believe an example prototype of what I'm looking for would look something like this:
#include <unistd.h>
#include <stdio.h>
#include <pwd.h>
// #include <something_else_maybe.h>
int main(void)
{
int num_iterations = 1000;
char *salt;
char *passphrase;
char *key;
passphrase = getpass("Password: ");
salt = get_some_random_bytes(8); // assumed arbitrary length
// the function in question
key = example_pbkdf(md5_function, num_iterations, salt, 8, passphrase, strlen(passphrase));
printf("Key (in hexadecimal or otherwise) is: %s\n", key);
free(key);
free(passphrase);
free(salt);
return (0);
}
Things I am specifically looking for:
(Knowing where to look for these answers would be more valuable than the answers themselves, but all help is appreciated. I do need the header/source/prototype/etc in C though.)
The function (if it exists) that operates like the one demonstrated above. It doesn't have to be a perfect match, I'm more concerned about what it does rather than the exact prototyping or usage.
Alternatively, (if it doesn't exist) the "recipe" or series of operations that could be summarized as "the algorithm" I'm looking for.
DES key generation. (though including multiple ciphers, say, AES, is awesome too)
How the salt is stored in an ecrypted RSA key, if it is (and if it isn't, how to recover it). I know the IV is stored in the header of a key encrypted with a cipher in CBC.

How to generate password for RSA / AES encryption

I want to send large data encrypted with RSA through sockets. I use openssl and c.
Because RSA decryption is quite slow I use the common and straight forward way to encrypt the data with AES first, and afterwards I encrypt the used AES password with RSA. Then I send both, the AES encrypted data and the RSA encrypted password, through the socket and do the encryption the other way around.
I do the AES encryption with:
EVP_CIPHER_CTX en;
unsigned char password[65];
int i, x = 0;
unsigned char key[32], iv[32];
unsigned char *ciphertext;
i = dataLength + AES_BLOCK_SIZE -1;
ciphertext = (unsigned char *)malloc(i);
EVP_CIPHER_CTX_init(&en);
EVP_EncryptInit_ex(&en, EVP_aes_256_cbc(), NULL, key, iv);
EVP_EncryptUpdate(&en, ciphertext, &i, (unsigned char*)data, dataLength);
EVP_EncryptFinal_ex(&en, ciphertext+i, &x);
But how do I create the key and the iv securely? Right now I Use the following function:
EVP_BytesToKey(EVP_aes_256_cbc(), EVP_sha1(), salt, password, 64, 9, key, iv);
My question is:
How do I create "password" correctly?
Because if I use rand() or something equal my attempt was completely useless because anybody who is able to get behind the "randomness" used for the "password" generation is able to decrypt the data anyway without caring about the RSA encryption of the "password".
Is there a function for secure passwordgeneration in openssl? Or is EVP_BytesToKey() just the wrong way to do what I want to do?
The default RAND_bytes method is fortunately seeded per thread, and by default uses the random number generator available from the operating system. The OpenSSL documentation seems to be out of date where Windows is involved, but you can find more information on this by looking at the answer of the venerable Thomas Pornin on security.stackoverflow.com.
EVP_BytesToKey is used to generate keys from passwords. EVP_BytesToKey is a key derivation function (KDF) that is specific to OpenSSL. OpenSSL also implements PBKDF2 which is the NIST approved method of password based key derivation function (PBKDF). But as you want a random key, not a derived key, none of those functions apply.
So please use rand(). If possible, try to check how the function is seeded for your specific platform.
Also note OpenSSL 1.1.0c changed the digest algorithm used in some internal components. Formerly, MD5 was used, and 1.1.0 switched to SHA256. Be careful the change is not affecting you in both EVP_BytesToKey and commands like openssl enc.

Is my encryption safe as openPGP/SMIME?

I'm trying to write a simple file enc/decryption within a larger project.
I'd like to avoid libgpgme because of license issues. The openPGP standard is to complex for the project timeframe i have.
I'd like to do my encryption stuff with openssl.
Now i've implemented the following:
encryption (pseude code):
RAND_bytes(aes_key)
RAND_bytes(aes_salt)
EVP_BytesToKey(EVP_aes_256_cbc(), EVP_sha1(), (const unsigned char *)aes_salt, aes_key, sizeof(aes_key), 5, key, iv);
then i aes256 my data
EVP_EncryptInit_ex(&e_ctx, EVP_aes_256_cbc(), NULL, key, iv);
then i encrypt the key and iv with RSA
RSA_public_encrypt(flen, (unsigned char *)key, encryptedKey, rsa, RSA_PKCS1_PADDING );
RSA_public_encrypt(flen, (unsigned char *)iv, encryptedIV, rsa, RSA_PKCS1_PADDING );
then i save the 128bit key and iv at the "top" of my file (256Bytes header).
decryption:
-> read the first 256bytes (split into key and iv)
-> decrypt the key and iv with the local RSA Private Key (of course the RSA Private Key IS NOT in the file)
-> use the key and iv to decrypt the data
Am i kind of safe with that code?
Since this is a new format, you should use OAEP padding. Just change RSA_PKCS1_PADDING to RSA_PKCS1_OAEP_PADDING. You actually don't need to encrypt the IV (it can't hurt as far as I can tell, and it might help).
Otherwise, this method should be fine so long as RSA_size(rsa)==16. Of course, the private key must not be knowable by anyone who should not be able to decrypt the file.
Encryption is a topic where things are easy to make "work" - but hard to make secure. When in doubt (and doubly so when not in doubt), pick a widely recognized standard and implement precisely to spec. The idea of encrypting the key with a public-private algorithm, then packing the IV in as well is sound in theory, but I'm not sure what the implications of encrypting the IV as well are, and what happens if the attacker starts flipping bits in the encrypted data? Etc. It looks sound, but again, I would strongly recommend simply implementing a published spec precisely.
I would recommend just implementing S/MIME, using a binary transfer encoding. S/MIME is recognized as being a secure specification, there are libraries implementing all the hard parts, and most importantly, you can test your implementation against other implementations to make sure you're not out of spec.
Since you are using the OpenSSL envelope-encryption functions anyway, you should just directly use the EVP_SealInit() / EVP_SealUpdate() / EVP_SealFinal() functions. These functions take care of generating the symmetric key and IV, encrypting the data with the symmetric key and encrypting the symmetric key with the recipient(s) RSA key(s).
Once thing that you are not taking care of is authenticity. Under CBC mode it is possible for an attacker to make certain predictable changes to the plaintext, even if they can't read it. To detect this, you should either calculate a HMAC over the encrypted message (using a seperate symmetric key to that used for encryption), or sign the encrypted message (eg. with EVP_SignInit() / EVP_SignUpdate() / EVP_SignFinal()).
Some observations:
The EVP_BytesToKey function is meant to create a key and initialization vector from a password and salt, not from random data. It will work, but you could also simply use the random bytes directly as key and initialization vector. (Make sure you are using a secure PRNG, I'm not sure what RAND_bytes actually does.)
The initialization vector does not need to be secret, CBC mode should be secure with a non-encrypted IV. (This does not hurt, though.)
The RSA encryption looks good (but you might want to use another padding, as David said).
As Serdalis said, you should also protect your file against modifications. Any keyed MAC will do (most common are HMAC build on a key and a hash function). Apply the MAC after encryption.

RSA to encrypt 500 bytes of data using openssl library

I am using RSA algorithm for shared key encryption/decryption.I am using open ssl libraries and c language.RSA_size() returns 256bytes.the key(data to enrypt) size is more than 256bytes but less than 500 bytes.The RSA keysize used for encryption is 1024.keypair is generated using openssl library.
If data to encrypt is less than or equals to 245 bytes the encryption works fine(because of padding).Otherwise i have to break the data(not sure it is ok as iam trying to exchange shared key)Is there any other way to increase RSA_size.my code doesn't call RAND_seed()(not sure what argment to pass) before calling rsa_public_encrypt. Any help is appreciated.
You can either break your data into chunks, or use the "seal"/"open" OpenSSL functions.
Essentially what they do is generate a random set of keys, encrypt the keys using the public key and then use those keys to encrypt whatever you want. That way you're not limited to specific sizes and decryption is not too slow (you only decrypt the random keys using RSA, the rest is done using symmetric encryption so it's a lot faster).
I suggest you have a look at EVP_SealInit and EVP_OpenInit.

C: DES Encryption problem

I was looking for examples how to encode stuff in C using DES cypher and openssl headers and I found this one: http://www.codealias.info/technotes/des_encryption_using_openssl_a_simple_example
The code is almost perfect but I'm not so expert in this stuff and my C knowledge in C is not so big since I use it on PIC and AVR micro controllers...
Anyway in the code:
printf("Clear text\t : %s \n",clear);
memcpy(encrypted,Encrypt(key,clear,sizeof(clear)), sizeof(clear));
printf("Encrypted text\t : %s \n",encrypted);
memcpy(decrypted,Decrypt(key,encrypted,sizeof(clear)), sizeof(clear));
printf("Decrypted text\t : %s \n",decrypted);
As you can see, sizeof(clear) is used as the size of the string... the problem is that on the example we know the size of the text string... but when I'm sending this text over the network the other computer don't know it...
How can solve this issue... I don't understand so well why I need to have the size of the original string to decrypt :S
Thanks!!
The world is full of bad security systems designed by people who read Applied Cryptography.
Don't send your own 'encryptyed' stuff on wire. You're missing an HMAC, you're missing a key exchange protocol, you're missing a wire frame protocol (which is exactly the answer to your question 'how do I know the size'). Just use an off-the-shelf protocol like TLS/SSL. gnu-tls offers a easy to use API for SSL/TLS, openssl also supports it but is notoriously cumbersome to use. Whatever you do, don't start writing your own protocol, you'll come up with yet another broken 'encryption' protocol because of a bad key exchange or a 'optimized nonce' or a missing frame signature or whatever.
Here is a simple example using gnu-tls: Simple client example using the C++ API
In the implementations I have seen of DES, I only ever recall seeing plaintext and ciphertext of the same size. Wikipedia seems to confirm this. Since DES works on 64-bit chunks, that would make since as long as the code implementing DES properly pads the input to match those 64-bit boundaries. In fact, that's pretty much the definition of a block cipher (which is what DES is).
Thus I would wager you will see it work flawlessly with the other computer using the size of the encrypted text. A few tests of your own should be able to confirm this absolutely.
Also, I firmly agree with the Jeremy's comment that DES is a poor choice of encryption algorithm for most situations. Triple DES or AES are much better options.

Resources