Erratic openssl/rsa behaviour: RSA_EAY_PRIVATE_DECRYPT:padding check failed - c

I have written a program that runs permanently using <openssl/rsa> C library.
It basically decrypts a password given in argument. The problem is that sometimes it works flawlessly, and some other times it fails (with the same pubkey/privkey/password, returning this error:
message: error:04065072:rsa routines:RSA_EAY_PRIVATE_DECRYPT:padding check failed
Has anyone ever experienced that?
Why this kind of error is returned, generally?
Some more details
It retrieves the private key at the initialisation of the program with the following:
#define PRIVFILE "<correct-path>/privkey.pem"
EVP_PKEY *privKey;
int size_key;
FILE *fp = fopen(PRIVFILE, "r");
if (!fp)
{
<logs>
return -1;
}
PEM_read_PrivateKey(fp, &privKey, 0, NULL);
fclose (fp);
if (privKey == NULL)
{
ERR_print_errors_fp (stderr);
return -1;
}
size_key = EVP_PKEY_size(privKey);
Later, during a listening loop, a method call the private decryption algorithm
int len_enc = size_key;
unsigned char* enc_pw;
unsigned char* dec_pw;
int len_dec = 8;
char* err = malloc(130);
enc_pw = malloc(len_enc);
dec_pw = malloc(len_dec);
memset(enc_pw, 0, len_enc);
memset(dec_pw, 0, len_dec);
memcpy(enc_pw, value, len_enc); //value being the raw ciphered data to decrypt
ERR_load_crypto_strings();
if (RSA_private_decrypt(len_enc, enc_pw, dec_pw, privKey->pkey.rsa,RSA_PKCS1_OAEP_PADDING) == -1)
{
ERR_error_string(ERR_get_error(), err);
radlog(L_ERR, "message: %s", err);
}
free(enc_pw);
free(dec_pw);
free(err);
I have done encryption on the data with perl using Crypt::OpenSSL::RSA:
my $rsa_pub = Crypt::OpenSSL::RSA->new_public_key( $key_string);
my $ciphertext = $rsa_pub->encrypt( $plaintext);
There is some base64 encoding/decoding that i didn't mention to make it a little bit shorter. The problem does not come from that.
private key and public key are generated with openssl genrsa:
openssl genrsa -out privkey.pem 1024 and openssl rsa -in privkey.pem -pubout > pubkey.pub
It seems to work for some time, but occasionally (during a peak of request, if that matters) i get these errors for ciphered data that seemed valid before:
message: error:04065072:rsa routines:RSA_EAY_PRIVATE_DECRYPT:padding check failed

Is it a multi threaded application?
I was getting the same problem yesterday and, in my case, it was related to more than one thread using the key (one for decript and many others for encript). The problem was solved protecting the key with a mutex semaphore.
The service is up and stable since yesterday.

Related

OpenAES/OpenSSL compatibility

I'm trying to use both OpenAES and OpenSSL in C, this way:
Application 1 encodes a password using OpenAES
Application 2 decodes the password using OpenSSL
The problem is that I can't decode it, actually I'm quite surprised as OpenAES generates a
different block each time when OpenSSL or any other AES implementation I've seen always generate
the same block.
Another difference I noticed is that the OpenAES generated block is 48 bytes, when OpenSSL generates 16.
This is how I encrypt the string (OpenAES):
pCtx = oaes_alloc();
if (pCtx == NULL)
return FALSE;
oRet = oaes_key_import(pCtx, sKey, szKey);
if (oRet != OAES_RET_SUCCESS)
{
return FALSE;
}
// Get the required buffer size
oRet = oaes_encrypt(pCtx, (const uint8_t*)csSource, szLen, NULL, pOutLen);
if (oRet != OAES_RET_SUCCESS)
{
oaes_free(&pCtx);
return FALSE;
}
*ppOut = (char*)calloc(*pOutLen, sizeof(char));
oRet = oaes_encrypt(pCtx, (const uint8_t*)csSource, szLen, (uint8_t*)*ppOut, pOutLen);
if (oRet != OAES_RET_SUCCESS)
{
oaes_free(&pCtx);
free(*ppOut);
return FALSE;
}
oaes_free(&pCtx);
And this is how I decrypt it (OpenSSL):
AES_KEY kDecrypt;
AES_set_decrypt_key(sKey, 128, &kDecrypt);
AES_decrypt(pEncoded, pDecoded, &kDecrypt);
sKey being the key generated with OpenAES
pEncoded the crypted block from OpenAES
pDecoded the output data.
I can't put my finger on the problem yet...
Is there a particular way to use OpenAES so the result can be decrypted by OpenSSL, or is it just not compatible?
OpenAES prefixes the encrypted data with a OpenAES specific header, followed by the iv and then the encrypted data (+ padding).
As the IV seems to be generated randomly by OpenAES that explains why the data differs with each encrypt.

How to use OpenSSL to validate a *.SF / *.RSA signature created by the Jarsigner

I have an archive I want to sign and I want to validate it in C with OpenSSL.
To sign the archive the jarsigner seemed like a good idea, considering I wouldn't have to create something on my own, and it seems to work great. With OpenSSL I can validate the different digest values, but I can't get it to validate the *.SF *.RSA signature.
The steps I have taken:
Create a keystore
$ keytool -genkeypair -alias <alias> -keystore <keystore> -validity 360 -keyalg RSA -keysize 2048 -sigalg SHA256withRSA
Sign the archive
$ jarsigner -keystore <keystore> -signedjar <signedFile>.zip <fileToSign>.zip <alias>
Snipped of C validation code
BIO *in = NULL, *indata = NULL;
PKCS7 *p7 = NULL;
int flags = PKCS7_DETACHED;
flags |= PKCS7_NOVERIFY;
flags |= PKCS7_BINARY;
OpenSSL_add_all_algorithms();
/* load *.RSA (PKCS7) file */
if (!(in = BIO_new_file(path, "r"))) {
printf ("Can't open input file %s\n", path);
status = FAILURE;
}
if (!(p7 = PEM_read_bio_PKCS7(in, NULL, NULL, NULL))) {
printf ("Error in reading PKCS7 PEM file.\n");
status = FAILURE;
}
/* load *.SF file */
if (!(indata = BIO_new_file(path, "r"))) {
printf("Can't read content file %s\n", path);
status = FAILURE;
}
/* validate signature */
if (PKCS7_verify(p7, NULL, NULL, indata, NULL, flags))
printf("Signature verification successful!\n");
else {
printf("Signature verification failed!\n");
status = FAILURE;
}
The error
It fails in "PEM_read_bio_PKCS7(...)".
I'm looking for either a way to validate it in the terminal or with C using OpenSSL. C is preferred ;) but I can always convert the command to code in case you only know how to do it manually.
If you want to do check the certificate chain using command-line tools, here is how:
unzip -p your.jar META-INF/*.RSA | openssl pkcs7 -inform DER -text -print_certs
I am an idiot. At the start of this project I knew that the signature format had to be either DER or PEM. I thought I had configured this correctly, but somehow it ended up in the situation where the Jarsigner's signature was in DER format when I wanted to verify a PEM signature.
My solution is to always expect a DER signature. This is default for the Jarsigner. For my OpenSSL signer/verifier I had to make sure the outform and inform was der: -outform der and -inform der.
Code wise I had to change this:
if (!(p7 = PEM_read_bio_PKCS7(in, NULL, NULL, NULL))) {
into this:
if (!(p7 = d2i_PKCS7_bio(in, NULL))) {

OpenSSL - find error depth in certificate chain

I am writing a C program to retrieve and verify an x509 certificate chain using OpenSSL. This is my first time programming in C and am relying heavily on the tutorial at http://www.ibm.com/developerworks/linux/library/l-openssl/
I am able to retrieve any error code from the connection using the code below:
if (SSL_get_verify_result(ssl) != X509_V_OK)
{
printf("\nError verifying certificate\n");
fprintf(stderr, "Error Code: %lu\n", SSL_get_verify_result(ssl));
}
however I also need to know which certificate is the offending one. Is there are way to determine the chain depth of the error like the command line s_client? Any example code would be greatly appreciated.
I found my answer in "Network Security with OpenSSL" by Chandra, Messier and Viega.
It uses SSL_CTX_set_verify to designate a callback function that gets run after the verification routine for each certificate in the chain.
SSL_CTX_set_verify(ctx, SSL_VERIFY_PEER, verify_callback);
int verify_callback(int ok, X509_STORE_CTX * store)
{
if (!ok) //if this particular cert had an error
{
int depth = X509_STORE_CTX_get_error_depth(store);
int err = X509_STORE_CTX_get_error(store);
}
}

Openssl/libcrypto AES 128 encoding using the KEY

I am encrypting a certain string using AES-128-ECB and then save the result in a file e.g test.enc
Here is my method that does the encryption:
int do_crypt(char *outfile) {
unsigned char outbuf[1024];
int outlen, tmplen;
unsigned char key[] = {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15};
char intext[] = "Some Text";
EVP_CIPHER_CTX ctx;
FILE *out;
EVP_CIPHER_CTX_init(&ctx);
EVP_EncryptInit_ex(&ctx, EVP_aes_128_ecb(), NULL, key, NULL);
if(!EVP_EncryptUpdate(&ctx, outbuf, &outlen, intext, strlen(intext))) {
/* Error */
return 0;
}
/* Buffer passed to EVP_EncryptFinal() must be after data just
* encrypted to avoid overwriting it.
*/
if(!EVP_EncryptFinal_ex(&ctx, outbuf + outlen, &tmplen))
{
/* Error */
return 0;
}
outlen += tmplen;
EVP_CIPHER_CTX_cleanup(&ctx);
/* Need binary mode for fopen because encrypted data is
* binary data. Also cannot use strlen() on it because
* it wont be null terminated and may contain embedded
* nulls.
*/
out = fopen(outfile, "wb");
fwrite(outbuf, 1, outlen, out);
fclose(out);
return 1;
}
As you can see the key is the actual password and in order to decode the encrypted file following command line should be executed:
openssl aes-128-ecb -in test.enc -K 000102030405060708090A0B0C0D0E0F -d
"000102030405060708090A0B0C0D0E0F" is a password hex representation I use in the code above 0123456789191112131415. As I understand the password can be encrypted as well using MD5 algorithm.
The question is how to encrypt data using actual KEY derived from password and not the password itself using libcrypto?
Take a look at EVP_BytesToKey.
My comments in an old app tell me that BytesToKey is out of date and you should perhaps consider looking at PKCS5_v2_PBE_keyivgen or similar. But essentially, a highly simplified way of doing it is you derive your Key as a hash from your password and a suitable salt value:
EVP_DigestInit_ex(...)
EVP_DigestUpdate(...Password...)
EVP_DigestUpdate(...Salt...)
EVP_DigestFinal_ex(...)
then you use the newly generated Key to derive your IV by:
EVP_DigestInit_ex(...)
EVP_DigestUpdate(...Key...)
EVP_DigestUpdate(...Password...)
EVP_DigestUpdate(...Salt...)
EVP_DigestFinal_ex(...)
A browse of the OpenSSL source code is most useful for looking up stuff like this.
Disclaimer: I'm not a C programmer (the app in question was Delphi using OpenSSL DLLs) nor a security expert so take these suggestions as a starting point, read the proper docs and use proper functions where possible!!...

How do I decrypt a private key file and sign some text using openssl calls in C?

I have 2 separate programs (spliced together below). The first generates the key pair and saves to files (works fine). The second opens the private key, decrypting with a pass phrase and then I need it to sign a string of text. The code below fails on the PEM_read_PrivateKey() (last) call (can't see why). Can anyone point me at what I am doing wrong and then what openssl calls I should make to use the private key to sign some text?
int main (int argc, char *argv[])
{
char *priv_pem = "priv.pem";
char *pub_pem = "pub.pem";
char *pass = "Password";
FILE *fp;
int bits = 4096;
unsigned long exp = RSA_F4;
RSA *rsa;
EVP_PKEY *pkey;
// GENERATE KEY
rsa=RSA_generate_key(bits,exp,NULL,NULL);
if (RSA_check_key(rsa)!=1)
Exit(1,"Error whilst checking key","");
pkey = EVP_PKEY_new();
EVP_PKEY_assign_RSA(pkey, rsa);
// WRITE ENCRYPTED PRIVATE KEY
if (!(fp = fopen(priv_pem, "w")))
Exit(2,"Error opening PEM file",priv_pem);
if (!PEM_write_PrivateKey(fp,pkey,EVP_aes_256_cbc(),NULL,0,NULL,pass))
Exit(3,"Error writing PEM file",priv_pem);
fclose(fp);
// WRITE PUBLIC KEY
if (!(fp = fopen(pub_pem, "w")))
Exit(4,"Error opening PEM file",pub_pem);
if (!PEM_write_PUBKEY(fp, pkey))
Exit(5,"Error writing PEM file",pub_pem);
fclose(fp);
// ------- End of key generation program -------
// ------- Start of text signing program -------
// READ IN ENCRYPTED PRIVATE KEY
if (!(fp = fopen(priv_pem, "r")))
Exit(6,"Error reading encrypted private key file",priv_pem);
if (!PEM_read_PrivateKey(fp,&pkey,NULL,pass))
Exit(7,"Error decrypting private key file",priv_pem);
fclose(fp);
// Sign some text using the private key....
// FREE
RSA_free(rsa);
return 0;
}
Have you initialised pkey to NULL before you pass &pkey to PEM_read_PrivateKey()? If not, it will attempt to re-use the EVP_PKEY structure that pkey points to - and if pkey is uninitialised, it will be looking at a random spot in memory.
You can use ERR_print_errors_fp(stderr); to dump the OpenSSL error stack to stderr when an error occurs - this is often helpful in finding the problem.
Thanks #caf for your help.
By trial and error I fixed PEM_read_PrivateKey() error by adding the following to the start:
if (EVP_get_cipherbyname("aes-256-cbc") == NULL)
OpenSSL_add_all_algorithms();
However, I'm still looking for the best (practice) way of generating the keys and then using the private key for signing. From my limited understanding, I am looking for openssl methods that adhere to RSA's "PKCS #1 v2.0: RSA Cryptography Standard"

Resources