Set up a specific key for RSA OpenSSL - c

I'm currentlyt trying to implement some test vectors for RSA implementations, and I wanted to test them over OpenSSL v1.1.0f implementation. However, when I try to set up the key (for e, n p, q or d), I have the following error :
erreur : dereferencing pointer to incomplete type « RSA {alias struct rsa_st} »
My code is the following :
int rsa_encrypt(byte *in, size_t in_len, byte *out, const char *n, const char *e, char padding){
int err = -1;
RSA *keys = NULL;
keys = RSA_new();
BN_hex2bn(&keys->n, n); // error here
BN_hex2bn(&keys->e, e); // and here
out = malloc(RSA_size(keys));
if (padding == OAEP ) {
err = RSA_public_encrypt(in_len, in, out, keys, RSA_PKCS1_OAEP_PADDING);
}
else if (padding == v1_5) {
err = RSA_public_encrypt(in_len, in, out, keys, RSA_PKCS1_PADDING);
}
RSA_free(keys);
return err;
}
Where n and e are a string representing my parameter in hexa.
I've looked for the structure corresponding to the RSA type, and found this.
I don't understand why I can't set up n and e... Any idea ?

The OpenSSL structs are opaque in 1.1.x, you can't talk about their fields. Among other things, it allows for new struct fields to be added during servicing releases, since caller code can't know the field offset.
For 1.1.x you'd need to do something akin to
BN* bnN = NULL;
BN* bnE = NULL;
RSA* keys = RSA_new();
BN_hex2bn(&bnN, n);
BN_hex2bn(&bnE, e);
RSA_set0_key(keys, bnN, bnE, NULL);
...
RSA_free(keys);
// do not free bnN or bnE, they were freed by RSA_free.
Note that I left out the requisite error checking.

Related

Error: "invalid use of incomplete type ‘RSA {aka struct rsa_st}" in OpenSSL 1.1.0

I have old code that was written to link against an old version of openssl. Part of this code loads a key from a PEM file, and tries to understand whether this key is a private or public key, by using the following code:
if( (prv->p==0 || prv->q==0) ) {
// This is not a private key!
throw error("No private key for decryption");
}
With the latest version of openssl, this (justifiably) doesn't compile:
crypto.cpp: In function ‘key* decrypt_header(file_t, RSA*)’:
crypto.cpp:158:13: error: invalid use of incomplete type ‘RSA {aka struct rsa_st}’
if( (prv->p==0 || prv->q==0) ) {
^~
I understand that direct access to the struct's private members was replaced with a function, but I am having a hard time figuring out which function that is.
crypto.cpp:158:13: error: invalid use of incomplete type ‘RSA {aka struct rsa_st}’
if( (prv->p==0 || prv->q==0) ) {
^~
As you are aware, OpenSSL 1.1.0 changed the visibility of a lot of struct members. You can no longer access the members directly. Instead, you have to use getter and setter functions.
Try RSA_get0_factors. The get0 means the reference counts are not incremented. Do not BN_free them.
void RSA_get0_factors(const RSA *r, const BIGNUM **p, const BIGNUM **q);
If the code supports multiple versions of OpenSSL, then you will need a guard because RSA_get0_factors is for OpenSSL 1.1.0 and above. Maybe something like the following. Also see OPENSSL_VERSION_NUMBER man page.
#include <openssl/opensslv.h>
#if OPENSSL_VERSION_NUMBER < 0x10100000L
/* OpenSSL 1.0.2 and below (old code) */
#else
/* OpenSSL 1.1.0 and above (new code) */
#endif
After 1.1.1 OpenSSL Supports getter what return each parameter like this.
const BIGNUM *RSA_get0_n(const RSA *d);
const BIGNUM *RSA_get0_e(const RSA *d);
const BIGNUM *RSA_get0_d(const RSA *d);
const BIGNUM *RSA_get0_p(const RSA *d);
const BIGNUM *RSA_get0_q(const RSA *d);
const BIGNUM *RSA_get0_dmp1(const RSA *r);
const BIGNUM *RSA_get0_dmq1(const RSA *r);
const BIGNUM *RSA_get0_iqmp(const RSA *r);
So if you don't need to considerate below version of OpenSSL less than 1.1.1 these code will make simple code. AND other structures support kind of this getter too. You can find more information of functions this. https://www.openssl.org/docs/man1.1.1/man3/
#if OPENSSL_VERSION_NUMBER < 0x10100005L
static void RSA_get0_key(const RSA *r, const BIGNUM **n, const BIGNUM **e, const BIGNUM **d, const BIGNUM **p, const BIGNUM **q)
{
if(n != NULL)
*n = r->n;
if(e != NULL)
*e = r->e;
if(d != NULL)
*d = r->d;
if(p != NULL)
*p = r->p;
if(q != NULL)
*q = r->q;
}
#endif
const BIGNUM *bn_p;
const BIGNUM *bn_q;
RSA_get0_key(key, NULL, NULL, NULL, &bn_p, &bn_q);
/* if( (prv->p==0 || prv->q==0) ) { */
if( (prv_p==0 || prv_q==0) ) {

Segmentation fault with generating an RSA and saving in ASN.1/DER?

#include <string.h>
#include <openssl/aes.h>
#include <openssl/rand.h>
#include <openssl/bio.h>
#include <openssl/rsa.h>
#include <openssl/evp.h>
#include <openssl/pem.h>
#define RSA_LEN 2048
#define RSA_FACTOR 65537
int genRSA2048(unsigned char **pub,unsigned int *pub_l,unsigned char **priv,unsigned int *priv_l){
RSA *pRSA = NULL;
pRSA = RSA_generate_key(RSA_LEN,RSA_FACTOR,NULL,NULL);
if (pRSA){
pub_l = malloc(sizeof(pub_l));
*pub_l = i2d_RSAPublicKey(pRSA,pub);
priv_l = malloc(sizeof(priv_l));
*priv_l = i2d_RSAPrivateKey(pRSA,priv);
return 0;
} else {
return 1;
}
}
int main(){
unsigned char *pub = NULL;
unsigned int publ;
unsigned char *priv = NULL;
unsigned int privl;
genRSA2048(&pub,&publ,&priv,&privl);
RSA *privrsa = NULL;
d2i_RSAPrivateKey(&privrsa,(const unsigned char **)&priv,privl);
RSA *pubrsa = NULL;
d2i_RSAPublicKey(&pubrsa,(const unsigned char **)&pub,publ);
unsigned char * data ="01234567890123456789012345678912";
unsigned char encrypted[256];
unsigned char decrypted[32];
int len = RSA_private_encrypt(32,data,encrypted,privrsa,RSA_PKCS1_PADDING);
RSA_public_decrypt(len,encrypted,decrypted,pubrsa,RSA_PKCS1_PADDING);
}
I've tried to find the bug by checking with gdb but as being fairly new to C I haven't find any clue to tell me what is happening but I believe it's an allocation problem, however according to the d2i_RSAPrivateKey and similar, they're supposed to allocate the space by itself.
Any help would be greatly appreciated.
compiled as cc foo.c -lcrypto
This is the follow up of this question:
Generate RSA public/private key with OpenSSL?
As I reference I used #WhozCraig example in the comments, which can be found here, even when it's quite different, it was a lot of help.
http://coliru.stacked-crooked.com/a/ae64a70076436165
pub_l = malloc(sizeof(pub_l)); is simply not needed. Nor is priv_l = malloc(sizeof(priv_l));. Remove them both from your function.
You should be populating your out-parameters; instead you're throwing out the caller's provided addresses to populate and (a) populating your own, then (b) leaking the memory you just allocated.
The result is the caller's privl and publ are untouched and thus the decoding back to RSA is dysfunctional, as both values are indeterminate.
pRSA = RSA_generate_key(RSA_LEN,RSA_FACTOR,NULL,NULL);
I think this is wrong. I know you are supposed to use RSA_generate_key_ex, and I think it needs a BIGNUM, not an integer. You should have gotten a warning. See RSA_generate_key(3) for details.
Your code should look something like:
BIGNUM* exp = BN_new();
ASSERT(exp != NULL);
int rc = BN_set_word(exp, RSA_F4);
ASSERT(rc == 1);
RSA* rsa = RSA_new();
ASSERT(rsa != NULL);
rc = RSA_generate_key_ex(rsa, 2048, exp, NULL);
ASSERT(rc == 1);
Be sure to call BN_free on the BIGNUM, and RSA_free on the RSA pointer.
RSA *privrsa = NULL;
d2i_RSAPrivateKey(&privrsa,(const unsigned char **)&priv,privl);
RSA *pubrsa = NULL;
d2i_RSAPublicKey(&pubrsa,(const unsigned char **)&pub,publ);
For this, it looks like you are trying to separate the public key and private key. For that, use RSAPublicKey_dup and RSAPrivateKey_dup. See Separating public and private keys from RSA keypair variable.
Its not clear to me what you are trying to do with the following. You should state what you are trying to do...
pub_l = malloc(sizeof(pub_l));
*pub_l = i2d_RSAPublicKey(pRSA,pub);
priv_l = malloc(sizeof(priv_l));
*priv_l = i2d_RSAPrivateKey(pRSA,priv);
I'm just guessing, but I'm going to say its all wrong. sizeof(priv_l) is the size of a pointer, so its 4 or 8 bytes. You're also overwriting the pointer passed in by the caller...
Also see OpenSSL's rsautl cannot load public key created with PEM_write_RSAPublicKey. It talks about saving the keys with SubjectPublicKeyInfo and PrivateKeyInfo in both ASN.1/DER and PEM formats.
By writing the {Public|Private}KeyInfo, the OID gets written to the key. That's important for interop. You also use the RSA* (and even an EVP_PKEY*), and not byte arrays.

PEM_read_RSAPrivateKey: Getting RSA key public modulus and exponent

I'm using PEM_read_RSAPrivateKey function in this way:
void test(void)
{
RSA * privateKey = NULL;
FILE * fp;
if(NULL != (fp= fopen("./my_file.key", "r")) )
{
privateKey=PEM_read_RSAPrivateKey(fp,NULL,NULL,NULL);
if(privateKey==NULL)
{
printf("\n\tCould NOT read RSA private key file");
}
else
{
printf("\n\tRSA structure filled");
}
// This is working OK and privateKey is NOT NULL
}
}
Then, I try to retrieve modulus and public exponent to fill them into a personal structure:
struct
{
unsigned char modulus[256];
unsigned char pub_exp[8];
} s;
But all accesses I tried (I tried a lot) to privateKey->n will result in a segmentation fault.
for example:
unsigned char modulus [2048];
unsigned char exp[2048];
BN_bn2bin(privateKey->n, modulus); // Segmentation fault results from this call
So my question is: how to copy modulus or public exponent from RSA structure to my structure "s" fields?
May someone help about this?
Many thanks,
Regards,
Sylvain
how to copy modulus or public exponent from RSA structure
int req = BN_num_bytes(rsa->n);
assert(rc > 0);
unsigned char* buff = malloc(req);
assert(buff != NULL);
int rc = BN_bn2bin(rsa->n, buff);
assert(req == rc);
Be wary of trying to copy the byte buffers into fixed sized arrays. Someone might come along and get you to copy a 4096-bit modulus into your 2048-bit array.
From OpenSSL 1.1.0 many structures were made opaque and you can no longer do things like look directly at the fields in a struct.
You should use the provided accessor functions instead. Here you can find more infos: https://wiki.openssl.org/index.php/OpenSSL_1.1.0_Changes
Acces rsa->n in OpenSSL 1.1.0:
RSA *pRsa;
BIGNUM *n;
FILE *pFile = fopen("private.key","r");
pRsa = PEM_read_RSAPrivateKey(pFile, NULL, NULL, password);
RSA_get0_key(pRSA, &n, NULL,NULL);
BN_print_fp(stdout, n);

How does one access the raw ECDH public key, private key and params inside OpenSSL's EVP_PKEY structure?

I'm using OpenSSL's c library to generate an elliptic curve Diffie-Hellman (ECDH) key pair, following the first code sample here. It glosses over the actual exchange of public keys with this line:
peerkey = get_peerkey(pkey);
The pkey variable and the return value are both of type EVP *. pkey contains the public key, private key, and params generated earlier, and the return value only contains the peer's public key. So this raises three questions:
How would get_peerkey() actually extract just the public key from pkey for sending to the peer?
How would the code extract the private key and params from pKey to store them for later use after the key exchange?
How would get_peerkey() generate a new EVP_PKEY structure from the peer's raw public key?
I've seen the OpenSSL functions EVP_PKEY_print_public(), EVP_PKEY_print_private(), and EVP_PKEY_print_params() but these are for generating human-readable output. And I haven't found any equivalent for converting a human-readable public key back into an EVP_PKEY structure.
To answer my own question, there's a different path for the private key and the public key.
To serialize the public key:
Pass the EVP_PKEY to EVP_PKEY_get1_EC_KEY() to get an EC_KEY.
Pass the EC_KEY to EC_KEY_get0_public_key() to get an EC_POINT.
Pass the EC_POINT to EC_POINT_point2oct() to get octets, which are just unsigned char *.
To deserialize the public key:
Pass the octets to EC_POINT_oct2point() to get an EC_POINT.
Pass the EC_POINT to EC_KEY_set_public_key() to get an EC_KEY.
Pass the EC_KEY to EVP_PKEY_set1_EC_KEY to get an EVP_KEY.
To serialize the private key:
Pass the EVP_PKEY to EVP_PKEY_get1_EC_KEY() to get an EC_KEY.
Pass the EC_KEY to EC_KEY_get0_private_key() to get a BIGNUM.
Pass the BIGNUM to BN_bn2mpi() to get an mpi, which is a format written to
unsigned char *.
To deserialize the private key:
Pass the mpi to BN_mpi2bn() to get a BIGNUM.
Pass the BIGNUM to EC_KEY_set_private_key() to get an EC_KEY.
Pass the EC_KEY to EVP_PKEY_set1_EC_KEY to get an EVP_KEY.
It is also possible to convert the BIGNUM to hex, decimal, or "bin", although I think that mpi used the fewest bytes.
OpenSSL 3.x.x
To serialize the public key:
// We assume the public and private keys have been already generated.
// EVP_PKEY* keyPair...
// Get the serialized public key length.
size_t serializedPublicKeyLen = 0;
if (EVP_PKEY_get_octet_string_param(keyPair, OSSL_PKEY_PARAM_PUB_KEY,
NULL, 0, &serializedPublicKeyLen) != 1) {
return;
}
// Allocate memory for the serialized public key.
unsigned char* serializedPublicKey = (unsigned char*)OPENSSL_malloc(serializedPublicKeyLen);
if (serializedPublicKey == NULL) {
return;
}
// Get the serialized public key.
if (EVP_PKEY_get_octet_string_param(keyPair, OSSL_PKEY_PARAM_PUB_KEY,
serializedPublicKey, serializedPublicKeyLen, &serializedPublicKeyLen) != 1) {
return;
}
// Deallocate the memory when you finish using the serialized public key.
OPENSSL_free(serializedPublicKey);
To deserialize the public key:
// A parameter build for the public key.
OSSL_PARAM_BLD* paramBuild = OSSL_PARAM_BLD_new();
if (paramBuild == NULL) {
return;
}
// This is just an example. Set the curve
// you used to generate the public and private keys.
const char curveName[] = "secp384r1";
// Set the curve name to the parameter build.
if (OSSL_PARAM_BLD_push_utf8_string(paramBuild,
OSSL_PKEY_PARAM_GROUP_NAME, curveName, 0) != 1) {
OSSL_PARAM_BLD_free(paramBuild);
return;
}
// Set the serialized public key.
if (OSSL_PARAM_BLD_push_octet_string(paramBuild, OSSL_PKEY_PARAM_PUB_KEY,
serializedPublicKey, serializedPublicKeyLen) != 1) {
OSSL_PARAM_BLD_free(paramBuild);
return;
}
// Convert the OSSL_PARAM_BLD to an OSSL_PARAM.
OSSL_PARAM* params = OSSL_PARAM_BLD_to_param(paramBuild);
if (params == NULL) {
OSSL_PARAM_BLD_free(paramBuild);
return;
}
// Create a EVP_PKEY context.
EVP_PKEY_CTX* publicKeyCtx = EVP_PKEY_CTX_new_from_name(NULL, "EC", NULL);
if (publicKeyCtx == NULL) {
OSSL_PARAM_BLD_free(paramBuild);
OSSL_PARAM_free(params);
return;
}
// Initialize the EVP_PKEY context.
if (EVP_PKEY_fromdata_init(publicKeyCtx) <= 0) {
OSSL_PARAM_BLD_free(paramBuild);
OSSL_PARAM_free(params);
EVP_PKEY_CTX_free(publicKeyCtx);
return;
}
// Create the peer public key object.
EVP_PKEY* publicKey = NULL;
if (EVP_PKEY_fromdata(publicKeyCtx, &publicKey,
EVP_PKEY_PUBLIC_KEY, params) <= 0) {
OSSL_PARAM_BLD_free(paramBuild);
OSSL_PARAM_free(params);
EVP_PKEY_CTX_free(publicKeyCtx);
return;
}
// Free auxiliary things...
OSSL_PARAM_BLD_free(paramBuild);
OSSL_PARAM_free(params);
EVP_PKEY_CTX_free(publicKeyCtx);
// Now you can use publicKey for EVP_PKEY_derive_set_peer.
// Call EVP_PKEY_free when you finish using it.
A full C++ OpenSSL 3 example: ec-diffie-hellman-openssl.h, ec-diffie-hellman-openssl.cpp.
A full C++ LibreSSL 3/OpenSSL 1 example: ec-diffie-hellman-libressl.h, ec-diffie-hellman-libressl.cpp.
To serialize the private key, you get BIGNUM instead:
BIGNUM* privateKey = NULL;
EVP_PKEY_get_bn_param(keyPair, OSSL_PKEY_PARAM_PRIV_KEY, &privateKey);
Then, you use one of the BIGNUM serialization function: https://www.openssl.org/docs/man3.0/man3/BN_bn2bin.html
To deserialize the private key, you use one of the BIGNUM deserialization function from the link above, then push it to the parameter build via OSSL_PARAM_BLD_push_BN with OSSL_PKEY_PARAM_PRIV_KEY.
The implementation above seems too complicated. openssl/evp.h has functions i2d_PublicKey() and d2i_PublicKey() to respectively convert to and from a binary representation of the public key (and there are equivalent functions for the private key - see: https://www.openssl.org/docs/manmaster/man3/d2i_PublicKey.html)
A small code example:
vector<unsigned char> ecdhPubkeyData(EVP_PKEY *key)
{
int len = i2d_PublicKey(key, 0); // with 0 as second arg it gives length
vector<unsigned char> ret(len);
unsigned char *ptr = ret.data();
len = i2d_PublicKey(key, &ptr);
return ret;
}
// Make sure you free the returned pointer when you are done with it
EVP_PKEY *ecdhPubkeyFromData(vector <unsigned char> const &pubkeyData)
{ // You do need to put in in an existing EVP_PKEY that is assigned
// an EC_KEY, because it needs to know what curve you use
EC_KEY *ec_key = EC_KEY_new_by_curve_name(NID_X9_62_prime256v1);
EVP_PKEY *ret = EVP_PKEY_new();
EVP_PKEY_assign_EC_KEY(ret, ec_key);
unsigned char const *ptr = pubkeyData.data();
d2i_PublicKey(EVP_PKEY_EC, &ret, &ptr, pubkeyData.size());
return ret;
}
// PS: In a real example you want to check if any of these functions
// return NULL or some error code
I am using C++ vectors to contain the binary data, but of course you could just use C-style arrays too :-)
I am absolutely not an OpenSSL expert, so let me know if I am doing something horribly wrong in this implementation :-p

Reading PEM-formatted RSA keyfile with the OpenSSL C API

I am trying to read in an AES-128-CBC encrypted PEM key file generated using Ruby with the OpenSSL API. The code that generated the PEM key is the following:
OpenSSL::PKey::RSA.new(2048).to_pem(OpenSSL::Cipher::AES.new('128-CBC'), "password")
Here's the code that reads the PEM file:
RSA *rsa;
BIO *pem = BIO_new(BIO_s_mem());
BIO_puts(pem, "-----BEGIN RSA PRIVATE KEY-----\n"
"Proc-Type: 4,ENCRYPTED\n"
"DEK-Info: AES-128-CBC,BB13D39833DD6ED1FF9843644E7981EE\n\n
"Eugt8JZNKQKErsabWkwfm3wQhU/Tmp9T0QaP5HM8VIWpZwKlDmRlSUDptADU6RPD\n"
"5VtG3DPieXcf+6deyARImid9sBBmQ9mK2omkNRcTMemqTOhAuaKBu78TMt9G4YSf\n"
"RjoXWSqu3jMwrlcGkpn7bIum8wImITRZ3p28oSzk9aDUNBrIU/2Si8DM4RYIZ/fK\n"
"Uvvgdok9dgcd0SjvucivX2HaGeg/IUz23q1jg9inDpimZvFJD1FJfGEUWDgyfJfa\n"
"M8JIxTKbWOPEopONDkT7u4dC5VcSjK29MVbfd7iCKFPMh5UN+c96rPxTng/OWyW5\n"
"0tvzHyyyvAG9p0Hx5Lr4pDbv21GHyu43sA6wbs9jWyqO3AB7CaoEEQhumwfLsdjj\n"
"YGrX6bWThpYv/XNBDmmvltHlKFfe01NCybivOb4KwBnvi45x21PBqaZCKDTFdEkL\n"
"iwDMTiG2iTxSUvPFLy30VFozE+pGyMcGDUyZDVqjsaqI/MRj8khnn5nyubXc27G3\n"
"8Kbsnlix2SW2M0VDxqiy9dyjcxXrkFRSnOFYVs1PFlgjFVTG4Mwh6CZxKw8mFVbi\n"
"EmLvUYwzoDZ1ve4VXSPp/vrKEh33JuHhM0vJOpqI6wqw0QR0I2o6etM1ZRJClPcw\n"
"VIcgcvwenEgLOkoHDqOr0IZQAtYWvAuqq822wKt258hc6z8+ALQf5iMroqk7ADd4\n"
"FlRLz4XTwqlg7pPtTde/emI1DT8dQWzq++QI0lr0CS/N1GXJKqTQDvauXLIiI3Qy\n"
"KfFYFpV9jyYfRfTjNtisI/edPtp98auK0mb9o/wS/hruFI9behgv63iW1IwAOXCW\n"
"ZlkWgobUH13gS864rL+AcrAXreo2j4dDQouTeRaJUEG0HoYTP65Zun/VsCi2aSOH\n"
"JwSnnmHz9OxvcGY80WJDN3kqOCBRIJoDKBv6jcOxGVCsVK+WSdGZ7cfb8lwp7aA8\n"
"8ND1bwL9FYkwkeIsoakj91iinqv4o3+3PUPgCU5oe68WYvAFjuU+criyf+EhmXJV\n"
"JQ1vFFZPrGzgntJz19uXXh1h2iwQPggRouJm2RozYwvv1nz4eQ40Y3eT1F9UOYJU\n"
"CKEhOtI2NpLeVOayqo8g9wO2oC+CQVhZhdYBE5o7pM7akFnYLvRg9s1UsWdcvT0G\n"
"IpFmejLSRJ/F954aQMHTUc6vBOJZH/VNC5Qt+ulFXl634Sr9wQQK2qlqSJyA04TR\n"
"1ixbCNOX71esvpFImsrlsO5oTA22T3h2GyJPUM10XhqGtDXtsTnal6smLna9U9B3\n"
"gTVxFWWukQOF5Lm8ZFQipo2loHWjkozTBc4REPYP44SoXJXstv7k4pt1cK7x6/2H\n"
"ElspXzjveqMhcrveWv1KaA2OGd+hGfUiNsCoIdapJjLz1Bd/+oIQ/ZWQeo0nRowE\n"
"R/HlbbED3V+fRIdJpgydFEAw6gK5E9sYJcgF7uf/n2NabFxxEZL3g6MJQ64Dtusg\n"
"DEH/MpvIYDSX4Navh1gTwCtOeG1CzW3diYaqbZK+UZCBLFU7j27YvVPSd6F2+Wud\n"
"WnAqU3S5BCPqk5OD3wqZv+sEcqJgGPGy1Gv0tl8ARJomdKAru03KsRn2eIWqR5/C\n"
"-----END RSA PRIVATE KEY-----\n");
// Retrieve RSA key from PEM file.
rsa = PEM_read_bio_RSAPrivateKey(pem, NULL, pem_password_callback, "password");
And here's a dummy password callback (not sure about this function's purpose, but I think it may return the length of the password):
int pem_password_callback(char *buf, int max_len, int flag, void *pwd)
{
return 8;
}
Currently, the rsa = ... part does not throw an error, but doesn't return a well-formed result either.
And here's a dummy password callback (not sure about this function's
purpose, but I think it may return the length of the password):
int pem_password_callback(char *buf, int max_len, int flag, void *pwd)
{
return 8;
}
No, its not a dummy. In your example, you just returned a buffer with 8 junk characters (whatever happened to be in buf).
The password callback is where you programmatically plug in your password. You are supplied with a buff of max_len, and you need to copy the password into the buffer and return the number of bytes copied.
int pem_password_callback(char *buf, int max_len, int flag, void *ctx)
{
const char* PASSWD = "password";
int len = strlen(PASSWD);
if(len > max_len)
return 0;
memcpy(buf, PASSWD, len);
return len;
}
The flag is a read/write flag to denote if you are reading a key or writing a key. In practice, I have never used it.
You will use it similar to:
FILE* file = ...;
EVP_PKEY* pkey = PEM_read_PrivateKey(file, NULL, pem_password_callback, NULL);
Unlike the write routine (which needs an EVP_* cipher), the read routine knows what you used to encrypt the key with because its encoded in the private key.
In my systems, I actually use the context for a label to ensure the same passwords arrive at different derived keys:
EVP_PKEY* pkey = PEM_read_PrivateKey(file, NULL, pem_password_callback, "Some Context");
Then, in my password callback:
int pem_password_callback(char *buf, int max_len, int flag, void *ctx)
{
// "Some Context" in the example above
char* label = (char*)ctx;
// Hash password and label
// ...
// Copy hash to buffer, return length
...
}

Resources