Reading PEM-formatted RSA keyfile with the OpenSSL C API - c

Question

I am trying to read in an AES-128-CBC encrypted PEM key file generated using Ruby with the OpenSSL API. The code that generated the PEM key is the following:
OpenSSL::PKey::RSA.new(2048).to_pem(OpenSSL::Cipher::AES.new('128-CBC'), "password")
Here's the code that reads the PEM file:
RSA *rsa;
BIO *pem = BIO_new(BIO_s_mem());
BIO_puts(pem, "-----BEGIN RSA PRIVATE KEY-----\n"
"Proc-Type: 4,ENCRYPTED\n"
"DEK-Info: AES-128-CBC,BB13D39833DD6ED1FF9843644E7981EE\n\n
"Eugt8JZNKQKErsabWkwfm3wQhU/Tmp9T0QaP5HM8VIWpZwKlDmRlSUDptADU6RPD\n"
"5VtG3DPieXcf+6deyARImid9sBBmQ9mK2omkNRcTMemqTOhAuaKBu78TMt9G4YSf\n"
"RjoXWSqu3jMwrlcGkpn7bIum8wImITRZ3p28oSzk9aDUNBrIU/2Si8DM4RYIZ/fK\n"
"Uvvgdok9dgcd0SjvucivX2HaGeg/IUz23q1jg9inDpimZvFJD1FJfGEUWDgyfJfa\n"
"M8JIxTKbWOPEopONDkT7u4dC5VcSjK29MVbfd7iCKFPMh5UN+c96rPxTng/OWyW5\n"
"0tvzHyyyvAG9p0Hx5Lr4pDbv21GHyu43sA6wbs9jWyqO3AB7CaoEEQhumwfLsdjj\n"
"YGrX6bWThpYv/XNBDmmvltHlKFfe01NCybivOb4KwBnvi45x21PBqaZCKDTFdEkL\n"
"iwDMTiG2iTxSUvPFLy30VFozE+pGyMcGDUyZDVqjsaqI/MRj8khnn5nyubXc27G3\n"
"8Kbsnlix2SW2M0VDxqiy9dyjcxXrkFRSnOFYVs1PFlgjFVTG4Mwh6CZxKw8mFVbi\n"
"EmLvUYwzoDZ1ve4VXSPp/vrKEh33JuHhM0vJOpqI6wqw0QR0I2o6etM1ZRJClPcw\n"
"VIcgcvwenEgLOkoHDqOr0IZQAtYWvAuqq822wKt258hc6z8+ALQf5iMroqk7ADd4\n"
"FlRLz4XTwqlg7pPtTde/emI1DT8dQWzq++QI0lr0CS/N1GXJKqTQDvauXLIiI3Qy\n"
"KfFYFpV9jyYfRfTjNtisI/edPtp98auK0mb9o/wS/hruFI9behgv63iW1IwAOXCW\n"
"ZlkWgobUH13gS864rL+AcrAXreo2j4dDQouTeRaJUEG0HoYTP65Zun/VsCi2aSOH\n"
"JwSnnmHz9OxvcGY80WJDN3kqOCBRIJoDKBv6jcOxGVCsVK+WSdGZ7cfb8lwp7aA8\n"
"8ND1bwL9FYkwkeIsoakj91iinqv4o3+3PUPgCU5oe68WYvAFjuU+criyf+EhmXJV\n"
"JQ1vFFZPrGzgntJz19uXXh1h2iwQPggRouJm2RozYwvv1nz4eQ40Y3eT1F9UOYJU\n"
"CKEhOtI2NpLeVOayqo8g9wO2oC+CQVhZhdYBE5o7pM7akFnYLvRg9s1UsWdcvT0G\n"
"IpFmejLSRJ/F954aQMHTUc6vBOJZH/VNC5Qt+ulFXl634Sr9wQQK2qlqSJyA04TR\n"
"1ixbCNOX71esvpFImsrlsO5oTA22T3h2GyJPUM10XhqGtDXtsTnal6smLna9U9B3\n"
"gTVxFWWukQOF5Lm8ZFQipo2loHWjkozTBc4REPYP44SoXJXstv7k4pt1cK7x6/2H\n"
"ElspXzjveqMhcrveWv1KaA2OGd+hGfUiNsCoIdapJjLz1Bd/+oIQ/ZWQeo0nRowE\n"
"R/HlbbED3V+fRIdJpgydFEAw6gK5E9sYJcgF7uf/n2NabFxxEZL3g6MJQ64Dtusg\n"
"DEH/MpvIYDSX4Navh1gTwCtOeG1CzW3diYaqbZK+UZCBLFU7j27YvVPSd6F2+Wud\n"
"WnAqU3S5BCPqk5OD3wqZv+sEcqJgGPGy1Gv0tl8ARJomdKAru03KsRn2eIWqR5/C\n"
"-----END RSA PRIVATE KEY-----\n");
// Retrieve RSA key from PEM file.
rsa = PEM_read_bio_RSAPrivateKey(pem, NULL, pem_password_callback, "password");
And here's a dummy password callback (not sure about this function's purpose, but I think it may return the length of the password):
int pem_password_callback(char *buf, int max_len, int flag, void *pwd)
{
return 8;
}
Currently, the rsa = ... part does not throw an error, but doesn't return a well-formed result either.

Answer 1

And here's a dummy password callback (not sure about this function's
purpose, but I think it may return the length of the password):
int pem_password_callback(char *buf, int max_len, int flag, void *pwd)
{
return 8;
}
No, its not a dummy. In your example, you just returned a buffer with 8 junk characters (whatever happened to be in buf).
The password callback is where you programmatically plug in your password. You are supplied with a buff of max_len, and you need to copy the password into the buffer and return the number of bytes copied.
int pem_password_callback(char *buf, int max_len, int flag, void *ctx)
{
const char* PASSWD = "password";
int len = strlen(PASSWD);
if(len > max_len)
return 0;
memcpy(buf, PASSWD, len);
return len;
}
The flag is a read/write flag to denote if you are reading a key or writing a key. In practice, I have never used it.
You will use it similar to:
FILE* file = ...;
EVP_PKEY* pkey = PEM_read_PrivateKey(file, NULL, pem_password_callback, NULL);
Unlike the write routine (which needs an EVP_* cipher), the read routine knows what you used to encrypt the key with because its encoded in the private key.
In my systems, I actually use the context for a label to ensure the same passwords arrive at different derived keys:
EVP_PKEY* pkey = PEM_read_PrivateKey(file, NULL, pem_password_callback, "Some Context");
Then, in my password callback:
int pem_password_callback(char *buf, int max_len, int flag, void *ctx)
{
// "Some Context" in the example above
char* label = (char*)ctx;
// Hash password and label
// ...
// Copy hash to buffer, return length
...
}