I'm trying to write a X509 Cert to DER format in memory.
Writing it to a file works perfectly.
I need the Cert in PEM format without the "-----BEGIN PRIVATE KEY-----" header, footer or newlines. I can't figure out how to do it directly so...
I'm outputting to der and base64 encoding.
THIS WORKS.
int X509_to_DER_file(X509 *cert) {
int res=0;
out = BIO_new(BIO_s_file());
if (NULL != out) {
if(BIO_write_filename(out, "my.der") > 0) {
res = i2d_X509_bio(out, cert);
}
BIO_free_all(out);
}
return (tres);
}
THIS DOES NOT.
It returns and mallocs the correct number of bytes and appears to write out to memory correctly but the resulting string is incorrect (the first 15 or so positions are correct).
char *X509_to_DER_mem(X509 *cert) {
char *der = NULL;
bio = BIO_new(BIO_s_mem());
if (NULL != bio) {
//load cert into bio
if (0 == i2d_X509_bio(bio, cert)) {
BIO_flush(bio);
BIO_free(bio);
return NULL;
}
der = (char *) malloc(bio->num_write + 1);
if (NULL == der) {
BIO_free(bio);
return NULL;
}
memset(der, 0, bio->num_write + 1);
BIO_read(bio, der, bio->num_write);
// Appears to work put "der" is incomplete.
BIO_free(bio);
}
return der;
}
It returns and mallocs the correct number of bytes and appears to
write out to memory correctly but the resulting string is incorrect
The result of i2d_X509_bio() is not a (zero-terminated) string, but a bunch of bytes. If you try to write it to a file as a string, it might look incomplete because you might encounter a 0-byte at a location before you reach the end. So in addition to the char * result, your function X509_to_DER_mem() will have to return the number of bytes that make up the result.
With regard to the memory BIO, another way of obtaining its data is with the BIO_get_mem_data() function. Something like this:
char *ptr = NULL;
long len = BIO_get_mem_data(bio, &ptr);
der = malloc(len);
memcpy(der, ptr, len);
Finally, your actual question is
I need the Cert in PEM format without the "-----BEGIN PRIVATE
KEY-----" header, footer or newlines.
Writing the certificate in DER format does not seem to give you what you need. This answer to another SO question explains how you could use the function PEM_read_bio() in combination with EVP_EncodeBlock() for that purpose.
Related
I'm trying to fetch 3 keys from SPIFFS that I stored from a string to a file (when fetched from an API endpoint).
I am able to read the files using the SPIFFS library and print the values out in the serial console. When I check the contents, they are just fine. When I use the contents in the secureClient.setCACert() function, they "break" and throw an exception; E][ssl_client.cpp:36] _handle_error(): [start_ssl_client():138]: (-8576) X509 - The CRT/CRL/CSR format is invalid, e.g. different type expected
My code looks like the following:
#include <WiFiClientSecure.h>
#include <FS.h>
#include <SPIFFS.h>
WiFiClientSecure net = WiFiClientSecure();
String read2String(const char * path) {
String output = "";
File file = SPIFFS.open(path, "r");
if (!file || file.isDirectory()) {
return "";
}
while (file.available()) {
char c = file.read();
output.concat(c);
}
file.close();
return output;
}
void readFilesToVariables() {
String awsRootCa = read2String("/AmazonRootCA1.pem");
String privateKey = read2String("/private.pem.key");
String certificate = read2String("/certificate.pem.crt");
net.setCACert(awsRootCa.c_str());
net.setCertificate(certificate.c_str());
net.setPrivateKey(privateKey.c_str());
}
void setup() {
Serial.begin(115200);
SPIFFS.begin();
readFilesToVariables();
mqttClient.begin(AWS_IOT_ENDPOINT, AWS_IOT_ENDPOINT_PORT, net); // crashes here (I think)
...
}
I have on the mqttClient.begin(...) line (the mqtt client and all the variables except for the net var) a couple of variables that are defined in my script but not shared here.
Anyone any idea what I'm doing wrong here?
the exact same code works when I define the certificates and private key using the following methodology:
static const char AWS_CERT_CA[] PROGMEM = R"EOF(
-----BEGIN CERTIFICATE-----
certificate_content_here
-----END CERTIFICATE-----
)EOF";
What am I doing wrong?
The same thing drove me crazy for the last few days.
Then I found a solution, but only supposed the problem.
I think that this issue has something to do with memory handling and where and how you store the certificates after you read it from flash.
Here's how I read the certificates. Using calloc I allocate the memory and never free it, so that the certificate will always be available to the MQTT client
char* getFileAsString(String path) {
File file = SPIFFS.open(path);
if (!file) {
Serial.println("Failed to open " + path);
return NULL;
}
char* buffer = (char*)calloc(file.size(), sizeof(char));
file.readBytes(buffer, file.size());
file.close();
return buffer;
}
Then I just use it in the config struct
const esp_mqtt_client_config_t mqttConf = {
.uri = MQTT_BROKER,
.client_id = MQTT_CLIENT_ID,
.cert_pem = getFileAsString("/caFile"),
.client_cert_pem = getFileAsString("/certificateFile"),
.client_key_pem = getFileAsString("/privateFile"),
};
Note: I did't put the *_len attributes, because I stored my certificates with a leadinig null character:
dd if=/dev/zero bs=1 count=1 >> [my cert name].pem
You can use regular certificates, but you must set the certificates length or adding 1 byte to the calloc instruction (calloc will set all the allocated memory to 0x00, so there will be a leading null character):
char* buffer = (char*)calloc(file.size() + 1, sizeof(char));
This is the code that i am using and the string i am trying to base64 decode is below which is a from the chrome localstate file,DPAPI key.
RFBBUEkBAAAA0Iyd3wEV0RGMegDAT8KX6wEAAADi+GGk7DWzQpfcOpvrA5QBAAAAAAIAAAAAABBmAAAAAQAAIAAAAB71Yux67n+/BdPaKkGQcjERQXhrLPqH+5oIpE0adTRXAAAAAA6AAAAAAgAAIAAAADw0+h5r3BV2XvQ5Dxou4g8lLEs+r7rMr4urasiaUVbLMAAAAIvSN/H/Q04j/4gknX3tPi/jPFo5zqEqsNFOqq3iMvVoAK5ZpYWdc5WBxzR+gXb+7UAAAAC6hU4zxFN3wWaxxwk1YVIB+ePgEFoe9SYynAXhkUIlxSheodOTXxCFij2DPCvxH8eAze6qw6lxHNC4NVHt7Yvu
/* A BASE-64 ENCODER AND DECODER USING OPENSSL */
#include <openssl/pem.h>
#include <string.h> //Only needed for strlen().
unsigned char *base64decode (const void *b64_decode_this, int decode_this_many_bytes){
BIO *b64_bio, *mem_bio; //Declares two OpenSSL BIOs: a base64 filter and a memory BIO.
unsigned char *base64_decoded = calloc( (decode_this_many_bytes*3)/4+1, sizeof(char) ); //+1 = null.
b64_bio = BIO_new(BIO_f_base64()); //Initialize our base64 filter BIO.
mem_bio = BIO_new(BIO_s_mem()); //Initialize our memory source BIO.
BIO_write(mem_bio, b64_decode_this, decode_this_many_bytes); //Base64 data saved in source.
BIO_push(b64_bio, mem_bio); //Link the BIOs by creating a filter-source BIO chain.
BIO_set_flags(b64_bio, BIO_FLAGS_BASE64_NO_NL); //Don't require trailing newlines.
int decoded_byte_index = 0; //Index where the next base64_decoded byte should be written.
while ( 0 < BIO_read(b64_bio, base64_decoded+decoded_byte_index, 1) ){ //Read byte-by-byte.
decoded_byte_index++; //Increment the index until read of BIO decoded data is complete.
} //Once we're done reading decoded data, BIO_read returns -1 even though there's no error.
BIO_free_all(b64_bio); //Destroys all BIOs in chain, starting with b64 (i.e. the 1st one).
return base64_decoded; //Returns base-64 decoded data with trailing null terminator.
}
int main(void){
char data_to_decode[]="RFBBUEkBAAAA0Iyd3wEV0RGMegDAT8KX6wEAAADi+GGk7DWzQpfcOpvrA5QBAAAAAAIAAAAAABBmAAAAAQAAIAAAAB71Yux67n+/BdPaKkGQcjERQXhrLPqH+5oIpE0adTRXAAAAAA6AAAAAAgAAIAAAADw0+h5r3BV2XvQ5Dxou4g8lLEs+r7rMr4urasiaUVbLMAAAAIvSN/H/Q04j/4gknX3tPi/jPFo5zqEqsNFOqq3iMvVoAK5ZpYWdc5WBxzR+gXb+7UAAAAC6hU4zxFN3wWaxxwk1YVIB+ePgEFoe9SYynAXhkUIlxSheodOTXxCFij2DPCvxH8eAze6qw6lxHNC4NVHt7Yvu";
int bytes_to_decode = strlen(data_to_decode); //Number of bytes in string to base64 decode.
unsigned char *base64_decoded = base64decode(data_to_decode, bytes_to_decode); //Base-64 decoding.
printf("Base-64 decoded string is: %s\n", base64_decoded); //Prints base64 decoded string.
printf("LENGHT:%d",strlen(base64_decoded));
int i=0;
while(base64_decoded[i]!='\0')
{
printf("\n%i",i);
i++;
}
}
This is the string returned when i run it through gdb
I was able to solve it by viewing the memory contents by gdb x /bx command and not p(print) since the print wont work because it is binary data and a null terminator would terminate the string when printing,thanks to commenter.
I'm using PEM_read_RSAPrivateKey function in this way:
void test(void)
{
RSA * privateKey = NULL;
FILE * fp;
if(NULL != (fp= fopen("./my_file.key", "r")) )
{
privateKey=PEM_read_RSAPrivateKey(fp,NULL,NULL,NULL);
if(privateKey==NULL)
{
printf("\n\tCould NOT read RSA private key file");
}
else
{
printf("\n\tRSA structure filled");
}
// This is working OK and privateKey is NOT NULL
}
}
Then, I try to retrieve modulus and public exponent to fill them into a personal structure:
struct
{
unsigned char modulus[256];
unsigned char pub_exp[8];
} s;
But all accesses I tried (I tried a lot) to privateKey->n will result in a segmentation fault.
for example:
unsigned char modulus [2048];
unsigned char exp[2048];
BN_bn2bin(privateKey->n, modulus); // Segmentation fault results from this call
So my question is: how to copy modulus or public exponent from RSA structure to my structure "s" fields?
May someone help about this?
Many thanks,
Regards,
Sylvain
how to copy modulus or public exponent from RSA structure
int req = BN_num_bytes(rsa->n);
assert(rc > 0);
unsigned char* buff = malloc(req);
assert(buff != NULL);
int rc = BN_bn2bin(rsa->n, buff);
assert(req == rc);
Be wary of trying to copy the byte buffers into fixed sized arrays. Someone might come along and get you to copy a 4096-bit modulus into your 2048-bit array.
From OpenSSL 1.1.0 many structures were made opaque and you can no longer do things like look directly at the fields in a struct.
You should use the provided accessor functions instead. Here you can find more infos: https://wiki.openssl.org/index.php/OpenSSL_1.1.0_Changes
Acces rsa->n in OpenSSL 1.1.0:
RSA *pRsa;
BIGNUM *n;
FILE *pFile = fopen("private.key","r");
pRsa = PEM_read_RSAPrivateKey(pFile, NULL, NULL, password);
RSA_get0_key(pRSA, &n, NULL,NULL);
BN_print_fp(stdout, n);
I am trying to read in an AES-128-CBC encrypted PEM key file generated using Ruby with the OpenSSL API. The code that generated the PEM key is the following:
OpenSSL::PKey::RSA.new(2048).to_pem(OpenSSL::Cipher::AES.new('128-CBC'), "password")
Here's the code that reads the PEM file:
RSA *rsa;
BIO *pem = BIO_new(BIO_s_mem());
BIO_puts(pem, "-----BEGIN RSA PRIVATE KEY-----\n"
"Proc-Type: 4,ENCRYPTED\n"
"DEK-Info: AES-128-CBC,BB13D39833DD6ED1FF9843644E7981EE\n\n
"Eugt8JZNKQKErsabWkwfm3wQhU/Tmp9T0QaP5HM8VIWpZwKlDmRlSUDptADU6RPD\n"
"5VtG3DPieXcf+6deyARImid9sBBmQ9mK2omkNRcTMemqTOhAuaKBu78TMt9G4YSf\n"
"RjoXWSqu3jMwrlcGkpn7bIum8wImITRZ3p28oSzk9aDUNBrIU/2Si8DM4RYIZ/fK\n"
"Uvvgdok9dgcd0SjvucivX2HaGeg/IUz23q1jg9inDpimZvFJD1FJfGEUWDgyfJfa\n"
"M8JIxTKbWOPEopONDkT7u4dC5VcSjK29MVbfd7iCKFPMh5UN+c96rPxTng/OWyW5\n"
"0tvzHyyyvAG9p0Hx5Lr4pDbv21GHyu43sA6wbs9jWyqO3AB7CaoEEQhumwfLsdjj\n"
"YGrX6bWThpYv/XNBDmmvltHlKFfe01NCybivOb4KwBnvi45x21PBqaZCKDTFdEkL\n"
"iwDMTiG2iTxSUvPFLy30VFozE+pGyMcGDUyZDVqjsaqI/MRj8khnn5nyubXc27G3\n"
"8Kbsnlix2SW2M0VDxqiy9dyjcxXrkFRSnOFYVs1PFlgjFVTG4Mwh6CZxKw8mFVbi\n"
"EmLvUYwzoDZ1ve4VXSPp/vrKEh33JuHhM0vJOpqI6wqw0QR0I2o6etM1ZRJClPcw\n"
"VIcgcvwenEgLOkoHDqOr0IZQAtYWvAuqq822wKt258hc6z8+ALQf5iMroqk7ADd4\n"
"FlRLz4XTwqlg7pPtTde/emI1DT8dQWzq++QI0lr0CS/N1GXJKqTQDvauXLIiI3Qy\n"
"KfFYFpV9jyYfRfTjNtisI/edPtp98auK0mb9o/wS/hruFI9behgv63iW1IwAOXCW\n"
"ZlkWgobUH13gS864rL+AcrAXreo2j4dDQouTeRaJUEG0HoYTP65Zun/VsCi2aSOH\n"
"JwSnnmHz9OxvcGY80WJDN3kqOCBRIJoDKBv6jcOxGVCsVK+WSdGZ7cfb8lwp7aA8\n"
"8ND1bwL9FYkwkeIsoakj91iinqv4o3+3PUPgCU5oe68WYvAFjuU+criyf+EhmXJV\n"
"JQ1vFFZPrGzgntJz19uXXh1h2iwQPggRouJm2RozYwvv1nz4eQ40Y3eT1F9UOYJU\n"
"CKEhOtI2NpLeVOayqo8g9wO2oC+CQVhZhdYBE5o7pM7akFnYLvRg9s1UsWdcvT0G\n"
"IpFmejLSRJ/F954aQMHTUc6vBOJZH/VNC5Qt+ulFXl634Sr9wQQK2qlqSJyA04TR\n"
"1ixbCNOX71esvpFImsrlsO5oTA22T3h2GyJPUM10XhqGtDXtsTnal6smLna9U9B3\n"
"gTVxFWWukQOF5Lm8ZFQipo2loHWjkozTBc4REPYP44SoXJXstv7k4pt1cK7x6/2H\n"
"ElspXzjveqMhcrveWv1KaA2OGd+hGfUiNsCoIdapJjLz1Bd/+oIQ/ZWQeo0nRowE\n"
"R/HlbbED3V+fRIdJpgydFEAw6gK5E9sYJcgF7uf/n2NabFxxEZL3g6MJQ64Dtusg\n"
"DEH/MpvIYDSX4Navh1gTwCtOeG1CzW3diYaqbZK+UZCBLFU7j27YvVPSd6F2+Wud\n"
"WnAqU3S5BCPqk5OD3wqZv+sEcqJgGPGy1Gv0tl8ARJomdKAru03KsRn2eIWqR5/C\n"
"-----END RSA PRIVATE KEY-----\n");
// Retrieve RSA key from PEM file.
rsa = PEM_read_bio_RSAPrivateKey(pem, NULL, pem_password_callback, "password");
And here's a dummy password callback (not sure about this function's purpose, but I think it may return the length of the password):
int pem_password_callback(char *buf, int max_len, int flag, void *pwd)
{
return 8;
}
Currently, the rsa = ... part does not throw an error, but doesn't return a well-formed result either.
And here's a dummy password callback (not sure about this function's
purpose, but I think it may return the length of the password):
int pem_password_callback(char *buf, int max_len, int flag, void *pwd)
{
return 8;
}
No, its not a dummy. In your example, you just returned a buffer with 8 junk characters (whatever happened to be in buf).
The password callback is where you programmatically plug in your password. You are supplied with a buff of max_len, and you need to copy the password into the buffer and return the number of bytes copied.
int pem_password_callback(char *buf, int max_len, int flag, void *ctx)
{
const char* PASSWD = "password";
int len = strlen(PASSWD);
if(len > max_len)
return 0;
memcpy(buf, PASSWD, len);
return len;
}
The flag is a read/write flag to denote if you are reading a key or writing a key. In practice, I have never used it.
You will use it similar to:
FILE* file = ...;
EVP_PKEY* pkey = PEM_read_PrivateKey(file, NULL, pem_password_callback, NULL);
Unlike the write routine (which needs an EVP_* cipher), the read routine knows what you used to encrypt the key with because its encoded in the private key.
In my systems, I actually use the context for a label to ensure the same passwords arrive at different derived keys:
EVP_PKEY* pkey = PEM_read_PrivateKey(file, NULL, pem_password_callback, "Some Context");
Then, in my password callback:
int pem_password_callback(char *buf, int max_len, int flag, void *ctx)
{
// "Some Context" in the example above
char* label = (char*)ctx;
// Hash password and label
// ...
// Copy hash to buffer, return length
...
}
I'm trying to write a wchar array to a file in C, however there is some sort of corruption and unrelevant data like variables and paths like this
c.:.\.p.r.o.g.r.a.m. .f.i.l.e.s.\.m.i.c.r.o.s.o.f.t. .v.i.s.u.a.l. .s.t.u.d.i.o. 1.0...0.\.v.c.\.i.n.c.l.u.d.e.\.x.s.t.r.i.n.g..l.i.s.t...i.n.s.e.r.t
are written on to the file along with the correct data (example) I have confirmed that the buffer is null-terminated and contains proper data.
Heres my code:
myfile = fopen("logs.txt","ab+");
fseek(myfile,0,SEEK_END);
long int size = ftell(myfile);
fseek(myfile,0,SEEK_SET);
if (size == 0)
{
wchar_t bom_mark = 0xFFFE;
size_t written = fwrite(&bom_mark,sizeof(wchar_t),1,myfile);
}
// in another func
while (true)
{
[..]
unsigned char Temp[512];
iBytesRcvd = recv(sclient_socket,(char*)&Temp,iSize,NULL);
if(iBytesRcvd > 0 )
{
WCHAR* unicode_recv = (WCHAR*)&Temp;
fwrite(unicode_recv,sizeof(WCHAR),wcslen(unicode_recv),myfile);
fflush(myfile);
}
[..]
}
What could be causing this?
recv() will not null-terminate &Temp, so wcslen() runs over the bytes actually written by recv(). You will get correct results if you just use iBytesReceived as byte count for fwrite() instead of using wcslen() and hoping the data received is correctly null-terminated (wide-NULL-terminated, that is):
fwrite(unicode_recv, 1, iBytesReceived, myfile);