C version of OpenSSL EVP_BytesToKey method - c

I'm looking for C implementation of the OpenSSL EVP_BytesToKey function.
This is pseudo-code explanation of the EVP_BytesToKey method (in /doc/ssleay.txt of the OpenSSL source):
/* M[] is an array of message digests
* MD() is the message digest function */
M[0]=MD(data . salt);
for (i=1; i<count; i++) M[0]=MD(M[0]);
i=1
while (data still needed for key and iv)
{
M[i]=MD(M[i-1] . data . salt);
for (i=1; i<count; i++) M[i]=MD(M[i]);
i++;
}
If the salt is NULL, it is not used.
The digests are concatenated together.
M = M[0] . M[1] . M[2] .......
this is my code(MD() is sha512. And i need the key is 32 bytes and the iv is 16 bytes):
int main()
{
unsigned long long out[8];
unsigned char key[9] = {0x4b,0x0c,0x0c,0x0c,0x0c,0x0c,0x0c,0x0c,0x0c};
int len, i;
len = sizeof(key);
sha512(out, key, len);
unsigned char a[sizeof(out)];
for (i = 0; i < count; i++)
{
long2char(out, a); // unsigned long long to unsigned char;
sha512(out, a, sizeof(out));
}
return 0;
}
After count times sha512(), the out is 64 bytes, so I think I don't need the rest of that pseudo-code. But the result is not correct, I don't know what went wrong.
I wish you can help me to figure it out. thanks!

I'm looking for C implementation of the OpenSSL EVP_BytesToKey function...
Perhaps you should just use the one OpenSSL provides? You know, the one called EVP_BytesToKey in evp_key.c (shown below).
OpenSSL 1.1.0c changed the digest algorithm used in some internal components. Formerly, MD5 was used, and 1.1.0 switched to SHA256. Be careful the change is not affecting you in both EVP_BytesToKey and commands like openssl enc
int EVP_BytesToKey(const EVP_CIPHER *type, const EVP_MD *md,
const unsigned char *salt, const unsigned char *data,
int datal, int count, unsigned char *key,
unsigned char *iv)
{
EVP_MD_CTX *c;
unsigned char md_buf[EVP_MAX_MD_SIZE];
int niv, nkey, addmd = 0;
unsigned int mds = 0, i;
int rv = 0;
nkey = EVP_CIPHER_key_length(type);
niv = EVP_CIPHER_iv_length(type);
OPENSSL_assert(nkey <= EVP_MAX_KEY_LENGTH);
OPENSSL_assert(niv <= EVP_MAX_IV_LENGTH);
if (data == NULL)
return nkey;
c = EVP_MD_CTX_new();
if (c == NULL)
goto err;
for (;;) {
if (!EVP_DigestInit_ex(c, md, NULL))
goto err;
if (addmd++)
if (!EVP_DigestUpdate(c, &(md_buf[0]), mds))
goto err;
if (!EVP_DigestUpdate(c, data, datal))
goto err;
if (salt != NULL)
if (!EVP_DigestUpdate(c, salt, PKCS5_SALT_LEN))
goto err;
if (!EVP_DigestFinal_ex(c, &(md_buf[0]), &mds))
goto err;
for (i = 1; i < (unsigned int)count; i++) {
if (!EVP_DigestInit_ex(c, md, NULL))
goto err;
if (!EVP_DigestUpdate(c, &(md_buf[0]), mds))
goto err;
if (!EVP_DigestFinal_ex(c, &(md_buf[0]), &mds))
goto err;
}
i = 0;
if (nkey) {
for (;;) {
if (nkey == 0)
break;
if (i == mds)
break;
if (key != NULL)
*(key++) = md_buf[i];
nkey--;
i++;
}
}
if (niv && (i != mds)) {
for (;;) {
if (niv == 0)
break;
if (i == mds)
break;
if (iv != NULL)
*(iv++) = md_buf[i];
niv--;
i++;
}
}
if ((nkey == 0) && (niv == 0))
break;
}
rv = EVP_CIPHER_key_length(type);
err:
EVP_MD_CTX_free(c);
OPENSSL_cleanse(md_buf, sizeof(md_buf));
return rv;
}

Related

Problem with decrypting using mbedtls on esp32

I am trying to write function to decrypt rsa2048 with mbedtls/pk.h
I am trying to write function to decrypt rsa2048 with mbedtls/pk.h esp32 but on the site they wrote Store data to be decrypted and its length in variables. This tutorial stores the data in to_decrypt, and its length in to_decrypt_len:
I idk what is the format of to_decrypt i tried to put encrypted with code upper buf but it didn't work. Tried co encode on rsa online sites into base64 and put into the program convert to unsigned char* It gaves to me error 0x4080 RSA - Bad input parameters to function
code :
#include <Arduino.h>
#include "mbedtls/pk.h"
#include "mbedtls/entropy.h"
#include "mbedtls/ctr_drbg.h"
#include "mbedtls/platform.h"
#include "mbedtls/base64.h"
void rsa2048_encrypt(const char *text)
{
// RNG (Random number generator init)
int ret = 0;
mbedtls_entropy_context entropy;
mbedtls_entropy_init(&entropy);
mbedtls_ctr_drbg_context ctr_drbg;
const char *personalization = "mgkegneljgnjlwgnjefdcmeg12313123dsggsd";
mbedtls_ctr_drbg_init(&ctr_drbg);
ret = mbedtls_ctr_drbg_seed(&ctr_drbg, mbedtls_entropy_func, &entropy,
(const unsigned char *)personalization,
strlen(personalization));
if (ret != 0)
{
// ERROR HANDLING CODE FOR YOUR APP
}
// Creating rsa context + Importing pub key
ret = 0;
mbedtls_pk_context pk;
mbedtls_pk_init(&pk);
/*
* Read the RSA public key
*/
const unsigned char *key = (const unsigned char *)"-----BEGIN PUBLIC KEY-----\n"
"MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAmwukduTT4dMD+EXuho4L\n"
"zEg9pH7Bk4y6TPw9pQqiw4b5Qx3+KM+SFi2I4AncfkOcjtuMqtSPdNSgFb1DErQD\n"
"+I+yOS7ztuAVy6hO/oRpVKAJNVl385pC/Ah8aiZts6cY8kjs47Mw4ufFNwIH8hOy\n"
"6f1+e8chBgeKxOVJBNiWr2nsPhvvAERTunw/CTvWsBLakyGs+OJwOcYsr0m5iOJx\n"
"XfBUEYOQ68XDIUTTLKdYsUOFSESwtIsPgqytj+SRcA/STH8eQJigNQNj8Zexpi+3\n"
"ykDWAxbmHQ8UWma1vV//oM6xy3DI/SXxvPusjNbKxDg+q5/e3hWoaBVq3ti9/ZTe\n"
"kQIDAQAB\n"
"-----END PUBLIC KEY-----\n";
if ((ret = mbedtls_pk_parse_public_key(&pk, key, strlen((const char *)key) + 1)) != 0)
{
printf(" failed\n ! mbedtls_pk_parse_public_key returned -0x%04x\n", -ret);
};
// Encrypting data
const unsigned char *to_encrypt = (const unsigned char *)text;
size_t to_encrypt_len = strlen((const char *)to_encrypt);
unsigned char buf[MBEDTLS_MPI_MAX_SIZE];
size_t olen = 0;
/*
* Calculate the RSA encryption of the data.
*/
printf("\n . Generating the encrypted value: \n");
fflush(stdout);
if ((ret = mbedtls_pk_encrypt(&pk, to_encrypt, to_encrypt_len,
buf, &olen, sizeof(buf),
mbedtls_ctr_drbg_random, &ctr_drbg)) != 0)
{
printf(" failed\n ! mbedtls_pk_encrypt returned -0x%04x\n", -ret);
}
for (size_t i = 0; i < olen; i++)
{
mbedtls_printf("%02X%s", buf[i],
(i + 1) % 16 == 0 ? "\r\n" : " ");
}
mbedtls_pk_free(&pk);
mbedtls_entropy_free(&entropy);
mbedtls_ctr_drbg_free(&ctr_drbg);
}
void decrypt_test()
{
const unsigned char *private_key = (const unsigned char *)"-----BEGIN RSA PRIVATE KEY-----\n"
"MIIEogIBAAKCAQEAmwukduTT4dMD+EXuho4LzEg9pH7Bk4y6TPw9pQqiw4b5Qx3+\n"
"KM+SFi2I4AncfkOcjtuMqtSPdNSgFb1DErQD+I+yOS7ztuAVy6hO/oRpVKAJNVl3\n"
"85pC/Ah8aiZts6cY8kjs47Mw4ufFNwIH8hOy6f1+e8chBgeKxOVJBNiWr2nsPhvv\n"
"AERTunw/CTvWsBLakyGs+OJwOcYsr0m5iOJxXfBUEYOQ68XDIUTTLKdYsUOFSESw\n"
"tIsPgqytj+SRcA/STH8eQJigNQNj8Zexpi+3ykDWAxbmHQ8UWma1vV//oM6xy3DI\n"
"/SXxvPusjNbKxDg+q5/e3hWoaBVq3ti9/ZTekQIDAQABAoIBADXzji5FICnDuOzq\n"
"wL6XrSlPtguIhCmo3acuWvEUS2EIlbIyPJ/M4wPOooN7Svuw4Uigw0kqoCTCXFZU\n"
"PoPCmmMi9ZyKZwoq3cq5bYuJXfGxoqKq2F+vPUHgXhK9/ox2R+r/T1dIomlCx1CF\n"
"52foTOi5agr+VtJ3S2WKd6c1CvJMuRRoIX9vI49L+NdA9FUcA4Ge2rJZPu7zd/Xj\n"
"VvqtIH63Y/4z+S5YqnBgYjk7xWf3f9ybrkdi9fiRNt9wq4LOet+OSiQXWyuX+ppL\n"
"im6Sl3O8XkaDWAFo8dUWkZf+6RpABxFnUy45CWZGs7W8MpVwykXdpxcn9iJ7jIaR\n"
"9dcmUgECgYEA3aAWxuiX081mFIdmQEpKkp1JFbZOoZpIzBXNx/M05FGkz8QPL7G3\n"
"9h8A8UCbTFG+cAM1vjMUPzWXbsytE8VC4qjGy+1RBDltBt5/XX4VECVDIwjTdmc0\n"
"RPfJ2vKkAFYPHjyQijQCZxAPM0E/IFGKnTP4Dt+rITxkYvoouFiR2eECgYEAsxfq\n"
"qk2d8K62DQPURHvNmFnct+QAlIF9i/XHGcLNvnzEASiIsZ75TiXUc0xIxY2/ewo4\n"
"CMbNUG98xiW0Q5pdyKFQ3qnhNNTKPCK1T4qqKarDU2pSXFC2afkCSBbfO7qWblRD\n"
"PMJJ7SG8fAeIWbqKVzPgSfwRw4xY+iEe7SvoerECgYAEIQhrmkfB3XDKbx9bkUbE\n"
"ZoPHEMd0QVCb5MgZspFIs7CzYj66L8ByqG83D3IVQOygX57vtTnqV5BDszKCTMmL\n"
"OYPCpuA8iOlcGGcdEc1IqLkQfQibix6xLkCngJ/HldLgSFaVDJUC4Iy38r4/VuWT\n"
"OjWj6Uzh6KMiKPD7RkMpYQKBgH2BSjM0l3U+ilfOkie39tlISDQaNQndQQUfJPr5\n"
"mENgnd8N79VBygYo3pw6HllLP5/TBneoEePHbVJSw+QIPqbF3a1csXTblinUTOlE\n"
"DIGMqLtBLByDd4IGPcIVPTVXSephZIkkwrfKR5NHmBcBcccwlIJkgnJeXVBUe57L\n"
"gWzRAoGAAmLnPNkIT5Nruqy7EdEeb90W0VA/7/CaESvZKUkAHUy4bIqwFSGGJz1Q\n"
"oKUx8cuK4tz79mHsjzlJoCSLUvI5Fpfz+CS9uvA15QBIHU+5G37Ga5WsuBTww+lx\n"
"XC6IQ356/xfs4CmAVD1xjhEuBjANSs8lgHMAQPGngU5EVaE1hPw=\n"
"-----END RSA PRIVATE KEY-----";
// RNG (Random number generator init)
int ret = 0;
mbedtls_entropy_context entropy;
mbedtls_entropy_init(&entropy);
mbedtls_ctr_drbg_context ctr_drbg;
const char *personalization = "mgkegneljgnjlwgnjefdcmeg12313123dsggsd";
mbedtls_ctr_drbg_init(&ctr_drbg);
ret = mbedtls_ctr_drbg_seed(&ctr_drbg, mbedtls_entropy_func, &entropy,
(const unsigned char *)personalization,
strlen(personalization));
if (ret != 0)
{
// ERROR HANDLING CODE FOR YOUR APP
}
ret = 0;
mbedtls_pk_context pk;
mbedtls_pk_init(&pk);
/*
* Read the RSA privatekey
*/
if ((ret = mbedtls_pk_parse_key(&pk, private_key, strlen((const char *)private_key) + 1, NULL, 0)) != 0)
{
printf(" failed\n ! mbedtls_pk_parse_keyfile returned -0x%04x\n", -ret);
}
unsigned char result[MBEDTLS_MPI_MAX_SIZE];
size_t olen = 0;
/*
* Calculate the RSA encryption of the data.
*/
printf("\n . Generating the decrypted value");
fflush(stdout);
const char *text = "fnianPxs/09bx75ufVLWPeFF9kbGEIL3+TQqW2+ZoeMpdvVnkifFToAii92ODVBPOL0RzQPfxlJcN/nVY3K5fWNSGHM8TTwTgCqvUc0ia5L5YHI1YSgDKzx2QPZlu7tEd06sjW7txRacnhilRfjFPp0CYeLwxYVBlPmKIE7oqQHrc8sal3X9NSqgwO7+03TBeH3beNanMCqQBRk9t+Z80XApEBMcZQHZ0lb+Z0C6DOuY0elH/fOp1SGlXuzf+tgcv7+TzL5uVVFCBNyMonTwMEp+zbLjX2Ck1IHhp8JXi3ovVi8HNcKCOQx/fxX1qTSt2NulHTwP2urCQSZbGjnYuw==";
const unsigned char *to_decrypt = (unsigned char *)text;
if ((ret = mbedtls_pk_decrypt(&pk, to_decrypt, (strlen(text) * 4) - 1, result, &olen, sizeof(result),
mbedtls_ctr_drbg_random, &ctr_drbg)) != 0)
{
printf(" failed\n ! mbedtls_pk_decrypt returned -0x%04x\n", -ret);
}
}
So what I do wrong ?

The Serial Version and The Parallel Result Not Showing the same output

I am actually new to openmp, I have a working aes-128-cbc toy code that decrypts an hardcoded ciphertext to 12345, this book was recommended for me by one of the community user, I also came across this openmp reference guide and lastly I was heavily guided by one of the community user. From those books and the guide, I was trying to parallelize the serial code below
SERIAL WORKING CODE:
#include <openssl/ssl.h>
#include <openssl/err.h>
#include <string.h>
#include <stdio.h>
#include <time.h>
int success = 0;
void handleOpenSSLErrors(void)
{
ERR_print_errors_fp(stderr);
abort();
}
unsigned char* decrypt(unsigned char *ciphertext, int ciphertext_len, unsigned char *key, unsigned char *iv ){
EVP_CIPHER_CTX *ctx;
unsigned char *plaintexts;
int len;
int plaintext_len;
unsigned char* plaintext = malloc(ciphertext_len);
bzero(plaintext,ciphertext_len);
/* Create and initialise the context */
if(!(ctx = EVP_CIPHER_CTX_new())) handleOpenSSLErrors();
/* Initialise the decryption operation. IMPORTANT - ensure you use a key
* and IV size appropriate for your cipher
* IV size for *most* modes is the same as the block size. For AES this
* is 128 bits */
if(1 != EVP_DecryptInit_ex(ctx, EVP_aes_128_cbc(), NULL, key, iv))
handleOpenSSLErrors();
EVP_CIPHER_CTX_set_key_length(ctx, EVP_MAX_KEY_LENGTH);
/* Provide the message to be decrypted, and obtain the plaintext output.
* EVP_DecryptUpdate can be called multiple times if necessary
*/
if(1 != EVP_DecryptUpdate(ctx, plaintext, &len, ciphertext, ciphertext_len))
handleOpenSSLErrors();
plaintext_len = len;
/* Finalise the decryption. Further plaintext bytes may be written at
* this stage.
*/
// return 1 if decryption successful, otherwise 0
if(1 == EVP_DecryptFinal_ex(ctx, plaintext + len, &len))
success = 1;
plaintext_len += len;
/* Add the null terminator */
plaintext[plaintext_len] = 0;
/* Clean up */
EVP_CIPHER_CTX_free(ctx);
//delete [] plaintext;
return plaintext;
}
size_t calcDecodeLength(char* b64input) {
size_t len = strlen(b64input), padding = 0;
if (b64input[len-1] == '=' && b64input[len-2] == '=') //last two chars are =
padding = 2;
else if (b64input[len-1] == '=') //last char is =
padding = 1;
return (len*3)/4 - padding;
}
void Base64Decode( char* b64message, unsigned char** buffer, size_t* length) {
BIO *bio, *b64; // A BIO is an I/O strean abstraction
int decodeLen = calcDecodeLength(b64message);
*buffer = (unsigned char*)malloc(decodeLen + 1);
(*buffer)[decodeLen] = '\0';
bio = BIO_new_mem_buf(b64message, -1);
b64 = BIO_new(BIO_f_base64());
bio = BIO_push(b64, bio);
//BIO_set_flags(bio, BIO_FLAGS_BASE64_NO_NL); //Do not use newlines to flush buffer
*length = BIO_read(bio, *buffer, strlen(b64message));
BIO_free_all(bio);
}
void initAES(const unsigned char *pass, unsigned char* salt, unsigned char* key, unsigned char* iv )
{
//initialisatio of key and iv with 0
bzero(key,sizeof(key));
bzero(iv,sizeof(iv));
EVP_BytesToKey(EVP_aes_128_cbc(), EVP_sha1(), salt, pass, strlen(pass), 1, key, iv);
}
int checkPlaintext(char* plaintext, char* result){
int length = 10; // we just check the first then characters
return strncmp(plaintext, result, length);
}
int main (void)
{
// password 12345
// it took 9 seconds to work out
char* ciphertext_base64 = (char*) "U2FsdGVkX19q3SzS6GhhzAKgK/YhFVTkM3RLVxxZ+nM6yXdfLZtvhyRR4oGohDotiifnR1iKyitSpiBM3hng+eoFfGbtgCu3Zh9DwIhgfS5A+OTl5a4L7pRFG4yL432HsMGRC1hy1RNPSzA0U5YyWA==\n";
char* plaintext = "This is the top seret message in parallel computing! Please keep it in a safe place.";
char dict[] = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz"; // changed
int decryptedtext_len, ciphertext_len, dict_len;
// cipher (binary) pointer and length
size_t cipher_len; // size_t is sizeof(type)
unsigned char* ciphertext;
unsigned char salt[8];
ERR_load_crypto_strings();
Base64Decode(ciphertext_base64, &ciphertext, &cipher_len);
unsigned char key[16];
unsigned char iv[16];
unsigned char plainpassword[] = "00000";
unsigned char* password = &plainpassword[0];
// retrive the slater from ciphertext (binary)
if (strncmp((const char*)ciphertext,"Salted__",8) == 0) { // find the keyword "Salted__"
memcpy(salt,&ciphertext[8],8);
ciphertext += 16;
cipher_len -= 16;
}
dict_len = strlen(dict);
time_t begin = time(NULL);
for(int i=0; i<dict_len; i++)
for(int j=0; j<dict_len; j++)
for(int k=0; k<dict_len; k++)
for(int l=0; l<dict_len; l++)
for(int m=0; m<dict_len; m++){
*password = dict[i];
*(password+1) = dict[j];
*(password+2) = dict[k];
*(password+3) = dict[l];
*(password+4) = dict[m];
initAES(password, salt, key, iv);
unsigned char* result = decrypt(ciphertext, cipher_len, key, iv);
if (success == 1){
if(checkPlaintext(plaintext, result)==0){
printf("Password is %s\n", password);
time_t end = time(NULL);
printf("Time elpased is %ld seconds", (end - begin));
return 0;
}
}
free(result);
}
// Clean up
EVP_cleanup();
ERR_free_strings();
return 0;
}
THIS IS THE PARALLEL VERSION:
int main (void)
{
// password 12345
// it took 9 seconds to work out
char* ciphertext_base64 = (char*) "U2FsdGVkX19q3SzS6GhhzAKgK/YhFVTkM3RLVxxZ+nM6yXdfLZtvhyRR4oGohDotiifnR1iKyitSpiBM3hng+eoFfGbtgCu3Zh9DwIhgfS5A+OTl5a4L7pRFG4yL432HsMGRC1hy1RNPSzA0U5YyWA==\n";
char* plaintext = "This is the top seret message in parallel computing! Please keep it in a safe place.";
char dict[] = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz"; // changed
int decryptedtext_len, ciphertext_len, dict_len;
// cipher (binary) pointer and length
size_t cipher_len; // size_t is sizeof(type)
unsigned char* ciphertext;
unsigned char salt[8];
ERR_load_crypto_strings();
Base64Decode(ciphertext_base64, &ciphertext, &cipher_len);
unsigned char key[16];
unsigned char iv[16];
omp_set_num_threads(NUM_THREADS);
double begin = omp_get_wtime();
// retrive the slater from ciphertext (binary)
if (strncmp((const char*)ciphertext,"Salted__",8) == 0)
{ // find the keyword "Salted__"
memcpy(salt,&ciphertext[8],8);
ciphertext += 16;
cipher_len -= 16;
}
dict_len = strlen(dict);
#pragma omp parallel for shared (key, iv)
for (int i=0; i<dict_len; i++)
{
unsigned char password[5] = {i};
for(int j=0; j<dict_len; j++)
{
password[1] = dict[j];
for(int k=0; k<dict_len; k++)
{
password[2] = dict[k];
for(int l=0; l<dict_len; l++)
{
password[3] = dict[l];
for(int m=0; m<dict_len; m++)
{
password[4] = dict[m];
{
*password = dict[i];
*(password+1) = dict[j];
*(password+2) = dict[k];
*(password+3) = dict[l];
*(password+4) = dict[m];
initAES(password, salt, key, iv);
unsigned char* result = decrypt(ciphertext, cipher_len, key, iv);
#pragma omp if(checkPlaintext(plaintext, result)==0)
{
printf("\nPassword is %s\n\n", password);
success == 1;
strcpy(result,password); // Copy thread-private copy to shared variable
time_t end = time(NULL);
printf("\nTime elpased is %ld seconds\n", (end - begin));
exit(0);
}
free(result);
}
}
}
}
#pragma omp cancellation point for
}
}
// Clean up
EVP_cleanup();
ERR_free_strings();
return 0;
}
Should bring my expected password, but when I run the code it does not output the expected . Kindly help.
You have only one buffer for the trial password (plainpassword), and all of the threads in your parallel for are trying to use it at the same time (via pointer password). There are massive data races here, and the resulting behavior is undefined.
One solution would be to declare the buffer inside the parallel loop instead of outside, as variables local to a parallel region are automatically private -- each thread will have its own.
#pragma omp parallel for shared (key, iv)
for (int i=0; i<dict_len; i++) {
unsigned char password[5] = { i };
for (int j=0; j<dict_len; j++) {
password[1] = dict[j];
for (int k=0; k<dict_len; k++) {
password[2] = dict[k];
for (int l=0; l<dict_len; l++) {
password[3] = dict[l];
for (int m=0; m<dict_len; m++) {
password[4] = dict[m];
// ...
}
}
}
}
}
Note also that
there is no apparent benefit in your original code to having both plainpassword and password. The above uses just an array, not a separate pointer as well, and it chooses the name "password" for that.
*(x + y), where x and y are primaries, means exactly the same thing as x[y] (and y[x]). The subscript form is easier to read, and that almost always makes it stylistically better.
Update:
I observe also that this code from the parallel version does not make sense, especially as compared to the serial code:
#pragma omp parallel if (strncmp((const char*)ciphertext,"Salted__",8) == 0) shared(ciphertext, plaintext, ciphertext_base64) private(salt)
{ // find the keyword "Salted__"
memcpy(salt,&ciphertext[8],8);
ciphertext += 16;
cipher_len -= 16;
}
The original code executes the if statement once per run, and its body at most once per run, so if the parallel version executes the body multiple times, and it has side effects (which it does), then the result of the program will be different. The original code should be restored for this part.

C: Output difference between printf and vfprintf when using the exact same inputs

I'm trying to write a logger function that logs stuff to a file in a printf manner but, if enabled, it will also log to the console. I'm trying it out with a custom function that uses a custom string struct representing a number and transforms it into an actual number.
The main function:
#define MSG "0xab45cdef"
int main(){
String s;
stringInit(&s);
stringSetString(&s,MSG,sizeof(MSG));
stringPrint(&s);
logOut("\nTransforming to value\n");
int64_t v = parseValue(s);
logOut("\n");
logOut("\nResult %li \n", v);
}
My output log function
void logOut(const char *control_string, ...){
FILE *fp;
fp = fopen(LOG_OUTPUT,"ab+");
va_list argptr;
va_start(argptr,control_string);
vfprintf(fp,control_string,argptr);
#ifdef LOG_CONSOLE
printf(control_string,argptr);
#endif
va_end(argptr);
fclose(fp);
}
My String related functions
typedef struct {
char *s;
unsigned int size;
} String;
void stringInit(String *s){
s->s = NULL;
s->size = 0;
}
void stringAddChar(String *s, char c){
if (s->size > 0){
// Adding one more char.
s->s = (char *) realloc (s->s, (s->size + 1)*sizeof(char));
}
else{
// First char.
s->s = (char *) malloc(sizeof(char));
}
s->size++;
s->s[s->size-1] = c;
}
void stringFree(String *s){
if (s->size == 0) return;
free(s->s);
s->s = NULL;
s->size = 0;
}
void stringSetString(String *s, char *str, uint32_t nsize){
// Clearing the previous string.
stringFree(s);
for (uint32_t i = 0; i < nsize; i++){
// This avoids the extra char in a null terminated string.
if ((i == nsize-1) && (str[i] == 0)) break;
stringAddChar(s,str[i]);
}
}
void stringPrint(String *s){
for (uint32_t i = 0; i < s->size; i++){
logOut("%c",s->s[i]);
}
}
And finally the parseValue function
int64_t power(int64_t base, int64_t exp){
int64_t ans = 1;
for (int i = 0; i < exp; i++){
ans = ans * base;
}
return ans;
}
int64_t parseValue(String input){
int64_t base = 10;
int64_t res = 0;
int64_t maxpow = input.size-1;
uint32_t start = 0;
if (input.size > 0){
// Must check if it is hex or not.
if (input.s[0] == '0' && input.s[1] == 'x'){
base = 16;
start = 2;
maxpow = input.size-3;
}
}
for (int i = start; i < input.size; i++){
int64_t p = maxpow;
maxpow--;
char c = toupper(input.s[i]);
// printf("Char %d is %d\n",i,c);
int64_t v = c - 48;
if ((v >= 0) && (v <= 9)){
res = res + v*power(base,p);
}
else if ((c >= 65) && (c <= 70)){
if (base == 16){
v = c - 55;
res = res + v*power(base,p);
}
else{
logOut("Invalid char %c in a decimal number\n",c);
return -1;
}
}
else{
logOut("Invalid digit %d\n",c);
return -1;
}
}
return res;
}
Now when I run the main the console outputs:
pppppppppp
Transforming to value
Result 140726926743712
While my log.txt file has this
0xab45cdef
Transforming to value
Result 2873478639
The content of the log.txt file is correct. So why is the console output different?
To summarize, your logOut function has some mistakes. You can't "re-use" va_list after it has been used in another function. The function could look like this:
void logOut(const char *control_string, ...){
FILE *fp;
fp = fopen(LOG_OUTPUT,"ab+");
if (fp == NULL) {
abort();
}
va_list argptr;
va_start(argptr, control_string);
if (vfprintf(fp, control_string, argptr) < 0) {
// handle error
}
va_end(argptr);
fclose(fp);
#ifdef LOG_CONSOLE
va_start(argptr, control_string); // do not re-use va_list
vprintf(control_string, argptr);
// ^ you pass va_list
va_end(argptr);
#endif
}
Notes:
realloc(NULL, ...) is equal to malloc(...). So there is no need to if (s->size > 0){ inside stringAddChar - just call realloc(s->s, and make sure that s->s is NULL when size is zero.
Your code misses a lot of error handling. A proper way to handle realloc is to use a temporary pointer so that original memory will not leak: void *p = realloc(s->s, ...); if (p == NULL) { free(s->s); /* handle errors */ abort(); } s->s = p;.
Try not to use magic numbers in code. c - 48; is better written as c - '0';.

readdir and getdents interfaces -- number of bytes read?

I'm working on Cygwin, which does not implement getdents, nor getdirentries.
The code I'm working on relies on knowing the number of bytes read, which is the return of these calls. All I seem to have is readdir.
Man pages are lacking in Cygwin. Any ideas, or existing documentation, on how to make these interfaces compatible, or how to get the number of bytes read from readdir?
Cygwin's struct dirent, in case it's relevant:
struct dirent
{
uint32_t __d_version; /* Used internally */
ino_t d_ino;
unsigned char d_type;
unsigned char __d_unused1[3];
__uint32_t __d_internal1;
char d_name[NAME_MAX + 1];
};
EDIT
The code that uses getdents is in function readdir (see the link for full file):
static int
mygetdents(int fd, struct dirent *buf, int n) {
return syscall (getdents, fd, (void*) buf, n);
}
long
dirread(int fd, Dir **dp)
{
char *buf;
struct stat st;
int n;
*dp = 0;
if(fstat(fd, &st) < 0)
return -1;
if(st.st_blksize < 8192)
st.st_blksize = 8192;
buf = malloc(st.st_blksize);
if(buf == nil)
return -1;
n = mygetdents(fd, (void*)buf, st.st_blksize);
if(n < 0){
free(buf);
return -1;
}
n = dirpackage(fd, buf, n, dp);
free(buf);
return n;
}
static int
dirpackage(int fd, char *buf, int n, Dir **dp)
{
int oldwd;
char *p, *str, *estr;
int i, nstr, m;
struct dirent *de;
struct stat st, lst;
Dir *d;
n = countde(buf, n);
if(n <= 0)
return n;
if((oldwd = open(".", O_RDONLY)) < 0)
return -1;
if(fchdir(fd) < 0)
return -1;
p = buf;
nstr = 0;
for(i=0; i<n; i++){
de = (struct dirent*)p;
memset(&lst, 0, sizeof lst);
if(de->d_name[0] == 0)
/* nothing */ {}
else if(lstat(de->d_name, &lst) < 0)
de->d_name[0] = 0;
else{
st = lst;
if(S_ISLNK(lst.st_mode))
stat(de->d_name, &st);
nstr += _p9dir(&lst, &st, de->d_name, nil, nil, nil);
}
p += de->d_reclen;
}
d = malloc(sizeof(Dir)*n+nstr);
if(d == nil){
fchdir(oldwd);
close(oldwd);
return -1;
}
str = (char*)&d[n];
estr = str+nstr;
p = buf;
m = 0;
for(i=0; i<n; i++){
de = (struct dirent*)p;
if(de->d_name[0] != 0 && lstat(de->d_name, &lst) >= 0){
st = lst;
if((lst.st_mode&S_IFMT) == S_IFLNK)
stat(de->d_name, &st);
_p9dir(&lst, &st, de->d_name, &d[m++], &str, estr);
}
p += de->d_reclen;
}
fchdir(oldwd);
close(oldwd);
*dp = d;
return m;
}
static int
countde(char *p, int n)
{
char *e;
int m;
struct dirent *de;
e = p+n;
m = 0;
while(p < e){
de = (struct dirent*)p;
if(de->d_reclen <= 4+2+2+1 || p+de->d_reclen > e)
break;
if(de->d_name[0]=='.' && de->d_name[1]==0)
de->d_name[0] = 0;
else if(de->d_name[0]=='.' && de->d_name[1]=='.' && de->d_name[2]==0)
de->d_name[0] = 0;
m++;
p += de->d_reclen;
}
return m;
}
I haven't made sense of dirpackage yet, but I think I may get around that detail if I could get the return of getdents in another way.
From the link you posted to the dirpackage method:
for(i=0; i<n; i++){
de = (struct dirent*)p;
memset(&lst, 0, sizeof lst);
if(de->d_name[0] == 0)
/* nothing */ {}
else if(lstat(de->d_name, &lst) < 0)
de->d_name[0] = 0;
else{
st = lst;
if(S_ISLNK(lst.st_mode))
stat(de->d_name, &st);
nstr += _p9dir(&lst, &st, de->d_name, nil, nil, nil);
}
p += de->d_reclen;
}
Here n comes from:
n = countde(buf, n);
... where the original value of n is supplied from the return of the getdents syscall. The name countde probably stands for "count directory entries".
As you can see from the loop, the return from countde represents the number of entries returned by the getdents call. Each iteration through the loop processes one directory entry (de = (struct dirent*)p;) and then finds the next one (p += de->d_reclen;).
That should be pretty straight-forward to translate to using readdir, since it only ever returns one entry.

WinDivert issue - redirecting DNS back to self on windows

I was looking at basil00's torwall, and for fun was trying to pare it down to just intercept DNS. (provide an answer back to myself of 127.0.0.1 for webfiltering purposes, learning project)
however, at this point, I have it hijacking the dns packet, but it does not return a correct address. for every "blocked" domain, it's different.
for example, I put cbc.ca in my hosts.deny file (blacklist), and it returns an address of 0.4.114.2
then blacklisting slashdot, it will return 0.4.0.1
this has been quite confusing and frustrating, and after three days of research, I am out of ideas.
Here is the code to the redirect portion of my program, which seems to be where things go awry.
(note some of the comments will be goofy as I was hacking down a program for a different purpose and haven't cleaned it up yet)
#include <stdbool.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include "windivert.h"
#include "domain.h"
#include "main.h"
#include "redirect.h"
#define MAX_PACKET 4096
#define NUM_WORKERS 4
// DNS headers
#define DNS_MAX_NAME 254
struct dnshdr
{
uint16_t id;
uint16_t options;
uint16_t qdcount;
uint16_t ancount;
uint16_t nscount;
uint16_t arcount;
} __attribute__((__packed__));
struct dnsq
{
uint16_t type;
uint16_t class;
} __attribute__((__packed__));
struct dnsa
{
uint16_t name;
uint16_t type;
uint16_t class;
uint32_t ttl;
uint16_t length;
uint32_t addr;
} __attribute__((__packed__));
static DWORD redirect_worker(LPVOID arg);
static int handle_dns(HANDLE handle, PWINDIVERT_ADDRESS addr,
PWINDIVERT_IPHDR iphdr, PWINDIVERT_UDPHDR udphdr, char *data,
size_t data_len);
// State:
static bool redirect_on = false;
static HANDLE handle = INVALID_HANDLE_VALUE;
static HANDLE workers[NUM_WORKERS] = {NULL}; // Worker threads
// Send a packet asynchronously:
static void send_packet(HANDLE handle, void *packet, size_t packet_len,
PWINDIVERT_ADDRESS addr)
{
addr->Direction = WINDIVERT_DIRECTION_INBOUND;
WinDivertHelperCalcChecksums(packet, packet_len, 0);
if (!WinDivertSend(handle, packet, packet_len, addr, NULL))
debug("Send packet failed (err=%d)\n", (int)GetLastError());
}
// Start traffic redirect through Tor:
extern void redirect_start(void)
{
debug("DNS divert START\n");
if (handle != INVALID_HANDLE_VALUE)
return;
handle = WinDivertOpen(
"outbound and udp.DstPort == 53 or inbound and udp.DstPort = 53", 0, 0, 0);
// Launch threads:
redirect_on = true;
for (size_t i = 0; i < NUM_WORKERS; i++)
{
workers[i] = CreateThread(NULL, MAX_PACKET*3,
(LPTHREAD_START_ROUTINE)redirect_worker, (LPVOID)handle, 0, NULL);
if (workers[i] == NULL)
{
exit(EXIT_FAILURE);
}
}
}
// Stop traffic redirect through Tor:
extern void redirect_stop(void)
{
debug("DNS divert STOP\n");
if (handle == INVALID_HANDLE_VALUE)
return;
// Close the WinDivert handle; will cause the workers to exit.
redirect_on = false;
if (!WinDivertClose(handle))
{
exit(EXIT_FAILURE);
}
handle = INVALID_HANDLE_VALUE;
for (size_t i = 0; i < NUM_WORKERS; i++)
{
WaitForSingleObject(workers[i], INFINITE);
workers[i] = NULL;
}
}
// Redirect worker thread:
static DWORD redirect_worker(LPVOID arg)
{
HANDLE handle = (HANDLE)arg;
// Packet processing loop:
char packet[MAX_PACKET];
UINT packet_len;
WINDIVERT_ADDRESS addr;
while (redirect_on)
{
if (!WinDivertRecv(handle, packet, sizeof(packet), &addr, &packet_len))
{
// Silently ignore any error.
continue;
}
PWINDIVERT_IPHDR iphdr = NULL;
PWINDIVERT_TCPHDR tcphdr = NULL;
PWINDIVERT_UDPHDR udphdr = NULL;
PVOID data = NULL;
UINT data_len;
WinDivertHelperParsePacket(packet, packet_len, &iphdr, NULL, NULL,
NULL, &tcphdr, &udphdr, &data, &data_len);
int dnshandle = 0;
if (udphdr != NULL && ntohs(udphdr->DstPort) == 53)
dnshandle = handle_dns(handle, &addr, iphdr, udphdr, data, data_len);
if(dnshandle != 1)
{
if (!WinDivertSend(handle, packet, packet_len, &addr, NULL))
{
}
}
}
return 0;
}
// Handle DNS requests.
// NOTES:
// - If anything goes wrong, we simply drop the packet without error.
// - An alternative approach would be to let Tor resolve the address, however,
// this would be slow.
static int handle_dns(HANDLE handle, PWINDIVERT_ADDRESS addr,
PWINDIVERT_IPHDR iphdr, PWINDIVERT_UDPHDR udphdr, char *data,
size_t data_len)
{
struct dnshdr *dnshdr = (struct dnshdr *)data;
data += sizeof(struct dnshdr);
data_len -= sizeof(struct dnshdr);
char name[DNS_MAX_NAME + 8]; // 8 bytes extra.
size_t i = 0;
while (i < data_len && data[i] != 0)
{
size_t len = data[i];
if (i + len >= DNS_MAX_NAME)
return -1;
name[i++] = '.';
for (size_t j = 0; j < len; j++, i++)
name[i] = data[i];
}
name[i++] = '\0';
// Generate a fake IP address and associate it with this domain name:
uint32_t fake_addr = domain_lookup_addr(name);
if (fake_addr == 0)
{
// This domain is blocked; so ignore the request.
// Construct a query response:
size_t len = sizeof(struct dnshdr) + data_len + sizeof(struct dnsa);
if (len > 512) // Max DNS packet size.
return -1;
len += sizeof(WINDIVERT_IPHDR) + sizeof(WINDIVERT_UDPHDR) + len;
char buf[len + 8]; // 8 bytes extra.
PWINDIVERT_IPHDR riphdr = (PWINDIVERT_IPHDR)buf;
PWINDIVERT_UDPHDR rudphdr = (PWINDIVERT_UDPHDR)(riphdr + 1);
struct dnshdr *rdnshdr = (struct dnshdr *)(rudphdr + 1);
char *rdata = (char *)(rdnshdr + 1);
UINT local_ip;
DivertHelperParseIPv4Address("127.0.0.1",&local_ip);
memset(riphdr, 0, sizeof(WINDIVERT_IPHDR));
riphdr->Version = 4;
riphdr->HdrLength = sizeof(WINDIVERT_IPHDR) / sizeof(uint32_t);
riphdr->Length = htons(len);
riphdr->Id = htons(0xF00D);
WINDIVERT_IPHDR_SET_DF(riphdr, 1);
riphdr->TTL = 64;
riphdr->Protocol = IPPROTO_UDP;
riphdr->SrcAddr = iphdr->DstAddr;
riphdr->DstAddr = iphdr->SrcAddr;
memset(rudphdr, 0, sizeof(WINDIVERT_UDPHDR));
rudphdr->SrcPort = htons(53); // DNS
rudphdr->DstPort = udphdr->SrcPort;
rudphdr->Length = htons(len - sizeof(WINDIVERT_IPHDR));
rdnshdr->id = dnshdr->id;
rdnshdr->options = htons(0x8180); // Standard DNS response.
rdnshdr->qdcount = htons(0x0001);
rdnshdr->ancount = htons(0x0001);
rdnshdr->nscount = 0;
rdnshdr->arcount = 0;
memcpy(rdata, data, data_len);
struct dnsa *rdnsa = (struct dnsa *)(rdata + data_len);
rdnsa->name = htons(0xC00C);
rdnsa->type = htons(0x0001); // (A)
rdnsa->class = htons(0x0001); // (IN)
rdnsa->ttl = htonl(0x00000258) ; // 1 second
rdnsa->length = htons(0x0004);
rdnsa->addr = htonl(local_ip); // Fake address
send_packet(handle, &buf, len, addr);
debug("address: %u\n",addr->Direction);
debug("Intercept DNS %s\n", (name[0] == '.'? name+1: name));
return 1;
}
// Re-inject the matching packet.
/*
/
*/
return 0;
}
Here's the domain lookup side of it (mostly just hacked down to try to get the results I want:
#include <stdbool.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <windows.h>
#include "domain.h"
#include "main.h"
#define RATE_LIMIT 8000
#define rand16() \
(rand() & 0xFF) | ((rand() & 0xFF) << 8)
// Domain blacklist:
struct blacklist
{
size_t size;
size_t len;
char **names;
};
static struct blacklist *blacklist = NULL;
// State:
static struct name *names[UINT16_MAX] = {NULL};
static HANDLE names_lock = NULL;
// Prototypes:
static struct blacklist *domain_blacklist_read(const char *filename);
static bool domain_blacklist_lookup(struct blacklist *blacklist,
const char *name);
static int __cdecl domain_blacklist_compare_0(const void *x, const void *y);
static int domain_blacklist_compare(const char *name0, size_t len,
const char *name1);
// Initialize this module:
extern void domain_init(void)
{
// Load the domain blacklist.
blacklist = domain_blacklist_read("hosts.deny");
}
// Lookup an address given a domain name. If the name does not exist then
// create one.
extern uint32_t domain_lookup_addr(const char *name0)
{
if (name0[0] == '.')
name0++;
if (domain_blacklist_lookup(blacklist, name0))
{
debug("Block %s\n", name0);
return 0; // Blocked!
}
return;
}
// Read the blacklist file:
static struct blacklist *domain_blacklist_read(const char *filename)
{
struct blacklist *blacklist =
(struct blacklist *)malloc(sizeof(struct blacklist));
if (blacklist == NULL)
{
exit(EXIT_FAILURE);
}
blacklist->size = 0;
blacklist->len = 0;
blacklist->names = NULL;
FILE *stream = fopen(filename, "r");
if (stream == NULL)
{
return blacklist;
}
// Read blocked domains:
int c;
char buf[256];
while (true)
{
while (isspace(c = getc(stream)))
;
if (c == EOF)
break;
if (c == '#')
{
while ((c = getc(stream)) != '\n' && c != EOF)
;
continue;
}
size_t i = 0;
while (i < sizeof(buf)-1 && (c == '-' || c == '.' || isalnum(c)))
{
buf[i++] = c;
c = getc(stream);
}
if (i >= sizeof(buf)-1 || !isspace(c))
{
exit(EXIT_FAILURE);
}
buf[i] = '\0';
if (blacklist->len >= blacklist->size)
{
blacklist->size = (blacklist->size == 0? 32: 2 * blacklist->size);
blacklist->names = (char **)realloc(blacklist->names,
blacklist->size * sizeof(char *));
if (blacklist->names == NULL)
{
exit(EXIT_FAILURE);
}
}
size_t size = (i+1) * sizeof(char);
char *name = (char *)malloc(size);
if (name == NULL)
{
exit(EXIT_FAILURE);
}
for (size_t j = 0; j < i; j++)
name[j] = buf[i - 1 - j];
name[i] = '\0';
blacklist->names[blacklist->len++] = name;
}
fclose(stream);
qsort(blacklist->names, blacklist->len, sizeof(char *),
domain_blacklist_compare_0);
return blacklist;
}
// Check if a domain matches the blacklist or not:
static bool domain_blacklist_lookup(struct blacklist *blacklist,
const char *name)
{
if (blacklist->len == 0)
return false;
size_t len = strlen(name);
ssize_t lo = 0, hi = blacklist->len-1;
while (lo <= hi)
{
ssize_t mid = (lo + hi) / 2;
int cmp = domain_blacklist_compare(name, len, blacklist->names[mid]);
if (cmp > 0)
hi = mid-1;
else if (cmp < 0)
lo = mid+1;
else
return true;
}
return false;
}
// Domain compare function(s):
static int __cdecl domain_blacklist_compare_0(const void *x, const void *y)
{
const char *name0 = *(const char **)x;
const char *name1 = *(const char **)y;
return strcmp(name0, name1);
}
static int domain_blacklist_compare(const char *name0, size_t len,
const char *name1)
{
size_t i = 0;
ssize_t j = (ssize_t)len - 1;
for (; j >= 0 && name1[i] != '\0'; i++, j--)
{
int cmp = (int)name1[i] - (int)name0[j];
if (cmp != 0)
return cmp;
}
if (j < 0 && name1[i] != '\0')
return 1;
return 0;
}
any assistance is appreciated.
Also, I have uploaded the code to: Github
Thank you.
There's one of two things going on here. You're reading and writing DNS packets incorrectly, and or you're failing to convert the addresses to or from host to network order before working with them.
I'm going to bet on you reading and writing DNS packets incorrectly. I've implemented my own filtering systems using WinDivert where I was hijacking and passing all DNS traffic through my own local DNS server and I've seen these fudged addresses come out exactly as the results you're getting when I was parsing and writing DNS packets incorrectly.
The bad news is, I can't point you to a full DNS library in C/C++ as I know of none that actually make the job easier (maybe, see edits). I personally had my diversion code written in C++ with WinDivert and then use Arsoft.Tools.Net C# DNS library for actually running a local DNS server and manipulating DNS responses.
There is one project in C++ that calls itself boost::net::dns because I think the writer was hoping it would be part of boost, which it isn't and probably won't be. Avoid this library as the internal mechanism for parsing and storing multiple A Records is bugged out and broken and you'll get the same wacky results you're getting here. I tried to work with him to get it fixed but, he was only interested in blaming my code.
I'll have a gander again and update my answer if I can find a decent lib for working with DNS packets in C++. Don't try to do it yourself, we're talking about entire protocols with entire books of RFC's to do this properly yourself.
Update
As promised, here are some results from my search:
Mozilla Necko (Formerly Netlib)
http://mxr.mozilla.org/mozilla-central/source/netwerk/dns/
C-Ares:
https://github.com/bagder/c-ares
A list of alternative libraries C-Ares compiled:
http://c-ares.haxx.se/otherlibs.html
Also, it is linux specific but pretty clean code. You might be able to pilfer through libcrafter's DNS classes and move them to your own project.
https://github.com/pellegre/libcrafter/blob/master/libcrafter/crafter/Protocols/DNS.h
Again, unless your "learning project" is understanding and recreating one of the foundational pillars of the internet, don't implement this yourself. Try and use one of the previously listed libraries to handle your DNS packets.

Resources