RAND_bytes not invoking though setting a RAND_set_rand_method()? - c

Question

Even though we set currentMethod.bytes with local function to generate random numbers, the RAND_bytes is not invoking. After we set RAND_set_rand_method(&cuurentMethod).
Here I attached link [https://github.com/openssl/openssl/blob/master/test/sm2_internal_test.c] which I already tried.
int main()
{
unsigned char rand[16];
int ret;
RAND_METHOD *oldMethod,currentMethod,*temp;
oldMethod = RAND_get_rand_method();/*getting default method*/
currentMethod = *oldMethod;
currentMethod.bytes = local_function_rand;
if((ret = RAND_set_rand_method(&currentMethod))!= 1)
return 0;
/* Now we are printing both address of local_function_method_rand() and
temp->bytes , those address are same after getting. */
temp = RAND_get_rand_method();
/* after we are comparing with RAND_SSLeay() function , to find default or not*/
if((ret = RAND_bytes(rand,16)) != 1)
return 0;
return 1;
}
Expecting result is our local function should invoke. Also, to invoke RAND_bytes() is it required to set fips mode in Linux system?

Answer 1

After cleaning up and minimizing your test program and filling in the missing parts:
#include <openssl/rand.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
int local_function_rand(unsigned char *buf, int num) {
printf("in local_function_rand(); requested %d bytes\n", num);
memset(buf, 4, num); // RFC 1149.5 standard random number
return 1;
}
int main(void) {
unsigned char rand[16];
RAND_METHOD currentMethod = {.bytes = local_function_rand};
RAND_set_rand_method(&currentMethod);
if (RAND_bytes(rand, sizeof rand) != 1) {
return EXIT_FAILURE;
}
return 0;
}
and running it (With OpenSSL 1.1.1):
$ gcc -Wall -Wextra rand.c -lcrypto
$ ./a.out
in local_function_rand(); requested 16 bytes
it works as expected; the user-supplied function is being called by RAND_bytes(). If you're getting different results from your code, there's probably a problem in the bits you didn't include in your question.