TL;DR...
I need to encrypt an in-RAM C struct, byte-for-byte, using OpenSSL / EVP RC4 stream-cipher. How do I implement EVP (e.g. EVP_CipherUpdate) to accomplish the actual encryption of bytes in the struct?
The Details...
I have a ISAM/BTree database that needs its records/data encrypted. Don't worry about that, just know that each "record" is a C struct with many members (fields). This has been working for like 15 years (don't ask, the codebase is from the K&R C days, I think). The ISAM overhead simply takes a bytestream (the struct w/data) as an argument when writing the record... specifically, the ISAM's "write" function accepts a pointer to the data/structure.
Anyway, I'm trying to implement an OpenSSL stream cipher (RC4) -via the EVP framework- that can just sit between my program and the ISAM, so as to simply hand the ISAM my encrypted bytes, without him knowing or caring. I might add that I think the ISAM doesn't care about the structure of the data or even that it's a struct... it just gets raw data, I believe.
My struct is like this simplified example (in reality there are many more varied members):
typedef struct
UCHAR flag;
char field2[30];
int qty;
} DB_REC;
How would I (if it's even possible) go about encrypting the entire structure (in-place, even), byte for byte, if necessary? I've tried testing with simple strings, even; but can't get that to work, either.
I have another file called crypto.c (and .h) where I'm building my functions to encrypt and decrypt whatever I "pass" to them (might be a string, a struct, whatever - that's why my arg is void). For example:
void encrypt_db_rawData(void *data_to_encrypt, size_t data_size)
{
unsigned char buf_out[data_size];
int buf_out_byteCount = 0;
buf_out_byteCount = sizeof(buf_out);
EVP_CIPHER_CTX ctx; //declare an EVP context
int keyBytes = 16; //size of RC4 key in bytes
/* ... my_keystr is acquired from a file elsewhere ... */
/* ... my_iv is a global defined elsewhere ... */
EVP_CIPHER_CTX_init(&ctx);
EVP_CipherInit_ex(&ctx, EVP_rc4(), NULL, NULL, NULL, 1);
EVP_CIPHER_CTX_set_key_length(&ctx, keyBytes);
EVP_CipherInit_ex(&ctx, NULL, NULL, my_keystr, my_iv, 1);
int byte_i;
for( byte_i = 0; byte_i < sizeof(data_to_encrypt); i++ )
{
EVP_CipherUpdate(&ctx, buf_out, &buf_out_byteCount, data_to_encrypt[byte_i], 1);
}
*data_to_encrypt = buf_out; //overwrite the original data with its encrypted version
}
C is not my first language ~ especially C89 (or older perhaps), and I'm in way over my head on this one - this got thrown into my lap due to some restructuring; so I appreciate any constructive help anyone can offer. I'm in pointer-HELL!
Related
I'm playing with libgcrypt (v1.6.1 on Gentoo x64) and i've already implemented (and tested thorugh the AEs test vectors) aes256-cbc and aes256-ctr. Now i am looking at aes256-gcm but i have some doubt about the workflow. Below there is a skeleton of a simple encryption program:
int main(void){
unsigned char TEST_KEY[] = {0x60,0x3d,0xeb,0x10,0x15,0xca,0x71,0xbe,0x2b,0x73,0xae,0xf0,0x85,0x7d,0x77,0x81,0x1f,0x35,0x2c,0x07,0x3b,0x61,0x08,0xd7,0x2d,0x98,0x10,0xa3,0x09,0x14,0xdf,0xf4};
unsigned char TEST_IV[] = {0x00,0x01,0x02,0x03,0x04,0x05,0x06,0x07,0x08,0x09,0x0a,0x0b,0x0c,0x0d,0x0e,0x0f};
unsigned char TEST_PLAINTEXT_1[] = {0x6b,0xc1,0xbe,0xe2,0x2e,0x40,0x9f,0x96,0xe9,0x3d,0x7e,0x11,0x73,0x93,0x17,0x2a};
unsigned char cipher[16] = {0};
int algo = -1, i;
const char *name = "aes256";
algo = gcry_cipher_map_name(name);
gcry_cipher_hd_t hd;
gcry_cipher_open(&hd, algo, GCRY_CIPHER_MODE_GCM, 0);
gcry_cipher_setkey(hd, TEST_KEY, 32);
gcry_cipher_setiv(hd, TEST_IV, 16);
gcry_cipher_encrypt(hd, cipher, 16, TEST_PLAINTEXT_1, 16);
char out[33];
for(i=0;i<16;i++){
sprintf(out+(i*2), "%02x", cipher[i]);
}
out[32] = '\0';
printf("%s\n", out);
gcry_cipher_close(hd);
return 0;
}
In GCM mode there want also these instruction:
gcry_cipher_authenticate (gcry cipher hd t h , const void * abuf , size t abuflen )
gcry_error_t gcry_cipher_gettag (gcry cipher hd t h , void * tag , size t taglen )
So the correct workflow of the encryption program is:
gcry_cipher_authenticate
gcry_cipher_encrypt
gcry_cipher_gettag
But what i haven't undestood is:
abuf is like a salt? (so have i to generate it using gcry_create_nonce or similar?)
If i want to encrypt a file, void *tag is what i have to write to the outfile?
1) gcry_cipher_authenticate is for supporting authenticated encryption with associated data. abuf is data that you need to authenticate but do not need to encrypt. For example, if you are sending a packet, you might want to encrypt the body, but you must send the header unencrypted for the packet to be delivered. The tag generated by the cipher will provide integrity for both the encrypted data and the data sent in plain.
2) The tag is used after decryption to make sure that the data has not been tampered with. You append the tag to the encrypted text. Note, that it is computed on encrypted data and associated (unencrypted) data, so you will need both when decrypting.
You can check these documents for more information on GCM:
http://csrc.nist.gov/groups/ST/toolkit/BCM/documents/proposedmodes/gcm/gcm-spec.pdf
http://csrc.nist.gov/publications/nistpubs/800-38D/SP-800-38D.pdf
Also, you can probably get faster answers to cryptography questions like this on http://crypto.stackexchange.com.
this is my first question so please tell me if I do something wrong :).
My problem is I use
EVP_DecryptInit(&ctx1, EVP_des_ecb(), tmpkey, NULL);
EVP_DecryptUpdate(&ctx1, keysigout, &outlu ,keysigin, keysigfilelength);
EVP_DecryptFinal(&ctx1, keysigout, &outlf);
printf("DECLEN:%i",outlu + outlf);
to decrypt a binary file. The file is 248 bytes long but the printf only tells me EVP decrypted 240 bytes. keysigfilelength is 248 and should tell the update that 248 bytes need to be decrypted.
I dont understand why this doesnt work and would be happy if you can enlighten me.
Edit:
I just encrypted a file manually with the command
openssl enc -e -des-ecb -in test.txt -out test.bin -K 00a82b209cbeaf00
and it grew by 8 bytes :O. I still don't know where they come from but I don't think the general error I have in my program is caused by this.
The context of this whole problem is an information security course at my university. We got similar Tasks with different algorithms, but even someone who has done his program successfully couldnt figure out where the problem in my program is.
Is it ok to post my whole program for you?
I hope its fine to answer my own question.
EVP_DecryptUpdate(&ctx1, keysigout, &outlu ,keysigin, keysigfilelength);
EVP_DecryptFinal(&ctx1, keysigout + outlu, &outlf);
The problem was the missing outlu, DecryptFinal tried to decrypt the whole block again. When i added the outlu i got 7 byte in outlf, and it worked.
For future reference i add the whole function below.
It expects the key and iv to be one block of data.
int decrypt(const EVP_CIPHER *cipher,unsigned char *key, unsigned char *encryptedData, int encryptedLength,unsigned int * length, unsigned char ** decryptedData)
{
int decryptedLength = 0, lastDecryptLength = 0, ret;
unsigned char * iv = NULL;
EVP_CIPHER_CTX *cryptCtx = EVP_CIPHER_CTX_new();
EVP_CIPHER_CTX_init(cryptCtx);
*decryptedData = malloc (encryptedLength * sizeof(char));
if(cipher->iv_len != 0) iv = key + cipher->key_len;
EVP_DecryptInit_ex(cryptCtx, cipher, NULL, key, iv);
EVP_DecryptUpdate(cryptCtx, *decryptedData, &decryptedLength, encryptedData, encryptedLength);
ret = EVP_DecryptFinal_ex(cryptCtx, *decryptedData + decryptedLength, &lastDecryptLength);
*length = decryptedLength + lastDecryptLength;
EVP_CIPHER_CTX_free(cryptCtx);
EVP_cleanup();
return ret;
}
Because block ciphers only really want to work on an input that is a multiple of their block size, the input is normally padded to meet this requirement.
The default for many programs (including openssl enc is to use PKCS #5 padding
If the plaintext is not a multiple of 8 bytes then padding bytes are added so that it is. If it is already a multiple of 8 bytes then 8 bytes of padding are added. Thus it is entirely normal for the encrypted data to be longer than the plaintext.
It seems like my previous post but issue here is different ..
This is the C structure for problem -
typedef struct ip_esp_private { /* keep track of things privately */
u_int32_t type;
u_int32_t ivlen;
u_int32_t icvlen;
u_int32_t keylen; /* length of "Encryption key */
u_int32_t akeylen; /*length of authn key */
u_int32_t key[0]; /* encryption key and authentication key both */
} esp_private;
The values are provided to structure contents at run time as follows -
case 'k': /* Key */
length = stringargument(arg, &temp);
priv->keylen = length;
priv = (esp_private *)realloc(priv,
sizeof(esp_private)+/*length*/priv->keylen);
/*This one is edited */
// if(priv->akeylen)
// memmove(&priv->key[priv->keylen],
// &priv->key[0],priv->akeylen);
/*These three are commented*/
memcpy(&priv->key[0], temp, priv->keylen);
pack->private = priv;
pack->modified |= ESP_MOD_KEY;
break;
case 'K': /* Authentication Key */
length = stringargument(arg, &temp);
priv->akeylen = length; // marked line(explained below)
priv = (esp_private *)realloc(priv,
sizeof(esp_private)+/*length*/priv->keylen+priv->akeylen);
/*this one edited too */
memcpy(&priv->key[priv->keylen/sizeof(u_int32_t)],
temp,priv->akeylen);
pack->private = priv;
pack->modified |= ESP_MOD_KEY;
Now there is a function which uses the value of authentication key.
The relevant part of the function is -
if (!epriv->akeylen) {
key = &fakekey;
keylen = 1;
} else {
key = (u_int8_t *)malloc(epriv->akeylen);
memcpy(key,&epriv->key[epriv->keylen/sizeof(u_int32_t)]
,epriv->akeylen);
Now when I tried to run the following program , getting this error about which I have no idea.
sendip: malloc.c:3574: mremap_chunk: Assertion `((size + offset)
& (mp_.pagesize-1)) == 0' failed.
I think may be there is a error in function part but what exactly it is I am not sure,
because when I comment the marked line (mentioned above) the akeylen is null
so taking that fakekey value and program runs fine.
Edit 1:
I have edited the code at three places (also edited in the above code ).
Now program works but an inconsistent output occurs.
Input :
Encryption key - qwerty
Authentication key - abcdef
Output:
Encryption key - qwerab
Authentication key - abcdef
The situation is more clear now .
The problem it means is surely there at realloc statements .
Please suggest on this.
Initially I added length at both realloc statements but now I changed it to priv->keylen at first place and priv->keylen+priv->akeylen at secone place.
But something still needs to be improved
Why this is overwriting ???
Since the key[0] struct hack appears to contain space for both keys, you'll need to allocate memory for both, too. In both cases ('k' and 'K' )
priv = realloc(priv, sizeof *priv +priv->keylen+priv->akeylen);
When concatenating the two keys, it is easiest to cast the u_int32_t key into a character pointer and do arithmatic on that one:
memcpy ( priv->key, source1, sizeofsource1);
/* and */
memcpy ( ((char*) priv->key) +priv->keylen, source2, sizeofsource2);
[and similar for the memmove()]
The rest of the casts in your program can be removed.
If you get assertion failures from within malloc, the problem is outside. The assertion is not about the parameters passed to malloc, but about the state of the memory, which is corrupted. This means that you previously wrote to a memory area you were not supposed to write. So even if you provided a proper traceback (using gdb for instance), this would not point you to the source of the problem. There are a number of tools for debugging memory problems. One of the most widely used tools is valgrind. It will make your program horribly slow and show you tons of possible problems by looking at every single memory access. Another more lightweight tool is mudflap, which is to be linked against. A very basic approach to narrowing down the problem is to add assert(condition) statements to your code and hope that you fail earlier. Sometimes you can solve this problem by looking at every single memory access in your code and ensure that it is not out of bounds (or if you are unsure, add an assertion statement).
I am working with libmcrypt in c and attempting to implement a simple test of encryption and decryption using rijndael-256 as the algorithm of choice. I have mirrored this test implementation pretty closely to the man pages examples with rijndael as opposed to their chosen algorithms. When compiled with the string gcc -o encryption_test main.c -lmcrypt, the following source code produces output similar to:
The encrypted message buffer contains j��A��8 �qj��%`��jh���=ZЁ�j
The original string was ��m"�C��D�����Y�G�v6��s��zh�
Obviously, the decryption part is failing, but as it is just a single function call it leads me to believe the encryption scheme is not behaving correctly as well. I have several questions for the libmcrypt gurus out there if you could point me in the right direction.
First, what is causing this code to produce this broken output?
Second, when dealing with mandatory fixed-sizes such as the key size and block-size, for example a 256-bit key does the function expect 32-bytes of key + a trailing null byte, 31-bytes of key + a trailing null byte, or 32-bytes of key with the 33rd byte being irrelevant? The same question holds true for block-size as well.
Lastly, one of the examples I noted used mhash to generate a hash of the key-text to supply to the encryption call, this is of course preferable but it was commented out and linking in mhash seems to fail. What is the accepted way of handling this type of key-conversion when working with libmcrypt? I have chosen to leave any such complexities out as to prevent further complicating already broken code, but I would like to incorporate this into the final design. Below is the source code in question:
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <mcrypt.h>
int main(int argc, char *argv[])
{
MCRYPT mfd;
char *key;
char *plaintext;
char *IV;
unsigned char *message, *buffered_message, *ptr;
int i, blocks, key_size = 32, block_size = 32;
message = "Test Message";
/** Buffer message for encryption */
blocks = (int) (strlen(message) / block_size) + 1;
buffered_message = calloc(1, (blocks * block_size));
key = calloc(1, key_size);
strcpy(key, "&*GHLKPK7G1SD4CF%6HJ0(IV#X6f0(PK");
mfd = mcrypt_module_open(MCRYPT_RIJNDAEL_256, NULL, "cbc", NULL);
if(mfd == MCRYPT_FAILED)
{
printf("Mcrypt module open failed.\n");
return 1;
}
/** Generate random IV */
srand(time(0));
IV = malloc(mcrypt_enc_get_iv_size(mfd));
for(i = 0; i < mcrypt_enc_get_iv_size(mfd); i++)
{
IV[i] = rand();
}
/** Initialize cipher with key and IV */
i = mcrypt_generic_init(mfd, key, key_size, IV);
if(i < 0)
{
mcrypt_perror(i);
return 1;
}
strncpy(buffered_message, message, strlen(message));
mcrypt_generic(mfd, buffered_message, block_size);
printf("The encrypted message buffer contains %s\n", buffered_message);
mdecrypt_generic(mfd, buffered_message, block_size);
printf("The original string was %s\n", buffered_message);
mcrypt_generic_deinit(mfd);
mcrypt_module_close(mfd);
return 0;
}
You need to re-initialize the descriptor mfd for decryption, you cannot use the same descriptor for both encryption and decryption.
I have been given a sample encryption module which simply xor a key with the packet data and place the result in data field of constructed packet itself....
The code for xoricv module is here.
void
xorcrypto(uint8_t *key, uint32_t keylen,
uint8_t *data, uint32_t datalen)
{
int d, k;
for (d=0, k=0; d < datalen; ++d, k = (k+1)%keylen) {
data[d] ^= key[k];
}
}
Now i need to change the module so that it performed the AES Encryption rather than simple xor operation.
Would you suggest possible transformation i need to do ????
It is a small part of my project and i am stuck in between..
The AES Command I have used on command line for encryption
openssl enc -aes-256-cbc -salt -in file.txt -out file.enc
Daily i am going through plenty of errors when i tried to implement it by my own and i have very limited time so please help me.......
Here is my implementation ...... although i am adding it here in answer column because may be there is small bug in my implementation which i could be able to solve through some valuable suggestions but still If any other way is possible please suggest and provide me some implementation code...
/* u_int8_t ... etc all are typedefs for uint8_t....etc
so don't bother about them */
void xorcrypto(u_int8_t *key, u_int32_t keylen,u_int8_t *data,
u_int32_t datalen)
{
int ch,i,j;
uint8_t modata[100];
FILE *fp,*fr,*fq,*ft;
fp=fopen("key","w");
fputs((char *)key,fp);
fq=fopen("file.txt","w");
fputs((char *)data,fq);
fclose(fp);
fclose(fq);
system("sudo openssl enc -aes-256-cbc -salt -in file.txt -out file.enc -pass file:key");
fr=fopen("file.enc","r");
memset(data,0,sizeof(data));
i=0;
while( (ch=fgetc(fr))==EOF) {
data[i]=ch;
i++;
}
fclose(fr);
system("sudo openssl enc -d -aes-256-cbc -salt -in file.enc
-out file1.txt -pass file:key");
ft=fopen("file1.txt","r");
memset(modata,0,sizeof(modata));
j=0;
while( (ch=fgetc(ft)) != EOF) {
modata[j]=ch;
j++;
}
fclose(ft);
}
Call to function in module is described as -
bool
espcrypto(esp_private *epriv, sendip_data *data, sendip_data *pack)
{
u_int32_t keylen;
u_int8_t *key;
static u_int8_t fakekey;
struct ip_esp_hdr *esp = (struct ip_esp_hdr *)pack->data;
if (!epriv->keylen) { /* This isn't going to be very productive... */
key = &fakekey;
keylen = 1;
} else {
key = (u_int8_t *)epriv->key;
keylen = epriv->keylen;
}
/* Encrypt everything past the ESP header */
xorcrypto(key, keylen,
(u_int8_t *)esp->enc_data,
pack->alloc_len + data->alloc_len -
sizeof(struct ip_esp_hdr));
return TRUE;
}
This is code in the xorcrypto.c file in packet generator tool i am using which is linked during packet construction through command line as -am xorcrypto.so .That is the reason why I was being so lazy.I am first looking for an implementation which can act as proof of concept .Rest all optimizations can be done later.
The output I am getting is - encryption not performed at all the data is still in plain text in packet.
udit#udit-Dabba ~/Downloads/sendip-2.5-mec-2/mec $ cat file.txt
udit#udit-Dabba ~/Downloads/sendip-2.5-mec-2/mec $ cat file.enc
Salted__����
}�#��G�����0����iudit#udit-Dabba ~/Downloads/sendip-2.5-mec-2/mec $ cat file1.txt
udit#udit-Dabba ~/Downloads/sendip-2.5-mec-2/mec $
Why file.txt is null even key file is not updated ???
If any other information needed i will add it here......but please help me getting out of the mesh .
This may not even be possible without making some additional changes to your software. It doesn't look like your function allows for the resulting encrypted data to be larger than the unencrypted data, for one, which will be required for RSA.
The key you pass to your function will also be quite different; one side of an RSA key will consist of the modulus and an exponent. The modulus will be a large number (not representable with the normal integer types), and the exponent is typically a large number for one side and a (relatively) small one for the other.
Aside from that, there are a number of issues and difficulties with RSA. You could try to deal with these yourself, but you'd probably be better off using an existing library like OpenSSL. Some of these issues include:
Implementing RSA (with usefully long keys) requires modular arithmetic on very large numbers, far larger than any of the normal integer types. You'd either have to write functions for this, or find a library.
The data has to be broken into pieces that are shorter than the modulus (part of the key), including whatever padding and such is encrypted along with the data. The length of each piece after encryption will be the length of the modulus. That's why the resulting encrypted data will be longer than the original data (and also padding).
Avoiding certain vulnerabilities requires additional steps, such as adding random padding into each piece of data and ensuring that the padded data, raised to the exponent from the key, would exceed the modulus (if it weren't done as modular exponentiation).
So first, you'll need to make it possible for your encryption function to return more data than you gave it. Then you should probably look at using an encryption library to do the actual encryption, to save a lot of work and reduce the chances of letting vulnerabilities slip through.
Well,
I feel the line :
while( (ch=fgetc(fr))==EOF)
// Also, Similar statements that you have used many times.
You are reading 1 character from the file and comparing that to EOF. That works fine untill you are working with normal text files.
But here you are working with an encrypted file, file.enc which can contain anything at all.
It can even posses EOF character itself as data content.
That means if file has 100 characters and 2nd character is EOF then it will terminate at 2nd character itself.
I feel this as a possible problem.
Remains.. the other problems - I suggest using fprintf() instead of fputs(), these are easy to use and hence avoids unseen bugs. Since you are working on strings instead of characters logically.
Moreover, using these you also get an advantage of string formatting when needed.
Try out and then get back.. :)