I tried to add a new digest algorithm to a new defined engine in Openssl.I have added new nids to file crypto/objects/obj_mac.num:
simple_cipher 920
id_simple_cipher 921
and added this line to crypto/objects/object.txt:
!Cname id-simple_cipher
cryptopro 100 : simple_cipher : SIMPLE 28147-89
and this to lvalues array :
0x2A,0x85,0x03,0x02,0x02,0x64, /* [5973] OBJ_id_simple_cipher */
and when I compiled Openssl the following lines was added to file crypto/objects/obj_mac.h:
#define SN_id_simple_cipher "simple_cipher"
#define LN_id_simple_cipher "SIMPLE 28147-89"
#define NID_id_simple_cipher 921
#define OBJ_id_simple_cipher OBJ_cryptopro,100L
and these two lines to nid_objs array in crypto/objects/obj_dat.h:
{NULL,NULL,NID_undef,0,NULL,0},
{"simple_cipher","SIMPLE 28147-89",NID_id_simple_cipher,6,
&(lvalues[5973]),0},
and also nid is added to the other arrays in this file.
but when I try to load engine or run this command openssl engine -c engine-name I cause segmentation fault. I think it's because of the added cipher and I'm doing something wrong while defining new nid for cipher.
there is an array in file engines/simple/eng_simple.c (where my engine is defined) named gost_cipher_nids when I add my new defined nid to this array this seg fault happens.
I'm new to openssl I really appreciate any help.
I'm working with version OpenSSL 1.0.1e-fips
Related
I am trying to convert some C code written to run on a Mac to an embedded device that does not have any encryption libraries. The code for the Mac is using libcrypto. I tried building libcrypto from openssl sources for the embedded device but I get hundreds of errors due to function pointer prototypes not matching. openssl is riddled with huge macros. As an alternative I am now trying to use mbedtls but I have not been able to get a decrypt function to work.
The code I am trying to port is a bit odd. It has what it calls a public key and is actually calling RSA_public_encrypt() with no padding to decrypt data. As a test, I changed the Mac code to call RSA_public_decrypt() and it worked so I assume the key is symmetric. The key it is using looks like this:
"-----BEGIN PUBLIC KEY-----\n"
5 lines of Base64 strings
"-----END PUBLIC KEY-----\n"
For mbedtls I am using mbedtls_pk_parse_public_key() to parse the key. If I inspect the low level RSA key structure after parsing the key there is a 128 byte N component and a 16 byte E component. I get the same key data with both openssl and mbedtls so it appears that the key is parsed properly. When decrypting on the Mac with RSA_public_decrypt(), the input and output are both 128 bytes. For mbedtls, I am calling mbedtls_pk_decrypt() to decrypt but when I trace through the code it calls mbedtls_rsa_rsaes_pkcs1_v15_decrypt() which forces the padding to be 11 bytes.
So my questions are: 1) exactly what kind of "public" key contains only N and E components and uses no padding; 2) Am I calling the correct mbedtls decryption function?
EDIT: Tried another approach and my output buffer just gets filled with zeros.
mbedtls_rsa_context rsa;
mbedtls_rsa_init(&rsa, MBEDTLS_RSA_PKCS_V15, 0);
mbedtls_rsa_import_raw(&rsa, modulus, sizeof(modulus), NULL, 0, NULL, 0, NULL, 0, exp, sizeof(exp));
mbedtls_rsa_complete(&rsa);
mbedtls_rsa_public(&rsa, inBfr, outBfr);
mbedtls_rsa_free(&rsa);
EDIT 2: My ultimate target is an embedded device with an ARM processor but I was testing on Windows to see if mbedtls would work. I started with VS 2010 because that is what was being used for the project I am working on. I switched to VS 2015 and the 2nd approach of importing the raw key data and calling mbedtls_rsa_public() worked perfectly. I guess the VS 2010 compiler just isn't good enough. I then ported the code to the devlepment system for my embedded device and it also worked.
The equivalent of OpenSSL's RSA_public_encrypt(…, RSA_NO_PADDING) would be mbedtls_rsa_public. The function mbedtls_pk_encrypt only lets you access encryption mechanisms based on RSA (RSAES-PKCS1-v1_5 and RSAES-OAEP), not the raw RSA primitive (“textbook RSA” a.k.a. “RSA with no padding”).
You need to call mbedtls_pk_parse_public_key to parse the key, then get a pointer to the RSA key object with mbedtls_pk_rsa, and call mbedtls_rsa_public using this RSA key object.
mbedtls_pk_context pk;
mbedtls_pk_init(&pk);
mbedtls_rsa_context *rsa = mbedtls_pk_rsa(&pk);
mbedtls_rsa_public(&rsa, inBfr, outBfr);
mbedtls_pk_free(&pk);
I think this should work from your description, but obviously I can't test this without sample data.
I am currently reading a book entitled "Linux device drivers" from O'Reilly.
Thing is that this book imo isn't really a guide on how to write drivers but it instead explains all the apis and their prinicples.
So I tried writing a small driver -which doesn't do anything interesting -with what I read so far.
Thing is:
I don't know which file I can execute cat on or echo to in order to invoke my callback functions
it looks nothing like all the other code snippets I found online
The different pieces of code:
my code (https://paste.ubuntu.com/p/8tVyTJTPBQ/)
creates:
$ls /sys/module/main/
oresize holders initsize initstate notes refcnt sections srcversion taint uevent
no new entry in /dev
code snippet using device_create: https://paste.ubuntu.com/p/cJxjdyXjhX/ source
creates:
$ ls /sys/module/main/
coresize holders initsize initstate notes refcnt sections srcversion taint uevent
$ ls -l /dev/ebbchar
crw------- 1 root root 238, 0 Mai 28 07:52 /dev/ebbchar
code using kobjects: https://paste.ubuntu.com/p/nt3XvZs7vF/ source
creates:
$ls -l /sys/kernel/
drwxr-xr-x 2 root root 0 Dec 17 16:29 etx_sysfs
I can see that my code successfully created a bunch of files under /sys/kernel. Now what is the difference in endgoal between my code and the two other snippets? Should I use device_create/kobjects or maybe none of those? The book I am reading doesn't mention anywhere the functions used by the 2 other pieces of code. So not sure which way I am supposed to follow...
Thanks_xe
device_create() creates a device and registers it with sysfs, and create necessary kobjects.
To create necessary kobjects, kobject-related functions(kobject_init(), kobject_add(), ...) are called in device_create().
If you need to create a device, you should call one of device creation functions like device_create().
Answering your question of how to keep up with latest api- Most api if renamed or updated retain similar name and reside mostly in the same header file so a quick grep on the source or easier is http://elixir.bootlin.com/ and search the source doc for a particular function in the Linux release you are working on. if you cant find the API in that release then go through the header to find the new API as the name will almost be the same for example when you will read the Timer chapter , you will find that setup_timer() has been changed to timer_setup(). and few other changes here and there.
If you feel like you can keep up with the latest discussion by subscribing to the kernel maillist or reading the documentation.
Quick question: How can I verify the functionality of a custom openSSL engine I'm writing from the command line?
Right now I am following along with this great tutorial, and am successfully able to exercise the engine (which returns a digest value of all 2's) using my test program (source code here, test program located at test/wssha256engine_test.c).
brett#wintermute:~/openssl_ws/wssha256engine$ make test
make[1]: Entering directory
/home/brett/openssl_ws/wssha256engine/test
make[1]: '../bin/test' is up to date.
make[1]: Leaving directory
/home/brett/openssl_ws/wssha256engine/test
brett#wintermute:~/openssl_ws/wssha256engine$ bin/test
Engine Loaded...
Initializing wssha256 engine...
*TEST: Successfuly initialized engine [A test engine for the ws sha256 hardware encryption module, on the Xilinx ZYNQ7000]
init result = 1
Digest NID=672 requested
SHA256 algorithm context initialized
*TEST: Digest INIT 1
SHA256 update
*TEST: Digest Update 1
SHA256 final: sizeof(EVP_MD)= 120
SHA256 cleanup
*TEST: Digest Final 1 Digest size:32
22222222222222222222222222222222
However, because reasons, I would like to also use the openssl command line interface to exercise the engine I just wrote, and have it compute the digest of a random string like in this other tutorial, just using sha256 and not md5.
But when I try and do this, the engine does not load and results in an error telling me that the NID of my digest doesn't exist, and just hashes the string with the standard algorithm instead:
brett#wintermute:~/openssl_ws/wssha256engine$ echo "Hello" | openssl dgst -engine /home/brett/Thesis/openssl_ws/wssha256engine/bin/libwssha256engine.so -sha256
ERROR: Digest is empty! (NID = 0)
engine "wssha256" set.
(stdin)= 66a045b452102c59d840ec097d59d9467e13a3f34f6494e539ffd32c1bb35f18
Why can't I use my engine on the command line, but I can create a C program to load it? And how can I use my engine to compute a digest from the command line?
Note: I am able to load the engine from the command line, just not use it.
After some digging, I was able to figure out why the CLI invocation of my engine did not work, and fix the issue.
The problem was in my digest selector function (original broken code is commented out below). Originally, I returned a failure status from the digest selector function, rather than returning the number of digest IDs that the engine supports.
static int wssha256engine_digest_selector(ENGINE *e, const EVP_MD **digest, const int **nids, int nid)
{
if (!digest)
{
*nids = wssha256_digest_ids;
// THIS ORIGINAL CODE CAUSES THE ENGINE TO NEVER INITIALIZE FROM A CLI INVOCATION
//printf("ERROR: Digest is empty! (NID = %d)\n",nid);
//return FAIL;
// THIS FIXES IT
int retnids = sizeof(wssha256_digest_ids - 1) / sizeof(wssha256_digest_ids[0]);
printf("wssha256engine: digest nids requested...returning [%d]\n",retnids);
return retnids;
}
Reading more closely, we can find that OpenSSL calls the digest selector function in following (separate) ways:
with digest being NULL. In this case, *nids is expected to be assigned a
zero-terminated array of NIDs and the call returns with the number of available NIDs. OpenSSL uses this to determine what digests are supported by this engine.
with digest being non-NULL. In this case, *digest is expected to be assigned the pointer to the EVP_MD structure corresponding to the NID given by nid. The call returns with 1 if the request NID was one supported by this engine, otherwise 0.
So even though my engine supported sha256, the CLI invocation didn't work because it must have not been initialized with a digest before the digest selector function is invoked (no clue as to the order of operations when using the CLI). However when I manually initialized it in my test program, all worked fine. Changing to a non-failure return value to support invocation 1. above fixed the problem.
First, let me start by stating that I am not a cryptographer by any means, and I am not very good at writing c code either, so please excuse me if the answer to this question is obvious or answered. I am developing a messaging program and cannot use TLS on the target platform. As a result, I need to find a way to encrypt each message using a symmetric pre shared key cipher, like AES.
I am seeking a method to encrypt and decrypt data between an mbedtls program (such as aescrypt2) on one end, and a nodejs program on the other. Mbedtls, formerly polarssl, is a library which provides encryption for embedded devices. Included with the source code are some sample programs, like aescrypt2, rsaencrypt, ecdsa and crypt_and_hash.
Aescrypt2 works fine when the resulting encrypted data is also decrypted using aescrypt2, but I cannot seem to get data encrypted with aescrypt to decrypt using nodejs crypto or any other program for that matter, including openssl. For example:
echo 'this is a test message' >test.txt
aescrypt 0 test.txt test.out hex:E76B2413958B00E193
aescrypt 1 test.out test.denc hex:E76B2413958B00E193
cat test.denc
this is a test message
With openssl:
openssl enc -in out.test -out outfile.txt -d -aes256 -k E76B2413958B00E193
bad magic number
Some sample node code that doesn't currently work
var crypto = require('crypto');
var AESCrypt = {};
AESCrypt.decrypt = function(cryptkey, iv, encryptdata) {
encryptdata = new Buffer(encryptdata, 'base64').toString('binary');
var decipher = crypto.createDecipheriv('aes-256-cbc', cryptkey, iv),
decoded = decipher.update(encryptdata, 'binary', 'utf8');
decoded += decipher.final('utf8');
return decoded;
}
AESCrypt.encrypt = function(cryptkey, iv, cleardata) {
var encipher = crypto.createCipheriv('aes-256-cbc', cryptkey, iv),
encryptdata = encipher.update(cleardata, 'utf8', 'binary');
encryptdata += encipher.final('binary');
encode_encryptdata = new Buffer(encryptdata, 'binary').toString('base64');
return encode_encryptdata;
}
var cryptkey = crypto.createHash('sha256').update('Nixnogen').digest(),
iv = 'a2xhcgAAAAAAAAAA',
buf = "Here is some data for the encrypt", // 32 chars
enc = AESCrypt.encrypt(cryptkey, iv, buf);
var dec = AESCrypt.decrypt(cryptkey, iv, enc);
console.warn("encrypt length: ", enc.length);
console.warn("encrypt in Base64:", enc);
console.warn("decrypt all: " + dec);
This results in either errors or garbage text every time. I have tried tweaking a variety of things as well.
I've tried this a hundred different ways, including using the -pass pass:password arg to no avail. Using nodejs, I have either gotten bad decrypt errors, or garbled nonsense back upon decryption. I have tried following many tutorials on the net, such as this one, and suggestions from this thread, and everything else I can find. I have read that different encryption programs use different standards, so compatibility across platforms/programs/languages is not always guaranteed, but I imagine somebody has been in this predicement before and knows a solution?
How would I, using nodejs, decrypt data encrypted by aescrypt2 (or a program like it)? I have only been able to make it work using a system exec call and having node execute aescrypt2 to decrypt/encrypt the data, which is not ideal, as it slows things down considerably. I am open to using a different program than aescrypt2. The only requirements are that it must run on Linux, cannot use openssl libs (because they are not supported on the target system), the program should be small and simple, due to space limitations, and foremost, the encryption/decryption needs to be compatible with nodejs. Any help would be much appreciated.
How would I, using nodejs, decrypt data encrypted by aescrypt2 (or a program like it)?
Sorry to say, but there's no better answer than: by doing the exact same thing that aescrypt2 does when decrypting a file. You've linked to the source by yourself, so just perform the same steps in node.js as they do in C in the decrypt branch.
First of all, get familiar with the layout of the file containing the encrypted data:
/*
* The encrypted file must be structured as follows:
*
* 00 .. 15 Initialization Vector
* 16 .. 31 AES Encrypted Block #1
* ..
* N*16 .. (N+1)*16 - 1 AES Encrypted Block #N
* (N+1)*16 .. (N+1)*16 + 32 HMAC-SHA-256(ciphertext)
*/
So you need to extract the IV, the encrypted blocks and the HMAC from the file, not try to decrypt the whole thing as you try with openssl (your openssl example also does not use the right IV but rather tries to derive it from the key provided - read the man page).
Next, get the key right. The actual key used to encrypt/decrypt is not the one provided on the command line, but rather 8192 iterations of hashing the IV with the key passed on the command line, using SHA256.
Finally, they decrypt, using AES-256-ECB (your openssl and node.js examples use CBC!), every 16 bytes and XOR the result with the pervious 16 bytes (the IV is used for the first 16 bytes).
There's maybe more to it, I just listed the most obvious things I saw when reading through the aescrypt2.c code.
So my advise is: try to write the same logic in node.js and try to find node.js crypto calls for the respective mbedtls counterparts.
I'm not a crypto expert, but I bet that the aescrypt implementation has so many steps that feel complicated (like generating the actual key used), because they know how to do crypto and are just doing it the right way.
I have configured a private APT repository (using resources on internet like http://inodes.org/2009/09/14/building-a-private-ppa-on-ubuntu/) and I'm uploading for the first time my package containing the sources of my C++ application.
So reprepro repository is empty.
I use the following command in order to start the build:
sudo reprepro -V -b /srv/reprepro processincoming incoming
Then the build start, a lot of output is genearated and I can see that pbuilder is compiling the project source code and everything is fine. I can even find in the result/ folder debian packages etc...
But the build failed with a POST_BUILD_FAILED because it seems that pbuilder has changed the douane-testing_0.8.1-apt1.debian.tar.gz file and the md5 sum is now different as shown here:
File "pool/main/d/douane-testing/douane-testing_0.8.1-apt1.debian.tar.gz" is already registered with different checksums!
md5 expected: 97257ae2c5790b84ed7bb1b412f1d518, got: df78f88b97cadc10bc0a73bf86442838
sha1 expected: ae93c44593e821696f72bee4d91ce4b6f261e529, got: d6f910ca5707ec92cb71601a4f4c72db0e5f18d9
sha256 expected: c3fac5ed112f89a8ed8d4137b34f173990d8a4b82b6212d1e0ada1cddc869b0e, got: ebdcc9ead44ea0dd99f2dc87decffcc5e3efaee64a8f62f54aec556ac19d579c
size expected: 2334, got: 2344
There have been errors!
I don't understand why it is failing as when I compare the 2 packages (having those md5 sums) the content is strictly the same (I used a diff tool but no differences and no new or removed files).
The only thing I can see is that the archive from pbuild is bigger of 10 Bytes than the orginal one I have uploaded:
On my development machine, the file with the md5 97257ae2c5790b84ed7bb1b412f1d518 :
-rw-r--r-- 1 zedtux zedtux 2334 Feb 3 23:38 douane-testing_0.8.1-apt1.debian.tar.gz
On my server, the file with the md5 df78f88b97cadc10bc0a73bf86442838 :
-rw-r--r-- 1 root root 2344 Feb 5 00:58 douane-testing_0.8.1-apt1.debian.tar.gz
I have pbuild version 0.213 on my server.
What could be the reason of this behavior and how can I fix it ?
Edit
I'm suspecting an issue with the GPG key which looks missing and then files aren't signed so md5sum is different.
During the build process I have the following lines:
I: Extracting source
gpgv: Signature made Wed Feb 5 22:04:37 2014 UTC using RSA key ID 9474CF36
gpgv: Can't check signature: public key not found
dpkg-source: warning: failed to verify signature on ./douane-testing_0.8.1-apt1.dsc
Edit 2
I have tried to find the command to create manually the .debian.tar.gz file.
The best I've found is the following:
tar cv debian | gzip --no-name --rsyncable -9 > douane-testing_0.8.1-apt1.debian.tar.gz
I don't get the same result than dpkg-source but I tried the same command on my server (I should at least have the same size) but it's not matching...
Could it be that Debian and Ubuntu aren't compressing the same way ?
Finally after some evenings of research I found the solution on launchpad.net !
Found the solution. By default pbuilder calls dpkg-buildpackage like so:
DEBBUILDOPTS="$DEBBUILDOPTS -rfakeroot"
dpkg-buildpackage -us -uc $DEBBUILDOPTS
That causes dpkg-buildpackage to rebuild the diff.gz and .dsc files. Add a -b in there, and it won't. It also means the resulting .changes file will only reference the .deb file. Which is what you want, I think.
The easy solution is to add a line to your .pbuilderrc:
DEBBUILDOPTS="-b"
My previous answer is alright but is not complete.
Then I had the issue that reprepro complains about the source tarball (.orig.tar.xz).
But it was normal as I wasn't doing the packages correctly.
I have written a bash script which I'm executing in VM for each Ubuntu series.
This script was always doing everything from scratch, and was using dh_make --createorig argument and here is the issue.
The correct way is to generate once (for example on Ubuntu precise) and then re-use the .orig.tar.xz file and no more use the --createorig argument of dh_make.
I hope this could help someone :-)