I need to calculate the md5 of a file saved locally in an windows8 app created in
javascript / html.
I need the md5 to compare it with an online file and see if the two objects are really the same.
What function should I use?
Here's the code I use to hash a string in MD5 using Windows.Security.Cryptography namespace:
var inputBuffer, outputBuffer, toHash, hashed,
hashProvider = Windows.Security.Cryptography.Core.HashAlgorithmProvider.openAlgorithm(Windows.Security.Cryptography.Core.HashAlgorithmNames.md5); // "open" an md5 hash provider
toHash = 'string'; // string to hash
inputBuffer = Windows.Security.Cryptography.CryptographicBuffer.convertStringToBinary(toHash, Windows.Security.Cryptography.BinaryStringEncoding.utf8); // convert string to binary
outputBuffer = hashProvider.hashData(inputBuffer); // hash the binary
hashed = Windows.Security.Cryptography.CryptographicBuffer.encodeToHexString(outputBuffer); // the hashed string
Now, all you need to do is to read the file in (see http://msdn.microsoft.com/en-us/library/windows/apps/hh464978.aspx). If reading the file into a buffer, then you wouldn't need the convertStringToBinary line.
The WinRT API provides SHA functionality in the Windows.Security.Cryptography.Core namespace, specifically through the static method HashAlgorithmProvider.openAlgorithm(Windows.Security.Cryptography.Certificates.HashAlgorithmNames.sha256).
This provides you with a HashAlgorithmProvider class that has methods like hashData.
Fin in this Link is all that's available in winjs in terms of cryptography.
And Here you can find an example on how to implement an MD5 calculator for a string, from there is a good start to make it work for a file.
Hope this helps.
Related
I'm looking for a for a way to generate a md5 hash (or equivalent 32 character) string using only the security module in Xcode 8 with Swift 3. In other words a method that does not require CommonCrypto.
I've only found one post that mentions this approach. This post claims this approach is only for OS X (not iOS).
I know md5 is not super secure but I need compatibility with an older site, so assistance would be greatly appreciated.
What is wrong with CommonCrypto? It is already available on every device, simple, fast and well tested:
extension Data {
var hexString: String {
return map { String(format: "%02hhx", $0) }.joined()
}
var md5: Data {
var digest = [Byte](repeating: 0, count: Int(CC_MD5_DIGEST_LENGTH))
self.withUnsafeBytes({
_ = CC_MD5($0, CC_LONG(self.count), &digest)
})
return Data(bytes: digest)
}
}
As far as i know the only other possibility to calculate md5 is to add the algorithm by yourself like the one used in CryptoSwift.
Some context: I have a PKCS #11-compliant cryptographic engine. This engine will be used to handle signed/enveloped data, i.e. verify the data's ECDSA/SHA1 signature, unwrap the symmetric key with RSAES-OAEP, and decrypt this data. This means the symmetric key will be wrapped with my engine's public key: hence I'd like the certificate for this public key to actually read "Subject Public Key Algorithm: RSAES-OAEP".
I'm looking for a C library which will let me manipulate objects comforming to the Cryptographic Message Syntax (CMS) and X.509 standards in the following way:
create a X.509 Certificate Signing Request (CSR), setting the Subject Public Key Algorithm to RSAES-OAEP
let me handle the signing part: my private key is only accessible via a PKCS #11 handle, so I need the library to give me the bytes to sign, and then let me set the CSR's ProofOfPossession field with what my crypto engine computed
export the complete CSR to something (DER or PEM)
create CMS structures to hold something like SignedData( EnvelopedData( stuff )). The library could handle the actual encryption/key wrapping/signature, or it could let some other software engine do it and just allow me to set the octet strings
let me easily parse back the message and recover those octet strings
meaning I want to open a DER/PEM file which contains this CMS message, and get the bytes for the signature, the wrapped key and the encrypted stuff, so that I can feed them to my PKCS #11 interface
Before anyone suggests OpenSSL's libcrypto, I've looked at it (looked as in, "spent the last week trying to understand how the structures work, how the ASN.1 representation works, how I can recover the bytes I'm interested in from OpenSSL's structures..."), and I have some issues with it (as of 1.0.1f):
(cf 1.) I cannot set the Subject Public Key Algorithm to RSAE-OAEP. Starting from demos/x509/mkreq.c, and going all the way back to the deep reaches of x509t.h's weird #define IMPLEMENT_ASN1_ENCODE_FUNCTIONS_fname(stname, itname, fname) macros, I think I can affirm that X509_REQ_set_pubkey() cannot handle OAEP.
(cf 2.) Neither can the CMS part of the crypto lib, for that matter. cms_RecipientInfo_ktri_encrypt() calls EVP_PKEY_CTX_ctrl(), which I guess resolves to crypto/rsa/rsa_pmeth.c:pkey_rsa_ctrl(): when p2 is EVP_PKEY_CTRL_CMS_ENCRYPT, p2 is not parsed, so RSA padding is not set (if it turns out I'm wrong and I just cannot read code correctly, please tell me).
So while I'm glad this guy managed to make it "work like a charm", I cannot share his enthusiasm.
I guess for 2. I could use OpenSSL to create CMS blank structures, compute the future EnvelopedData content (ie stuff encrypted with a symmetric key + symmetric key wrapped with RSA OAEP), and stuff this content into the structures, bypassing CMS_encrypt() completely, before encapsulating into a SignedData (I'm assuming CMS_sign() will handle ECDSA/SHA1). As long as I don't want to use fancy parameters for OAEP (although this other guy managed to patch the lib to use OAEP with SHA-256).
But this might end up requiring a tad too much fiddling with OpenSSL's ASN.1 API. Hence my interrogation: does anyone know of a C library to build CMS structures and feed them the octet strings computed by some other engines? Also how to build certificates which read "THIS KEY IS MEANT TO BE USED WITH RSAES-OAEP".
I've looked at libksba and cryptlib and while I guess they could work, I cannot see how to use them yet (might have something to do with my eyes bleeding from staring at OpenSSL's code so much - I do not mean to say that OpenSSL's code is bad or anything, just that I've been looking at it hard, and the doc is slightly lacking).
Actually, I guess, I could drop the C requirement (mainly there because communicating with the crypto engine is done in PKCS #11/C). Here's what the library should be able to do:
build a CSR
... featuring "RSAES-OAEP" as Subject Public Key Algorithm
give me the bytes to sign for the Proof-of-Possession part
take the signature and output a complete X.509 CSR
parse a CMS structure
(SignedData) give me the bytes corresponding to the signedInfo->signature and encapsulatedContentInfo fields so that I can verify them with some other engine
(EnvelopedData) give me the bytes corresponding to the keyTransRecipientInfo->encryptedKey and encryptedContentInfo->encryptedContent fields so that I can unwrap/decrypt with some other engine
build a CMS structure, either...
letting some external engine set the fields mentioned above, and letting me specify the algorithms
actually implementing the algorithms, and building the CMS from just the data (... with RSAES-OAEP for key wrapping)
Right now I'm going with a "all-OpenSSL" approach, because I feel like I'm in too deep and should not start wandering somewhere else. If anybody has a better idea, I am all ears.
As for setting that subject public key algorithm... Well, either I'll just leave regular RSA and have my application "know" that wrapping is RSAES-OAEP, or... I don't know. And as for signing the request... is POP all that useful anyway?
(This is not a serious question)
NB: edited to remove the whole "I want my certificate to read OAEP", since I just found out about RFC 5756 (also found an interesting discussion from 2006 when this RFC was not out yet).
Here's what I managed to get working so far.
1. Building a CSR, signing it with some other engine
I mostly followed demos/x509/mqreq.c, with some twists.
(NB: error checking, fancy modulus length/label/subject DN generation/handling has been left out for brevity and focus on actual flow).
unsigned char* mod = NULL;
unsigned char* exp = NULL;
size_t mod_l = 0;
size_t exp_l = 0;
P11_handle h_key = P11_gen_rsa(&mod, &mod_l, &exp, &exp_l);
RSA* rsa = RSA_new();
rsa->n = BN_bin2bn(rsa_mod, rsa_mod_l, NULL);
rsa->e = BN_bin2bn(rsa_exp, rsa_exp_l, NULL);
EVP_PKEY* pkey = EVP_PKEY_new();
EVP_PKEY_assign_RSA(pkey, rsa);
X509_REQ* csr = X509_REQ_new();
X509_REQ_set_pubkey(csr, pkey);
/* Boring X509_NAME/X509_EXTENSION stuff */
X509_REQ_INFO* csr_req = csr->req_info;
unsigned char* pop_in = NULL;
size_t pop_in_l = ASN1_item_i2d((void*)csr_req, &pop_in,
ASN1_ITEM_rptr(X509_REQ_INFO));
unsigned char* sig = NULL;
size_t sig_l = 0;
P11_make_pop(SIGN_RSA_PKCS, DIGEST_SHA256,
pop_in, pop_in_l, &sig, &sig_l,
h_key);
/* Add signature to CSR (heavily inspired from ASN1_item_sign_ctx())
* (please don't ask about the flags) */
if (csr->signature->data != NULL) OPENSSL_free(csr->signature->data);
csr->signature->data = sig;
csr->signature->length = sig_l;
csr->signature->flags&= ~(ASN1_STRING_FLAG_BITS_LEFT|0x07);
csr->signature->flags|= ASN1_STRING_FLAG_BITS_LEFT;
/* Add signature algorithm information to CSR */
int sig_algo_nid = 0;
OBJ_find_sigid_by_algs(&sig_algo_nid,
EVP_MD_nid(EVP_sha256()), EVP_PKEY_RSA);
X509_ALGOR_set0(csr->sig_alg, OBJ_nid2obj(sig_algo_nid),
V_ASN1_NULL, NULL));
After that, the X509_REQ structure is good for PEM export. openssl req -verify seems to validate the process, so as far as I'm concerned this works.
2. Building nested CMS structures (as in SignedData(EnvelopedData(Data)))
Finally got it, using 1.0.2 (any previous version would have needed patching or ASN.1-level parsing). Many thanks to Dr. Stephen Henson and Tom Francis for helping me with this via the mailing list.
/* Make EnvelopedData structure */
BIO* in = BIO_new_file(in_path, "rb");
int flags = CMS_BINARY | CMS_USE_KEYID | CMS_PARTIAL | CMS_KEY_PARAM;
CMS_ContentInfo* edata = CMS_encrypt(NULL, NULL, cipher, flags);
CMS_RecipientInfo* r_info = CMS_add1_recipient_cert(edata, r_cert, flags);
EVP_PKEY_CTX* wrap_ctx = CMS_RecipientInfo_get0_pkey_ctx(r_info);
EVP_PKEY_CTX_set_rsa_padding(wrap_ctx, RSA_PKCS1_OAEP_PADDING);
EVP_PKEY_CTX_set_rsa_oaep_md(wrap_ctx, EVP_sha256());
EVP_PKEY_CTX_set_rsa_mgf1_md(wrap_ctx, EVP_sha256());
EVP_PKEY_CTX_set0_rsa_oaep_label(wrap_ctx, oaep_label, oaep_label_l);
/* NB: oaep_label must be heap-allocated, and will be freed by OSSL */
CMS_final(edata, in, NULL, flags);
BIO* tmp = BIO_new(BIO_s_mem());
i2d_CMS_bio(tmp, edata);
/* Make SignedData structure */
flags|= CMS_NOSMIMECAP | CMS_NOCERTS;
flags&= ~(CMS_KEY_PARAM);
CMS_ContentInfo* sdata = CMS_sign(NULL, NULL, NULL, NULL, flags);
ASN1_OBJECT* ectype_edata = OBJ_nid2obj(NID_pkcs7_enveloped);
CMS_set1_eContentType(sdata, ectype_edata);
CMS_SignerInfo* s_info =
CMS_add1_signer(sdata, s_cert, s_key, NULL, flags);
CMS_SignerInfo_sign(s_info);
CMS_final(sdata, tmp, NULL, flags);
BIO* out = BIO_new_file(out_path, "wb");
i2d_CMS_bio(out, sdata);
BIO_flush(out);
3. Parsing the structure and getting the fields I need.
I basically wrote my own CMS parser. ASN.1 is actually simple to parse when you know the spec. I've tried compiling the ASN.1 modules in the RFC using some "ASN.1 to C structs" compilers but had no luck (they kept choking on the syntax).
Currently I am sending the UART the strings I want to log and reading it on the host with any terminal.
In order to reduce the logging time and hopefully the image size as well (my flash is tiny), I figured out that the strings are unused in the embedded system, so why storing them on the flash?
I want to implement a server, whom I can send a hashed-key of any string (for example - it's ROM address) and the string will be output to file or screen.
My questions are:
How to create the key2string converter out of the image file (the OS is CMX, but can be answered generally)
Is there a recomended way to generate image, that will know the strings addresses but will exclude them from ROM?
Is there a known generic (open-source or other) that implemented a similar logger?
Thanks
Rather than holding hard-coded strings, then trying to hash the answers and sent it via a UART, then somehow remove the strings from the resulting image, I suggest the following.
Just send an index for an error code. The PC side can look up that index and determine what the string is for that condition. If you want the device code to be more clear, the index can be an enumeration.
For example:
enum errorStrings
{
ES_valueOutOfLimits = 1,
ES_wowItsGettingWarm = 2,
ES_randomError = 3,
ES_passwordFailure = 4
};
So, if you were sending data to the UART via printf, you could do the following:
printf("%d\n",(int)ES_wowItsGettingWarm);
Then your PC software just needs to decode the "2" that comes across the UART back into a useful string of "Wow it's getting warm."
This keeps the firmware small, but you need to manually keep the file containing the enum and the file with the strings in sync.
My solution is sending file name and line (which should be 14-20 Byte) and having a source parser on the server side, which will generate map of the actual texts. This way the actual code will contain no "format" strings, but single "filename" string for each file. Furthermore, file names can be easily replaced with enum (unlike replacing every string in the code) to reduce the COMM throughput.
I hope the sample psaudo-code will help clarifying the idea:
/* target code */
#define PRINT(format,...) send(__FILE__,__LINE__,__VA_ARGS__)
...
/* host code (c++) */
void PrintComm(istream& in)
{
string fileName;
int line,nParams;
int* params;
in>>fileName>>line>>nParams;
if (nParams>0)
{
params = new int[nParams];
for (int i=0; i<nParams; ++i)
in>>params[i];
}
const char* format = FindFormat(fileName,line);
...
delete[] params;
}
My requirement is to compare the MD5 hashes of a file on the local disk and a file downloaded from a database.
The file is stored on SQL Server in a VARBINARY(MAX) column. The file can be any type. I'm currently testing with a PDF file. I get the file from the database using a HttpPost request. A JSONObject is built using the HttpResponse object. The JSONObject contains the file contents in binary format.
Now I have to compare the MD5 hash of the received binary data against the MD5 hash of the same file on disk. I have written the following code but the MD5 hashes do not match.
I think I'm going wrong in simply calculating the MD5 of the downloaded binary contents. Is there a correct way to do this? Thanks in advance.
// Read response from a HttpResponse object 'response'
BufferedReader reader = new BufferedReader(new InputStreamReader(response.getEntity().getContent()));
String line="";
StringBuilder sb = new StringBuilder();
while((line=reader.readLine())!=null) {
sb.append(line);
}
// construct jsonobject
JSONObject jsonResponse = new JSONObject(sb.toString());
//Read file from disk
FileInputStream fis = new FileInputStream(new File(this.getClass().getResource("C:\\demo.pdf").getPath()));
// Calculate MD5 of file read from disk
String md5Request = org.apache.commons.codec.digest.DigestUtils.md5Hex(fis);
// Calculate MD5 of binary contents. "binfile" is name of key in the JSONObject
// and binary contents of downloaded file are in its corresponding value field
String md5Response = org.apache.commons.codec.digest.DigestUtils.md5Hex(jsonResponse.getString("binfile"));
Assert.assertEquals("Hash sums of request and response must match", md5Request, md5Response);
When I debug, I see this value against the binfile key in the JSONObject 'jsonResponse'
binfile=[37,80,68,70,45,49,46,52,13,37,-30,-29,-49,-45,13,10,52,48...]
and what follows is a lengthy stream of binary data.
OK, in SQL there's a build-in function that looks like this:
select *,
convert(varchar(50),master.sys.fn_repl_hash_binary(a.BinaryField),2) as 'MD5Hash'
from SomeTable a
You give the fn_repl_hash_binary the name of the binary field you're reading, plus "2" as an argument which tells SQL to calc the value as an MD5; I think "1" is SHA.
And in Java, you can use something like this:
private String getMD5Hash(byte[] bytes) throws java.lang.Exception{
String s="This is a test";
MessageDigest m=MessageDigest.getInstance("MD5");
m.update(bytes,0,bytes.length);
return new BigInteger(1,m.digest()).toString(16);
}
This should do the trick. Best of luck, CodeWarrior.
It is not a new post but here is a possible solution, as I faced this problem too on python and made a bunch of test to find how to do...
As you treat all data in binary, you need to open the file to compare in binary mode.
My original code that was failing every time to read the correct MD5 checksum:
with open(filepath, "r") as file_to_check:
tile_file = file_to_check.read()
Corrected code:
with open(filepath, "rb") as file_to_check:
tile_file = file_to_check.read()
Simply adding the b (binary) after the read (r) flag to let python know it need to read the file as binary and now it works.
This might be what will help you find your problem... Hope it helps!
I'm looking for one line code examples in various languages for getting a valid MD5 result (as a string, not a bytehash or what have you). For instance:
PHP:
$token = md5($var1 . $var2);
I found VB especially troublesome to do in one line.
C#:
string hash = System.Web.Security.FormsAuthentication.HashPasswordForStoringInConfigFile(input, "md5");
VB is virtually the same.
Here it is not using the System.Web namespace:
string hash = Convert.ToBase64String(new System.Security.Cryptography.MD5CryptoServiceProvider().ComputeHash(System.Text.Encoding.UTF8.GetBytes(input)));
Or in readable form:
string hash =
Convert.ToBase64String
(new System.Security.Cryptography.MD5CryptoServiceProvider()
.ComputeHash
(System.Text.Encoding.UTF8.GetBytes
(input)
)
);
There is a kind of universality in how this is to be accomplished. Typically, one defines a routine called md5_in_one_line (or Md5InOneLine) once, and uses it all over the place, just as one would use a library routine.
So for example, once one defines Md5InOneLine in C#, it's an easy one-liner to get the right results.
Python
token = __import__('md5').new(var1 + var2).hexdigest()
or, if md5 is alrady imported:
token = md5.new(var1 + var2).hexdigest()
Thanks to Greg Hewgill
Aren't you really just asking "what languages have std. library support for MD5?" As Justice said, in any language that supports it, it'll just be a function call storing the result in a string variable. Even without built-in support, you could write that function in any language!
Just in case you need VBScript:
download the MD5 class from webdevbros and then with one line:
hash = (new MD5).hash("some value")
Does it really matter if you can do MD5 in one line. If it's that much trouble that you can't do it in VB in 1 line, then write your own function. Then, when you need to do MD5 in VB in one line, just call that function.
If doing it all in 1 line of code is all that important, here is 1 line of VB. that doesn't use the System.Web Namespace.
Dim MD5 As New System.Security.Cryptography.MD5CryptoServiceProvider() : Dim HashBytes() As Byte : Dim MD5Str As String = "" : HashBytes = MD5.ComputeHash(System.Text.Encoding.UTF8.GetBytes("MyString")) : For i As Integer = 0 To HashBytes.Length - 1 : MD5Str &= HashBytes(i).ToString("x").PadLeft(2, "0") : Next
This will hash "MyString" and store the MD5 sum in MD5Str.
Coldfusion has a bunch of hashing algorithms, MD5 is the default.
cfset var md5hashresult = hash("string to hash")