Salted sha512 in C, cannot synchronise with Symfony2's FOSUserBundle - c

My developement is separated into two components :
The website, a Symfony application using FOSUserBundle, which encrypts password using SHA512, and a salt.
An authentication module, programmed in C, which should be able to reproduce the SHA512 salted hash once it's given the salt, and the cleartext password.
Some information about my environment
I'm using Linux Ubuntu 12.04.
ldd --version answers EGLIBC 2.15-0ubuntu10.4 2.15 (maybe I need 2.7 ? But apt-get is a real PAIN when it comes to upgrading packages correctly).
The crypt.h header files mentions #(#)crypt.h 1.5 12/20/96
The problem itself
My problem occurs in the authentication module : I'm unable to get the same hash as the one produced by Symfony's FOSUserBundle. Here's my example :
The password salt, used by Symfony, is bcccy6eiye8kg44scw0wk8g4g0wc0sk.
The password itself is test
With this information, Symfony stores this final hash :
fH5vVoACB4e8h1GX81n+aYiRkSWxeu4TmDibNChtLNZS3jmFKBZijGCXcfzCSJFg+YvNthxefHOBk65m/U+3OA==
Now, in my C authentication module, I run this piece of code (crypt.h is included) :
char* password = "test";
char* salt = "$6$bcccy6eiye8kg44scw0wk8g4g0wc0sk";
char* hash = malloc(256);
memset(hash, 0, 256);
encode64(crypt(password, salt), hash, strlen(password));
fprintf(stdout, "%s\n", hash);
(here is my base64 encoder : http://libremail.tuxfamily.org/sources/base64-c.htm)
And this outputs...
JDYkYg==
Which is completely different from my Symfony2 hash.
Browsing Stack Overflow, I found this question (Symfony2 (FOSUserBundle) SHA512 hash doesn't match C# SHA512 hash) written by someone encountering the same issue (with C# though). So I decided to run this test...
char* password = "test{bcccy6eiye8kg44scw0wk8g4g0wc0sk}";
char* salt = "$6$bcccy6eiye8kg44scw0wk8g4g0wc0sk"; // I tried without salt, or with "$6$" as well.
char* hash = malloc(256);
memset(hash, 0, 256);
encode64(crypt(password, salt), hash, strlen(password));
fprintf(stdout, "%s\n", hash);
Of course, it was a complete failure, I got :
JDYkYmNjY3k2ZWl5ZThrZzQ0cyRycmN6TnpJUXFOYU1VRlZvMA==
I've tried mixing the password and the salt in various ways, but I could never get the Symfony's salt in the authentication module. Is there something I've missed on the way ? Have I misunderstood the way Symfony's FOSUserBundle stores passwords ?

Not really an answer but I'm guessing you have not looked into how Symfony encodes passwords in any great detail? The encoding process is tucked away into an encoder object. For SHA512 we use:
namespace Symfony\Component\Security\Core\Encoder;
class MessageDigestPasswordEncoder extends BasePasswordEncoder
{
/**
* Constructor.
*
* #param string $algorithm The digest algorithm to use
* #param Boolean $encodeHashAsBase64 Whether to base64 encode the password hash
* #param integer $iterations The number of iterations to use to stretch the password hash
*/
public function __construct($algorithm = 'sha512', $encodeHashAsBase64 = true, $iterations = 5000)
{
$this->algorithm = $algorithm;
$this->encodeHashAsBase64 = $encodeHashAsBase64;
$this->iterations = $iterations;
}
public function encodePassword($raw, $salt)
{
if (!in_array($this->algorithm, hash_algos(), true)) {
throw new \LogicException(sprintf('The algorithm "%s" is not supported.', $this->algorithm));
}
$salted = $this->mergePasswordAndSalt($raw, $salt);
$digest = hash($this->algorithm, $salted, true);
// "stretch" hash
for ($i = 1; $i < $this->iterations; $i++) {
$digest = hash($this->algorithm, $digest.$salted, true);
}
return $this->encodeHashAsBase64 ? base64_encode($digest) : bin2hex($digest);
}
public function isPasswordValid($encoded, $raw, $salt)
{
return $this->comparePasswords($encoded, $this->encodePassword($raw, $salt));
}
protected function mergePasswordAndSalt($password, $salt)
{
if (empty($salt)) {
return $password;
}
if (false !== strrpos($salt, '{') || false !== strrpos($salt, '}')) {
throw new \InvalidArgumentException('Cannot use { or } in salt.');
}
return $password.'{'.$salt.'}';
}
As you can see, one immediate problem is that hashing is repeated 5000 times by default. This (as well as the other inputs can all be adjusted in you app/config/security.yml file).
You can also see where the salt and password get merged together. Which explains the other stackoverflow answer.
It would be trivial to make a symfony command to just run this encoding algorithm from the symfony console for testing. After that is just a question of adjusting the inputs or tweaking your C code until the results match.
If you are lucky then all your will have to do is add the iteration loop.

Related

Find specific number of a word from the beginning in string

I've been gathering information using api calls from my jira. Information gathered is saved in a body file and it has the following content:
No tickets:
{"startAt":0,"maxResults":50,"total":0,"issues":[]}{"startAt":0,"maxResults":50,"total":0,"issues":[]}
One Ticket:
{"expand":"names,schema","startAt":0,"maxResults":50,"total":1,"issues":[{"expand":"operations,versionedRepresentations,editmeta,changelog,renderedFields","id":"456881","self":"https://myjira...com","key":"TICKET-1111","fields":{"summary":"[TICKET] New Test jira","created":"2018-12-17T01:47:09.000-0800"}}]}{"expand":"names,schema","startAt":0,"maxResults":50,"total":1,"issues":[{"expand":"operations,versionedRepresentations,editmeta,changelog,renderedFields","id":"456881","self":"https://myjira...com","key":"TICKET-1111","fields":{"summary":"[TICKET] New Test jira","created":"2018-12-17T01:47:09.000-0800"}}]}
Two Tickets:
{expand:schema,names,startAt:0,maxResults:50,total:2,issues:[{expand:operations,versionedRepresentations,editmeta,changelog,renderedFields,id:456881,self:https://myjira...com,key:TICKET-1111,fields:{summary:[TICKET] New Test jira,created:2018-12-17T01:47:09.000-0800}},{expand:operations,versionedRepresentations,editmeta,changelog,renderedFields,id:320281,self:https://myjira...com,key:TICKET-2222,fields:{summary:[TICKET] Test jira,created:2016-03-18T07:58:52.000-0700}}]}{expand:schema,names,startAt:0,maxResults:50,total:2,issues:[{expand:operations,versionedRepresentations,editmeta,changelog,renderedFields,id:456881,self:https://myjira...com,key:TICKET-1111,fields:{summary:[TICKET] New Test jira,created:2018-12-17T01:47:09.000-0800}},{expand:operations,versionedRepresentations,editmeta,changelog,renderedFields,id:320281,self:https://myjira...com,key:TICKET-2222,fields:{summary:[TICKET] Test jira,created:2016-03-18T07:58:52.000-0700}}]}
etc..
Using this code I've been able to gather total open tickets:
std::ifstream t("BodyOpenIssues.out");
std::string BodyString((std::istreambuf_iterator<char>(t)),
std::istreambuf_iterator<char>());
// Removing Quotes
BodyString.erase(std::remove(BodyString.begin(), BodyString.end(), '"'), BodyString.end());
int Result = 0;
unsigned first = BodyString.find("total:");
unsigned last = BodyString.find(",issues");
std::string TotalOpenIssues = BodyString.substr(first + 6, last - (first + 6));
Result = std::stoi(TotalOpenIssues);
return Result;
Using a second function I'm trying to get the keys based on total open tickets.
if (GetOpenIssuesNumber() > 0)
{
std::ifstream t("BodyOpenIssues.out");
std::string BodyString((std::istreambuf_iterator<char>(t)),
std::istreambuf_iterator<char>());
// Removing Quotes
BodyString.erase(std::remove(BodyString.begin(), BodyString.end(), '"'), BodyString.end());
unsigned first = BodyString.find("key:TICKET-");
unsigned last = BodyString.find(",fields");
std::string TotalOpenIssues = BodyString.substr(first + 11, last - (first + 11));
String^ Result = gcnew String(TotalOpenIssues.c_str());
return "TICKET-" + Result;
}
else
{
return "No open issues found";
}
What I mean is:
If Total is 1 to search from the beginning and find the first key TICKET-1111.
If Total is 2 to search from the beginning and get the first key TICKET-1111 then to continue from there and to find the next key TICKET-2222.
And based on that total to find that many keys in that string.
I got lost from all the casting between the types as ifstream reads the file and I save the result in std::string. After the find I save the result in System::String to use it in my Label.. I've been researching and found out that I can use char array but I can't make it dynamic based on BodyString.length().
If more information is required please let me know.
Any suggestions are really appreciated! Thank you in advance!
I went for nlohmann json library. It has everything I need. Thank you Walnut!
These are formatted as JSON. You should use a JSON library for C++ and parse the files with that. Using search/replace is unnecessary complicated and you will likely run into corner cases you haven't considered sooner or later (do you really want the code to randomly miss tickets, etc.?). Also String^ is not C++. Are you writing C++/CLI instead of C++? If so, please tag c++-cli instead of c++. – walnut

How can I password protect a file regardless of its extension in Java 8 ro Java 10

I have tried doing this by encrypting individual files but I have a lot of data (~20GB) and hence it would take a lot of time. In my test it took 2.28 minutes to encrypt a single file of size 80MB.
Is there a quicker way to be able to password protect that would apply to any any file (text/binary/multimedia)?
If you are just trying to hide the file from others, you can try to encrypt the file path instead of encrypting the whole huge file.
For the path you mentioned: text/binary/multimedia, you can try to encrypt it by a method as:
private static String getEncryptedPath(String filePath) {
String[] tokens = filePath.split("/");
List<String> tList = new ArrayList<>();
for (int i = 0; i < tokens.length; i++) {
tList.add(Hashing.md5().newHasher() // com.google.common.hash.Hashing;
.putString(tokens[i] + filePath, StandardCharsets.UTF_8).hash().toString()
.substring(2 * i, 2 * i + 5)); // to make it impossible to encrypt, add your custom secret here;
}
return String.join("/", tList);
}
and then it becomes an encrypted path as:
72b12/9cbb3/4a5f3
Once you know the real path text/binary/multimedia, any time you want to access the file, you can just use this method to get the real file path 72b12/9cbb3/4a5f3.

TextEncodings.Base64Url.Decode vs Convert.FromBase64String

I was working on creating a method that would generate a JWT token. Part of the method reads a value from my web.config that services as the "secret" used to generate the hash used to create the signature for the JWT token.
<add key="MySecret" value="j39djak49H893hsk297353jG73gs72HJ3tdM37Vk397" />
Initially I tried using the following to convert the "secret" value to a byte array.
byte[] key = Convert.FromBase64String(ConfigurationManager.AppSettings["MySecret"]);
However, an exception was thrown when this line was reached ...
The input is not a valid Base-64 string as it contains a non-base 64 character, more than two padding characters, or an illegal character among the padding characters.
So I looked into the OAuth code and so another method being used to change a base64 string into a byte array
byte[] key = TextEncodings.Base64Url.Decode(ConfigurationManager.AppSettings["MySecret"]);
This method worked without issue. To me it looks like they are doing the same thing. Changing a Base64 text value into an array of bytes. However, I must be missing something. Why does Convert.FromBase64String fail and TextEncodings.Base64Url.Decode work?
I came across the same thing when I migrated our authentication service to .NET Core. I had a look at the source code for the libraries we used in our previous implementation, and the difference is actually in the name itself.
The TextEncodings class has two types of text encoders, Base64TextEncoder and Base64UrlEncoder. The latter one modifies the string slightly so the base64 string can be used in an url.
My understanding is that it is quite common to replace + and / with - and _. As a matter of fact we have been doing the same with our handshake tokens. Additionally the padding character(s) at the end can also be removed. This leaves us with the following implementation (this is from the source code):
public class Base64UrlTextEncoder : ITextEncoder
{
public string Encode(byte[] data)
{
if (data == null)
{
throw new ArgumentNullException("data");
}
return Convert.ToBase64String(data).TrimEnd('=').Replace('+', '-').Replace('/', '_');
}
public byte[] Decode(string text)
{
if (text == null)
{
throw new ArgumentNullException("text");
}
return Convert.FromBase64String(Pad(text.Replace('-', '+').Replace('_', '/')));
}
private static string Pad(string text)
{
var padding = 3 - ((text.Length + 3) % 4);
if (padding == 0)
{
return text;
}
return text + new string('=', padding);
}
}

How do I print values from C extensions?

Every Ruby object is of type VALUE in C. How do I print it in a readable way?
Any other tips concerning debugging of Ruby C extensions are welcome.
You can call p on Ruby objects with the C function rb_p. For example:
VALUE empty_array = rb_ary_new();
rb_p(empty_array); // prints out "[]"
Here's what I came up with:
static void d(VALUE v) {
ID sym_puts = rb_intern("puts");
ID sym_inspect = rb_intern("inspect");
rb_funcall(rb_mKernel, sym_puts, 1,
rb_funcall(v, sym_inspect, 0));
}
Having it in a C file, you can output VALUEs like so:
VALUE v;
d(v);
I've borrowed the idea from this article.
I've found an interesting way using Natvis files in Visual Studio.
I have created C++ wrapper objects over the Ruby C API - this gives me a little bit more type safety and the syntax becomes more similar to writing actual Ruby.
I won't be posting the whole code - too long for that, I plan on open sourcing it eventually.
But the gist of it is:
class Object
{
public:
Object(VALUE value) : value_(value)
{
assert(NIL_P(value_) || kind_of(rb_cObject));
}
operator VALUE() const
{
return value_;
}
// [More code] ...
}
Then lets take the String class for example:
class String : public Object
{
public:
String() : Object(GetVALUE("")) {}
String(VALUE value) : Object(value)
{
CheckTypeOfOrNil(value_, String::klass());
}
String(std::string value) : Object( GetVALUE(value.c_str()) ) {}
String(const char* value) : Object( GetVALUE(value) ) {}
operator std::string()
{
return StringValueCStr(value_);
}
operator std::string() const
{
return operator std::string();
}
static VALUE klass()
{
return rb_cString;
}
// String.empty?
bool empty()
{
return length() == 0;
}
size_t length() const
{
return static_cast<size_t>(RSTRING_LEN(value_));
}
size_t size() const
{
return length();
};
};
So - my wrappers make sure to check that the VALUE they wrap is of expected type or Nil.
I then wrote some natvis files for Visual Studio which will provide some real time debug information for my wrapper objects as I step through the code:
<?xml version="1.0" encoding="utf-8"?>
<AutoVisualizer xmlns="http://schemas.microsoft.com/vstudio/debugger/natvis/2010">
<Type Name="SUbD::ruby::String">
<DisplayString Condition="value_ == RUBY_Qnil">Ruby String: Nil</DisplayString>
<DisplayString Condition="value_ != RUBY_Qnil">Ruby String: {((struct RString*)value_)->as.heap.ptr,s}</DisplayString>
<StringView Condition="value_ != RUBY_Qnil">((struct RString*)value_)->as.heap.ptr,s</StringView>
<Expand>
<Item Name="[VALUE]">value_</Item>
<Item Name="[size]" Condition="value_ != RUBY_Qnil">((struct RString*)value_)->as.heap.len</Item>
<Item Name="[string]" Condition="value_ != RUBY_Qnil">((struct RString*)value_)->as.heap.ptr</Item>
<Item Name="[capacity]" Condition="value_ != RUBY_Qnil">((struct RString*)value_)->as.heap.aux.capa</Item>
</Expand>
</Type>
</AutoVisualizer>
Note that this is all hard-coded to the exact internal structure of Ruby 2.0. This will not work in Ruby 1.8 or 1.9 - haven't tried with 2.1 or 2.2 yet. Also, there might be mutations of how the String can be stored which I haven't added yet. (Short strings can be stored as immediate values.)
(In fact - the natvis posted above only works for 32bit - not 64bit at the moment.)
But once that is set up I can step through code and inspect the Ruby strings almost like they are std::string:
Getting it all to work isn't trivial. If you noticed in my natvis my RUBY_Qnil references - they would not work unless I added this piece of debug code to my project:
// Required in order to make them available to natvis files in Visual Studio.
#ifdef _DEBUG
const auto DEBUG_RUBY_Qnil = RUBY_Qnil;
const auto DEBUG_RUBY_FIXNUM_FLAG = RUBY_FIXNUM_FLAG;
const auto DEBUG_RUBY_T_MASK = RUBY_T_MASK;
const auto DEBUG_RUBY_T_FLOAT = RUBY_T_FLOAT;
const auto DEBUG_RARRAY_EMBED_FLAG = RARRAY_EMBED_FLAG;
const auto DEBUG_RARRAY_EMBED_LEN_SHIFT = RARRAY_EMBED_LEN_SHIFT;
const auto DEBUG_RARRAY_EMBED_LEN_MASK = RARRAY_EMBED_LEN_MASK;
#endif
You cannot use macros in natvis definitions unfortunately, so that's why I had to manually expand many of them into the natvis file by inspecting the Ruby source itself. (The Ruby Cross Reference is of great help here: http://rxr.whitequark.org/mri/ident?v=2.0.0-p247)
It's still WIP, but it's already saved me a ton of headaches. Eventually I want to extract the debug setup on GitHub: https://github.com/thomthom (Keep an eye on that account if you are interested.)

Dart VM itself implement `eval` in `dart:mirrors` and developers use it. Are planned to make this method public?

Here is code that use this eval method in Dart platform.
This is done via reflection.
runtime/lib/mirrors_impl.dart
_getFieldSlow(unwrapped) {
// ..... Skipped
var atPosition = unwrapped.indexOf('#');
if (atPosition == -1) {
// Public symbol.
f = _eval('(x) => x.$unwrapped', null);
} else {
// Private symbol.
var withoutKey = unwrapped.substring(0, atPosition);
var privateKey = unwrapped.substring(atPosition);
f = _eval('(x) => x.$withoutKey', privateKey);
}
// ..... Skipped
}
static _eval(expression, privateKey)
native "Mirrors_evalInLibraryWithPrivateKey";
runtime/lib/mirrors.cc
DEFINE_NATIVE_ENTRY(Mirrors_evalInLibraryWithPrivateKey, 2) {
GET_NON_NULL_NATIVE_ARGUMENT(String, expression, arguments->NativeArgAt(0));
GET_NATIVE_ARGUMENT(String, private_key, arguments->NativeArgAt(1));
const GrowableObjectArray& libraries =
GrowableObjectArray::Handle(isolate->object_store()->libraries());
const int num_libraries = libraries.Length();
Library& each_library = Library::Handle();
Library& ctxt_library = Library::Handle();
String& library_key = String::Handle();
if (library_key.IsNull()) {
ctxt_library = Library::CoreLibrary();
} else {
for (int i = 0; i < num_libraries; i++) {
each_library ^= libraries.At(i);
library_key = each_library.private_key();
if (library_key.Equals(private_key)) {
ctxt_library = each_library.raw();
break;
}
}
}
ASSERT(!ctxt_library.IsNull());
return ctxt_library.Evaluate(expression);
runtime/vm/bootstrap_natives.h
V(Mirrors_evalInLibraryWithPrivateKey, 2) \
P.S.
I ask question here becuase I cannot ask it at Dart mail lists.
P.S.
As we can see it static private method in mirrors_impl.dart:
static _eval(expression, privateKey) native "Mirrors_evalInLibraryWithPrivateKey";
Does anyone want that this method should be public? (this is not a question but just a thought aloud).
According to the Dart FAQ a pure string eval like that is not likely to make it into the language, even though other dynamic features will likely be added:
So, for example, Dart isn’t likely to support evaluating a string as
code in the current context, but it may support loading that code
dynamically into a new isolate. Dart isn’t likely to support adding
fields to a value, but it may (through a mirror system) support adding
fields to a class, and you can effectively add methods using
noSuchMethod(). Using these features will have a runtime cost; it’s
important to us to minimize the cost for programs that don’t use them.
This area is still under development, so we welcome your thoughts on
what you need from runtime dynamism.

Resources