segmentation fault when calling c function on ruby - c

Question

I am trying to extends my Ruby code with a C functions. The C code compiles without warning. But When I try to run the ruby code, I got a segmentation fault:
I have this c code:
#include <ruby.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <time.h>
#include "nessie.h"
/* #define TRACE_INTERMEDIATE_VALUES */
/*
* The number of rounds of the internal dedicated block cipher.
*/
#define R 10
VALUE rb_mExample;
VALUE rb_cClass;
// ...
static char* displayHash(const unsigned char array[], int length){
int i, k;
char *str;
str = malloc(3 * length + 1);
if (str == NULL) {
return NULL;
}
k = 0;
str[0] = '\0';
for (i = 0; i < length; i++){
char hex[3];
if (i % 32 == 0){
str[k++] = ' ';
}
if (i % 8 == 0){
str[k++] = ' ';
}
snprintf(hex, sizeof(hex), "%02X", array[i]);
str[k++] = hex[0];
str[k++] = hex[1];
}
str[k] = '\0';
return str;
}
VALUE
print_string(VALUE class, VALUE *valor) {
struct NESSIEstruct w;
u8 digest[DIGESTBYTES];
int i;
for (i = 0; valor[i] != '\0'; i++);
int sizeo = i;
NESSIEinit(&w);
NESSIEadd((u8*)valor, 8*sizeo, &w);
NESSIEfinalize(&w, digest);
return (VALUE) displayHash(digest, DIGESTBYTES);
}
void
Init_example(){
rb_mExample = rb_define_module("Example");
rb_cClass = rb_define_class_under(rb_mExample, "Class", rb_cObject);
rb_define_method(rb_cClass, "print_string", print_string, 1);
}
and this Ruby code:
require "example"
def print
e = Example::Class.new
e.print_string("ruby")
end
When I run the ruby code, I got a segmentation fault.
EDIT: gist with log info
https://gist.github.com/psantos10/f07484afa26ce0e55181
Where I failing? I am new in C language.
EDIT:
I changed my "print_string" to look like this:
VALUE
print_string(VALUE class, VALUE *valor) {
struct NESSIEstruct w;
u8 digest[DIGESTBYTES];
int i;
for (i = 0; valor[i] != '\0'; i++);
int sizeo = i;
NESSIEinit(&w);
NESSIEadd((u8*)valor, 8*sizeo, &w);
NESSIEfinalize(&w, digest);
return rb_str_new(displayHash(digest, DIGESTBYTES), 128);
}
whith that, segmentation fault gone. But the string returned is coming with a strange characters like:
"ruby\x00\x00\x00\x00\x00No error detected.\x00\n\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xD80x\xC0\x18`\x18\x18&F\xAF\x05#\x8C##\xB8\x91\xF9~\xC6?\xC6\xC6\xFB\xCDo\x13\xE8\x87\xE8\xE8\xCB\x13\xA1L\x87&\x87\x87\x11mb\xA9\xB8\xDA\xB8\xB8\t\x02\x05\b\x01\x04\x01\x01\r\x9EnBO!OO\x9Bl\xEE\xAD6\xD866\xFFQ\x04Y\xA6\xA2\xA6\xA6\f\xB9\xBD\xDE\xD2o\xD2"
When the correct must be only:
"ruby"
EDIT 3:
Making this change:
VALUE
print_string(VALUE class, VALUE *valor) {
struct NESSIEstruct w;
u8 digest[DIGESTBYTES];
/*
int i;
for (i = 0; valor[i] != '\0'; i++);
int sizeo = i;
*/
NESSIEinit(&w);
NESSIEadd((u8*)"ruby", 8*4, &w);
NESSIEfinalize(&w, digest);
return rb_str_new(displayHash(digest, DIGESTBYTES), 128);
}
the correct value are returned.
Then I try to to that:
VALUE
print_string(VALUE class, VALUE *valor) {
struct NESSIEstruct w;
u8 digest[DIGESTBYTES];
/*
int i;
for (i = 0; valor[i] != '\0'; i++);
int sizeo = i;
*/
NESSIEinit(&w);
NESSIEadd((u8*)"ruby", 8*4, &w);
NESSIEfinalize(&w, digest);
return rb_str_new2(valor);
}
Expecting "ruby" string to be returned. But not. It returns: "\x05"
What that mean?

Answer 1

for (i = 0; i < length; i++)
Inside this loop for each passing if condition you increment k so by doing this you will be having array out of bound access hence the crash.
Make sure the
char *str = malloc( 3 * length + 1);
is not
char *str = malloc( 3 * (length + 1));
For Eg:
Have value of length = 2;
char *str = malloc(7);
Now in the for loop you increment k 4 times.
k = 4;
Now after the second iteration if you exit the loop then
k=8;
So str[8] is not a valid access and might lead to crash

Answer 2

Before all, let me thanks #FrederickCheung for give me the right direction.
The solution code is:
VALUE
print_string(VALUE class, VALUE valor) {
struct NESSIEstruct w;
u8 digest[DIGESTBYTES];
VALUE info;
// Note here I must convert the Ruby VALUE type to C type. Thats is the trick.
char* valor2 = RSTRING_PTR(valor);
int i;
for (i = 0; valor2[i] != '\0'; i++);
int sizeo = i;
NESSIEinit(&w);
NESSIEadd((u8*)valor2, 8*sizeo, &w);
NESSIEfinalize(&w, digest);
info = rb_str_new_cstr(displayHash(digest, DIGESTBYTES));
return info;
}
For who want to see where this code was used, can access: https://github.com/AngoDev/kryptonita