morphline#flume - looking for regexp change and a hash function - solr

Question

Fluming data to Solr. Data get changed using morphline.
Looking for a couple of basic functions in morphline library:
create a hash value based on other attribute values (e.g. hash=("sha-1", timestamp,message,host,..)
change case of an attribute's string value (something more generic like regexp_replace would do as well).
Don't want yet to write a custom Java handler.. I think there is should be an easier way :)

Answer 1

(1) Non-generic solution for hash function as I wasn't able to find out-of-the-box morphline implementation, hard-coded SHA-1 (eg. no for loop, hard-coded 20 bytes):
{ java {
imports : "import java.security.*;"
code: """
try {
MessageDigest digest = MessageDigest.getInstance("SHA-1");
String value;
value = (String) record.getFirstValue("message");
if (value != null) { digest.update(value.getBytes("ISO-8859-1"), 0, value.length()); }
value = (String) record.getFirstValue("timestamp");
if (value != null) { digest.update(value.getBytes("ISO-8859-1"), 0, value.length()); }
value = (String) record.getFirstValue("hostname");
if (value != null) { digest.update(value.getBytes("ISO-8859-1"), 0, value.length()); }
byte[] a = digest.digest();
record.replaceValues("id"
, String.format("%02X%02X%02X%02X%02X%02X%02X%02X%02X%02X%02X%02X%02X%02X%02X%02X%02X%02X%02X%02X"
,a[0] ,a[1] ,a[2] ,a[3] ,a[4] ,a[5] ,a[6] ,a[7] ,a[8] ,a[9] //SHA-1 has exactly 20 bytes
,a[10],a[11],a[12],a[13],a[14],a[15],a[16],a[17],a[18],a[19]) );
}
catch (java.security.NoSuchAlgorithmException e) { logger.error("hash to id: caught NoSuchAlgorithmException for SHA-1"); }
catch (java.io.UnsupportedEncodingException e) { logger.error("hash to id: caught UnsupportedEncodingException"); }
finally {
return child.process(record);
}
"""
}
}
(2) Non-generic implementaion for lower case transformation (I would hope morphline had just something like regexp_replace) :
java {
code: """
String program = (String) record.getFirstValue("program");
String program_lc = program.toLowerCase();
if (! program.equals(program_lc) )
{ record.replaceValues("program", program_lc); }
return child.process(record);
"""
}