How can I customize the completion of a GtkComboBoxText with both a "static" aspect and a "dynamic" one? The static aspect is because some entries are known and added to the combo-box-text at construction time with gtk_combo_box_text_append_text. The dynamic aspect is because I also need to complete thru some callback function(s), that is to complete dynamically -after creation of the GtkComboBoxText widget- once several characters has been typed.
My application uses Boehm's GC (except for GTK objects of course) like Guile or SCM or Bigloo are doing. It can be seen as an experimental persistent dynamic-typed programming language implementation with an integrated editor coded on and for Debian/Linux/x86-64 with the system GTK3.21 library, it is coded in C99 (some of which is generated) and is compiled with GCC6.
(I don't care about non-Linux systems, GTK3 libraries older than GTK3.20, GCC compiler older that GCC6)
question details
I'm entering (inputting into the GtkComboBoxText) either a name, or an object-id.
The name is C-identifier-like but starts with a letter and cannot end with an underscore. For example, comment, if, the_GUI, the_system, payload_json, or x1 are valid names (but _a0bcd or foobar_ are invalid names, because they start or end with an underscore). I currently have a big dozen of names, but I could have a few thousands of them. So it would be reasonable to offer a completion once only a single or perhaps two letters has been typed, and completion for names can happen statically because they are not many of them (so I feel reasonable to call gtk_combo_box_append_text for each name).
The object-id starts with an underscore followed by a digit and has exactly 18 alphanumeric (sort-of random) characters. For example, _5Hf0fFKvRVa71ZPM0, _8261sbF1f9ohzu2Iu, _0BV96V94PJIn9si1K are possible object-ids. Actually it is 96 almost random bits (probably only 294 are possible). The object-id plays the role of UUIDs (in the sense that it is assumed to be world-wide unique for distinct objects) but has a C friendly syntax. I currently have a few dozen of objects-ids, but I could have a few hundred of thousands (or maybe a million) of them. But given a prefix of four characters like _6S3 or _22z, I am assuming that only a reasonable number (probably at most a dozen, and surely no more than a thousand) object-ids exist in my application with that prefix. Of course it would be unreasonable to register (statically) a priori all the object ids (the completion has to happen after four characters have been typed, and should happen dynamically).
So I want a completion that works both on names (e.g. typing one letter perhaps followed by another alphanum character should be enough to propose a completion of at most a hundred choices), and on object-ids (typing four characters like _826 should be enough to trigger a completion of probably at most a few dozen choices, perhaps a thousand ones if unlucky).
Hence typing the three keys p a tab would offer completion with a few names like payload_json or payload_vectval etc... and typing the five keys _ 5 H f tab would offer completion with very few object-ids, notably _5Hf0fFKvRVa71ZPM0
sample incomplete code
So far I coded the following:
static GtkWidget *
mom_objectentry (void)
{
GtkWidget *obent = gtk_combo_box_text_new_with_entry ();
gtk_widget_set_size_request (obent, 30, 10);
mo_value_t namsetv = mo_named_objects_set ();
I have Boehm-garbage-collected values, and mo_value_t is a pointer to any of them. Values can be tagged integers, pointers to strings, objects, or tuples or sets of objects. So namesetv now contains the set of named objects (probably less than a few thousand of named objects).
int nbnam = mo_set_size (namsetv);
MOM_ASSERTPRINTF (nbnam > 0, "bad nbnam");
mo_value_t *namarr = mom_gc_alloc (nbnam * sizeof (mo_value_t));
int cntnam = 0;
for (int ix = 0; ix < nbnam; ix++)
{
mo_objref_t curobr = mo_set_nth (namsetv, ix);
mo_value_t curnamv = mo_objref_namev (curobr);
if (mo_dyncast_string (curnamv))
namarr[cntnam++] = curnamv;
}
qsort (namarr, cntnam, sizeof (mo_value_t), mom_obname_cmp);
for (int ix = 0; ix < cntnam; ix++)
gtk_combo_box_text_append_text (GTK_COMBO_BOX_TEXT (obent),
mo_string_cstr (namarr[ix]));
at this point I have sorted all the (few thousands at most) names and added "statically" them using gtk_combo_box_text_append_text.
GtkWidget *combtextent = gtk_bin_get_child (GTK_BIN (obent));
MOM_ASSERTPRINTF (GTK_IS_ENTRY (combtextent), "bad combtextent");
MOM_ASSERTPRINTF (gtk_entry_get_completion (GTK_ENTRY (combtextent)) ==
NULL, "got completion in combtextent");
I noticed with a bit of surprise that gtk_entry_get_completion (GTK_ENTRY (combtextent)) is null.
But I am stuck here. I am thinking of:
Having some mom_set_complete_objectid(const char*prefix) which given a prefix like "_47n" of at least four characters would return a garbage collected mo_value_t representing the set of objects with that prefix. This is very easy to code for me, and is nearly done.
Make my own local GtkEntryCompletion* mycompl = ..., which would complete like I want. Then I would put it in the text entry combtextent of my gtk-combo-box-text using gtk_entry_set_completion(GTK_ENTRY(combtextent), mycompl);
Should it use the entries added with gtk_combo_box_text_append_text for the "static" name completion role? How should I dynamically complete using the dynamic set value returned from my mom_set_complete_objectid; given some object-pointer obr and some char bufid[20]; I am easily and quickly able to fill it with the object-id of that object obr with mo_cstring_from_hi_lo_ids(bufid, obr->mo_ob_hid, obr->mo_ob_loid)..
I don't know how to code the above. For reference, I am now just returning the combo-box-text:
// if the entered text starts with a letter, I want it to be
// completed with the appended text above if the entered text starts
// with an undersore, then a digit, then two alphanum (like _0BV or
// _6S3 for example), I want to call a completion function.
#warning objectentry: what should I code here?
return obent;
} /* end mom_objectentry */
Is my approach the right one?
The mom_objectentry function above is used to fill modal dialogs with short lifetime.
I am favoring simple code over efficiency. Actually, my code is temporary (I'm hoping to bootstrap my language, and generate all its C code!) and in practice I'll probably have only a few hundred names and at most a few dozen of thousands of object-ids. So performance is not very important, but simplicity of coding (some conceptually "throw away" code) is more important.
I don't want (if possible) to add my own GTK classes. I prefer using existing GTK classes and widgets, customizing them with GTK signals and callbacks.
context
My application is an experimental persistent programming language and implementation with a near Scheme or Python (or JavaScript, ignoring the prototype aspect, ...) semantics but with a widely different (not yet implemented in september 7th, 2016) syntax (to be shown & input in GTK widgets), using the Boehm garbage collector for values (including objects, sets, tuples, strings...)... Values (including objects) are generally persistent (except the GTK related data : the application starts with a nearly empty window). The entire language heap is persisted in JSON-like syntax in some Sqlite "database" (generated at application exit) dumped into _momstate.sql which is re-loaded at application startup. Object-ids are useful to show object references to the user in GTK widgets, for persistence, and to generate C code related to the objects (e.g. the object of id _76f7e2VcL8IJC1hq6 could be related to a mo_76f7e2VcL8IJC1hq6 identifier in some generated C code; this is partly why I have my object-id format instead of using UUIDs).
PS. My C code is GPLv3 free software and available on github. It is the MELT monitor, branch expjs, commit e2b3b99ef66394...
NB: The objects mentioned here are implicitly my language objects, not GTK objects. The all have a unique object-id, and some but not most of them are named.
I will not show exact code on how to do it because I never did GTK & C only GTK & Python, but it should be fine as the functions in C and Python functions can easily be translated.
OP's approach is actually the right one, so I will try to fill in the gaps. As the amount of static options is limited probably won't change to much it indeed makes sense to add them using gtk_combo_box_text_append which will add them to the internal model of the GtkComboBoxText.
Thats covers the static part, for the dynamic part it would be perfect if we could just store this static model and replace it with a temporay model using gtk_combo_box_set_model() when a _ was found at the start of the string. But we shouldn't do this as the documentation says:
You should not call gtk_combo_box_set_model() or attempt to pack more cells into this combo box via its GtkCellLayout interface.
So we need to work around this, one way of doing this is by adding a GtkEntryCompletion to the entry of the GtkComboBoxText. This will make the entry attempt to complete the current string based on its current model. As an added bonus it can also add all the character all options have in common like this:
As we don't want to load all the dynamic options before hand I think the best approach will be to connect a changed listener to the GtkEntry, this way we can load the dynamic options when we have a underscore and some characters.
As the GtkEntryCompletion uses a GtkListStore internally, we can reuse part of the code Nominal Animal provided in his answer. The main difference being: the connect is done on the GtkEntry and the replacing of GtkComboText with GtkEntryCompletion inside the populator. Then everything should be fine, I wish I would be able to write decent C then I would have provided you with code but this will have to do.
Edit: A small demo in Python with GTK3
import gi
gi.require_version('Gtk', '3.0')
import gi.repository.Gtk as Gtk
class CompletingComboBoxText(Gtk.ComboBoxText):
def __init__(self, static_options, populator, **kwargs):
# Set up the ComboBox with the Entry
Gtk.ComboBoxText.__init__(self, has_entry=True, **kwargs)
# Store the populator reference in the object
self.populator = populator
# Create the completion
completion = Gtk.EntryCompletion(inline_completion=True)
# Specify that we want to use the first col of the model for completion
completion.set_text_column(0)
completion.set_minimum_key_length(2)
# Set the completion model to the combobox model such that we can also autocomplete these options
self.static_options_model = self.get_model()
completion.set_model(self.static_options_model)
# The child of the combobox is the entry if 'has_entry' was set to True
entry = self.get_child()
entry.set_completion(completion)
# Set the active option of the combobox to 0 (which is an empty field)
self.set_active(0)
# Fill the model with the static options (could also be used for a history or something)
for option in static_options:
self.append_text(option)
# Connect a listener to adjust the model when the user types something
entry.connect("changed", self.update_completion, True)
def update_completion(self, entry, editable):
# Get the current content of the entry
text = entry.get_text()
# Get the completion which needs to be updated
completion = entry.get_completion()
if text.startswith("_") and len(text) >= completion.get_minimum_key_length():
# Fetch the options from the populator for a given text
completion_options = self.populator(text)
# Create a temporary model for the completion and fill it
dynamic_model = Gtk.ListStore.new([str])
for completion_option in completion_options:
dynamic_model.append([completion_option])
completion.set_model(dynamic_model)
else:
# Restore the default static options
completion.set_model(self.static_options_model)
def demo():
# Create the window
window = Gtk.Window()
# Add some static options
fake_static_options = [
"comment",
"if",
"the_GUI",
"the_system",
"payload_json",
"x1",
"payload_json",
"payload_vectval"
]
# Add the the Combobox
ccb = CompletingComboBoxText(fake_static_options, dynamic_option_populator)
window.add(ccb)
# Show it
window.show_all()
Gtk.main()
def dynamic_option_populator(text):
# Some fake returns for the populator
fake_dynamic_options = [
"_5Hf0fFKvRVa71ZPM0",
"_8261sbF1f9ohzu2Iu",
"_0BV96V94PJIn9si1K",
"_0BV1sbF1f9ohzu2Iu",
"_0BV0fFKvRVa71ZPM0",
"_0Hf0fF4PJIn9si1Ks",
"_6KvRVa71JIn9si1Kw",
"_5HKvRVa71Va71ZPM0",
"_8261sbF1KvRVa71ZP",
"_0BKvRVa71JIn9si1K",
"_0BV1KvRVa71ZPu2Iu",
"_0BV0fKvRVa71ZZPM0",
"_0Hf0fF4PJIbF1f9oh",
"_61sbFV0fFKn9si1Kw",
"_5Hf0fFKvRVa71ozu2",
]
# Only return those that start with the text
return [fake_dynamic_option for fake_dynamic_option in fake_dynamic_options if fake_dynamic_option.startswith(text)]
if __name__ == '__main__':
demo()
Gtk.main()
Here is my suggestion:
Use a GtkListStore to contain a list of GTK-managed strings (essentially, copies of your identifier string) that match the current prefix string.
(As documented for gtk_list_store_set(), a G_TYPE_STRING item is copied. I consider the overhead of the extra copy acceptable here; it should not affect real-world performance much anyway, I think, and in return, GTK+ will manage the reference counting for us.)
The above is implemented in a GTK+ callback function, which gets an extra pointer as payload (set at the time the GUI is created or activated; I suggest you use some structure to keep references you need to generate the matches). The callback is connected to the combobox popup signal, so that it gets called whenever the list is expanded.
Note that as B8vrede noted in a comment, a GtkComboBoxText should not be modified via its model; that is why one should/must use a GtkComboBox instead.
Practical example
For simplicity, let's assume all the data you need to find or generate all known identifiers matched against is held in a structure, say
struct generator {
/* Whatever data you need to generate prefix matches */
};
and the combo box populator helper function is then something like
static void combo_box_populator(GtkComboBox *combobox, gpointer genptr)
{
struct generator *const generator = genptr;
GtkListStore *combo_list = GTK_LIST_STORE(gtk_combo_box_get_model(combobox));
GtkWidget *entry = gtk_bin_get_child(GTK_BIN(combobox));
const char *prefix = gtk_entry_get_text(GTK_ENTRY(entry));
const size_t prefix_len = (prefix) ? strlen(prefix) : 0;
GtkTreeIter iterator;
/* Clear the current store */
gtk_list_store_clear(combo_list);
/* Initialize the list iterator */
gtk_tree_model_get_iter_first(GTK_TREE_MODEL(combo_list), &iterator);
/* Find all you want to have in the combo box;
for each const char *match, do:
*/
gtk_list_store_append(combo_list, &iterator);
gtk_list_store_set(combo_list, &iterator, 0, match, -1);
/* Note that the string pointed to by match is copied;
match is not referred to after the _set() returns.
*/
}
When the UI is built or activated, you need to ensure the GtkComboBox has an entry (so the user can write text into it), and a GtkListStore model:
struct generator *generator;
GtkWidget *combobox;
GtkListStore *combo_list;
combo_list = gtk_list_store_new(1, G_TYPE_STRING);
combobox = gtk_combo_box_new_with_model_and_entry(GTK_TREE_MODEL(combo_list));
gtk_combo_box_set_id_column(GTK_COMBO_BOX(combobox), 0);
gtk_combo_box_set_entry_text_column(GTK_COMBO_BOX(combobox), 0);
gtk_combo_box_set_button_sensitivity(GTK_COMBO_BOX(combobox), GTK_SENSITIVITY_ON);
g_signal_connect(combobox, "popup", G_CALLBACK(combo_box_populator), generator);
On my system, the default pop-up accelerator is Alt+Down, but I assume you've already changed that to Tab.
I have a crude working example here (a .tar.xz tarball, CC0): it reads lines from standard input, and lists the ones matching the user prefix in reverse order in the combo box list (when popped-up). If the entry is empty, the combobox will contain all input lines. I didn't change the default accelerators, so instead of Tab, try Alt+Down.
I also have the same example, but using GtkComboBoxText instead, here (also CC0). This does not use a GtkListStore model, but uses gtk_combo_box_text_remove_all() and gtk_combo_box_text_append_text() functions to manipulate the list contents directly. (There is just a few different lines in the two examples.) Unfortunately, the documentation is not explicit whether this interface references or copies the strings. Although copying is the only option that makes sense, and this can be verified from the current Gtk+ sources, the lack of explicit documentation makes me hesitant.
Comparing the two examples I linked to above (both grab some 500 random words from /usr/share/dict/words if you compile and run it with make), I don't see any speed difference. Both use the same naïve way of picking prefix matches from a linked list, which means the two methods (GtkComboBox + model, or GtkComboBoxText) should be about equally fast.
On my own machine, both get annoyingly slow with more than 1000 or so matches in the popup; with just a hundred or less matches, it feels instantaneous. This, to me, indicates that the slow/naïve way of picking prefix matches from a linked list is not the culprit (because the entire list is traversed in both cases), but that the GTK+ combo boxes are just not designed for large lists. (The slowdown is definitely much, much worse than linear.)
The issue is a bit complicated so I need to give you the full story:
I'm trying to create basic sound using the package pyaudio.
The pyaudio module asks for a callback function that reads audio data.
It basically looks like this:
def callback():
return sound.readframes()
I have a class that provides audio data. It creates a signal by indexing a numpy array that contains a wave length of a basic wave form. It basically looks like this:
class Wave(object):
data = [] # one wavelength of a basic waveform like a sine wave will be stored here
def readframes(self):
indices = # the indices for the next frames
# based on a frequency and the current position
return self.data[indices]
class SineWave(Wave):
data = # points of a sine wave in a numpy array
sound = SineWave()
# pyaudio now reads audio data in a non-blocking callback mode
Now to the problem:
There was noise between each call to readframes or each block of audio data. The sound itself was still intact, but there was an additional clicking. Increasing the size of the returned data block also increased the time between each click. I still don't quite know what caused the noise, but I fixed it by storing the returned numpy array in the instance instead of returning it directly.
This causes noise:
return self.data[indices]
This does not cause noise:
+ self.result = self.data[indices]
+ return self.result
It can't be an alteration of the data that causes the noise or else this wouldn't fix it. Why does the result have to be stored in the instance?
Update:
I also have to mention that the noise doesn't occur everytime I run the script. It occurs randomly about 50% of the runs. Either the noise appears right at the start and won't go away or it doesn't appear at all.
It looks like an issue with the initialization of the sound device, but how that could be related to my fix is a mistery.
Here is code that reproduces the noise problem:
import pyaudio
import time
from numpy import sin, linspace, arange, pi, float32, uint32
SR = 44100 # sampling rate
FPB = 2048 # frames per buffer
class SineWave(object):
"""A sine wave"""
data_size = 2**10
data = sin(linspace(0, 2*pi, data_size, endpoint=False, dtype=float32))
def __init__(self, frequency):
self.pos = 0
self.fq = frequency
self.ppf = self.data_size * self.fq / SR # points per frame
def readframes(self):
size = FPB
pos = self.pos
indices = (arange(pos, size+pos)*self.ppf).astype(uint32) & (self.data_size-1)
self.pos = (self.pos + size) % SR
self.result = self.data[indices]
return self.data[indices]
def callback(in_data, frame_count, time_info, status):
return sound.readframes(), pyaudio.paContinue
p = pyaudio.PyAudio()
stream = p.open(format=p.get_format_from_width(4),
channels=1,
rate=SR,
output=True,
stream_callback=callback,
frames_per_buffer=FPB)
sound = SineWave(440)
stream.start_stream()
time.sleep(2)
stream.stop_stream()
stream.close()
p.terminate()
On my system the noise appears in about 90% of the runs.
Changing
return self.data[indices]
to
return self.result
fixes it.
There shouldn't be a difference whether you store the intermediate array in the instance or not. I suspect there is some other problem going on that causes the error you're experiencing. Can you provide code that actually runs and exhibits the problem?
If you want to see a (hopefully!) working implementation, have a look at the play() function of the sounddevice module, especially at the loop argument. With this, you should be able to play your waveform repeatedly.
The only thing you need to store in the instance, is the frame index where you left off, just to continue with the following frame index in the next callback (and of course wrap around in the end). You should also consider the case where your wave form is shorter than an audio block. It might even be much shorter. I solved that by using a recursive function call, see the code.
UPDATE:
It's indeed quite strange that assigning to self "solves" your problem. I think this has something to do with how PyAudio interprets the returned array as Python buffer.
You should rewrite your code using the sounddevice module, which properly supports NumPy arrays and has a sane API for the callback function.
It's easy, just use this callback function:
def callback(outdata, frames, time, status):
if status:
print(status)
outdata[:, 0] = sound.readframes()
And start the stream with:
import sounddevice as sd
stream = sd.OutputStream(
channels=1, samplerate=SR, callback=callback, blocksize=FPB)
Apart from that, your code is flawed since it doesn't do any interpolation when reading from the wavetable.
Currently I am sending the UART the strings I want to log and reading it on the host with any terminal.
In order to reduce the logging time and hopefully the image size as well (my flash is tiny), I figured out that the strings are unused in the embedded system, so why storing them on the flash?
I want to implement a server, whom I can send a hashed-key of any string (for example - it's ROM address) and the string will be output to file or screen.
My questions are:
How to create the key2string converter out of the image file (the OS is CMX, but can be answered generally)
Is there a recomended way to generate image, that will know the strings addresses but will exclude them from ROM?
Is there a known generic (open-source or other) that implemented a similar logger?
Thanks
Rather than holding hard-coded strings, then trying to hash the answers and sent it via a UART, then somehow remove the strings from the resulting image, I suggest the following.
Just send an index for an error code. The PC side can look up that index and determine what the string is for that condition. If you want the device code to be more clear, the index can be an enumeration.
For example:
enum errorStrings
{
ES_valueOutOfLimits = 1,
ES_wowItsGettingWarm = 2,
ES_randomError = 3,
ES_passwordFailure = 4
};
So, if you were sending data to the UART via printf, you could do the following:
printf("%d\n",(int)ES_wowItsGettingWarm);
Then your PC software just needs to decode the "2" that comes across the UART back into a useful string of "Wow it's getting warm."
This keeps the firmware small, but you need to manually keep the file containing the enum and the file with the strings in sync.
My solution is sending file name and line (which should be 14-20 Byte) and having a source parser on the server side, which will generate map of the actual texts. This way the actual code will contain no "format" strings, but single "filename" string for each file. Furthermore, file names can be easily replaced with enum (unlike replacing every string in the code) to reduce the COMM throughput.
I hope the sample psaudo-code will help clarifying the idea:
/* target code */
#define PRINT(format,...) send(__FILE__,__LINE__,__VA_ARGS__)
...
/* host code (c++) */
void PrintComm(istream& in)
{
string fileName;
int line,nParams;
int* params;
in>>fileName>>line>>nParams;
if (nParams>0)
{
params = new int[nParams];
for (int i=0; i<nParams; ++i)
in>>params[i];
}
const char* format = FindFormat(fileName,line);
...
delete[] params;
}
I try to connect CLIPS to my C program. Set of rules will be loaded from external .clp file into CLIPS. My new C program will in fixed time intervals set new facts (example (temperature 35C)) which will represent current measurements from some temperature sensors. Then expert system will be started and make some conclusions and give necessary actions based on provided measurements. Conclusions will be in the form of facts (Answer-is x).
How I can read desired fact field from CLIPS in the form of C variable ? For example, if fact of interest is (Answer-is, x), and if I know that first field is Answer-is, how can I make my C program to find that fact and read x ?
Example code which can help you to understand what I would like to do is the following:
int main(int argc,char *argv[]) {
void *theEnv;
theEnv = CreateEnvironment();
InitializeEnvironment();
Clear();
Reset();
// Load rules to CLIPS
Load("example.clp");
// Set facts and start CLIPS
AssertString("(TemperatureSensor1 35)");
AssertString("(TemperatureSensor2 32)");
AssertString("(TemperatureSensor2 41)");
Run(-1L);
// Extract expert system answer to C
// Read second field of generated fact from the example.clp
// which is in the form like (Answer-is xxx)
// ????????????????????????????
// This part I do not know ....
// ????????????????????????????
return(-1);
}
You can use the Eval API function to invoke the find-all-facts fact CLIPS query function. The return value from Eval will have the list of facts satisfying your query. You can then iterate over each fact (if there is more than one) and use the GetFactSlot function to retrieve the slot values of the fact.
I want to feed OpenSSL specific data for use as random seed during the signing of data with an EC key. I'm doing this to compare my application with another reference one (closed source). That utility takes the file with private key, file with data to sign and file with random data as parameters.
I've got the generation of EC keys, and signing of data down, but can't compare the two applications since I have no common ground. OpenSSL generates random data used in signing the data (probably from /dev/random) and thus gives me a different signature every run.
I've tried RAND_clear() in combination with RAND_add(), but keep getting changing signatures. Either I don't understand the whole ECDSA concept, or I'm doing something wrong.
My second option for comparing the applications is to import the public key and verify the signature generated by the reference program. This is the better option, but I'm unable to import the given example public key (83 character hex string). EC_POINT_oct2point() keeps giving me null results.
Any help/pointers/references would be greatly appreciated.
char * key_as_binary_data; //369368AF243193D001E39CE76BB1D5DA08A9BC0A63307AB352338E5EA5C0E05A0C2531866F3E3C2702
int data_size; //Size of the key buffer
EC_POINT * ecpoint = NULL;
EC_GROUP * ecgroup = NULL;
EC_KEY * eckey = NULL;
point_conversion_form_t form = POINT_CONVERSION_UNCOMPRESSED;
int asn1_flag = OPENSSL_EC_NAMED_CURVE;
eckey = EC_KEY_new();
ecpoint = EC_POINT_new(ecgroup);
ecgroup = EC_GROUP_new_by_curve_name(OBJ_sn2nid("sect163k1"));
EC_GROUP_set_asn1_flag(ecgroup, asn1_flag);
EC_GROUP_set_point_conversion_form(ecgroup, form);
EC_KEY_set_group(eckey,ecgroup);
EC_KEY_generate_key(eckey);
//This gives me a null ecpoint
EC_POINT_oct2point(ecgroup,ecpoint,key_as_binary_data,data_size-1,ctx);
EC_KEY_set_public_key(eckey,ecpoint);
This is how you should go about loading that public key:
EC_KEY *key = NULL;
EC_POINT *pub_key;
const EC_GROUP *group;
SSL_library_init();
SSL_load_error_strings();
key = EC_KEY_new_by_curve_name(NID_sect163k1);
group = EC_KEY_get0_group(key);
pub_key = EC_POINT_new(group);
EC_POINT_hex2point(group,
"369368AF243193D001E39CE76BB1D5DA08A9BC0A63307AB352338E5EA5C0E05A0C2531866F3E3C2702", pub_key, NULL);
EC_KEY_set_public_key(key, pub_key);
if (!EC_KEY_check_key(key)) {
printf("EC_KEY_check_key failed:\n");
printf("%s\n",ERR_error_string(ERR_get_error(),NULL));
} else {
printf("Public key verified OK\n");
}
It seems to verify OK, so it should work for checking a signature.
I think your bug might have just been passing a NULL (in ecgroup) to EC_POINT_new().
The reason you get different results despite the fact that you are clearing the pool and resetting it is that by default OpenSSL's RAND implementation will hash the pid into the output block (precisely to ensure that even applications that use the same seed do not get the same PRNG output, since 99.9% of the time that happening is a Bad Thing).
In addition, even if this was not the case, it is unlikely that your reference application uses the same PRNG that OpenSSL uses to turn the seed file into a series of random bytes. (Unless your reference application actually uses OpenSSL as well, of course). What you would have to do is first figure out what kind of PRNG the reference app uses - this might be a standard PRNG design like the ones from X9.31 or FIPS-186, or might be something totally custom. Then reimplement that design for OpenSSL and plug it in via RAND_set_rand_method.
As to verification: it looks like you need to transpose the lines:
ecpoint = EC_POINT_new(ecgroup);
ecgroup = EC_GROUP_new_by_curve_name(OBJ_sn2nid("sect163k1"));
Otherwise ecpoint is set to NULL right from the start, and this causes EC_KEY_generate_key to fail, because the group is set to NULL. Quoting from openssl-0.9.8k's crypto/ec/ec_key.c:
if (!eckey || !eckey->group)
{
ECerr(EC_F_EC_KEY_GENERATE_KEY, ERR_R_PASSED_NULL_PARAMETER);
return 0;
}
You can control the random data that OpenSSL produces during signing by using the method:
ECDSA_SIG* ECDSA_do_sign_ex(const unsigned char *dgst, int dgstlen, const BIGNUM *kinv, const BIGNUM *rp, EC_KEY *eckey);
where kinv is the random number that is used during signing.
The signing procedure is purely to allow someone else to confirm that you signed it ie. it was your private key that was used to sign the message (or any data) without actually having your private key.
The algorithm is outlined on wikipedia Elliptic_Curve_DSA
Reading "Signature generation algorithm" it appears the random data is used to assist in the strength of the signature and to make it harder to attack to figure out the private key.
Therefore you should expect the signature to be different each time since this is not just simply a hash.
See the section "Signature verification algorithm" to see that the verification steps are the ones you wish to use to confirm that your openssl version is outputing valid signatures w.r.t. the ECDSA method and the closed source program.
I can't find the docs for RAND_clear, so can't comment why it's not resulting in reproducible random numbers.
But even if you get that done, what you want may not be possible. ECDSA signature generation requires choosing a random integer in a particular range. Two different implementations of the algorithm might have completely different ideas about how to generate such an integer from their entropy source. AFAIK the means of transforming bits of entropy into the required number is not part of the ECDSA algorithm.
So for example (very bad examples of course), one implementation might do this:
int random_number = rand() % n;
and another might do this:
int random_number = (rand() * n) / RAND_MAX;
So even if you give them the same seed data, you might still get different signatures from different implementations. All you can do is validate whether you have generated a valid signature.