Lib using Tcl C API is crashing maybe due bad refCount usage - c

I'm a beginner at Tcl C API and I'm trying to understand how to use it. I have this code that gets a tcl list from get_my_list proc and then I iterate over it to dispatch some info, in this case, info related to attributes A, B and C that I get with get_attr_info proc. When I run it in a very simple example that basically has only this code, everything works perfectly, but when I add this lib in a big tcl project, eventually it crashes. I suspect that it's due to bad usage of reference counting of tcl objects, I mean, their lifetimes. What could I be doing wrong in the example below? I'm using Tcl 8.6.
Tcl_Obj* Get_Info(Tcl_Interp *interp, const char* info, const char* attr) {
char cmd[256];
sprintf(cmd, "get_attr_info %s %s", info, attr);
Tcl_Eval(interp, cmd);
return Tcl_GetObjResult(interp);
}
static int Copy_Info(ClientData cdata, Tcl_Interp *interp, int objc, Tcl_Obj *const objv[]) {
Tcl_Eval(interp, "get_my_list");
Tcl_Obj *const my_list = Tcl_GetObjResult(interp);
int my_list_size;
Tcl_ListObjLength(interp, my_list, &my_list_size);
Tcl_IncrRefCount(my_list);
for (int i = 0; i < my_list_size; ++i) {
Tcl_Obj* current_info_obj;
Tcl_ListObjIndex(interp, my_list, i, &current_info_obj);
const char* current_info_ctr = Tcl_GetStringFromObj(current_info_obj, NULL);
/* getting info A */
Tcl_Obj* info_a_obj = Get_Info(current_info_ctr, "A");
const char* info_a = Tcl_GetStringFromObj(info_a_obj, NULL);
Copy_Info_A(info_a);
/* getting info B */
Tcl_Obj* info_b_obj = Get_Info(current_info_ctr, "B");
const char* info_b = Tcl_GetStringFromObj(info_b_obj, NULL);
Copy_Info_B(info_b);
/* getting info C */
Tcl_Obj* info_c_obj = Get_Info(current_info_ctr, "C");
const char* info_c = Tcl_GetStringFromObj(info_c_obj, NULL);
Copy_Info_C(info_c);
}
Tcl_DecrRefCount(my_list);
Tcl_FreeResult(interp);
return TCL_OK;
}

The tricky bit is that you call Tcl_Eval() inside Get_Info(), which can do all sorts of things with reference count management of values that you're not holding your own reference to, including shimmering away the list representation of the value you're holding a reference to in my_list, which can pull the rug out from under its elements. In particular, the object referred to in current_info_obj must have its reference count incremented from after the Tcl_ListObjIndex() to the end of the loop body.
// ...
for (int i = 0; i < my_list_size; ++i) {
Tcl_Obj* current_info_obj;
Tcl_ListObjIndex(interp, my_list, i, &current_info_obj);
// HOLD THE REFERENCE; IT OWNS THE current_info_ctr STRING FOR US
Tcl_IncrRefCount(current_info_obj);
const char* current_info_ctr = Tcl_GetStringFromObj(current_info_obj, NULL);
/* getting info A */
Tcl_Obj* info_a_obj = Get_Info(current_info_ctr, "A");
const char* info_a = Tcl_GetStringFromObj(info_a_obj, NULL);
Copy_Info_A(info_a);
/* getting info B */
Tcl_Obj* info_b_obj = Get_Info(current_info_ctr, "B");
const char* info_b = Tcl_GetStringFromObj(info_b_obj, NULL);
Copy_Info_B(info_b);
/* getting info C */
Tcl_Obj* info_c_obj = Get_Info(current_info_ctr, "C");
const char* info_c = Tcl_GetStringFromObj(info_c_obj, NULL);
Copy_Info_C(info_c);
// RELEASE THE REFERENCE
Tcl_DecrRefCount(current_info_obj);
}
// ...
Improving your code
You can also do a short-cut, and use Tcl_GetString(objPtr) in place of Tcl_GetStringFromObj(objPtr, NULL). You might put that inside Get_Info. You probably also ought to consider switching to using Tcl_EvalObjv(), as that's a faster API (it avoids a level of parsing), though a more complex one to use.
const char *Get_Info(Tcl_Interp *interp, Tcl_Obj* info, const char* attr) {
Tcl_Obj *argv[3], *result;
argv[0] = Tcl_NewStringObj("get_attr_info", -1); // cacheable
argv[1] = info;
argv[2] = Tcl_NewStringObj(attr, -1);
Tcl_IncrRefCount(argv[0]);
Tcl_IncrRefCount(argv[1]);
Tcl_IncrRefCount(argv[2]);
// It's very bad form to omit error handling
if (Tcl_EvalObjv(interp, 3, argv, 0) != TCL_OK) {
Tcl_Panic("problem: %s", Tcl_GetString(Tcl_GetObjResult(interp)));
}
result = Tcl_GetObjResult(interp);
Tcl_DecrRefCount(argv[0]);
Tcl_DecrRefCount(argv[1]);
Tcl_DecrRefCount(argv[2]);
return Tcl_GetString(result);
}
There are cases where it's possible to reduce the amount of reference count handling. Don't worry about doing that to start with! Better to definitely avoid crashes!
Tcl_EvalObjv is (effectively) what Tcl_Eval calls to actually dispatch a command after parsing in interactive mode, and it's what the bytecode engine calls to evaluate any non-inlined command. It's much more efficient than parsing a string into words.

Related

How can I have multiple threads working on the same for loop in C which in turn calls a function

This code is part of nvme-cli(its built on linux api)
I am trying to build an option to repeat a nvme-command say n times with p number of threads.
For example, if I say repeat 50-times with 5-threads, then each of the 5 threads should repeat 10-times individually.
So for that, I need to run the below for loop with multiple threads. Since I am new to multi-threading I thought of using P-threads. Can I know the syntax for doing so?
int* identify(int fd, int name_space, void *ctrl, int cns, int rc)
{
int *err,count=0;
struct nvme_admin_cmd cmd;
static const char *perrstr;
memset(&cmd, 0, sizeof(cmd));
cmd.opcode = nvme_admin_identify;
cmd.nsid = name_space;
cmd.addr = (__u64)(uintptr_t)ctrl;
cmd.data_len = 4096;
cmd.cdw10 = cns;
for(count=0; count<rc; count++){
*(err+count) = ioctl(fd, NVME_IOCTL_ADMIN_CMD, &cmd);
}
return err;
}

creating callbacks and structs for repeated field in a protobuf message in nanopb in c

I have a proto message defined as:
message SimpleMessage {
repeated int32 number = 1;}
now, after compiling, the field is of pb_callback_t and I suppose to write that function. (without .options file)
now, where and what should the function contain? where does the data itself being stored and how can I access it/ assign new data to it?
* EDIT *
according to #Groo 's answer, this is the code I tried:
typedef struct {
int numbers_decoded;
} DecodingState;
bool read_single_number(pb_istream_t *istream, const pb_field_t *field, void **arg)
{
// get the pointer to the custom state
DecodingState *state = (DecodingState*)(*arg);
int32_t value;
if (!pb_decode_varint32(istream, &value))
{
const char * error = PB_GET_ERROR(istream);
printf("Protobuf error: %s", error);
return false;
}
printf("Decoded successfully: %d", value);
state->numbers_decoded++;
return true;
}
int main(void) {
int32_t arr[3] = {10, 22, 342};
uint8_t buffer[128];
size_t message_length;
bool status;
SimpleMessage simple = SimpleMessage_init_zero;
printf("\nbefore : arr[0] = %d\n",arr[0]);
// set the argument and the callback fn
simple.number.arg = &arr;
simple.number.funcs.decode = read_single_number;
pb_ostream_t ostream = pb_ostream_from_buffer(buffer, sizeof(buffer));
status = pb_encode(&ostream, SimpleMessage_fields, &simple);
message_length = ostream.bytes_written;
SimpleMessage simple1 = SimpleMessage_init_zero;
simple = simple1;
arr[0] = 0;
pb_istream_t istream = pb_istream_from_buffer(buffer, message_length);
// this function will call read_single_number several times
status = pb_decode(&istream, SimpleMessage_fields, &simple);
printf("\nafter : arr[0] = %d\n",arr[0]);
return EXIT_SUCCESS;
}
and the output is:
before : arr[0] = 10
Decoded successfully: 17
after : arr[0] = 0
what do I do wrong?
You can use some nanopb-specific proto flags to force nanopb to generate structs with statically allocated arrays.
However, the default behavior of nanopb's protogen is to generate a callback function which is called by nanopb during encoding (once for the entire list) and decoding (once for each item in the list). This is sometimes preferred in low-memory embedded systems, because you don't need to allocate more than one item at a time.
So, for your .proto file:
message SimpleMessage {
repeated int32 number = 1;
}
You might get something like:
typedef struct _SimpleMessage {
pb_callback_t number;
} SimpleMessage;
Meaning you will have to create your own callback function which will be called for each item in succession.
So for simplicity, let's say you have a simple "variable length" list like this:
#define MAX_NUMBERS 32
typedef struct
{
int32_t numbers[MAX_NUMBERS];
int32_t numbers_count;
}
IntList;
// add a number to the int list
void IntList_add_number(IntList * list, int32_t number)
{
if (list->numbers_count < MAX_NUMBERS)
{
list->numbers[list->numbers_count] = number;
list->numbers_count++;
}
}
Obviously, for such an example, using callbacks wouldn't make any sense, but it makes the example simple.
Encoding callback must iterate through the list, and write the protobuf tag and the value for each item in the list:
bool SimpleMessage_encode_numbers(pb_ostream_t *ostream, const pb_field_t *field, void * const *arg)
{
IntList * source = (IntList*)(*arg);
// encode all numbers
for (int i = 0; i < source->numbers_count; i++)
{
if (!pb_encode_tag_for_field(ostream, field))
{
const char * error = PB_GET_ERROR(ostream);
printf("SimpleMessage_encode_numbers error: %s", error);
return false;
}
if (!pb_encode_svarint(ostream, source->numbers[i]))
{
const char * error = PB_GET_ERROR(ostream);
printf("SimpleMessage_encode_numbers error: %s", error);
return false;
}
}
return true;
}
Decoding callback is called once for each item, and "appends" to the list:
bool SimpleMessage_decode_single_number(pb_istream_t *istream, const pb_field_t *field, void **arg)
{
IntList * dest = (IntList*)(*arg);
// decode single number
int64_t number;
if (!pb_decode_svarint(istream, &number))
{
const char * error = PB_GET_ERROR(istream);
printf("SimpleMessage_decode_single_number error: %s", error);
return false;
}
// add to destination list
IntList_add_number(dest, (int32_t)number);
return true;
}
With these two in place, you must be careful to assign the right callback to the right function:
uint8_t buffer[128];
size_t total_bytes_encoded = 0;
// encoding
{
// prepare the actual "variable" array
IntList actualData = { 0 };
IntList_add_number(&actualData, 123);
IntList_add_number(&actualData, 456);
IntList_add_number(&actualData, 789);
// prepare the nanopb ENCODING callback
SimpleMessage msg = SimpleMessage_init_zero;
msg.number.arg = &actualData;
msg.number.funcs.encode = SimpleMessage_encode_numbers;
// call nanopb
pb_ostream_t ostream = pb_ostream_from_buffer(buffer, sizeof(buffer));
if (!pb_encode(&ostream, SimpleMessage_fields, &msg))
{
const char * error = PB_GET_ERROR(&ostream);
printf("pb_encode error: %s", error);
return;
}
total_bytes_encoded = ostream.bytes_written;
printf("Encoded size: %d", total_bytes_encoded);
}
And similar for decoding:
// decoding
{
// empty array for decoding
IntList decodedData = { 0 };
// prepare the nanopb DECODING callback
SimpleMessage msg = SimpleMessage_init_zero;
msg.number.arg = &decodedData;
msg.number.funcs.decode = SimpleMessage_decode_single_number;
// call nanopb
pb_istream_t istream = pb_istream_from_buffer(buffer, total_bytes_encoded);
if (!pb_decode(&istream, SimpleMessage_fields, &msg))
{
const char * error = PB_GET_ERROR(&istream);
printf("pb_decode error: %s", error);
return;
}
printf("Bytes decoded: %d", total_bytes_encoded - istream.bytes_left);
}
If you have a repeated struct inside your message, your callback will not use
nanopb primitive functions (like pb_decode_varint32 above), but again pb_decode for each concrete message type. Your callback can also attach new callbacks to those nested structs, if needed.
To complement Groo's answer, here are answers to your specific questions.
1. Now, where and what should the function contain?
Groo provided good explanation of the callback functions. The network_server example in nanopb repository also uses callbacks and can be a useful reference: network_server/server.c network_server/client.c
2. Where does the data itself being stored?
Wherever you want! The whole point of nanopb's callbacks is that it gives you full flexibility in deciding how to store your data. In some cases you may want to even process the data on the fly, not storing it anywhere.
For example, the network_server example above gets the filenames from filesystem and sends them to the network directly - this way it can handle any amount of files without requiring much memory.
3. How can I access it/ assign new data to it?
Now this is the downside of callbacks - you'll have to implement your own access and allocation functions for whatever storage you use. That's why for the most common cases, either static allocation (with fixed maximum size) or dynamic allocation (which malloc()s required amount of memory) are more convenient.

Execv failing even when command is available

I'm trying to call execv after manually saerching for the program to execute.
In my case,
c is a struct which has args as an array of strings having the arguments passed while receiving input. nargs is the number of arguments.
c->args[0] would contain "ls","cat" etc.
I tried printing the value of the args[0], fullPath etc. in my child process. They all show values like "/bin/ls","/bin/cat" etc. But when I call execv, it returns -1 with an errno of 2, which I understand is the error for "No such file or directory". But I'm sure the file is there because thats what my PathResolver is returning after checking all permissions.
Can anyone point where I might have made a mistake.
//The part happening inside child
char *fullPath = PathResolver(c->args[0],1,&permission);
printf("FullPath: %s -- Permission: %d\n",fullPath,permission);
if(permission==0)
{
fprintf(stderr, "%s: Command not found\n",c->args[0]);
}
else if(permission==-1)
{
fprintf(stderr, "%s: Permission denied\n",c->args[0]);
}
else
{
char* args[c->nargs+1];
int m=0;
for(m=0;m<c->nargs;m++)
{
strcpy(args[m],c->args[m]);
}
args[c->nargs] = NULL;
printf("%d\n",execv(args[0], args));
printf("errno: %d\n",errno);
}
PathResolver function
char* PathResolver(char *command, int ResolverMode, int *Permission)
{
*Permission = 0;
char *returnString;
returnString = malloc((sizeof(char)));
char *strPath = getenv("PATH");
char *del = ":";
char *strToken = strtok(strPath,del);
FILE *f;
while(strToken)
{
char filePath[100];
sprintf(filePath,"%s/%s",strToken,command);
if(access(filePath,F_OK)>=0)
{
if(access(filePath,X_OK)>=0)
{
*Permission = 1;
sprintf(returnString,"%s%s ",returnString,filePath);
if(ResolverMode == 1)
break;
}
else
{
*Permission = -1;
}
}
strToken = strtok(NULL,del);
}
sprintf(returnString,"%s\b",returnString);
return returnString;
}
strcpy(args[m],c->args[m]); is undefined behaviour, because args[m] is not a pointer to valid memory.
The following might be simpler:
char * args[c->nargs + 1];
for (size_t m = 0; m != c->nargs; ++m)
{
args[m] = c->args[m];
}
args[c->nargs] = NULL;
There's no need to copy the strings.
(This may not be your actual problem, but it certainly prevents your program from being correct.)
execv() expects the program name to be prefixed by a full path as 1st parameter.
To have PATH searched instead of providing a path use execvp().
Update:
Also this line
returnString = malloc((sizeof(char)));
does only allocate 1 byte to returnString, which is way to few for how you use returnString.

User entered string run a particular function in c

Guys so I'm working on the web service assignment and I have the server dishing out random stuff and reading the uri but now i want to have the server run a different function depending on what it reads in the uri. I understand that we can do this with function pointers but i'm not exactly sure how to read char* and assign it to a function pointer and have it invoke that function.
Example of what I'm trying to do: http://pastebin.com/FadCVH0h
I could use a switch statement i believe but wondering if there's a better way.
For such a thing, you will need a table that maps char * strings to function pointers. The program segfaults when you assign a function pointer to string because technically, a function pointer is not a string.
Note: the following program is for demonstration purpose only. No bounds checking is involved, and it contains hard-coded values and magic numbers
Now:
void print1()
{
printf("here");
}
void print2()
{
printf("Hello world");
}
struct Table {
char ptr[100];
void (*funcptr)(void)
}table[100] = {
{"here", print1},
{"hw", helloWorld}
};
int main(int argc, char *argv[])
{
int i = 0;
for(i = 0; i < 2; i++){
if(!strcmp(argv[1],table[i].ptr) { table[i].funcptr(); return 0;}
}
return 0;
}
I'm gonna give you a quite simple example, that I think, is useful to understand how good can be functions pointers in C. (If for example you would like to make a shell)
For example if you had a struct like this:
typedef struct s_function_pointer
{
char* cmp_string;
int (*function)(char* line);
} t_function_pointer;
Then, you could set up a t_function_pointer array which you'll browse:
int ls_function(char* line)
{
// do whatever you want with your ls function to parse line
return 0;
}
int echo_function(char* line)
{
// do whatever you want with your echo function to parse line
return 0;
}
void treat_input(t_function_pointer* functions, char* line)
{
int counter;
int builtin_size;
builtin_size = 0;
counter = 0;
while (functions[counter].cmp_string != NULL)
{
builtin_size = strlen(functions[counter].cmp_string);
if (strncmp(functions[counter].cmp_string, line, builtin_size) == 0)
{
if (functions[counter].function(line + builtin_size) < 0)
printf("An error has occured\n");
}
counter = counter + 1;
}
}
int main(void)
{
t_function_pointer functions[] = {{"ls", &ls_function},
{"echo", &echo_function},
{NULL, NULL}};
// Of course i'm not gonna do the input treatment part, but just guess it was here, and you'd call treat_input with each line you receive.
treat_input(functions, "ls -laR");
treat_input(functions, "echo helloworld");
return 0;
}
Hope this helps !

Use of sqlite3_exec

I have the next SQLITE3 commands that generates a file with more than 60 million records:
.mode csv
.output matppp.csv
select mat, ppp from matppp order by mat;
.output stdout
How can I include these commands into a C program using:
sqlite3_exec(db, "..........", NULL, 0, &db_err);
?
When I attempt to do it myself, the c program generates an expression error when executing.
Thanks!!
If you want to do this within C (as opposed to piping something to sqlite3's command line program that has those nifty dot commands) then you will have to use a callback.
For your cutting and pasting convenience, here is the code, hacked out of the Apophenia library for statistical computing.
Part I:
sqlite3 *db=NULL; //The global database handle.
static int apop_db_open(char *filename){
if (!filename)
sqlite3_open(":memory:",&db);
else
sqlite3_open(filename,&db);
if (!db)
printf("Not sure why, but the database didn't open.\n");
return 0;
}
//From the SQLite manual:
#define ERRCHECK {if (err!=NULL) {printf("%s\n",err); sqlite3_free(err); return 0;}}
apop_data * apop_sqlite_query_to_screen(char *query){
char *err = NULL;
if (db==NULL)
apop_db_open(NULL);
sqlite3_exec(db, query, The_Callback, a_param, &err);
ERRCHECK
}
Part II:
The callback will have the following form, and runs once for each line returned. Notice how the parameter a_param transfers; if you don't need it (as in this example), just set it to NULL above.
int The_Callback(void *a_param, int argc, char **argv, char **column){
for (int i=0; i< argc; i++)
printf("%s,\t", argv[i]);
printf("\n");
return 0;
}
The companion web site of the book Using SQLite has some examples. In particular, chapter 7 has some examples of the C/C++ API.
Example code: http://examples.oreilly.com/9780596521196/
I think you really want to use a callback function and perhaps fprintf() to write your formatted output to a file. Fortunately, the prototype for the callback pointer contains an extra (optional) void * which could serve as a FILE * stream, making the callback more re-usable in the future.
AFAIK, sqlite3_exec() does not offer the same interface as the sqlite3 CLI. Its just for queries, not output modifiers.
Check out the example code at the bottom of the link I gave, its very easy to use a callback function.
I am doing some experiments with SQLite with a simple test harness using a single table that contains a char string key and a single integer value. The following are pieces of source from the experimental test harness that I am using. I pulled these pieces in order to show the creation of the table along with the function I use to create a record set from a select SQL statement using the call back functionality of SQLite. There are printf() statements and fprintf() statements in various places so that I can see the results of actions as this is a simple console type application for the test harness.
Notice that there are times when you do not need the call back argument so SQLite allows you to specify a NULL pointer indicating not to bother with the call back.
And as you read over the source just remember this is an experimental hack!
The function to create the table looks like:
int CreateSetupTable (sqlite3 *db)
{
char *zErrMsg = 0;
int rc;
char *aszSqlCreate = "create table tbl1(one varchar(10), two smallint)";
char *aszSqlCreateIndex01 = "create unique index index1 on tbl1 (one)";
do {
rc = sqlite3_exec(db, aszSqlCreate, 0, 0, &zErrMsg);
if( rc!=SQLITE_OK ){
fprintf(stderr, "SQL error: %s\n", zErrMsg);
sqlite3_free(zErrMsg);
break;
}
rc = sqlite3_exec(db, aszSqlCreateIndex01, 0, 0, &zErrMsg);
if( rc!=SQLITE_OK ){
fprintf(stderr, "SQL error: %s\n", zErrMsg);
sqlite3_free(zErrMsg);
break;
}
} while (0); // loop only once to allow breaks on errors
return rc;
}
I insert some records into this table and then have a function that I call to get one or more records from the table using a select SQL statement. The select function retrieves the records and uses a call back to convert each record returned into a C struct. The C struct look like:
typedef struct {
char cKey[20];
int iValue;
} Tbl1Record;
The call back used for the record set uses a struct that contains record select management data. By this what I mean is that the call back takes as its first argument a pointer to a struct that in turn points to the location to put the transformed data along with some information about the size of the memory area. Since a select might return more than one record depending on the where clause, the call back function uses the call back struct to know how many transformed records it can put into the memory area as well as an index so that as it is putting records, it can index through the memory area in order to return multiple transformed records.
The call back management struct looks like this:
typedef struct _RecordProcessor {
void *pRecordSet;
int nRecordSetMax;
int nRecordSetActual;
} RecordProcessor;
The select function looks like:
int SelectRecord (sqlite3 *db, char *cSelect, char *cKey)
{
char *zErrMsg = 0;
int rc;
char aszSqlSelect[128];
Tbl1Record myRec[20];
RecordProcessor myProcessor;
myProcessor.pRecordSet = myRec;
myProcessor.nRecordSetActual = 0;
myProcessor.nRecordSetMax = 20;
if (cKey) {
sprintf (aszSqlSelect, "select %s from tbl1 where one='%s'", cSelect, cKey);
} else {
sprintf (aszSqlSelect, "select %s from tbl1", cSelect);
}
rc = sqlite3_exec(db, aszSqlSelect, MyRecordProcessor, &myProcessor, &zErrMsg);
if( rc!=SQLITE_OK ){
fprintf(stderr, "SQL error SelectRecord: %s\n", zErrMsg);
sqlite3_free(zErrMsg);
} else {
int i;
for (i = 0; i < myProcessor.nRecordSetActual; i++) {
printf ("Rec #%d cKey = %s iValue = %d\n", i+1, myRec[i].cKey, myRec[i].iValue);
}
}
return rc;
}
The call back that processes each record returned by the select looks like this:
static int MyRecordProcessor (void *callBackArg, int argc, char **argv, char **azColName)
{
int iRetStatus = 0;
char *colNameTable[] = {
"one",
"two"
};
Tbl1Record *pTbl1Record = (Tbl1Record *)((RecordProcessor *)callBackArg)->pRecordSet;
if (((RecordProcessor *)callBackArg)->nRecordSetActual < ((RecordProcessor *)callBackArg)->nRecordSetMax) {
int i, j;
int iIndex = ((RecordProcessor *)callBackArg)->nRecordSetActual;
memset (pTbl1Record + iIndex, 0, sizeof(Tbl1Record));
((RecordProcessor *)callBackArg)->nRecordSetActual++;
for (i = 0; i < argc; i++){
int j;
for (j = 0; j < sizeof (colNameTable)/sizeof(colNameTable[0]); j++) {
if (strcmp (azColName[i], colNameTable[j]) == 0) {
switch (j) {
case 0:
strncpy (pTbl1Record[iIndex].cKey, (argv[i] ? argv[i] : "NULL"), 19);
break;
case 1:
pTbl1Record[iIndex].iValue = atoi (argv[i] ? argv[i] : "0");
break;
default:
break;
}
break;
}
}
}
}
return iRetStatus;
}

Resources