Casting a Host Structure array into a table - c

I am trying to tune a piece of large code written in Pro*C, specifically a bottleneck loop and an UPDATE statement inside it. The for loop loops through a "Host Structure array" which may contain several thousand at times millions of records and the update executes too many times. The update could be executed more frequently throughout the program but that would need drastic change to the code and I am not in a liberty to make major changes.
So I have something like this
....
#define NULL_REF_NO 10
#define NULL_ERR 256
....
....
struct s_errors
{
char s_ref_id [NULL_REF_NO];
char s_ref_seq_no [NULL_REF_NO];
char s_err_msg [NULL_ERR];
};
....
....
struct s_errors *ps_errors = NULL;
....
....
/*The part below happens throughout the program to collect all errors*/
/*ls_ref_id, ls_ref_seq_no, and ls_err_msg are local variables of same data type. and i_curr_index is the array index variable*/
strcpy(ls_ref_id, ps_errors[i_curr_index].s_ref_id);
strcpy(ls_ref_seq_no, ps_errors[i_curr_index].s_ref_seq_no);
strcpy(ls_err_msg, ps_errors[i_curr_index].s_err_msg);
.....
/* At this point ps_error contains thousands or even millions of rows*/
/* The final part is to update all these rows back to the table like below*/
/* pl_err_count is a Global var which keeps track of the total number of records in the host structure array*/
int i_curr_index = 0;
char l_ref_id [NULL_REF_NO];
char l_ref_seq_no [NULL_REF_NO];
char l_err_msg [NULL_ERR];
for(i_curr_index = 0; i_curr_index < pl_err_count; i_curr_index++)
{
strcpy(l_ref_id, ps_errors[i_curr_index].s_ref_id);
strcpy(l_ref_seq_no, ps_errors[i_curr_index].s_ref_seq_no);
strcpy(l_err_msg, ps_errors[i_curr_index].s_err_msg);
EXEC SQL
UPDATE some_table
SET status = 'E',
error_message = :l_err_msg
WHERE ref_id = :l_ref_id
AND ref_seq_no = :l_ref_seq_no;
if (SQL_ERROR_FOUND)
{
sprintf(err_data, "Updation failed with sql errors ");
strcpy(table, "some_table");
WRITE_ERROR(SQLCODE,function,"",err_data);
return(FATAL);
}
}
The bottleneck is the for loop above (and it's the last step in the program) which loops too many times causing the program to run long. I was wondering if there is a way to CAST the Host Structure array ps_errors to an Oracle table type so that I can easily do bulk UPDATE or even do a MERGE with some parallel DML without having to loop through each and every record.

Extrapolating from some other code I have seen at work which does something sort of similar, you could do something like this instead of the for loop:
EXEC SQL for :pl_err_count
UPDATE some_table
SET status = 'E',
error_message = :ps_errors.s_err_msg
WHERE ref_id = :ps_errors.s_ref_id
AND ref_seq_no = :ps_errors.s_ref_seq_no;
This might rely on the content added to ps_errors being null terminated. Given the use of strcpy() in the existing code rather that strncpy() (or similar), I'm guessing they already are.
If s_err_msg could be null, you should also consider using indicator variables. eg.
error_message = :ps_errors.s_err_msg INDICATOR :indicator_variable

You could use an array update http://docs.oracle.com/cd/B28359_01/appdev.111/b28427/pc_08arr.htm#i1879 but you'll need to change your array of structs ps_errors to be a struct of arrays
E.g:
EXEC SQL
UPDATE some_table
SET status = 'E',
error_message = :ps_errors.s_err_msg
WHERE ref_id = :ps_errors.s_ref_id
AND ref_seq_no = :ps_errors.s_ref_seq_no;

Related

How do I correctly create a copy of an array in C or set a reference to one?

so, I'm very new to C, coming from a Java/C# background and I can't quite wrap my head around it so far.
What I'm trying to do is program a microcontroller (Adafruit Feather running an atmega32u4 in this case) to pose as a USB-coupled controller for Nintendo Switch and run automated commands.
The project I'm trying to expand upon is using a struct array of commands like this:
typedef struct {
Buttons_t button;
uint16_t duration; // 1 equals 0.025s on ATmega32u4 => 1s = 40
} command;
static const command loop[] = {
// do stuff
{ NOTHING, 150 },
{ TRIGGERS, 15 }, { NOTHING, 150 },
{ TRIGGERS, 15 }, { NOTHING, 150 },
{ A, 5 }, { NOTHING, 250 }
};
Now initially this was all there was to it, the program would loop through the commands, send a button to the console and "hold" it for the defined period of time. When the program ran to the end of the array, it would simply reset the index and start anew.
Now I'm trying to send different commands to the console, based on a few easy if..else queries. Specifically, the program will start with a day, month and year variable (the date the Switch console is currently set to) and roll days forward individually to get to a set date in the future. To this end, I want to check at every 'step' if the date +1 day is valid as described in this tutorial and based on the result either roll one day, one day and one month or one day, one month and one year forward. Then I want it to end after a set amount of days.
I wrote several arrays of commands to represent the different steps needed for setting up the controller, moving to where it's supposed to loop, rolling a day, a month or a year like this:
static const command setupController[] = {
// Setup controller
...
};
static const command moveToLoop[] = {
// Go into date settings
...
};
static const command rollDay[] = {
//roll to next day
...
};
static const command rollMonth[] = {
//roll to next month
...
};
static const command rollYear[] = {
//roll to next year
...
};
And another array I want to copy those to like this:
#define COMMANDMAXSIZE 100
static command activeCommand[COMMANDMAXSIZE];
I know this is (extremely) wasteful of memory, but I'm definitely not good enough at C to come up with fancier, more conservative solutions yet.
Then I go into my program, which looks like this:
int main(void) {
SetupHardware(); //Irrelevant, because it is exactly like I downloaded it and it works even with the bumbling changes I've made
GlobalInterruptEnable(); //Ditto
RunOnce(setupController);
RunOnce(moveToLoop);
while (daysSkipped != stopDay)
{
if (datevalid((dayOfMonth + 1), month, year)) {
dayOfMonth++;
RunOnce(rollDay);
}
else if (datevalid(1, (month + 1), year)) {
dayOfMonth = 1;
month++;
RunOnce(rollMonth);
}
else if (datevalid(1, 1, (year + 1))) {
dayOfMonth = 1;
month = 1;
year++;
RunOnce(rollYear);
}
daysSkipped++;
}
}
and finally (I swear I'll be done soon), the start of RunOnce looks like this
void RunOnce(command stepsToRun[]) {
memcpy(activeCommand, stepsToRun, sizeof(activeCommand)); //set the setup commands to be active
activeBounds = sizeof(stepsToRun) / sizeof(stepsToRun[0]);
...
Later in the program, the task that translates commands into button presses for the console actually runs one fixed array, so I figured I'd just "mark" the commands to run as active, and only ever run the active array. Only, it doesn't work as expected:
The program runs, sets up the controller, moves to the date settings and indeed starts to roll a date, but then, regardless if the next day is valid or not, it rolls forward a month, then a year and then it gets stuck moving the simulated analog stick upwards and pressing A indefinitely.
I figure the problem lies in my memcpy to overwrite the active array with the steps I want to run next, but I can't think of a way to solve it. I tried writing a function that was supposed to overwrite the active array element by element using a for loop, but this way the controller wouldn't even set itself up correctly and effectively nothing happened. Usually with any kind of output capabilities I'd try to fit in prints at points of interest, but I have virtually no way of getting feedback on my microcontroller.
Any help would be greatly appreciated.
Ignoring that doing a hard copy of the data is incredibly slow and wasteful, it is also incorrect indeed.
memcpy(activeCommand, stepsToRun, sizeof(activeCommand));
Here you need to copy the size of the data you pass on, not the size of the target buffer! Right now you end up copying more data than you have, because all of these declarations static const command rollDay[] etc get a variable size depending on the number of items in the initializer list.
The quick & dirty fix to your immediate problem would be to pass along the size:
void RunOnce(size_t size, command stepsToRun[size])
{
memcpy(activeCommand, stepsToRun, size);
and then call this function with RunOnce(sizeof rollDay, rollDay); etc.
The activeBounds = sizeof(stepsToRun) / sizeof(stepsToRun[0]); part is also incorrect but not the immediate reason for the bug. See How to find the 'sizeof' (a pointer pointing to an array)? and What is array to pointer decay? etc.
When you pass array to function it decays to a pointer.
RunOnce(rollYear);
Thus
void RunOnce(command stepsToRun[]) {
memcpy(activeCommand, stepsToRun, sizeof(activeCommand)); //set the setup commands to be active
activeBounds = sizeof(stepsToRun) / sizeof(stepsToRun[0]);
}
sizeof(stepsToRun) doesn't yield the correct result as you expected, since it is now sizeof(pointer) in function.
You will have to pass the size of the array as an extra argument to RunOnce function.

MySQL connector(libmysql/C) is very slow in get RES

"select * from tables" query in MySQL connector/libmysql C is very slow in getting the results:
Here is my code in C :
int getfrommysql() {
time_t starttime, endtime;
time(&starttime);
double st;
st = GetTickCount();
MYSQL *sqlconn = NULL;
MYSQL_RES * res = NULL;
MYSQL_ROW row = NULL;
MYSQL_FIELD * field;
/*char ipaddr[16];
memset(ipaddr,0,sizeof(ipaddr));*/
char * sqlquery = "select * from seat_getvalue";
sqlconn = malloc(sizeof(MYSQL));
sqlconn = mysql_init(sqlconn);
mysql_real_connect(sqlconn, "111.111.111.111", "root", "password", "database", 0, NULL, 0);
char query[100];
memset(query, 0, 100);
strcpy(query, "select * from seat_getvalue");
mysql_query(sqlconn, query);
res = mysql_store_result(sqlconn);
int col_num, row_num;
if (res) {
col_num = res->field_count;
row_num = res->row_count;
printf("\nthere is a %d row,%d field table", res->row_count, res->field_count);
}
for (int i = 0; i < row_num; i++) {
row = mysql_fetch_row(res);
for (int j = 0; j < col_num; j++) {
printf("%s\t", row[j]);
}
printf("\n");
}
mysql_close(sqlconn);
time(&endtime);
double et = GetTickCount();
printf("the process cost time(get by GetTickCount):%f",et-st);
printf("\nthere is a %d row,%d field table", res->row_count, res->field_count);
}
Apart from the fact, that there isn't even a question given in your post, you are comparing apples to oranges. Mysql gives you (I think - correct me if I am wrong) the time needed to execute the query, while in your C code you measure the time that passed between the start and the end of the program. This is wrong for at least two reasons:
Difference between two GetTickCount() calls gives you the time that has passed between the calls in the whole system, not time spent executing your software. These are two different things, because your process does not have to executed from the beginning to the end uninterrupted - it can (and probably will be) swapped for another process in the middle of the execution, it can be interrupted etc. The whole time the system spent doing stuff outside your program will be added to your measurements. To get time spent on the execution of your code you could probably use GetProcessTimes or QueryProcessCycleTime.
Even if you did use an appropriate method of retrieving your time, you are timing the wrong part of the code. Instead of measuring the time spent just on executing the query and retrieving the results, you measure the whole execution time: estabilishing connection, copying the query, executing it, storing the results, fetching them, printing them and closing the connection. That's quite different from what mysql measures. And printing hundreds of lines can take quite a lot of time, depending on your shell - more than the actual SQL query execution. If you want to know how much time the connector needs to retrieve the data, you should benchmark only the code responsible for executing the query and data retrieval. Or, even better, use some dedicated performance monitoring tools or libraries. I can't point a specific solution, because I never performed tests like that, but there certainly must be some.

Unique String generator

I want to make a program (network server-client).
One of the specification for this program is next:
The server will receive the sent packages and save it into a file, with a unique name (generated by the server at the moment the transfer starts.
Ex __tf_"unique_random_string".txt
I made a function that returns a pointer to a "unique" string created.
The problem is: If i stop the server and then start it again it will generate the same names.
Ex:this file names were generated and then i stopped the server.
__ft_apqfwk.txt
__ft_arzowk.txt
__ft_cdyggx.txt
I start it again and i try to generate 3 file names. Them will be the same.
Sorry for my english. I'm still learning it.
My function to generate this "unique string" is:
char *create_random_name(void)
{
const char charset[] = "abcdefghijklmnopqrstuvwxyz";
char *file_name;
int i=0;
int key;
if((file_name = malloc(16 * sizeof ( char )) ) == NULL)
{
printf("Failed to alloc memory space\n");
return NULL;
}
strcpy(file_name,"__ft_");
for(i=5 ; i<11 ; i++)
{
key = rand() % (int)(sizeof(charset)-1);
file_name[i]=charset[key];
}
strcat(file_name,".txt");
file_name[15] = '\0';
return file_name;
}
One option is saving to a file the names that have been used, and using them as a checklist. You also want to seed rand with something like srand(time(NULL)).
another is ignoring the randomisation, and just going in order, e.g. aaa, aab aac...aba ,abb etc. Again, save where your cycle is up to on a file.
Your question seems a little bit unclear but if you want to generate a unique string there are a couple of things you can consider:
Get System timestamp ( yyyy-MM-dd-HH-mm-ss-fff-tt)
Use Random function to generate a random number
Combine this with your function and I am sure you will get a unique string.
Hope it helps !
If it's available, you could avoid manually generating random names that might collide and let the system do it for you (and handle collision resolution by creating a new name) by using mkstemps. This is also safer because it opens the file for you, removing the risk of a random name being generated, verified to be unique, then trying to open it and discovering another thread/process raced in and created it.
char name[] = "/path/to/put/files/in/__ft_XXXXXX.txt";
int fd = mkstemps(name, strlen(".txt"));
if (fd == -1) { ... handle error ... }
After mkstemps succeeds, name will hold the path to the file (it's mutated in place, replacing the XXXXXX string), and fd will be an open file descriptor to that newly created file; if you need a FILE*, use fdopen to convert to a stdio type.
Before calling rand(),--- once and only once---, call srand(time()) to initialize the random number generator.
Before settling on any specific file name, call stat() to assure that file name does not already exist.

Exception on prepared statement reuse in wxSQLite3

I'm writing an application that uses the wxSQLite3 library, which is a wrapper around libsqlite3 for the wxWidgets cross-platform GUI programming framework. When attempting to reuse a prepared statement, a wxSQLite3Exception is thrown.
This example illustrates the problem:
#include <wx/string.h>
#include <wx/wxsqlite3.h>
int main() {
wxSQLite3Database::InitializeSQLite();
//create in-memory test database & populate it
wxSQLite3Database db;
db.Open(wxT(":memory:"));
db.ExecuteUpdate(wxT("CREATE TABLE SimpleTable (id INT PRIMARY KEY, val INT);"));
db.ExecuteUpdate(wxT("INSERT INTO SimpleTable VALUES (1, 10);"));
db.ExecuteUpdate(wxT("INSERT INTO SimpleTable VALUES (2, 20);"));
//create a prepared statement we can reuse
wxSQLite3Statement stmt;
stmt = db.PrepareStatement(wxT("SELECT * FROM SimpleTable WHERE id = ?;"));
//first use of statement (works)
stmt.Bind(1, 1);
wxSQLite3ResultSet r_set = stmt.ExecuteQuery();
if (r_set.NextRow()) {
wxPrintf(wxT("id: %i value: %i\n"), r_set.GetInt(wxT("id")), r_set.GetInt(wxT("val")));
}
r_set.Finalize();
//reset and reuse statement
stmt.Reset();
stmt.Bind(1, 2); //**EXCEPTION THROWN HERE**
wxSQLite3ResultSet r_set2 = stmt.ExecuteQuery();
if (r_set2.NextRow()) {
wxPrintf(wxT("id: %i value: %i\n"), r_set2.GetInt(wxT("id")), r_set2.GetInt(wxT("val")));
}
r_set2.Finalize();
//cleanup
stmt.Finalize();
db.Close();
wxSQLite3Database::ShutdownSQLite();
return 0;
}
The exception handling was removed for brevity, but the message from the exception is:
WXSQLITE_ERROR[1000]: Statement not accessible
I wrote roughly equivalent code in plain C using libsqlite3 and it ran without problem. Does anyone know what I'm doing wrong, or if this is a bug of some sort in wxSQLite3? Thank you in advance for your help!
In SQLite itself, a statement and a result set actually are the same object.
wxSQLite3 uses reference counting so that the statement is freed only when the last wxSQLite3Statement or wxSQLite3ResultSet object is freed.
This happens automatically in the respective destructors.
However, calling Finalize() explicitly bypasses the reference counting.
While not necessary, if you want to ensure that wxSQLite3ResultSet resources are freed correctly before the next statement execution, just destruct this object:
wxSQLite3Statement stmt = ...;
...
{
wxSQLite3ResultSet r_set = stmt.ExecuteQuery();
... r_set.NextRow() ...
// r_set destructed here
}
...
As long as you intend to reuse a prepared SQL statement, that is, to reset the statement and to bind new values to statement variables, you must not call method Finalize - neither on the prepared statement object itself nor on a result set retrieved from that statement.
As the method name, Finalize, suggests, the method finalizes the underlying SQLite statement object by calling sqlite3_finalize (quotation from the SQLite docs: "The sqlite3_finalize() function is called to delete a prepared statement.") After the underlying SQLite statement object has been deleted, it obviously can't be accessed anymore. Therefore you get the exception.
Usually you don't need to call method Finalize explicitly. wxSQLite3 takes care of finalizing statements through reference counting.

Behaviour of an hashtable using glib

I want to update the Volume to each #IP. So that for example after each 5 s I add V(i) of each #IP(i). Ok Now the hash table works fine it keeps updated after every T seconds. But the problem is that after a certain period I find that sometimes the same ip adress is repeated twice or even a lot of times within the hash table. So that when I close the process I find the same #IP repeated too many times. It is like there is a problem with the hash table or something like that.
Here is the code this funcion "update_hashTable()" is so important it is called every X seconds I suspect in fact a memory leak ... because I always call malloc for IP#.
but it keeps working ... any idea ???
int update_hashTable( ... ) {
u_int32_t *a;
... //declarations
struct pf_addr *as;
as = ks->addr[0];
a = (u_int32_t*)malloc(sizeof(u_int32_t));
*a = ntohl(as->addr32[0]);
sz = value; // no matter it is... an int for example
if (ReturnValue=(u_int32_t)g_hash_table_lookup(hashtable, a)) {
ReturnValue +=sz;
g_hash_table_insert(hashtable, (gpointer)a, gpointer)ReturnValue);
}
else {
g_hash_table_insert(hashtable, (gpointer)a, (gpointer)sz);
}
Indeed, you appear to have a memory leak, but this isn't your problem. The problem is that the true-path of your if statement simply reinserts a second value associated with the same key, which is not what you want.
The typical pattern for this check-if-exists and increment algorithm is usually something like
gpointer val = g_hash_table_lookup(hash_table, key);
if (val == NULL) {
val = g_malloc0(...);
g_hash_table_insert(hash_table, key, val);
}
*val = /* something */;
The important thing to take away from this is that once you have a pointer to the value associated with some key, you can simply modify it directly.
If this code will be executed by multiple threads in parallel, then the entire block should be protected by a mutex, perhaps with GMutex: http://developer.gnome.org/glib/2.28/glib-Threads.html
gcc provides atomic builtin intrinsics, say for atomically incrementing the value, see http://gcc.gnu.org/onlinedocs/gcc/Atomic-Builtins.html

Resources