I am trying to suppress the logging of the tensorflow in C-API when it loads a saved model. The logging looks like this
2020-07-24 13:06:39.805191: I tensorflow/cc/saved_model/reader.cc:31] Reading SavedModel from: /home/philgun/tf-C-API/my_model
2020-07-24 13:06:39.806627: I tensorflow/cc/saved_model/reader.cc:54] Reading meta graph with tags { serve }
2020-07-24 13:06:39.819994: I tensorflow/cc/saved_model/loader.cc:202] Restoring SavedModel bundle.
2020-07-24 13:06:39.875249: I tensorflow/cc/saved_model/loader.cc:151] Running initialization op on SavedModel bundle at path: /home/philgun/tf-C-API/my_model
2020-07-24 13:06:39.884401: I tensorflow/cc/saved_model/loader.cc:311] SavedModel load for tags { serve }; Status: success. Took 79210 microseconds.
Below is the part of my code that loads the saved model
//*********************Read Model
TF_Graph* Graph = TF_NewGraph();
TF_Status* Status = TF_NewStatus();
TF_SessionOptions* SessionOpts = TF_NewSessionOptions();
TF_Buffer* RunOpts = NULL;
const char* tags = "serve"; // default model serving tag
int ntags = 1;
TF_Session* Session = TF_LoadSessionFromSavedModel(SessionOpts, RunOpts, saved_model_dir, &tags, ntags, Graph, NULL, Status);
Since there's so little documentation on TF C-API, I am now stuck in this problem. Does anybody know how to do it?
After some hustling I found a way to do it by setting a new environment variable called TF_CPP_MIN_LOG_LEVEL. Here's how I did it:
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include "tensorflow/c/c_api.h"
int main()
{
<your main code>
}
void CallSavedModel(double raw_input[], int inputsize, char* saved_model_dir)
{
char* new_environment = "TF_CPP_MIN_LOG_LEVEL=3";
int ret;
ret = putenv(var);
IMPORT YOUR SAVED MODEL START FROM HERE
}
I got the answer by combining https://pubs.opengroup.org/onlinepubs/009695399/functions/putenv.html and Disable Tensorflow debugging information
Cheers!
Hope this is helpful for those who faced the same headache like I had.
Phil
Related
I recently started working on a small driver for the raspberry pico and the bme280 sensor. I wanted to use the official bosh API written in C and therefore decided to write all the code in C using the micropython C api to write usermodules. I managed to get my code compiled into a UF2 file and my module shows up when I try to list the modules with help('modules'). When I import my module the class with the driver code shows up in dir(mymodule) but when I try to create an object the terminal connected to the PICO hangs and doesn't respond anymore.
typedef struct {
mp_obj_base_t base;
uint8_t sda;
uint8_t scl;
uint8_t i2c_address;
} BME280_obj_t;
const mp_obj_type_t BME280_class_type;
STATIC mp_obj_t BME280_make_new(const mp_obj_type_t *type, size_t n_args, size_t n_kw, const mp_obj_t *args) {
mp_arg_check_num(n_args, n_kw, 2, 2, true);
BME280_obj_t* self = m_new_obj(BME280_obj_t);
self->base.type = &BME280_class_type;
self->sda = mp_obj_get_int(args[0]);
self->scl = mp_obj_get_int(args[1]);
self->i2c_address = n_args <= 2? BME280_I2C_ADDR_SEC : mp_obj_get_int(args[2]);
return MP_OBJ_FROM_PTR(self);
}
STATIC const mp_rom_map_elem_t BME280_locals_dict_table[] = {
// for testing purpose i removed all methods from the class
};
STATIC MP_DEFINE_CONST_DICT(BME280_locals_dict, BME280_locals_dict_table);
const mp_obj_type_t BME280_type = {
{ &mp_type_type },
.name = MP_QSTR_BME280,
.print = BME280_print,
.make_new = BME280_make_new,
.locals_dict = (mp_obj_dict_t*) &BME280_locals_dict,
};
STATIC const mp_rom_map_elem_t bme280_module_globals_table[] = {
{ MP_ROM_QSTR(MP_QSTR___name__), MP_ROM_QSTR(MP_QSTR_bme280) },
{ MP_OBJ_NEW_QSTR(MP_QSTR_BME280), (mp_obj_t)&BME280_class_type }
};
STATIC MP_DEFINE_CONST_DICT(bme280_module_globals, bme280_module_globals_table);
// module registration
const mp_obj_module_t bme280_user_cmodule = {
.base = { &mp_type_module },
.globals = (mp_obj_dict_t*)&bme280_module_globals,
};
MP_REGISTER_MODULE(MP_QSTR_melopero_bme280, melopero_bme280_user_cmodule, 1);
I think the problem relies somewhere in the initialization procedure since it does not go further... Maybe there is something that micropython is doing behind the scenes that I'm ignoring. There is not so many documentation on writing usermodules in C... any help/hints/ideas are greatly apreciated :)
EDIT+ANSWER:
Yes I got the example to build and so I started to strip down my code to the minimum necessary to get it almost identical to the example and so I found the error...
The problem was that I used a different name for the class type in the declaration: BME280_class_type and in the definition: BME280_type
You have .print defined but it doesn't exist in your code.
const mp_obj_type_t BME280_type = {
{ &mp_type_type },
.name = MP_QSTR_BME280,
.print = BME280_print,
.make_new = BME280_make_new,
.locals_dict = (mp_obj_dict_t*) &BME280_locals_dict,
};
There is an example of writing a proper class print function here
However, this should be enough to at least satisfy the print requirement. Paste this into your code and see if it stops hanging.
STATIC void BME280_print(const mp_print_t *print, mp_obj_t self_in, mp_print_kind_t kind) {
(void)kind;
BME280_obj_t *self = MP_OBJ_TO_PTR(self_in);
mp_print_str(print, "BME280");
}
Also, I'm sure you just left this out of your example, otherwise you wouldn't have been able to build it, at all, but for the sake of being thorough ~ you have to include the proper header files.
#include <stdio.h>
#include "py/runtime.h"
#include "py/obj.h"
Edit: I was curious. You said you got everything to build, but it hangs. I commented out the print function for one of my C MODULE classes, and it would not build. This tells me that you (like the header) decided to just leave this part out of your example. This wastes time. How can you be helped if your example is creating errors that do not exist for you? However, even though this answer is now wrong, I am going to leave it up as an example of why answer-seekers shouldn't pick and choose what to post. Just post ALL of the relevant script to your problem and let us figure out the rest. I'd probably have an answer for you if I wasn't solving the wrong problem. Actually, I do see your problem, you're creating your module with mixed namespaces.
I am trying to write a small extension for PostgreSQL.
As a way to test if my module loads correctly I am writing some stuff in files in the void _PG_init(void) and void _PG_fini(void) functions. Here is the code of these two functions:
#include "postgres.h"
#include "executor\executor.h"
#include "fmgr.h"
#include "funcapi.h"
#include <stdio.h>
PG_MODULE_MAGIC;
extern void _PG_init(void);
extern void _PG_fini(void);
static void myExecutorStart(QueryDesc *queryDesc, int eflags);
static void myExecutorRun(QueryDesc *queryDesc, ScanDirection direction, uint64 count);
static void myExecutorFinish(QueryDesc *queryDesc);
static void myExecutorEnd(QueryDesc *queryDesc);
static ExecutorStart_hook_type prevExecutorStart = NULL;
static ExecutorRun_hook_type prevExecutorRun = NULL;
static ExecutorFinish_hook_type prevExecutorFinish = NULL;
static ExecutorEnd_hook_type prevExecutorEnd = NULL;
void _PG_init(void) {
FILE *file = NULL;
file = fopen("F:\\init.txt", "a+");
fprintf(file, "Init started!\n");
fclose(file);
prevExecutorStart = ExecutorStart_hook;
ExecutorStart_hook = myExecutorStart;
prevExecutorRun = ExecutorRun_hook;
ExecutorRun_hook = myExecutorRun;
prevExecutorFinish = ExecutorFinish_hook;
ExecutorFinish_hook = myExecutorFinish;
prevExecutorEnd = ExecutorEnd_hook;
ExecutorEnd_hook = myExecutorEnd;
}
void _PG_fini(void) {
FILE *file = NULL;
file = fopen("F:\\fini.txt", "a+");
fprintf(file, "Fini started!\n");
fclose(file);
ExecutorStart_hook = prevExecutorStart;
ExecutorRun_hook = prevExecutorRun;
ExecutorFinish_hook = prevExecutorFinish;
ExecutorEnd_hook = prevExecutorEnd;
}
Those functions are in a file called "myextension.c", compiled into "myextension.dll". I built it in Visual Studio 2015, with following settings:
Configuration Properties -> General, “Configuration Type” = “Dynamic
Library (.dll)”.
C/C++ -> Code Generation, “Enable C++ Exceptions” = “No”,“Advanced”
set “Compile As” = “Compile as C Code (/TC)”.
Linker -> Manifest File, “Generate Manifest” = “No”.
Linker -> Input -> Additional Dependencies, added “postgres.lib” to the
library list.
Configuration Properties -> C/C++ -> General, Additional Include
Directories, added: “include\server\port\win32_msvc”, “include\server\port\win32”, “include\server”, “include”
Solution configuration = Release
Solution Platform = x64 (Installed 64 bit version of PostgreSQL 9.6 on
Windows 10)
In myExecutorXXX functions I check if there are previous ExecutorXXX functions, call them if they exist, if they don't I call the standard_ExecutorXXX function. Here is an example of one of the functions:
static void myExecutorStart(QueryDesc *queryDesc, int eflags) {
if (prevExecutorStart) prevExecutorStart(queryDesc, eflags);
else standard_ExecutorStart(queryDesc, eflags);
FILE *file = NULL;
file = fopen("F:\\query.txt", "a+");
fprintf(file, "Query: %s started!\n", queryDesc->sourceText);
fclose(file);
}
I copied the "myextension.dll" in "../PostgreSQL/9.6/lib" directory, and added a "myextension.control" and "myextension--1.0.sql" to "../PostgreSQL/9.6/share/extension" directory.
myextension.control:
# pg_extension extension
comment = 'myextension!!!'
default_version = '1.0'
myextension--1.0.sql:
-- complain if script is sourced in psql, rather than via CREATE EXTENSION
\echo Use "CREATE EXTENSION myextension" to load this file. \quit
In the "postgresql.conf" I added shared_preload_libraries = 'myextension'. After that I connected to a test DB and ran: CREATE EXTENSION myextension;, and restarted the server.
If anyone has any idea what might be causing this, please help.
A couple of comments to get you on the right track:
_PG_fini() will never get called, because modules don't get unloaded.
_PG_init(), however, does get called when the module is loaded. Your main question seems to be why nothing is written to F:\init.txt and other files you use for logging.
On Windows, PostgreSQL normally runs as service. I suspect that the operating system user doesn't have the privileges to write to these files. I know little about Windows and its permission management, but I notice that you do not check the return code of fopen(), so it might well have silently failed.
My recommendation is to use the logging infrastructure instead, e.g. with
elog(LOG, "Init started!");
That will write the message to the PostgreSQL server log and is much more comfortable and less error prone.
Two more comments:
There is no point in creating an extension, because your code does not provide any SQL functions. CREATE EXTENSION myextension is a no-operation.
Don't forget to restart the PostgreSQL server after changing shared_preload_libraries.
Hello.
First I'm sorry for my ita-english.
I want use gwan with aerospike but when run the servlet...problem.
I start with this example.c of aerospike. In file example.c I put gwan.h and this is the output ./gwan:
loading
hello.cs: to use .cs scripts, install C#..
hello.lua: to use .lua scripts, install Lua
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Linking example.c: undefined symbol: g_namespace
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
To run G-WAN, you must fix the error(s) or remove this Servlet.
Inside example.c:
#include "gwan.h"
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#include <stdlib.h>
#include <aerospike/aerospike.h>
#include <aerospike/aerospike_key.h>
#include <aerospike/aerospike_query.h>
#include <aerospike/as_error.h>
#include <aerospike/as_key.h>
#include <aerospike/as_query.h>
#include <aerospike/as_record.h>
#include <aerospike/as_status.h>
#include <aerospike/as_val.h>
#include "example_utils.h"
const char TEST_INDEX_NAME[] = "test-bin-index";
bool query_cb(const as_val* p_val, void* udata);
void cleanup(aerospike* p_as);
bool insert_records(aerospike* p_as);
int
main(int argc, char* argv[])
{
if (! example_get_opts(argc, argv, EXAMPLE_MULTI_KEY_OPTS)) {
exit(-1);
}
aerospike as;
example_connect_to_aerospike(&as);
example_remove_test_records(&as);
example_remove_index(&as, TEST_INDEX_NAME);
if (! example_create_integer_index(&as, "test-bin", TEST_INDEX_NAME))
{
cleanup(&as);
exit(-1);
}
if (! insert_records(&as)) {
cleanup(&as);
exit(-1);
}
if (! example_read_test_records(&as)) {
cleanup(&as);
exit(-1);
}
as_error err;
as_query query;
as_query_init(&query, g_namespace, g_set);
as_query_where_inita(&query, 1);
as_query_where(&query, "test-bin", as_integer_equals(7));
LOG("executing query: where test-bin = 7");
if (aerospike_query_foreach(&as, &err, NULL, &query, query_cb, NULL)
!= AEROSPIKE_OK) {
LOG("aerospike_query_foreach() returned %d - %s", err.code,
err.message);
as_query_destroy(&query);
cleanup(&as);
exit(-1);
}
LOG("query executed");
as_query_destroy(&query);
cleanup(&as);
LOG("simple query example successfully completed");
return 0;
}
bool
query_cb(const as_val* p_val, void* udata)
{
if (! p_val) {
LOG("query callback returned null - query is complete");
return true;
}
as_record* p_rec = as_record_fromval(p_val);
if (! p_rec) {
LOG("query callback returned non-as_record object");
return true;
}
LOG("query callback returned record:");
example_dump_record(p_rec);
return true;
}
void
cleanup(aerospike* p_as)
{
example_remove_test_records(p_as);
example_remove_index(p_as, TEST_INDEX_NAME);
example_cleanup(p_as);
}
bool
insert_records(aerospike* p_as)
{
set
as_record rec;
as_record_inita(&rec, 1);
for (uint32_t i = 0; i < g_n_keys; i++) {
as_error err;
as_key key;
as_key_init_int64(&key, g_namespace, g_set, (int64_t)i);
as_record_set_int64(&rec, "test-bin", (int64_t)i);
if (aerospike_key_put(p_as, &err, NULL, &key, &rec) != AEROSPIKE_OK) {
LOG("aerospike_key_put() returned %d - %s", err.code, err.message);
return false;
}
}
LOG("insert succeeded");
return true;
}
how can connect aerospike with gwan?
Thank you
You need to #pragma link your aerospike library, and make sure all your required header files are in the right place. See G-WAN FAQ or read example code in the G-WAN tarball.
Also, in G-WAN the return code of the main function will be used as HTTP response code, so avoid return -1;.
undefined symbol: g_namespace
the error message is clear. As long as this variable is undefined your C servlet won't compile.
I don't know your library but this variable is probably defined in a library include file - or must be defined by the end user (you). Check the library documentation.
Detailed steps to run Aerospike C-client example with G-WAN,
Download and extract G-WAN server tar on your system
You can start the G-WAN server using ./gwan script present in extracted folder, e.g. ./gwan_linux64-bit/
Get Aerospike C-client from https://github.com/aerospike/aerospike-client-c, and install on your system
Copy example.c to ./gwan_linux64-bit/0.0.0.0_8080/#0.0.0.0/csp/
Make following changes to example.c,
Add following #pragma directive,
#pragma include "/home/user/aerospike-client-c/examples/utils/src/include/"
This will help search example_utils.h, which is necessary for all the example scripts in C-client.
Add following #pragma directive,
#pragma link "/home/user/aerospike-client-c/examples/utils/src/main/example_utils.c"
We shall have to link example_utils.c, as it has definitions of all util functions used in example scripts.
Make changes to the return values. Retun proper HTTP error codes.
Now, you are good to go. Run ./gwan server and access your webservice through browser, http://127.0.0.1:8080/?example.c
I have some C code trying to use libharu. Although I can use every function of this library (even UTF8) I can hardly draw images. Here is some very basic code:
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <math.h>
#include <setjmp.h>
#include "hpdf.h"
jmp_buf env;
#ifdef HPDF_DLL
void __stdcall
#else
void
#endif
error_handler (HPDF_STATUS error_no,
HPDF_STATUS detail_no,
void *user_data)
{
printf ("ERROR: error_no=%04X, detail_no=%u\n", (HPDF_UINT)error_no,
(HPDF_UINT)detail_no);
longjmp(env, 1);
}
int main (int argc, char **argv)
{
HPDF_Doc pdf;
HPDF_Font font;
HPDF_Page page;
char fname[256];
HPDF_Image image;
strcpy (fname, argv[0]);
strcat (fname, ".pdf");
pdf = HPDF_New (error_handler, NULL);
if (!pdf) {
printf ("error: cannot create PdfDoc object\n");
return 1;
}
/* error-handler */
if (setjmp(env)) {
HPDF_Free (pdf);
return 1;
}
font = HPDF_GetFont (pdf, "Helvetica", NULL);
page = HPDF_AddPage (pdf);
HPDF_Page_SetWidth (page, 550);
HPDF_Page_SetHeight (page, 500);
image = HPDF_LoadPngImageFromFile (pdf, "img.png");
HPDF_SaveToFile (pdf, fname);
HPDF_Free (pdf);
return 0;
}
When I compile this I have ERROR: error_no=1015, detail_no=0. I have found a similar post in stackoverflow: this. However although original poster said the problem is solved it hardly helped mine. I moved img.png to a folder and recompiled the file. Changed the code that says /home/name/path/to/img.png which is the direct path to image. Nothing works. I "always" have the same error, but when I change the name of file I have ERROR: error_no=1017, detail_no=2 which basicly means program cannot find image (according to reference of libharu) So I deduce that program finds img.png; but, it's strange but, cannot allocate the necessary memory. Which is weird because I cannot see any reason for this program not to allocate memory. I have every kind of permission.
I am using GCC 4.7.2 under Ubuntu Quantal Quetzal and libharu 2.3.0 RC2. Thank you for your help.
Hello Equalities of polynomials .
I also encountered the same problem when i integrated the haru sdk in my macOS environment.
The error_handler returned ERROR: error_no=1017, detail_no=2,and then i checked the official document for haru at http://libharu.sourceforge.net/error_handling.html query 0x1017 indicates that the file failed to open, so i suspect that the second parameter of the HPDF_LoadPngImageFromFile method needs to pass an exact png image file path, so after I modified it, the problem was solved, and I hope to help you.
code ad follow:
char filename1[255];
strcpy(filename1, "/Users/xx/Downloads/lusaceg.com.png");
image = HPDF_LoadPngImageFromFile (pdf, filename1);
Faithfully yours.
I am porting an existing windows based C++ application to 64 bit environment, and this is one of those weird errors.
In the code snippet you can that I am using openforwardonly and it used to work fine with our old setup but in the 64 bit environment it gives the problem of fetching only ONE recordset. Or it could be a problem with the MoveNext(); of ADO.
To circumvent it we started using adOpenStatic, and it worked fine for me for a while but only later realized that it has a performance hit and it is taking forever to get/iterative through values.
I request someone to try this code with both the flags and validate the behavior I am seeing.
Information about ado flags:
http://www.w3schools.com/ADO/met_rs_open.asp
Another EE topic
http://www.experts-exchange.com/Programming/Languages/Visual_Basic/VB_DB/Q_11520558.html
I remember seeing a MS support case, but I can't get to it now.
I would appreciate any help or suggestions. I know we are using old technology, but we want to move to the additional capabilities without changing code much.
// Dbtest.cpp : Defines the entry point for the console application.
//
#include "stdafx.h"
#include <windows.h>
#include <stdio.h>
#include <time.h>
#import "C:\Program Files\Common Files\System\ADO\msado15.dll" \
no_namespace rename("EOF", "EndOfFile")
int main(int argc, char* argv[])
{
HRESULT hr = S_OK;
try
{
CoInitialize(NULL);
_bstr_t strCnn("Provider=OraOLEDB.Oracle;Data Source =****; User Id=****; password=***");
_ConnectionPtr m_pConn;
hr=m_pConn.CreateInstance(__uuidof(Connection));
if(FAILED(hr))
{
printf("Failed creating record set instance\n");
return 0;
}
m_pConn->Open(strCnn,"","",NULL);
//Open the Record set for getting records from Author table
_RecordsetPtr pRstDoctors = NULL;
time_t start,end1,end2;
pRstDoctors.CreateInstance(__uuidof(Recordset));
time(&start);
pRstDoctors->Open("select logid from log",strCnn, adOpenForwardOnly,
**adLockReadOnly**,adCmdText);
//Declare a variable of type _bstr_t
int valField1;
//int valField2;
pRstDoctors->MoveFirst();
//Loop through the Record set
if (!pRstDoctors->EndOfFile)
{
while(!pRstDoctors->EndOfFile)
{
valField1 = pRstDoctors->Fields->GetItem("logid")->Value.intVal;
// valField2 = pRstDoctors->Fields->GetItem("reportid")->Value.intVal;
// printf("%d - \n",valField1);
pRstDoctors->MoveNext();
}
}
time(&end1);
double dif=difftime(end1,start);
printf("time difference is %.2lf",dif);
}
catch(_com_error e)
{
printf("Error:%s\n",e);
}
CoUninitialize();
return 0;
}
Using "adOpenStatic" instead of "adOpenForwardOnly" will work
pRstDoctors->Open("select logid from log",strCnn, adOpenStatic,
**adLockReadOnly**,adCmdText);