I'm trying to learn sqlite c api and written some code to test multiple database connections to same db file. this files compiles with no error but when i run it i get:
./sq: symbol lookup error: ./sq: undefined symbol: pthread_mutex_lock, version GLIBC_2.2.5
here is the code:
#include "../sqlite3.h"
int main()
{
sqlite3 **my_db;
sqlite3 **my_db2;
sqlite3_stmt **ppStmt;
sqlite3_stmt **ppStmt2;
int res;
double res2;
int rc;
char pzTail[100];
sqlite3_open("../test.db", my_db);
sqlite3_open("../test.db", my_db2);
sqlite3_prepare_v2(*my_db, "select * from tbl1 where one='goog';", 50, ppStmt, &pzTail);
rc = sqlite3_step(*ppStmt);
if (rc != SQLITE_OK) {
fprintf(stderr, "Failed to fetch data: %s\n", sqlite3_errmsg(my_db));
sqlite3_close(my_db);
return 1;
}
res = sqlite3_column_int(*ppStmt, 1);
printf("we have %d\n",res);
sqlite3_step(*ppStmt);
res = sqlite3_column_int(*ppStmt, 1);
printf("we have %f\n",res);
sqlite3_step(*ppStmt);
res = sqlite3_column_int(*ppStmt, 1);
printf("we have %d\n",res);
sqlite3_finalize(*ppStmt);
sqlite3_close(*my_db);
return 0;
}
When i run code without second sqlite3_open statement everything is right but when i add it,the error mentioned above appears at runtime and gdb shows this:
(gdb) run
Starting program: .../sqlite-amalgamation3130000/playground/sq
dl-debug.c:74: No such file or directory.
dl-debug.c:74: No such file or directory.
dl-debug.c:74: No such file or directory.
dl-debug.c:74: No such file or directory.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
dl-debug.c:74: No such file or directory.
dl-debug.c:74: No such file or directory.
(process 4636) exited with code 0177]
I compiled this code with a copy of sqlite_amalgamation source instead of systems installed sqlite library for some reason with this command:
gcc -g -I.. sqlite_c_api.c ../sqlite3.c -ldl -pthread -o sq
and i tried 'pthread_mutex_lock()' on some machine in some multi-threaded application before.
Where is/are my mistake(s)?
I solved it. The error was because i used ** pointers when i replaced them with * pointers and changed the code accordingly everything seems to be right.
Related
TL;DR
One has to compile their custom library as shared library:
gcc -c -fPIC warp_client.c -o warp_client.o
gcc -shared warp_client.o libwarp-client.so
Include the shared library and additional dependencies of that shared library in the Postgresql Makefile with the flags SHLIB_LINK and PG_LDFLAGS(Here the bachelor_fdw.c is the extension to compile):
EXTENSION = bachelor_fdw
MODULE_big = bachelor_fdw
DATA = bachelor_fdw--0.1.sql
OBJS = bachelor_fdw.o
PG_LIBS = -lpq
SHLIB_LINK = -lwarp_client -lucp
PG_LDFLAGS += -L/usr/lib/warpdrive/ -L/usr/lib/ucx/
PG_CONFIG = pg_config
PGXS := $(shell $(PG_CONFIG) --pgxs)
include $(PGXS)
Include the directories of the shared libraries into the environment variable LD_LIBRARY_PATH of Postgresql. For that, one has to add a line to the file 'environment' in the main Postgresql directory and restart Postgresql. Here is mine:
$ cat /etc/postgresql/12/main/environment
# environment variables for postgres processes
# This file has the same syntax as postgresql.conf:
# VARIABLE = simple_value
# VARIABLE2 = 'any value!'
# I. e. you need to enclose any value which does not only consist of letters,
# numbers, and '-', '_', '.' in single quotes. Shell commands are not
# evaluated.
LD_LIBRARY_PATH='/usr/include/:/usr/include/ucx/:/usr/lib/:/usr/lib/ucx/'
I am trying to create a foreign data wrapper, which uses a custom library from me. The fdw compiles and installs fine, but when using it, symbols to my library are undefined. What is the proper way of using custom c code as library in a postgresql extension and what am i doing wrong? Here are the steps i took:
Compile my library (warp_client.c) with flag -fPIC into an object file.
gcc -c -fPIC warp_client.c -o static/warp_client.o
Create static library from the object file.
ar -rcs out/libwarp_client.a static/warp_client.o
Copy libwarp_client.a and warp_client.h into the postgresql extension project root.
Compile postgresql extension with the following makefile.
EXTENSION = bachelor_fdw
MODULE_big = bachelor_fdw
DATA = bachelor_fdw--0.1.sql libwarp_client.a
OBJS = bachelor_fdw.o
HEADERS = warp_client.h
ifdef DEBUG
$(info $(shell echo "debug ist an"))
endif
PG_LIBS = -lpq
PG_CONFIG = pg_config
PGXS := $(shell $(PG_CONFIG) --pgxs)
include $(PGXS)
make USE_PGXS=1 install
Try to create the extension. The extension makes a call to a library function in it's _PG_INI() function. Error comes up:
CREATE EXTENSION IF NOT EXISTS bachelor_fdw;
psql:only_create.sql:3: ERROR: could not load library "/usr/lib/postgresql/12/lib/bachelor_fdw.so": /usr/lib/postgresql/12/lib/bachelor_fdw.so: undefined symbol: warpclient_getData
The warp_client.h has the function headers and warp_client.c has the functions. warp_client.c includes "warp_client.h", bachelor_fdw.c (the extension) includes "warp_client.h".
warp_client.h:
#ifndef TEST_FIELD_UCP_WARP_CLIENT_H
#define TEST_FIELD_UCP_WARP_CLIENT_H
#include <ucp/api/ucp.h>
int warpclient_queryServer(char *server_addr_local, int port, int useINet6, char *query);
void *warpclient_getData();
int warpclient_cleanup();
#endif //TEST_FIELD_UCP_WARP_CLIENT_H
Any more desired info? I would be really glad for any help.
EDIT 1
I use the functions from warp_client.h inside of bachelor_fdw.c. Do i still need to export them? I thought only functions, which get called from the postgresql server needs to be exported.
Here is part of bachelor_fdw.c:
#include <warp_client.h>
#include "postgres.h"
#include "foreign/fdwapi.h"
#include "foreign/foreign.h"
#include "nodes/nodes.h"
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
...
PG_MODULE_MAGIC;
/*
* SQL functions
*/
PG_FUNCTION_INFO_V1(bachelor_fdw_handler);
PG_FUNCTION_INFO_V1(bachelor_fdw_validator);
/*
* Extension initialization functions
*/
extern void _PG_init(void);
extern void _PG_fini(void);
/*
* FDW callback routines
*/
static void bachelorBeginForeignScan(ForeignScanState *node, int eflags);
static TupleTableSlot *bachelorIterateForeignScan(ForeignScanState *node);
static void bachelorReScanForeignScan(ForeignScanState *node);
static void bachelorEndForeignScan(ForeignScanState *node);
static void bachelorGetForeignRelSize(PlannerInfo *root, RelOptInfo *baserel, Oid foreigntableid);
static void bachelorGetForeignPaths(PlannerInfo *root, RelOptInfo *baserel, Oid foreigntableid);
static ForeignScan* bachelorGetForeignPlan(PlannerInfo *root, RelOptInfo *baserel, Oid foreigntableid, ForeignPath *best_path, List *tlist, List *scan_clauses, Plan *outer_plan);
void _PG_init(void){
int ret = 0;
void *data;
ret = warpclient_queryServer(NULL, -1, 0, "SELECT TEST FROM TEST;");
elog_debug("Testquery for server. Return code (%d)...\n", ret);
while(NULL != (data = warpclient_getData())){
elog_debug("Data received as fdw: %s\n", data);
}
elog_debug("Finished receiving data.\n");
/* Call cleanup */
ret = warpclient_cleanup();
elog_debug("Warpclient cleanup (%d)...\n", ret);
}
And here is part of warp_client.c:
#include "warp_client.h"
...
int warpclient_cleanup(){
int ret = 0;
//free buffers
free(recvbuffer->buffer);
free(recvbuffer);
/* Close the endpoint to the server */
debugmsg("Close endpoint.\n");
ep_close();
/* releasing UCX ressources */
ucp_worker_destroy(ucp_worker);
ucp_cleanup(ucp_context);
return ret;
}
int warpclient_queryServer(char *server_addr_local, int port, int useINet6, char *query){
/*
* Initialize important connection variables
*/
debugmsg("Initializing connection variables...\n");
if(NULL != server_addr_local) server_addr = server_addr_local;
if((port >= 0) && (port <= UINT16_MAX)) server_port = port;
if(useINet6) ai_family = AF_INET6;
int ret;
/* Initialize the UCX required objects worker and context*/
debugmsg("Initializing context and worker...\n");
ret = init_context_and_worker();
if (ret != 0) {
fprintf(stderr, "Initializing worker or context failed! Exiting..\n");
return -2;
}
/*
* UCP objects: client_ep as communication endpoint for the worker.
* status for function error code check.
*/
ucs_status_t status;
/* ep initialization and exchange with server over sockets */
debugmsg("Creating Client endpoint.\n");
status = create_client_endpoint();
if (status != UCS_OK) {
fprintf(stderr, "failed to start client (%s)\n", ucs_status_string(status));
return -1;
}
ret = send_query(query);
if(ret!=0){
debugmsg("Failed to connect to Server.\n");
}
return ret;
}
EDIT 2
I managed to get a good step forward thanks to Laurenz Albe. But i still have a problem with a shared library used in my shared library. Do I also need to link to shared libraries used in my own shared library, even though i linked that as i compiled my shared library before distribution?
what I did:
I added SHLIB_LINK = -lwarp_client to the Makefile and also needed the line PG_LDFLAGS += -L. for the linker to find libwarp_client.so.
I also managed to include the environment variable LD_LIBRARY_PATH for the postgres service, so that it can find my library in the standard places. And removed the library from the DATA flag in the Makefile.
New Makefile:
EXTENSION = bachelor_fdw
MODULE_big = bachelor_fdw
DATA = bachelor_fdw--0.1.sql
OBJS = bachelor_fdw.o
ifdef DEBUG
$(info $(shell echo "debug ist an"))
endif
PG_LIBS = -lpq
SHLIB_LINK = -lwarp_client
PG_LDFLAGS += -L.
PG_CONFIG = pg_config
PGXS := $(shell $(PG_CONFIG) --pgxs)
include $(PGXS)
Enrivonment variables:
/proc/1551/environ | xargs -0 -n 1 echo
LD_LIBRARY_PATH=/usr/include/:/usr/include/ucx/:/usr/lib/:/usr/lib/ucx/
...
When using CREATE on the extension, my library gets used but postgres complains about another shared library, which my library uses.
psql:only_create.sql:3: ERROR: could not load library "/usr/lib/postgresql/12/lib/bachelor_fdw.so": /usr/lib/warpdrive/libwarp_client.so: undefined symbol: ucp_ep_create
The error clearly says, it uses my shared library from a subdirectory "warpdrive" in the included standard directory. The shared library from UCP is also in that standard directory:
ls /usr/lib/ucx
cmake libjucx.so.0.0.0 libucp.a libucs.la libuct.so
jucx-1.12.1.jar libucm.a libucp.la libucs.so libuct.so.0
libjucx.a libucm.la libucp.so libucs.so.0 libuct.so.0.0.0
libjucx.la libucm.so libucp.so.0 libucs.so.0.0.0 pkgconfig
libjucx.so libucm.so.0 libucp.so.0.0.0 libuct.a ucx
libjucx.so.0 libucm.so.0.0.0 libucs.a libuct.la
That looks like warpclient_getData gets used in your code, but you didn't link your shared object with the library that provides the function. Add the library to the SHLIB_LINK variable:
SHLIB_LINK = -lwarp
(That example assumes a library called libwarp.so.)
I recoded malloc free and realloc in c using mmap and munmap. I compiled them as a shared library .so file.
Here is a simple test:
#include <stdlib.h>
int main() {
int i;
char *addr;
i = 0;
while (i < 1024)
{
addr = (char*)malloc(1024);
addr[0] = 42;
i++;
}
return (0);
}
Here is a run.sh that must replace the stdlib by my shared library:
#/bin/sh
export DYLD_LIBRARY_PATH=.
export DYLD_INSERT_LIBRARIES="libft_malloc.so"
export DYLD_FORCE_FLAT_NAMESPACE=1
$#
The problem is that when i am compiling directly the test file with my shared library and replacing the header in it, it is working well:
-> gcc test1.c libft_malloc.so
-> ./a.out
-> no error
But when i am running it with the run.sh that should just replace the official malloc library by my libft_malloc.so file, i am getting a segfault:
-> gcc test1.c
-> ./run.sh ./a.out
-> ./run.sh: line 5: 73502 Segmentation fault: 11 $#
I know the error is in my code and not in the run.sh or in the test.c because they are the officials files i must use to test my library in my school and those files are working well on other malloc repositories, but i can't find what can be the problem.
Here is my repository: https://github.com/Shirakawa42/malloc.git
I tried debugged by placing write() everywhere but the segfault don't seems to be in the malloc, so i'm lost.
edit:
It also segfault if we run a test without any malloc, but just by loading my library:
#include <stdlib.h>
int main() {
int i;
i = 0;
while (i < 1024)
{
i++;
}
return (0);
}
-> gcc test1.c
-> ./run.sh ./a.out
-> ./run.sh: line 5: 74764 Segmentation fault: 11 $#
edit 2:
Compiling with flag fsanitize=address repair the segfault, but it's absolutely not optimal
edit 3:
Setting the 2 first export manually in shell tell me:
dyld: warning: could not load inserted library 'libft_malloc.so' into library validated process because no suitable image found. Did find:
libft_malloc.so: code signing blocked mmap() of 'libft_malloc.so'
and after setting the third export all my actions make me segfault, like ls and vim, cd made me abort
dyld: warning: could not load inserted library 'libft_malloc.so' into library validated process because no suitable image found. Did find:
libft_malloc.so: code signing blocked mmap() of 'libft_malloc.so'
This error append when there are a segfault in your malloc or free, repairing free made it work.
You can debug it in gdb. First build your code with debug options:
gcc -g -O0
Above options should be used for both your lib and test program. Then you can try running you program in gdb:
gdb a.out
(gdb) r <arguments to a.out>
(gdb) bt <-- when it crashes
Linux loads you program in the memory and calls the entry point main. Startup code that compiler adds for the platform may have calls to malloc. Hence, the crash without malloc in your test code.
Refering to Connect Postgresql in C
I asked some help to compile my script of Postgres database connection. This is now compiling well but now I have a problem when I execute it.
There is the code :
#include <stdio.h>
#include "libpq-fe.h"
#include <stdlib.h>
int main() {
PGconn *conn;
PGresult *res;
int rec_count;
int row;
int col;
conn = PQconnectdb("dbname=ljdata host=localhost user=dataman password=supersecret");
if (PQstatus(conn) == CONNECTION_BAD) {
puts("We were unable to connect to the database");
exit(0);
}
res = PQexec(conn,
"update people set phonenumber=\'5055559999\' where id=3");
res = PQexec(conn,
"select lastname,firstname,phonenumber from people order by id");
if (PQresultStatus(res) != PGRES_TUPLES_OK) {
puts("We did not get any data!");
exit(0);
}
rec_count = PQntuples(res);
printf("We received %d records.\n", rec_count);
puts("==========================");
for (row=0; row<rec_count; row++) {
for (col=0; col<3; col++) {
printf("%s\t", PQgetvalue(res, row, col));
}
puts("");
}
puts("==========================");
PQclear(res);
PQfinish(conn);
return 0;
}
And this is the compilation line it's actually working :
gcc -m64 -I "C:\Program Files\PostgreSQL\9.6\include" -L "C:\Program Files\PostgreSQL\9.6\lib" test.c -lpq -o test.exe
Now when I double-click on the .exe I have the following error :
"Unable to start the program because LIBPQ.dll is missing"
So it clearly means that I have to link the libpq.dll
But what I'm trying to do is compiling all the methods from the .c/.dll etc in the binary .exe which I plan to distribute and the postgresql libs will not be installed on the target machines.
After some web search, I found the "-static" parameter but I'm not sure that it's what I'm looking for and I'm not able to have it compiling with this parameter.
Then I tried to add "-static-libgcc" which results as :
gcc -m64 -static-libgcc -I "C:\Program Files\PostgreSQL\9.6\include" -L "C:\Program Files\PostgreSQL\9.6\lib" test.c -lpq -o test.exe
But I still have the "System error" when I try to run it :
"Unable to run the script because LIBPQ.dll is missing on your computer"
(Not sure if my french translate is exact)
Do you know the parameter or the way I can compile my C program into a binary .exe which would be totally standalone and without any library requirement ?
EDIT
Screenshot :
Stick with dynamic linking.
Your problem is that the system cannot find libpq.dll at runtime.
There are two solutions:
Add C:\Program Files\PostgreSQL\9.6\lib to the PATH environment variable.
Put a copy of libpq.dll in the same path as your executable.
I'm trying to compile this code which call func from "libcfmapi.so" to decrypt "cfg" file
#include <stdlib.h>
#include <stdio.h>
int restorebackup(const char *tmp_cfg_name,const char *xml_cfg_name);
int ATP_CFM_ExtCustomImportEncryptedUserCfgFile(const char *tmp_cfg_name);
int main(int argc, char **argv)
{
int ret;
if(argc < 3)
{
printf("specify temp config file name.\n");
exit(1);
}
ret=restorebackup(argv[1],argv[2]);
return ret;
}
int restorebackup(const char *tmp_cfg_name,const char *xml_cfg_name)
{
int ret=0;
//ret = ATP_CFM_ExtDigVerifyFile(tmp_cfg_name,tmp_cfg_name);
if(ret != 0)
{
printf("Verify File failed.\n");
return ret;
}
ret = ATP_CFM_ExtCustomImportEncryptedUserCfgFile(tmp_cfg_name);
return ret;
}
but got error regarding func type declare
root#kali:~/debian-qemu# gcc h.c -o demo
/tmp/ccVbt5NT.o: In function `restorebackup':
h.c:(.text+0x8c): undefined reference to `ATP_CFM_ExtCustomImportEncryptedUserCfgFile'
collect2: error: ld returned 1 exit status
any help appreciated
The reason you're getting this error is because you're not linking against the required library, libcfmapi.so.
This is not a library you would expect to find in your Debian system as it is unique to the BT device you're trying to hack.
In short - get the lib from your device, cross compile to the device architecture against the lib you extracted from the device and you should be fine.
More information based on Ishay Peled answer:
readelf -s <pulled library> | grep ATP_CFM_ExtCustomImportEncryptedUserCfgFile
I suspect the problem isn't that function you call doesn't exist but rather there are no functions being displayed it is most likely empty just like nm result:
nm: libcfmapi.so: no symbols
do the command without piping to grep, my bet is your output is:
readelf -s libcfmapi.so
Dynamic symbol information is not available for displaying symbols.
If someone knows the way of getting the headers from the file, i believe then you can find your function you require then link and run (i too am trying to use libcfmapi.so, but lack the programming/reversing knowledge required).
when using a library, then must:
include that library in the link statement via
-l cfmapi
include the header file for that library in the source code:
#include <cfmapi.h>
I need to create an application to extract one file from zip archive, after which I want to compile it for Android.
I'm using Ubuntu, with libzip-0.10.1 pre-installed.
I created C project in Eclipse, added include path and found simple script for extracting file. Unfortunately I cannot get the following to build and I could use some advice.
// zip.c file
#include <stdio.h>
#include <stdlib.h>
#include <zip.h>
int main(int argc, char **argv) {
struct zip *zip_file;
struct zip_file *file_in_zip;
int err;
int files_total;
int file_number;
int r;
char buffer[10000];
if (argc < 3) {
fprintf(stderr,"usage: %s <zipfile> <fileindex>\n",argv[0]);
return -1;
};
zip_file = zip_open(argv[1], 0, &err);
if (!zip_file) {
fprintf(stderr,"Error: can't open file %s\n",argv[1]);
return -1;
};
file_number = atoi(argv[2]);
files_total = zip_get_num_files(zip_file);
if (file_number > files_total) {
printf("Error: we have only %d files in ZIP\n",files_total);
return -1;
};
file_in_zip = zip_fopen_index(zip_file, file_number, 0);
if (file_in_zip) {
while ( (r = zip_fread(file_in_zip, buffer, sizeof(buffer))) > 0) {
printf("%s",buffer);
};
zip_fclose(file_in_zip);
} else {
fprintf(stderr,"Error: can't open file %d in zip\n",file_number);
};
zip_close(zip_file);
return 0;
};
Also I added few .h files to include directory in my project and few .c files to directory with zip.c file. After that all dependences was good, but I have an error:
‘struct zip’ has no member named ‘default_password’ in file zip_fopen_index.c
The file zip_fopen_index.c is:
#include <errno.h>
#include <stdio.h>
#include <stdlib.h>
#include "zipint.h"
ZIP_EXTERN struct zip_file *
zip_fopen_index(struct zip *za, zip_uint64_t fileno, int flags)
{
return zip_fopen_index_encrypted(za, fileno, flags, za->default_password); // error here
}
First of all allow me some comments:
Your program is not compiled and linked by Eclipse.
Compiling is done by the compiler (gcc using option -c):
make all
Building file: ../zip.c
Invoking: GCC C Compiler
gcc -O0 -g3 -Wall -c -fmessage-length=0 -MMD -MP -MF"zip.d" -MT"zip.d" -o "zip.o" "../zip.c"
Finished building: ../zip.c
Linking is done by the linker (via the compiler using option -o):
Invoking: GCC C Linker
gcc -o "unzipper" ./zip.o
./main.o: In function `zip':
/home/alk/workspace/unzipper/Debug/../zip.c:20: undefined reference to `zip_open'
/home/alk/workspace/unzipper/Debug/../zip.c:27: undefined reference to `zip_get_num_files'
/home/alk/workspace/unzipper/Debug/../zip.c:33: undefined reference to `zip_fopen_index'
/home/alk/workspace/unzipper/Debug/../zip.c:35: undefined reference to `zip_fread'
/home/alk/workspace/unzipper/Debug/../zip.c:38: undefined reference to `zip_fclose'
/home/alk/workspace/unzipper/Debug/../zip.c:43: undefined reference to `zip_close'
collect2: ld returned 1 exit status
Eclipse provides a framework helping you in managing all sources and their references as also spawing compiler and linker tasks and setting their options.
When the linker told you there where undefined references to the zip_*function during the build of your program, the cause for this was, you were missing to tell the linker (via the compiler, via Eclipse) where those zip_* functions could be found.
Those zip_* functions are located in a library, namely libzip.
So what you as the programmer need to tell the linker (via the compiler, via Eclipse) is to link those functions against what the compiler compiled from your sources.
As the result the linker is able to create a runnable program from your compiled sources together with all libraries needed. Certain libraries are know to Eclipse (and therfore to the linker) by default, for example the one containing the C standard functions, namely libc.
To get things going:
1 Remove the source files you pulled from the libzip librarie's sources from your project. Those sources had been compiled into the library libzip, which you will use in your project.
2 Tell the linker (via Eclipse) to use libzip for your project.
Do so by following the steps below:
open the project's properties
click 'C/C++ General'
click 'Path and Symbols', on the left select the 'Libraries' tab, there click 'Add' and enter zip
finally click 'OK'
3 Then try to build your program:
Building target: unzipper
Invoking: GCC C Linker
gcc -o "unzipper" ./zip.o -lzip
Finished building target: unzipper
(Please note additional option -lzip!)
If the developement version of 'libzip' had been installed properly before, you should be fine.
PS: unzipper was the name I used for the Eclispe project to produce the examples.
PSS: I used Eclipse Juno SR1