I am a beginner in library linking and have spent two days trying to link the libcrypto library to my C program on Visual Studio 2017.
I have had to include applink.c to solve my issues. But I think it is very weird to include something which is not a header. After searching a bit on the internet, I learnt that including a source file is something that exist but that one should avoid doing. Why then the openssl library does not ?
I don't think it is particularly relevant, but here is a small overview of this source file, you can find the whole source file here.
#define APPLINK_OPEN 18 /* formally can't be used, as flags can vary */
#define APPLINK_READ 19
#define APPLINK_WRITE 20
#define APPLINK_LSEEK 21
#define APPLINK_CLOSE 22
#define APPLINK_MAX 22 /* always same as last macro */
#ifndef APPMACROS_ONLY
# include <stdio.h>
# include <io.h>
# include <fcntl.h>
static void *app_stdin(void)
{
return stdin;
}
static void *app_stdout(void)
{
return stdout;
}
if (once) {
OPENSSL_ApplinkTable[APPLINK_STDIN] = app_stdin;
OPENSSL_ApplinkTable[APPLINK_STDOUT] = app_stdout;
OPENSSL_ApplinkTable[APPLINK_STDERR] = app_stderr;
OPENSSL_ApplinkTable[APPLINK_FPRINTF] = fprintf;
OPENSSL_ApplinkTable[APPLINK_FGETS] = fgets;
OPENSSL_ApplinkTable[APPLINK_FREAD] = fread;
OPENSSL_ApplinkTable[APPLINK_FWRITE] = fwrite;
OPENSSL_ApplinkTable[APPLINK_FSETMOD] = app_fsetmod;
OPENSSL_ApplinkTable[APPLINK_FEOF] = app_feof;
OPENSSL_ApplinkTable[APPLINK_FCLOSE] = fclose;
OPENSSL_ApplinkTable[APPLINK_FOPEN] = fopen;
OPENSSL_ApplinkTable[APPLINK_FSEEK] = fseek;
OPENSSL_ApplinkTable[APPLINK_FTELL] = ftell;
OPENSSL_ApplinkTable[APPLINK_FFLUSH] = fflush;
OPENSSL_ApplinkTable[APPLINK_FERROR] = app_ferror;
OPENSSL_ApplinkTable[APPLINK_CLEARERR] = app_clearerr;
OPENSSL_ApplinkTable[APPLINK_FILENO] = app_fileno;
OPENSSL_ApplinkTable[APPLINK_OPEN] = _open;
OPENSSL_ApplinkTable[APPLINK_READ] = _read;
OPENSSL_ApplinkTable[APPLINK_WRITE] = _write;
OPENSSL_ApplinkTable[APPLINK_LSEEK] = _lseek;
OPENSSL_ApplinkTable[APPLINK_CLOSE] = _close;
once = 0;
}
What is the difference between including a header and a source file ? What are the cons and pros to do that ?
This post isn't asking for a way to fix the link problems
Including a file is just like copying the file into your C file.
This means that anything that includes the C file will have its own sort of copy of the functions in applink.c.
You'll also notice that they're declared 'static', meaning that these copies are local to their respective files, and cannot create conflicts. They'll likely be inlined, even.
In general, this is a trade-off between binary size and compile time on one hand, and runtime speed and code simplicity on the other.
This isn't a rare thing to see, what surprises me more is that it's .c and not .h.
Related
I am trying to use EVP_* APIs from OpenSSL, but I have encountered a weird strange behavior when trying to dump the Public/Private key from the EVP_PKEY struct.
Issue:: After populating the EVP_PKEY struct, on calling PEM_write_PUBKEY API (see TRIAL1), the program exits. The same happens on calling of PEM_write_PrivateKey API (see TRIAL2).
Output: I am left with a temp.pem 0 bytes file and a message on cmd prompt saying OPENSSL_Uplink(5D8C7000,08): no OPENSSL_Applink
#define TRIAL1
void InitOpenSSLLib(void)
{
SSL_library_init();
SSL_load_error_strings();
OpenSSL_add_all_algorithms();
}
int main(int argc, char** argv)
{
EVP_PKEY_CTX* ctx = NULL;
EVP_PKEY* pKeyPair = EVP_PKEY_new();
BIO *mem = BIO_new(BIO_s_mem());
FILE* fp = fopen("temp.pem", "wb");
InitOpenSSLLib();
ctx = EVP_PKEY_CTX_new_id(EVP_PKEY_RSA, 0);
EVP_PKEY_keygen_init(ctx);
EVP_PKEY_CTX_set_rsa_keygen_bits(ctx, 2048);
EVP_PKEY_keygen(ctx, &pKeyPair);
// Succeeds till here... all of the above called APIs return value greater than 0
#ifdef TRIAL1
// Program exits even before printing any error
if (PEM_write_PUBKEY(fp, pKeyPair) <= 0)
printf("PEM_write_PUBKEY failed\n");
#elif TRIAL2
// same behavior with this API too
PEM_write_PrivateKey(fp, pKeyPair, NULL, NULL, 0, 0, NULL);
#endif
// Tried this too.... but the control never reaches here
fflush(fp);
// Tried this too... most of the mem struct members are NULL.
EVP_PKEY_print_private(mem, pKeyPair, 0, 0);
//.... Cleanup codes
fclose(fp);
BIO_free_all(mem);
return 0;
}
Any pointers? Am I doing anything wrong here? Is there any other way to dump private/public key or the pair PEM format to a file?
I am using VC++ 2015. Also, on hitting Ctrl+F5, the prompt shows the message:: OPENSSL_Uplink(5D8C7000,08): no OPENSSL_Applink
Answering my own question for future devs
The key question here is the error (read, message) thrown by OpenSSL, i.e., no OPENSSL_Applink.
As documented here,
As per 0.9.8 the above limitation is eliminated for .DLLs. OpenSSL
.DLLs compiled with some specific run-time option [we insist on the
default /MD] can be deployed with application compiled with different
option or even different compiler. But there is a catch! Instead of
re-compiling OpenSSL toolkit, as you would have to with prior
versions, you have to compile small C snippet with compiler and/or
options of your choice. The snippet gets installed as
/include/openssl/applink.c and should be either added to
your application project or simply #include-d in one [and only one] of
your application source files. Failure to link this shim module into
your application manifests itself as fatal "no OPENSSL_Applink"
run-time error. An explicit reminder is due that in this situation
[mixing compiler options] it is as important to add CRYPTO_malloc_init
prior first call to OpenSSL.
you should check your compiling options for /MD (assuming you know the Code Generation options under VC++ Project Properties).
I did the same, but still my issue didn't get resolved. The answer to this is the second paragraph to the link given [https://www.openssl.org/docs/faq.html#PROG2], where it instructs us to include <install-root>/include/openssl/applink.c
Where to get this applink.c file if not found in openssl's
dir?
In my case, I installed OpenSSL using exe binary and I couldn't find this specific applink.c file anywhere in my <install-root>, so I started looking and found it here on GitHub.
All you have to do is, include this file as source file in your project or better save it in openssl install-root dir, so that you can include it from the same location for other projects too.
// applink.c from https://github.com/openssl/openssl/blob/master/ms/applink.c
/*
* Copyright 2004-2016 The OpenSSL Project Authors. All Rights Reserved.
*
* Licensed under the OpenSSL license (the "License"). You may not use
* this file except in compliance with the License. You can obtain a copy
* in the file LICENSE in the source distribution or at
* https://www.openssl.org/source/license.html
*/
#define APPLINK_STDIN 1
#define APPLINK_STDOUT 2
#define APPLINK_STDERR 3
#define APPLINK_FPRINTF 4
#define APPLINK_FGETS 5
#define APPLINK_FREAD 6
#define APPLINK_FWRITE 7
#define APPLINK_FSETMOD 8
#define APPLINK_FEOF 9
#define APPLINK_FCLOSE 10 /* should not be used */
#define APPLINK_FOPEN 11 /* solely for completeness */
#define APPLINK_FSEEK 12
#define APPLINK_FTELL 13
#define APPLINK_FFLUSH 14
#define APPLINK_FERROR 15
#define APPLINK_CLEARERR 16
#define APPLINK_FILENO 17 /* to be used with below */
#define APPLINK_OPEN 18 /* formally can't be used, as flags can vary */
#define APPLINK_READ 19
#define APPLINK_WRITE 20
#define APPLINK_LSEEK 21
#define APPLINK_CLOSE 22
#define APPLINK_MAX 22 /* always same as last macro */
#ifndef APPMACROS_ONLY
# include <stdio.h>
# include <io.h>
# include <fcntl.h>
static void *app_stdin(void)
{
return stdin;
}
static void *app_stdout(void)
{
return stdout;
}
static void *app_stderr(void)
{
return stderr;
}
static int app_feof(FILE *fp)
{
return feof(fp);
}
static int app_ferror(FILE *fp)
{
return ferror(fp);
}
static void app_clearerr(FILE *fp)
{
clearerr(fp);
}
static int app_fileno(FILE *fp)
{
return _fileno(fp);
}
static int app_fsetmod(FILE *fp, char mod)
{
return _setmode(_fileno(fp), mod == 'b' ? _O_BINARY : _O_TEXT);
}
#ifdef __cplusplus
extern "C" {
#endif
__declspec(dllexport)
void **
# if defined(__BORLANDC__)
/*
* __stdcall appears to be the only way to get the name
* decoration right with Borland C. Otherwise it works
* purely incidentally, as we pass no parameters.
*/
__stdcall
# else
__cdecl
# endif
OPENSSL_Applink(void)
{
static int once = 1;
static void *OPENSSL_ApplinkTable[APPLINK_MAX + 1] =
{ (void *)APPLINK_MAX };
if (once) {
OPENSSL_ApplinkTable[APPLINK_STDIN] = app_stdin;
OPENSSL_ApplinkTable[APPLINK_STDOUT] = app_stdout;
OPENSSL_ApplinkTable[APPLINK_STDERR] = app_stderr;
OPENSSL_ApplinkTable[APPLINK_FPRINTF] = fprintf;
OPENSSL_ApplinkTable[APPLINK_FGETS] = fgets;
OPENSSL_ApplinkTable[APPLINK_FREAD] = fread;
OPENSSL_ApplinkTable[APPLINK_FWRITE] = fwrite;
OPENSSL_ApplinkTable[APPLINK_FSETMOD] = app_fsetmod;
OPENSSL_ApplinkTable[APPLINK_FEOF] = app_feof;
OPENSSL_ApplinkTable[APPLINK_FCLOSE] = fclose;
OPENSSL_ApplinkTable[APPLINK_FOPEN] = fopen;
OPENSSL_ApplinkTable[APPLINK_FSEEK] = fseek;
OPENSSL_ApplinkTable[APPLINK_FTELL] = ftell;
OPENSSL_ApplinkTable[APPLINK_FFLUSH] = fflush;
OPENSSL_ApplinkTable[APPLINK_FERROR] = app_ferror;
OPENSSL_ApplinkTable[APPLINK_CLEARERR] = app_clearerr;
OPENSSL_ApplinkTable[APPLINK_FILENO] = app_fileno;
OPENSSL_ApplinkTable[APPLINK_OPEN] = _open;
OPENSSL_ApplinkTable[APPLINK_READ] = _read;
OPENSSL_ApplinkTable[APPLINK_WRITE] = _write;
OPENSSL_ApplinkTable[APPLINK_LSEEK] = _lseek;
OPENSSL_ApplinkTable[APPLINK_CLOSE] = _close;
once = 0;
}
return OPENSSL_ApplinkTable;
}
#ifdef __cplusplus
}
#endif
#endif
That's it, this will eliminate the strange behavior of program exit.
I am trying to learn to program in C (not C++!). I've read about external variables, which should (according to the writer) give a nicer code. In order to use the external variables, I must #define them in the .h file, before I can use them in main.c file, using the extern command in front of the variable. I am trying to create an array in the .h file like this:
#define timeVals[4][2];
timeVals[0][0] = 7;
timeVals[0][1] = 45;
timeVals[1][0] = 8;
timeVals[1][1] = 15;
timeVals[2][0] = 9;
timeVals[2][1] = 30;
timeVals[3][0] = 10;
timeVals[3][1] = 25;
(it's a clock I'm trying to make, simple program in console). The first column indicates hours and the second indicates minutes. In my main I have written
extern int timeVals[][];
but I get an error telling me that " expected identifier or '(' before '[' token|" and I can't see what the issue is... any ideas or advices?
I am using the .h file to learn how to use external variables, so I can't move the values back into main.c
First, this:
#define timeVals[4][2];
Is a confusion. You mean this:
int timeVals[4][2];
Put that in your .h file, then in your .c file, something like this:
int timeVals[4][2] = {
{ 1, 2 }, // ...
};
That's how you initialize the entire array (any unspecified elements will be zero).
I am using the latest LibClang to parse some C header files. The code I process comes from CXUnsavedFile's (it is all generated dynamically and nothing lives on disk). For Example:
FileA.h contains:
struct STRUCT_A {
int a;
struct STRUCT_B foo;
};
FileB.h contains:
struct STRUCT_B {
int b;
};
When parsing fileA.h with the following code snippet:
CXUnsavedFile unsaved_files[2];
unsaved_files[0].Filename = "fileA.h";
unsaved_files[0].Contents = fileA_contents;
unsaved_files[0].Length = strlen( fileA_contents );
unsaved_files[1].Filename = "fileB.h";
unsaved_files[1].Contents = fileB_contents;
unsaved_files[1].Length = strlen( fileB_contents );
tu = clang_parseTranslationUnit(
index,
"fileA.h",
argv, // "-x c-header -target i386-pc-win32"
argc,
(CXUnsavedFile *)&unsaved_files,
2,
CXTranslationUnit_None
);
CXCursor cur = clang_getTranslationUnitCursor( tu );
clang_visitChildren( cur, visitor, NULL );
I get the error "field has incomplete type 'struct STRUCT_B'" which makes sense as I have not included fileB.h in order to define struct STRUCT_B.
Adding an "#include <fileB.h>" does not work (fatal error: 'fileB.h' file not found).
How do I get parsing fileA.h to work when one or more needed definitions are present in another CXUnsavedFile fileB.h?
Not sure this will help you, but here are two remarks:
Although it isn't explicitly mentioned in the documentation, I think that the Filename field should contain a full path to the file (which could be important for inclusions, especially when there are "-I" switches in the command-line)
from libclang's documentation (emphasis mine):
const char* CXUnsavedFile::Filename
The file whose contents have not yet been saved.
This file must already exist in the file system.
I suspect libclang relies on the filesystem for almost everything (finding the correct file to include, checking it exists, ...) and only account for CXUnsavedFiles at the last step, when actual content must be read.
If you can, I would suggest creating empty files in a memory filesystem. This would not incur much resource usage, and could help libclang find the correct include files.
I'm trying to do a checkpoint/restart program in C and I'm studying cryoPID's code to see how a process can be restarted. In it's code, cryoPID creates the ELF header of the process to be restarted in a function that uses some global variable and it's really confusing.
I have been searching for an easy way to create an ELF executable file, even trying out libelf, but I find that most of the times some necessary information is vague in the documentation of these programs and I cannot get to understand how to do it. So any help in that matter would be great.
Seeing cryoPID's code I see that it does the whole creation in an easy way, not having to set all header fields, etc. But I cannot seem to understand the code that it uses.
First of all, in the function that creates the ELF the following code is relevant (it's in arch-x86_64/elfwriter.c):
Elf64_Ehdr *e;
Elf64_Shdr *s;
Elf64_Phdr *p;
char* strtab;
int i, j;
int got_it;
unsigned long cur_brk = 0;
e = (Elf64_Ehdr*)stub_start;
assert(e->e_shoff != 0);
assert(e->e_shentsize == sizeof(Elf64_Shdr));
assert(e->e_shstrndx != SHN_UNDEF);
s = (Elf64_Shdr*)(stub_start+(e->e_shoff+(e->e_shstrndx*e->e_shentsize)));
strtab = stub_start+s->sh_offset;
stub_start is a global variable defined with the macro declare_writer in cryopid.h:
#define declare_writer(s, x, desc) \
extern char *_binary_stub_##s##_start; \
extern int _binary_stub_##s##_size; \
struct stream_ops *stream_ops = &x; \
char *stub_start = (char*)&_binary_stub_##s##_start; \
long stub_size = (long)&_binary_stub_##s##_size
This macro is used in writer_*.c which are the files that implement writers for files. For example in writer_buffered.c, the macro is called with this code:
struct stream_ops buf_ops = {
.init = buf_init,
.read = buf_read,
.write = buf_write,
.finish = buf_finish,
.ftell = buf_ftell,
.dup2 = buf_dup2,
};
declare_writer(buffered, buf_ops, "Writes an output file with buffering");
So stub_start gets declared as an uninitialized global variable (the code above is not in any function) and seeing that all the variables in declare_writer are not set in any other part of the code, I assume that stub_start just point to some part of the .bss section, but it seems like cryoPID use it like it's pointing to its own ELF header.
Can anyone help me with this problem or assist me in anyway to create ELF headers easily?
As mentioned in the comment, it uses something similar to objcopy to set those variables (it doesn't use the objcopy command, but custom linkers that I think could be the ones that area "setting" the variables). Couldn't exactly find what, but I could reproduce the behavior by mmap'ing an executable file previously compiled and setting the variables stub_start and stub_size with that map.
I was trying to write a small debug utility and for this I need to get the function/global variable address given its name. This is built-in debug utility, which means that the debug utility will run from within the code to be debugged or in plain words I cannot parse the executable file.
Now is there a well-known way to do that ? The plan I have is to make the .debug_* sections to to be loaded into to memory [which I plan to do by a cheap trick like this in ld script]
.data {
*(.data)
__sym_start = .;
(debug_);
__sym_end = .;
}
Now I have to parse the section to get the information I need, but I am not sure this is doable or is there issues with this - this is all just theory. But it also seems like too much of work :-) is there a simple way. Or if someone can tell upfront why my scheme will not work, it ill also be helpful.
Thanks in Advance,
Alex.
If you are running under a system with dlopen(3) and dlsym(3) (like Linux) you should be able to:
char thing_string[] = "thing_you_want_to_look_up";
void * handle = dlopen(NULL, RTLD_LAZY | RTLD_NOLOAD);
// you could do RTLD_NOW as well. shouldn't matter
if (!handle) {
fprintf(stderr, "Dynamic linking on main module : %s\n", dlerror() );
exit(1);
}
void * addr = dlsym(handle, thing_string);
fprintf(stderr, "%s is at %p\n", thing_string, addr);
I don't know the best way to do this for other systems, and this probably won't work for static variables and functions. C++ symbol names will be mangled, if you are interested in working with them.
To expand this to work for shared libraries you could probably get the names of the currently loaded libraries from /proc/self/maps and then pass the library file names into dlopen, though this could fail if the library has been renamed or deleted.
There are probably several other much better ways to go about this.
edit without using dlopen
/* name_addr.h */
struct name_addr {
const char * sym_name;
const void * sym_addr;
};
typedef struct name_addr name_addr_t;
void * sym_lookup(cost char * name);
extern const name_addr_t name_addr_table;
extern const unsigned name_addr_table_size;
/* name_addr_table.c */
#include "name_addr.h"
#define PREMEMBER( X ) extern const void * X
#define REMEMBER( X ) { .sym_name = #X , .sym_addr = (void *) X }
PREMEMBER(strcmp);
PREMEMBER(printf);
PREMEMBER(main);
PREMEMBER(memcmp);
PREMEMBER(bsearch);
PREMEMBER(sym_lookup);
/* ... */
const name_addr_t name_addr_table[] =
{
/* You could do a #include here that included the list, which would allow you
* to have an empty list by default without regenerating the entire file, as
* long as your compiler only warns about missing include targets.
*/
REMEMBER(strcmp),
REMEMBER(printf),
REMEMBER(main),
REMEMBER(memcmp),
REMEMBER(bsearch),
REMEMBER(sym_lookup);
/* ... */
};
const unsigned name_addr_table_size = sizeof(name_addr_table)/sizeof(name_addr_t);
/* name_addr_code.c */
#include "name_addr.h"
#include <string.h>
void * sym_lookup(cost char * name) {
unsigned to_go = name_addr_table_size;
const name_addr_t *na = name_addr_table;
while(to_to) {
if ( !strcmp(name, na->sym_name) ) {
return na->sym_addr;
}
na++;
to_do--;
}
/* set errno here if you are using errno */
return NULL; /* Or some other illegal value */
}
If you do it this way the linker will take care of filling in the addresses for you after everything has been laid out. If you include header files for all of the symbols that you are listing in your table then you will not get warnings when you compile the table file, but it will be much easier just to have them all be extern void * and let the compiler warn you about all of them (which it probably will, but not necessarily).
You will also probably want to sort your symbols by name such that you can use a binary search of the list rather than iterate through it.
You should note that if you have members in the table which are not otherwise referenced by the program (like if you had an entry for sqrt in the table, but didn't call it) the linker will then want (need) to link those functions into your image. This can make it blow up.
Also, if you were taking advantage of global optimizations having this table will likely make those less effective since the compiler will think that all of the functions listed could be accessed via pointer from this list and that it cannot see all of the call points.
Putting static functions in this list is not straight forward. You could do this by changing the table to dynamic and doing it at run time from a function in each module, or possibly by generating a new section in your object file that the table lives in. If you are using gcc:
#define SECTION_REMEMBER(X) \
static const name_addr_t _name_addr##X = \
{.sym_name= #X , .sym_addr = (void *) X } \
__attribute__(section("sym_lookup_table" ) )
And tack a list of these onto the end of each .c file with all of the symbols that you want to remember from that file. This will require linker work so that the linker will know what to do with these members, but then you can iterate over the list by looking at the begin and end of the section that it resides in (I don't know exactly how to do this, but I know it can be done and isn't TOO difficult). This will make having a sorted list more difficult, though. Also, I'm not entirely certain initializing the .sym_name to a string literal's address would not result in cramming the string into this section, but I don't think it would. If it did then this would break things.
You can still use objdump to get a list of the symbols that the object file (probably elf) contains, and then filter this for the symbols you are interested in, and then regenerate the table file the table's members listed.