I am using musl: https://www.musl-libc.org
If i browse the repository i can see that there are a bunch of crypt related source files (including crypt_sha256.c).
The problem is that there are no header files for them. How am i supposed to use them?
These are the symbols from the lib on my system:
I could also not find any code samples from google how to use the SHA-256 features of musl.
Thanks!
You are correct, that the three functions
sha256_init()
sha256_update()
sha256_sum()
are indeed generic SHA256 hash functions.
Unfortunately, these symbols are not exported publically, but only used internally to generate a salted password hash of the form $5$0rXgD0/KkyyT0$5PPj3bke0vPxsMDlSXzBz2D3TFNahLrXSs7.elU3u2/
For that reason, no public header files for use of these functions are provided. Only the higher-level function crypt_sha256() is exported.
Why they decided to not export the generic interface can only be speculated about, at least I could not find an explanation for that.
That's not generic SHA-256 message digest algorithm but a specific algorithm used by the crypt(3) password hashing function. See the documentation for that function on how it is used.
As Ctx wrote and commented about not knowing the reason for, these functions are not exported. I can fill in that reason.
Generally, musl does not unilaterally invent new interfaces that will almost surely end up differing in subtle ways from similar interfaces other libc providers end up inventing. We are in the process of launching a cross-libc collaboration group less formal than the POSIX standardization process that might make it reasonable to offer some interfaces like this in the future, and that might eventually funnel some of the consensus that emerges upstream to POSIX.
Short of that, anyone wanting to use these implementations is welcome to copy the code and use it under the terms of the license. They're small, self-contained, and permissively-licensed, and by using them this way you don't lock in the signature for any external interface boundary. As usual with cryptographic code, though, you should be careful of any risk of side channels leaking secrets. As used in musl I don't believe that's an issue, but I haven't analyzed other possible uses, and it might be safer to pick an implementation designed for use in arbitrary cryptographic settings.
Related
gcc (GCC) 4.7.2
PJ SIP 2.1
Hello,
I am developing an application that will use the PJSIP API.
Just looking at the API documentation and I see some functions that seem to be just wrappers for the standard C library. i.e. pj_memset, pj_strncpy, pj_strlen, etc.
I can see some alternatives that might be worth considering pj_strncpy_with_null() which will always NULL terminate a string. A another advantage could be is that the pjsip uses a pj_str_t structure to store the string and the size. Which could be better than using a normal C string.
And is there any point using pj_size_t over size_t which is portable anyway?
The link for quick reference is here:
http://www.pjsip.org/pjlib/docs/html/group__PJ__PSTR.htm
It there any real advantage using PJSIP over the standard C library?
Many thanks for any suggestions,
Short answer: Use the PJSIP API (all of it).
Long answer: It depends.
If you were programming an application for standard Desktops, that is, x86/x64 Windows/Mac/Linux, then no, it wouldn't really matter too much if you used the standard C library or wrappers like the PJSIP functions. Practically, of course, there might be functions that take (as you pointed out) the pj_str_t struct instead of a char *; it would be easier then to use the PJSIP API just to simplify and remove the need for conversions.
The reason for wrappers, I'm assuming, is to make it easier to develop on embedded devices. I don't mean just ARM or other non-x86 processors—though it could apply there as well; I mean custom embedded devices: things that have a very specific purpose and change infrequently. These embedded devices have very limited capabilities and sometimes even lack an OS. Without an OS, these processors might not have a malloc function or the like. Frequently, the libraries associated with the devices, since they are customized so much, are not entirely "standard" and differ in some small way. By having wrappers for everything, PJSIP can avoid most issues and even provide implementations across the board for things such as strcpy or malloc such that all devices run the "same" code.
Wrappers also provide the means for "hooks." Hooks enable better error messaging (and possibly handling). It's unclear whether PJSIP is doing this (I have never used PJSIP—I am talking from experience using other frameworks), but I am pointing it out just to show why a framework might bother wrapping everything.
In the end, it boils down to your purpose: if you chose to use PJSIP in the first place, then I would go all out and use all of its API. If you are only using it in a few places (for whatever reason) then it probably doesn't matter. Again, it appears that PJSIP is targeting embedded devices (it lists Nokia and even RTOS systems), where it is fairly common to provide wrappers for even "standard" functions. If this is the case, and you are using it in this way, definitely use the entire API.
Will you be sticking with pjsip?
PJSIP source code ("The Software") is licensed under both General
Public License (GPL) version 2 or later and a proprietary license that
can be arranged...
If you think the GPL may be too restrictive for future expansion (such as Android's no-GPL-in-userspace policy) and their proprietary license is not acceptable, you may benefit from using your own portable code/wrappers that you could use with a less restrictive BSD stlye library like Baresip
There are plenty of other methods to provide needed functionality where the standard C library does not support it, many of which will be better tested (I hate to mention autotools, but... it does support most platforms - some would say too many) Or you could include implementations/adaptations from musl-libc
Another thing to consider is the C api is based on standards and fairly set in stone while the wrappers in a given project are much more free to break API compatibility from version to version (just ask a glib/gtk programmer)
I'm implementing a pthread replacement library which is extremely lightweight. There are several reasons why I want to disable __thread completely.
It's a waste of memory. If I'm creating a thousand threads which has nothing to do with the context that declares a variable with __thread they will still allocate the program will still have allocated 1000*the size of that data bytes and never use it. It's simply not memory compatible with the mass-concurrency model. If we need extremely lightweight fibers with only 8K of stack, a TLS block of just 4K would be an overhead of 50% of the memory used by each thread. In some scenarios the TLS overhead would be enormous.
TLS is a complex standard and I simply don't have the time/resources to support it. It's to expensive. Personally I think the standard is poorly designed. It should have defined standard functions that had to be provided by the linker so thread libraries can take control over where TLS allocation takes place and insert relevant offsets and addresses it requires. Also the standard ELF implementation has been infected with pthread, expecting pthread sized structs to calculate offsets making it really hard to adapt to something else.
It's just a bad pattern. It encourages using globals and creating functions with static functions/side effects. This is not a territory we want to be in if we're creating correct programs that are easy to analyze.
If we really need "thread context" for some magic that tracks thread state behind the scenes (like for allocation or cancellation tracking) why not just expose the magic that TLS uses to understand that context in the first place? Personally I'm just going to use the %fs register directly. This would not be possible in libraries for obvious reasons but why should they be thread aware to begin with? Why not just design them correctly so they get the context related data they need in the first place right in the argument list?
My question is simply: What is the simplest way to disable __thread support and make clang emit errors if you accidentally used it? How can I get errors if I load a dynamic library which happens to require TLS?
I believe the simplest way is adding something like this unconditionally to your CFLAGS (perhaps from clang's equivalent of the gcc specfile if you want it to be system-global):
-D__thread='^-^'
where the righthand side can be anything that's syntactically invalid (a constraint violation) at any point in a C program.
As for preventing loading of libraries with TLS, you'd have to patch the linker and/or dynamic linker to reject them. If you're just talking about dlopen, your program could first read the file and parse the ELF headers for TLS relocations, then reject the library (without passing it to dlopen) if it has any. This might even be possible with an LD_PRELOAD wrapper.
I agree with you that, especially in its current implementation, TLS is something whose use should generally be avoided, but may I ask if you've measured the costs? I think stamping it out completely will be fairly difficult on a system that's designed to use it, and there's much lower-hanging fruit for cutting bloat. Which libc are you using? If it's glibc, I'm pretty sure glibc has lots of TLS it uses internally itself these days... Of course if you're writing your own threads implementation, that's going to entail a lot of interaction with the rest of the standard library, so perhaps you're already patching it out...?
By the way (shameless plug follows), we have an extremely light-weight threads implementation in musl libc which presently does not have TLS. I don't think it would be easy to integrate with another libc (and I'm sure if you're writing your own you will find it difficult to integrate with glibc, especially glibc's dynamic linker, which expects TLS to be supported) but if you can use the whole library as-is, it may meet your needs for specific projects, or have useful code you can borrow (license is MIT).
Assume you have a function read_key and normally it does some stuff. You someone should be able to replace it with his function read_key_myfunction and it does other stuff.
The general approach would of course be to build an array and register function pointers or using simple switch statements (or both).
But my target is a bit broader: People should be able to write their C-stuff and NOT interfere with my code and it should still register. Of course, I tell them which interface to implement.
What they now basically do is program a library for my software which I dynamically load based on a configuration option. Think of it like OpenSSLs engines: Anyone can write their own engine, compile it as a dll/so and distribute it. They don't need to modify (or know) OpenSSLs code, as long as they stick to the defined interface.
I just want the same (it will in the end be a wrapper for OpenSSL engine functions) for my program.
A colleague suggested I should use the same function in every file and load the libraries dynamically. This sounds like a good solution to me, but I am not quite satisfied since I don't see OpenSSL using any non-engine-specific function in their engine-code.
If some things are unclear here is my specific example:
I am extending a program called sscep which implements a protocol for automatic certificate renewal. A lot of cryptography should take place in HSMs in the future (and right now it should take place within the Windows Key Management (which is accessed by the capi-engine from OpenSSL)).
While OpenSSL already serves a generic interface, there is some stuff I need to do beforehand and it depends on the engine used. I also want to open the possibility for everyone else to extend it quickly without having to dig into my code (like I had from the person before me).
If anyone has any idea, it would be greatly appreciated to see some kind of guideline. Thanks in advance.
What you are describing is commonly called a plugin architecture/plugin framework. You need to combine cross-platform dlopen/LoadLibrary functionality with some logic for registering and performing lookup of exported functions. You should be able to find examples on how to do this on the internet.
I'm trying to use public key crypto to sign and later verify a file. The file is a simple plaintext file that contains user information for authoring purposes.
I tried different sites for a C implementation of a public key crypto algorithm but I haven't found anything. A lot of sites point to using certificates (x.509, etc) but that is way beyond what I need. I am just looking for a way to generate and public and private keys and use a relatively well known algorithm to sign and verify a file.
Any pointers to a pure C implementation out there? The focus is on code that I can reuse and not external libs. The main problem being that I don't want to have to link against a full lib and its dependencies in order to have a very basic public key system.
Thanks.
OpenSSL is a very good package. You can just use the crypto library portion, which provides basic RSA implementations. That might be in line with what you are looking for.
Cryptlib is another alternative that could work for you. It has some strange licensing issues though, so consider those depending on how you will be using it.
Crypto++ is a set of different crypto technologies, and includes RSA, so you might try that.
Finally, RSA is not terribly complex to implement, so you could even implement it yourself using GMP, which provides the necessary mathematical functions you would need.
You may want to look at the well-respected, debugged, and tested OpenSSL libraries. Although OpenSSL is primarily for SSL/TLS networking, it contains extremely good implementations of many cryptographic protocols, which are often used by themselves for general cryptography.
Hope this helps!
DJ Bernstein's curve25519 lets you create public/private key pairs. It does not have functions for signing, but you should be able to figure that part out with not too much hassle.
Update: In the mean time, there's also Ed25519 which already has the signature generation stuff figured out, without you having to jump through hoops. Same author, same availability of software (also e.g. "Donna" implementation and python binding), same ease of use, comparable speed.
The original implementation as well as the "Donna" implementation are both available under very liberal licenses.
You need to compile one file and call exactly one function to generate a key pair, and it's very fast. No obscure requirements for the public key. All one ever needs for some "cheap, fast, easy public key crypto".
I think that there was an answer[1] that fitted your question on :: Small RSA or DSA lib without dependencies
You may find LibTomCrypt useful. It's written in C, supports RSA and
DSA (along with a host of other algorithms), and is public domain
software. You can read about its features here: http://libtom.org/?page=features
[1] https://stackoverflow.com/a/1735526/68338 ( courtesy of https://stackoverflow.com/users/33837/emerick-rogul )
The answers on this question contain some interesting links to other libraries.
However, I remember that there exists some reference source code in C for RSA and private key cryptography. I will add a link as soon as I have found it ;-)
EDIT
I just found "this link" (http://www.hackchina.com/en/cont/93068 - open on your own risk) - not sure about the source and details of that code. But, however, in the past the link to the original RSA reference implementation was contained somewhere in OpenSSL source or its documentation. Which is based on cryptsoft.com's library. I am sure the source can still be found somewhere on www.rsa.com/rsalabs/ - but I could not find it, and I am running out of time for now. Good luck ;-)
The longer I'm working as a C developer I find myself lacking some source of middle sized code chunks.
I have source of code snippets and libraries, but I can't find a good source for code sized in between. Something that is a header, or a header+implementation file but isn't a library but is included into the project.
Stuff like a dynamic array, or linked list or some debugging or logging helpers.
I know that its partially due to C developers DIY mentality, but I just don't believe that people don't share stuff like this.
You might want to check out http://nothings.org for some single file (moderately sized) projects that include (image) decompression, font rasterization and other useful things.
You may also want to look at CCAN.
http://www.koders.com/ is worth checking. You might find something usefull now and then.
You can also sort the results by license which is pretty handy feature.
There's a handful of utility libraries that spring to mind quickly; glib provides a wide variety of useful little utilities, including:
doubly- and singly-linked lists, hash tables, dynamic strings and string utilities, such as a lexical scanner, string chunks (groups of strings), dynamic arrays, balanced binary trees, N-ary trees
(And yes, glib is useful even in non-graphical environments; don't let its GNOME-background fool you. :)
The Apache portable runtime is a library that helps abstract away platform-specific knowledge; I've seen a handful of programs use it. It feels like enough programmers are content with "It runs on Linux" to not really worry about platform differences, and forgo learning Yet Another Library as a result. It feels more like a systems-level toolkit:
Memory allocation and memory pool functionality, Atomic operations, Dynamic library handling, File I/O, Command argument parsing, Locking, Hash tables and arrays, Mmap functionality, Network sockets and protocols, Thread, process and mutex functionality, Shared memory functionality, Time routines, User and group ID services
I always look at the Python (C) source code first when I am looking for the "best" way to code something up in C. Guido van Rossum's C coding style is concise and clear and given the number functions and features supported in the standard python libraries there is nearly always a useful/relevent snippet of code in there.