The longer I'm working as a C developer I find myself lacking some source of middle sized code chunks.
I have source of code snippets and libraries, but I can't find a good source for code sized in between. Something that is a header, or a header+implementation file but isn't a library but is included into the project.
Stuff like a dynamic array, or linked list or some debugging or logging helpers.
I know that its partially due to C developers DIY mentality, but I just don't believe that people don't share stuff like this.
You might want to check out http://nothings.org for some single file (moderately sized) projects that include (image) decompression, font rasterization and other useful things.
You may also want to look at CCAN.
http://www.koders.com/ is worth checking. You might find something usefull now and then.
You can also sort the results by license which is pretty handy feature.
There's a handful of utility libraries that spring to mind quickly; glib provides a wide variety of useful little utilities, including:
doubly- and singly-linked lists, hash tables, dynamic strings and string utilities, such as a lexical scanner, string chunks (groups of strings), dynamic arrays, balanced binary trees, N-ary trees
(And yes, glib is useful even in non-graphical environments; don't let its GNOME-background fool you. :)
The Apache portable runtime is a library that helps abstract away platform-specific knowledge; I've seen a handful of programs use it. It feels like enough programmers are content with "It runs on Linux" to not really worry about platform differences, and forgo learning Yet Another Library as a result. It feels more like a systems-level toolkit:
Memory allocation and memory pool functionality, Atomic operations, Dynamic library handling, File I/O, Command argument parsing, Locking, Hash tables and arrays, Mmap functionality, Network sockets and protocols, Thread, process and mutex functionality, Shared memory functionality, Time routines, User and group ID services
I always look at the Python (C) source code first when I am looking for the "best" way to code something up in C. Guido van Rossum's C coding style is concise and clear and given the number functions and features supported in the standard python libraries there is nearly always a useful/relevent snippet of code in there.
Related
Facebook created an opensource fast lossless compression algorithm, targeting real-time compression scenarios at zlib-level and better compression ratios, called ZStandard.
I have been looking for a tutorial that describes the c to swift wrapping, like this, but looks not comprehensive enough, what are the prerequests do I have to know before writing a wrapper?
when I finish writing it, I will also make it open source.
Thank you for a good question. I've looked at the library and played with it, it seems pretty interesting.
I would say you need to be comfortable using the ZSTD C library in a C program. You need to be also comfortable programming in Swift. Depending on the parts of the API you want to wrap, you may need to understand how to deal with raw memory in Swift (Unsafe... types).
Some challenges to consider when wrapping ZSTD:
The streaming API with dictionaries is experimental and subject to
change, yet use of dictionaries is one of the advantages of ZSTD.
When dealing with memory buffers, we want to minimize copying them,
since the buffers may be quite large and copying them would adversely
affect performance. This, of course, complicates memory management.
There is a multitude of approaches you can choose from when writing a wrapper. For example, you can write wrappers in C that will expose simple APIs you will wrap in Swift. You can include C code in your wrapper framework, or you can keep it in separate C libraries.
I have come up with a simple example of a wrapper around a small subset of the API, you can take a look at https://github.com/omniprog/SwiftZSTD.
I want to write a library that is dynamically loadable and callable from C code, but I really don't want to write it in C - the code is security critical, so I want a language that makes it easier to have confidence that my code is correct. What are my options?
To be more specific, I want C programmers to be able to #include this, and -l that, and start using my library just as if I had written it in C. I'd like programmers in other languages to be able to use their favourite tools for linking to C libraries to link to it. Ideally I'd like that to be possible on every platform that supports C, but I'll settle for Linux, Windows and MacOS.
Anything that compiles to native code. So you might Google for that - "languages that compile to native code." See, e.g., Programming languages that compile to native code and have the batteries included
C++ is often the choice for this. Compiles to native code and provided you keep your interfaces simple, easy to write an adapter layer.
Objective C and Fortran are also possible.
It sounds like you are looking for a language with ABI compatibility or which can be described as resulting in native code. So long as it can be compiled to a valid object file (typically an .obj or .o file) which is accepted by the linker, that should be the main criteria. You also then want to write a header file as a convenience for any client code which is written in C (or a closely related language/variant thereof).
As mentioned by others, you need a pretty good reason for choosing a language other than C as it is the lingua-franco of low-level/systems software. Assembler is an option, although harder to port between platforms. D is a more portable - but less widespread - alternative which is designed to produce secure, efficient native code with a minimum of fuss. There are many others.
Almost every security critical application I know of is written in C. I don't believe that there are any other language that has higher real status in producing secure applications.
C is being said to be a poor language for security by people who don't understand.
If you want C programmers to use your library, use C. Doing anything else is tying one hand behind your back whilst trying to walk on a balance beam (the gymnastics equipment). Sure, there are dozens of other languages that are CAPABLE of interfacing to C, but it typically involves using a C layer and then stuffing the C data types into a language specific data type (Java Objects, Python Objects, etc, etc), and when the call is finished, you use the same conversion back to a C data type. Just makes it harder to work with, and potentially slower if you don't get all the design decisions right. And people won't understand the source code, so won't like to use it (see more about this below).
If you want security, then write very good code, wearing your "security aspects" hat firmly on at all times, find a security mailing list or website and post it there for review, take the review comments on board, understand the comments, and fix any comments that are meaningful to fix. Distribute the source code to the users, so people can see what your code does. Those that understand security will know what to look for and understand that you have done a good job (or a bad job, whichever is applicable) - and those who don't will hopefully trust the right pople. If it's good, people will use it. If it's "hidden", and not easy to access, you won't get many customers, no matter what language you use.
Don't worry, you won't reveal anything more from releasing source. If there is a flaw in the code, and it is popular (or important) enough, someone will find the flaw, even if you publish only binaries. For those skilled in reverse engineering, not having source code is only a small obstacle.
Security doesn't stem from using a specific language or a specific tool, it stems from good design and good basic understanding of the problems with security.
And remember security by obscurity (whether that means "hidden source code" or "unusual language" or something else obscure) is false security.
You might be interested in ATS, http://ats-lang.sourceforge.net/. ATS compiles via C, can be as efficient as C, and can be used in a way that is ABI-compatible with C. From the project website:
ATS is a statically typed programming language that unifies implementation with formal specification. It is equipped with a highly expressive type system rooted in the framework Applied Type System, which gives the language its name. In particular, both dependent types and linear types are available in ATS. The current implementation of ATS (ATS/Anairiats) is written in ATS itself. It can be as efficient as C/C++ (see The Computer Language Benchmarks Game for concrete evidence) and supports a variety of programming paradigms
ATS's dependent and linear type system helps produce static guarantees about your code, including various aspects of resource management safety.
Chris Double has been writing a series of articles exploring the power of ATS's type system for systems programming here: http://bluishcoder.co.nz/tags/ats/. Of particular note is this article: http://bluishcoder.co.nz/2012/08/30/safer-handling-of-c-memory-in-ats.html
This document covers aspects of calling back and forth between ATS and C code: https://docs.google.com/document/d/1W6DYQApEqKgyBzMbvpCI87DBfLdNAQ3E60u1hUiMoU0
The main downside is that dependently-typed programming is still a daunting prospect, even for non-systems programming. The syntax of the language is also a bit weird: consider lexical quirks such as the use of abst#ype as a keyword. Finally, ATS is to some degree a research project, and I personally don't know whether it would be sensible to adopt for a commercial endeavour.
Theoretically, it's going to be Fortran: less indirection (as in: my array is [here], not just a pointer to here, and this is true of most but not all of your data structures and variables).
However... There are many gotchas and quirks in Fortran: not, perhaps, as many as in C but you probably know your way around C rather better than Fortran. Which is the point behind most of the comments saying 'Know your code' - but do you really know what your compiler is doing?
Knowing you, I'm prepared to take it on trust that you do, for C. Most programers don't. You do not know and cannot know what a local JVM or JIT compiler does, and that's a black hole in your security model if you're using Java or C# r scripting languages.
Ignore anyone who tells you that the hairy-chested he-men of secure computing write their own assembler: they probably don't even know the security errors they're making in any and all nontrivial projects they release. Know your compiler, indeed.
You could write it in lua - providing a C API to a Lua library is relatively straight forward. C++ is also an option, though of course you'd have to write C wrappers and make sure no exceptions can escape your functions. But honestly, if it's security critical the minor inconveniences of the C language shouldn't be that much of a big deal. What you really should be doing is prove the correctness of your program where feasible, and test extensively where it's not.
You can write a library in Java. JNI is normally used to call C from Java, but it can be used the other way around.
There is finally a decent answer to this question: Rust.
I'm searching for something on the level of GNU extensions for C, but a little beyond (some basic data structure management). Best would be something BSD/MIT licensed.
If there is something for just strings containing GNU extensions equivalents plus adding some more it would be great.
I would prefer something that can be simply compiled into a project (no external libraries) based totally on the C standard (ANSI C89 if possible).
Edit: its for an OpenSource project that has a weird license, so no GPL code can't be added and working with plain K&R/ANSI C is pure pain.
This question already seems to be addressed here.
I actually wrote a somewhat lengthy response (recommending Glib and mentioning that Lua since 5.0 is MIT, not BSD), however my machine crashed half-way through :(
Thinking outside the box, a viable but off-the-wall approach is to use Lua. It is small, written in the subset of ANSI C that also happens to be valid C++, and supplies a rich garbage collected environment for strings, and associative arrays.
It can be built as a shared library, but it can also be statically linked.
It can admittedly feel a tad verbose when driving its data types entirely from the C side, but it is easy to move some of the higher level logic of your application into the Lua side where its data just works. Its VM is highly tuned, allowing it to perform better than one would expect for an interpreted scripting language, and there is a JIT compiler available as well for those times when its existing VM just isn't quite fast enough.
It is also open source and MIT licensed.
I'm looking for a C library that provides generic collection data structures such as lists, associative arrays, sets, etc. The library should be stable and well tested. I'm basically looking for something better than the crappy C standard library.
What C libraries fit this description?
EDIT: I'd prefer that the library was cross-platform, but failing that, anything that works on Mac/Linux.
if you are on linux, i would use GLIB. it provides all the helper libraries you would expect, plus its very well maintained and very active (always a +). the link above is straight to the section on data structures, but there is plenty of other things in GLIB so take a look around!
Apache Portable Runtime has collections (e.g. hash table, linked list) and is, well, portable.
Is there a standard way of reading a kind of configuration like INI files for Linux using C?
I am working on a Linux based handheld and writing code in C.
Otherwise, I shall like to know about any alternatives.
Final update:
I have explored and even used LibConfig. But the footprint is high and my usage is too simple. So, to reduce the footprint, I have rolled out my own implementation. The implementation is not too generic, in fact quite coupled as of now. The configuration file is parsed once at the time of starting the application and set to some global variables.
Try libconfig:
a simple library for processing structured configuration files, like this one: test.cfg. This file format is more compact and more readable than XML. And unlike XML, it is type-aware, so it is not necessary to do string parsing in application code.
Libconfig is very compact — a fraction of the size of the expat XML parser library. This makes it well-suited for memory-constrained systems like handheld devices.
The library includes bindings for both the C and C++ languages. It works on POSIX-compliant UNIX and UNIX-like systems (GNU/Linux, Mac OS X, Solaris, FreeBSD), Android, and Windows (2000, XP and later)...
No, there isn't one standard way. I'm sorry, but that is probably the most precise answer :)
You could look at this list of Linux configuration file libraries, though. That might be helpful.
Here are four options:
Iniparser
libini
sdl-cfg
RWini
If you can use the (excellent, in any C-based application) glib, it has a key-value file parser that is suitable for .ini-style files. Of course, you'd also get access to the various (very nice) data structures in glib, "for free".
There is an updated fork of iniparser at ccan, the original author has not been able to give it much attention over the years. Disclaimer - I maintain it.
Additionally, iniparser contains a dictionary that is very useful on its own.
If you need a fast and small code just for reading config files I suggest the inih
It loads the config file content just once, parse the content and calls a callback function for each key/value pair.
Really small. It can be used on embedded systems too.
I hate to suggest something entirely different in suggesting XML, but libexpat is pretty minimal, but does XML.
I came to this conclusion as I had the same question as you did, but then I realized the project already had libexpat linked-in--and I should probably just use that.