What are some common general purpose library as Boost is to C++ but for C? It should be a cross-platform library and include collections such as trees, linked-lists, queues, etc...
What are the advantages/disadvantages to those libraries?
glib is pretty nice because:
liberally licensed (LGPL)
constant development
tons of data structures
trees
lists
queues
caches
etc.
good documentation
lots of sample code
development "assistance"
logging
thread abstraction
thread pools
test framework
timers
Unicode support
many supported platforms
regular expressions
tons more...
The Apache portable runtime project
http://apr.apache.org/
is good. Covers basic datastructures and is very good at network and IO abstraction.
The later is a magnitude better then glib.
Unfortunately most document links on the apache websites are broken at the moment :-(
But this one works work http://apr.apache.org/docs/apr/1.4/modules.html
Related
I don't understand how BLAS, LAPACK and ATLAS are related and how I should use them together! I have been looking through all of their manuals and I have a general idea of BLAS and LAPACK and how to use them with the very few examples I find, but I can't find any actual examples using ATLAS to see how it is related with these two.
I am trying to do some low level work on matrixes and my primary language is C. First I wanted to use GSL, but it says that if you want the best performance you should use BLAS and ATLAS. Is there any good webpage giving some nice examples of how to use these (in C) all together? In other words I am looking for a tutorial on using these three (or any subset of them!). In short I am confused!
BLAS is a collection of low-level matrix and vector arithmetic operations (“multiply a vector by a scalar”, “multiply two matrices and add to a third matrix”, etc ...).
LAPACK is a collection of higher-level linear algebra operations. Things like matrix factorizations (LU, LLt, QR, SVD, Schur, etc) that are used to do things like “find the eigenvalues of a matrix”, or “find the singular values of a matrix”, or “solve a linear system”. LAPACK is built on top of the BLAS; many users of LAPACK only use the LAPACK interfaces and never need to be aware of the BLAS at all. LAPACK is generally compiled separately from the BLAS, and can use whatever highly-optimized BLAS implementation you have available.
ATLAS is a portable reasonably good implementation of the BLAS interfaces, that also implements a few of the most commonly used LAPACK operations.
What “you should use” depends somewhat on details of what you’re trying to do and what platform you’re using. You won't go too far wrong with “use ATLAS + LAPACK”, however.
While ago, when I started doing some linear algebra in C, it came to me as a surprise to see there are so few tutorials for BLAS, LAPACK and other fundamental APIs, despite the fact that they are somehow the cornerstones of many other libraries. For that reason I started collecting all the examples/tutorials I could find all over the internet for BLAS, CBLAS, LAPACK, CLAPACK, LAPACKE, ATLAS, OpenBLAS ... in this Github repo.
Well, I should warn you that as a mechanical engineer I have little experience in managing such a git repository or GitHub. It will first seem as a complete mess to you guys. However if you manage to get over the messy structure you will find all kind of examples and instructions which might be a help. I have tried most of them, to be sure they compile. And the ones which do not compile I have mentioned. I have modified many of them to be compilable with GNU compilers (gcc, g++ and gfortran). I have made MakeFiles which you can read to learn how you can call individual Fortran/FORTRAN routines in a C or C++ program. I have also put some installations instructions for mac and linux (sorry windows guys!). I have also made some bash .sh files for automatic compilation of some of these libraries.
But going to your other question: BLAS and LAPACK are rather APIs not specific SDKs. They are just a list of specifications or language extensions rather than an implementations or libraries. With that said, there are original implementations by Netlib in FORTRAN 77, which most people refer to (confusingly!) when talking about BLAS and LAPACK. So if you see a lot of strange things when using these APIs is because you were actually calling FORTRAN routines in C rather than C libraries and functions. ATLAS and OpenBLAS are some of the best implementations of BLAS and LACPACK as far as I know. They conform to the original API, even though, to my knowledge they are implemented on C/C++ from scratch (not sure!). There are GPGPU implementations of the APIs using OpenCL: CLBlast, clBLAS, clMAGMA, ArrayFire and ViennaCL to mention some. There are also vendor specific implementations optimized for specific hardware or platform, which I strongly discourage anybody to use them.
My recommendation to anyone who wants to learn using BLAS and LAPACK in C is to learn FORTRAN-C mixed programming first. The first chapter of the mentioned repo is dedicated to this matter and there I have collected many different examples.
P.S. I have been working on the dev branch of the repository time to time. It seems slightly less messy!
ATLAS is by now quite outdated. It was developed at a time when it was thought that optimizing the BLAS for various platforms was beyond the ability of humans, and as a result autogeneration and autotuning was the way to go.
In the early 2000s, along came Kazushige Goto, who showed how highly efficient implementations can be coded by hand. You may enjoy an interesting article in the New York Times: https://www.nytimes.com/2005/11/28/technology/writing-the-fastest-code-by-hand-for-fun-a-human-computer-keeps.html.
Kazushige's on the one hand had better insights into the theory behind high-performance implementations of matrix-matrix multiplication, and on the other hand engineered these better. His approach, which on current CPUs is usually the highest performing, is not in the search space that ATLAS autotunes. Hence, ATLAS is inherently inferior. Kazushige's implementation of the BLAS became known as the GotoBLAS. It was forked as the OpenBLAS when he joined industry.
The ideas behind the GotoBLAS were refactored into a new implementation, the BLAS-like Library Instantiation Software (BLIS) framework (https://github.com/flame/blis), which implements the same algorithms, but structures the code so that less needs to be custom implemented for a new architecture. BLIS is coded in C.
What this discussion shows is that there are many implementation of the BLAS. The BLAS themselves are a de facto standard for the interface. ATLAS was once the state of the art. It is no longer.
As far as I'm aware, and after working through the ATLAS repository, it seems that it includes a re-implementation of BLAS in C. There's bit more to it than that but I hope it answers the question.
I'm looking for a "core" C cross-platform toolbox, because I need lots of "useful" functions (read/write ini files, network routines, arrays, lists...)
I was thinking about GLib, may be Qt (the core part) but I would appreciate a pure "C" stuff
I think glib is a fantastic choice, but it's perhaps somewhat less widely scoped than what you're after, there is no networking in glib as far as I know.
For that, you need to add GIO from the same family of libraries.
Apache Portable Runtime (APR) may be the solution you are looking for. It is used, among other projects, for the Apache http server and Subversion.
I think glib is great. It comes with a great deal of core algorithms and data types. There are many libraries build around GLib and GObject. You have indeed GIO for all kinds of different input and output on files, over network etc (will remind you a bit of the Java like IO operations), and you gtk for Gui applications. All These libraries were designed with portability in mind and extendibility to other languages (eg. python, perl etc). But learn to use GOjbect as well, because the best work comes from using these libraries ( http://www.gtk.org/documentation.php ) together.
I'm looking to implement 'branch and bound' over a cluster (like Amazon's say), as I want it to be horizontally scalable, not limited to a single CPU. There's a paper "Task Pool Teams: A Hybrid Programming Environment for Irregular Algorithms on SMP Clusters" by Judith Hippold and Gudula Runger. It's basically a bottom-up, task-stealing framework like Intel's TBB, except for ad-hoc networks instead of shared memory. If this library was available I'd use it (replacing the local, threaded part with TBB). Unfortunately they don't seem to have made it available for download anywhere that I could find, so I wonder are there other implementations, or similar libraries out there?
It doesn't look like Microsoft's Task Parallel Library has the equivalent, either, to steal from.
(I tried to make a tag 'taskpool' after 'threadpool', the most-used variant (before 'thread-pool') but, didn't have enough points. Anyone heavy enough think it's worth adding?)
edit:
I haven't tried it yet, but it PEBBL (under here: software.sandia.gov/trac/acro/wiki/Packages) claims to scale really high. The paper that the answerer mentions from the Wiley book 'Parallel Branch-and-Bound Algorithms', Crainic, Le Cun and Roucairol, 2006, from "Parallel Combinatorial Optimization", 2006 edited by El-Ghazali Talbi was where I found it, and there are other libraries listed; some may be better, I reserve the right to update this :). Funny that Google didn't find these libs, either my Googling was weak or Google itself fails to be magic sometimes.
When you say "over a cluster" it sounds like you mean distributed memory, and parallelizing branch and bound is a notoriously difficult problem for distributed memory - at least in a way that guarantees scalability. The seminal paper on this topic is available here, and there's an excerpt from a Wiley book on the topic here.
Shared memory branch is bound is an easier problem because you can implement a global task queue. A good high level description of how to do both shared memory and message passing implementations is available here. If nothing else, the references section is worth purusing for ideas and existing implementations.
One thing you might consider is investigating shared message queues like RabbitMQ. It is a AMQP server (a messaging protocol developed so distributed applications can send messages to each other).
you basically need some kind of distributed synchronization/queue
I suggest looking into armci as a low-level distributed memory interface with synchronization and build on top of that.
Alternative is to allocate mpi process as Master to distribute work allocation.
http://www.cs.utk.edu/~dongarra/ccgsc2008/talks/Talk10-Lusk.pdf
I have heard of Boost and ACE as two of the well known C++ libraries. What are the other good C/C++ libraries available?
Does Boost and ACE support session management for web applications written in C/C++?
EDIT: Ok I will try to be domain specific. I am looking for a C/C++ library which could help me maintain session state for a C++ based CGI web application.
When you're trying to build a web application in C++ I'd recommend Wt, a Qt-like framework for creating web applications in C++.
It handles sessions either in one process per session (when security matters) or multiple sessions per process.
You can either use the built-in webserver or use it with any webserver that supports FastCGI.
(Also, I'd recommend it over Boost.CGI as it seems to be maintained and feature-complete).
Depends if you are talking about general purpose or domain specific libraries. For general purpose Boost is best of breed (and don't forget about the good old STL), so I don't see the point of looking for something else that will cover much of the same ground, but is not as polished. As for domain specific you'd have to specify the domain :-)
If you're interested in C (not C++) as well, glib (the Gnome project's utility library) provides a number of useful data structures and constructs.
C++ has libraries for anything you could imagine, so the scope of your question is rather undefined. What interests you? Web applications, scientific programs, GUIs? Specify what you need exactly if you want a good answer.
Boost is a general-purpose library for relatively low-level things. It's rather complex and advanced though, so you should have a good grasp of C++ before you start with it. ACE is mainly for synchronization and communication between threads/processes/applications.
If web applications is what you need, I recommend you to strongly consider the language you're picking. C++ may not be the best direction to go here, unless you have very specific constraints that force your hand.
There is also GTK which is good if you need to have a gui or use unicode. (although c++0x should have better unicode support natively when the standard is complete).
Boost doesn't yet support sessions, but a CGI library has been proposed which should have sessions.
If you want to use C++ for web applications, consider using CGICC
Ok I will try to be domain specific. I am looking for a C/C++ library which could help me maintain session state for a C++ based CGI web application.
CppCMS?
Example of session management: http://cppcms.sourceforge.net/wikipp/en/page/tut_sessions
Reference: http://cppcms.sourceforge.net/wikipp/en/page/ref_cppcms_session_iface
Configuration: http://cppcms.sourceforge.net/wikipp/en/page/ref_config#sessions
Poco is an excellent C++ Library with data access, xml, networking, compression and crypto all wrapped up in once nice little package.
Boost evidently, QT for GUI (that's not clearly a library I know), Electronic Arts Standard Template Library and
Blitz++ if you want to do scientific computation :
Blitz++ is a C++ class library for
scientific computing which provides
performance on par with Fortran 77/90.
The C++ programming language offers
many features useful for tackling
complex scientific computing problems:
inheritance, polymorphism, generic
programming, and operator overloading
are some of the most important.
Unfortunately, these advanced features
came with a hefty performance
pricetag: until recently, C++ lagged
behind Fortran's performance by
anywhere from 20% to a factor of ten.
As a result, the adoption of C++ for
scientific computing has been slow.
Is there a way to soup up C++ so that
we can keep the advanced language
features but ditch the poor
performance? This is the goal of the
Blitz++ project: to develop techniques
which will enable C++ to rival -- and
in some cases even exceed -- the speed
of Fortran for numerical computing,
while preserving an object-oriented
interface. The Blitz++ Numerical
Library is being constructed as a
testbed for these techniques.
Recent benchmarks show C++ encroaching
steadily on Fortran's high-performance
monopoly, and for some benchmarks, C++
is even faster than Fortran! These
results are being obtained not through
better optimizing compilers,
preprocessors, or language extensions,
but through the use of template
techniques. By using templates
cleverly, optimizations such as loop
fusion, unrolling, tiling, and
algorithm specialization can be
performed automatically at compile
time.
Another goal of Blitz++ is to extend
the conventional dense array model to
incorporate new and useful features.
Some examples of such extensions are
flexible storage formats, tensor
notation and index placeholders.
I don't ever recall coming across anything for C that's as extensive as C++'s Boost library.
You can have a look at glib. It provides quite a few interesting things like containers, unicode support, threading...
Have a look at its documentation
I've never seen anything. Really, C++ templates are the enabling technology for STL and much of boost, and C doesn't have them. I've seen many C-interface libraries for things like image processing, 3D graphics, networking, etc., but never a real general-purpose library like boost. You can find pieces though - threading libraries, string libraries, etc. - that are similar to sections of boost.
Depending on what exactly you're trying to do there is APR (The Apache Portable Runtime library) which is what the Apache http daemon is built on
http://apr.apache.org/
For GUI, there is wxWidgets (formerly wxWindows)
http://www.wxwidgets.org/
Glib does compensate it partially in form of various unicode, string types. If you add Gobject you could get some object programming even some garbage collection.
ACE (Adaptive Communication Environment) is sometimes mentioned. It's not quite an apples-to-apples comparison. Boost provides more "basic building blocks" whereas ACE provides more of an tightly integrated framework geared towards telco products. I have used both extensively and I find Boost vastly superior. One big advantage of Boost is that many of its features will show up in the STL extensions for the upcoming C++0x standard (see, for example, http://en.wikipedia.org/wiki/Technical_Report_1).
or you can look at ccan http://ccan.ozlabs.org/
but no, there really isn't any complete package, glib comes closest