Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 5 years ago.
Improve this question
I'm looking for a library in C that will do optimization of an objective function (preferrably Levenberg-Marquardt algorithm) and will support box constraints, linear inequality constraints and non-linear inequality constraints.
I've tried several libraries already, but none of them do employ the necessary constraint types for my application:
GNU GSL (does not support constraints at all)
cMPFIT (only supports box constraints)
levmar (does not support non-linear constraints at all)
I am currently exploring NLopt, but I'm not sure if I can achieve a least-squares approach with any of the supplied algorithms.
I find it hard to believe that there's not a single library supporting the full range of constraints in this problem, so I guess I did a mistake somewhere while googling.
I recently discovered I can call Matlab functions from C. While that would solve the problem quite easily, I don't want to have to call Matlab functions from C. It's not fast in my experience.
Any help will be greatly appreciated.
Some time ago I was researching the state of C/C++ least squares fitting libraries. I noted down a few links, including the ones you gave and also:
ALGLIB/optimization -- Lev-Mar with boundary constraints.
WNLIB/wnnlp -- a constrained non-linear optimization package in C (general optimization, not least squares). Constraints are handled by adding a penalty function.
I haven't used any of the libraries yet, but NLopt seems the most promising for me. It would be great if it had specialized interface and algorithms for (weighted) least-squares fitting.
BTW, does your note about Matlab mean that it has Lev-Mar with non-linear constraints?
The approach I finally followed is the following:
I used NLopt for the optimization and the objective function was constructed to compute the squared error of the problem.
The algorithm that showed the most promising results was COBYLA (Local derivative-free optimization). It supports box constraints and non-linear constraints. The linear inequity constraints were introduced as non-linear constraints, which should be generally feasible.
Simple benchmarking shows that it does converge a little slower than a Lev-Mar approach, but speed is sacrificed due to the need for constraints.
MPFIT: A MINPACK-1 Least Squares Fitting Library in C
MPFIT uses the Levenberg-Marquardt technique to solve the least-squares problem. In its typical use, MPFIT will be used to fit a user-supplied function (the "model") to user-supplied data points (the "data") by adjusting a set of parameters. MPFIT is based upon MINPACK-1 (LMDIF.F) by More' and collaborators.
http://cow.physics.wisc.edu/~craigm/idl/cmpfit.html
OPTIF9 can be converted to C (from Fortran) and may already have been by somebody.
If what you mean by box constraints is that it supports upper and lower limits on parameter values, I believe there is a version that does that.
That is a tricky problem, because it means whenever a parameter gets to a boundary, it effectively reduces the degrees of freedom by 1.
It can get "stuck on a wall" when you didn't really want it to.
What we've found is that it's better to use an unconstrained minimizer and transform parameters, via something like a log or logit transform, so that in the search space they are unconstrained, but in the model space they are constrained.
As far as the other types of constraints, I don't know, although one option is, as part of your objective function, to make it get really bad when constraints are violated, so the optimizer avoids those areas.
I've found when I have a really flexible set of constraints, if I want a good trouble-free algorithm, I use Metropolis-Hastings.
Unless I'm wrong, if it generates a sample that violates constraints, you can simply discard the sample.
It takes longer, but it's simple and always works.
Related
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 8 years ago.
Improve this question
I used to program in Java and Python earlier, but recently have started using MATLAB for lots of stuff (specifically computer vision algorithms).
However MATLAB has indexing of arrays beginning from 1 instead of 0, which is the norm in almost every programming languages I have encountered so far.
The reason 0-based indexing made sense to me was like this:
In every processor I have seen, the address index begins with 0x00000000 say in an 8-bit processor and If I want to save 5 numbers, they would be stored in address 0x00000000 to 0x00000004 . Hence it makes sense to have indexing from 0 in programming languages.
While I was searching for this question I found List of 1-indexed programming languages? and http://en.wikipedia.org/wiki/Comparison_of_programming_languages_(array) to be useful.
For me it is not a very convenient notation, maybe because I used to code in other languages before. But I still don't understand why MATLAB (and even Julia now) has a 1-based array indexing and what advantages it provides. Can anyone list out the advantages?
Why does it have 1-based indexing? Historical reasons. (Cleve Moler decided so). It probably has to do with 1-based indexing being the convention for matrix notation.
Why does it not implement 0-based indexing too?
Up for some nice light reading?
In particular, look for comments from Cleve Moler (the creator of MATLAB) and Steve Lord (MathWorks Engineer). After much exchange, the basic reasoning (at least in 2001) for not implementing a supplementary 0-based indexing was not for any philosophical or compatibility reasons, although it started as a backward compatibility argument, but because it would require just too darn much effort to update MATLAB built-in functions:
I agree that it would be possible to add zero-based indexing to MATLAB,
using either new notation or an object that overloaded all of our
existing indexing notation. It could be backwards compatible in the
sense that old code that didn't know about or use the new stuff would still
work.
But here is the key point: none of our existing code would work with
the new object. You couldn't plot it; you couldn't print it; you
certainly couldn't do any matrix operations with it. Yes, over time,
everything could be rewritten to handle the extension, but that
is what we want to avoid. It would be like AM and FM radio -- two
systems living in the same box, but with separate and independent
underlying technology.
-- Cleve Moler
moler#mathworks.com
Translation: They don't want to write all new plot, print, etc. routines.
Matlab is intended as a way of making the functionality of programming languages accessible to people who haven't learnt to code. Having indices start at 0 can be a common source of bugs for people not familiar with programming, and the whole point of Matlab is to enable people to apply maths they understand to problems they need to solve, without much additional knowledge.
Most of the tasks that Matlab is used for, e.g. physics and engineering, use the convention of indices starting at 1 in most of the literature, so people working in these fields do not need to convert their algorithms by subtracting 1 from everything.
In addition to this, Matlab is short for "matrix laboratory", and in mathematics, the convention is typically to index matrices starting at 1, so it kind of makes sense that they will follow this convention.
Your rationale for why it makes sense to start at 0 is perfectly good, but it's the purpose of a high-level language like matlab to obscure aspects like these that the user does not need to know.
I want to code a genetic algorithm in C for optimizing a function of 10 variables (x1 to x10). However I am not able to figure out which encoding I should use. I have mostly seen binary encoding being used in example but the variables in my case can take real values. Also, is value encoding a good option for these types of problems?
For real valued problems I would suggest to try CMA-ES or another ES variant. CMA-ES certainly is the current state of the art for real-valued problems. It is designed to find good solutions in multidimensional problems quickly. There are implementations available on Hansen's page. There's also a C# implementation in the work for HeuristicLab. Evolution strategies are algorithms that were specifically designed for real-valued optimization problems. They are very similar to genetic algorithms (both were invented around the same time, but in different places). The main distinction is that for ES the main driver is mutation and it features a clever adaption of the mutation strength. Without this adaption the (local) optimum cannot be located in time. CMA-ES is easy to configure, all it needs is the initial standard deviation and optionally the population size (otherwise there's a formula that estimates this given the problem size).
Genetic algorithms can of course also be applied, but you have to use some specific operators which are able to mutate variables only with very small degree. For example there's the Breeder Genetic Algorithm from Mühlenbein. In general however genetic algorithms are more suited for problems that need a right combination of things. E.g. which items to include in a knapsack problem or which functions and terminals to combine to a formula (genetic programming). Less for problems, where you need to find the right value for something. Although of course there are variants of the genetic algorithm to solve these, look for Real coded Genetic Algorithm (RCGA or RGA).
Another algorithm suited for real-valued problems is Particle Swarm Optimization, but in my opinion it is harder to configure. I'd start with SPSO-2011 the 2011 standard PSO.
If your problem contains integer variables choices become more difficult. Evolution strategies do not perform so well when variables are discrete, because the adaptation schemes for integer variables are different. A genetic algorithm becomes an interesting first-choice algorithm again.
A genetic algorithm is best used when two answers that are pretty close to optimal will make something else pretty close to optimal when combined. The problem with a pure binary encoding is that if you don't check your crossover you end up getting two answers which may not have all that much to do with the original answers.
That said, this is only really an issue if your number of variables is very small and the amount of data in your variables is large. As far as picking an encoding, it's more of an art than a science and it depends on your problem. I would suggest going with an encoding that fits the amount of precision you want. With 10 variables you won't got that far wrong however you encode it, an 8-bit ASCII encoder would probably work fine.
Hope that helps.
I'm looking for a tool that can statically discover invariants in C programs. I checked out Daikon but it discovers invariants only dynamically.
Is there a tool available for what I'm looking for? Thanks!
See The SLAM project: debugging system software via static analysis. It claims to infer invariants statically, for just what you asked for, the C language. The author, Tom Ball, is widely known for stellar work in program analysis.
If you mean "invariant" in the widest sense, as the linked page to Daikon is using, then the work of many static analysis tools can be described as "discovering invariants", just perhaps not the expressive invariants you were looking for.
Frama-C's value analysis accumulates its results, the possible values of all variables, for each statement. At the end of the analysis, it can thus present non-relational information about the domain variation of each variable in the program, at each statement. In this screenshot, an invariant is that S is always 0, 1, 3 or 6 just before the selected instruction, for all executions of this deterministic program.
The two hidden parameters in your question are the shape of the invariants you are interested in, and the shape of the programs you want to find these invariants for. For instance, SLAM, mentioned in Ira's answer, was designed to work on device driver code, and to infer invariants that just contain the necessary information for verifying proper use of system APIs. Another tool, Astrée, is famous for doing a very good job at inferring just the right invariants to demonstrate runtime safety of flight control software.
The two degrees of freedom make for a very large design space. You won't find anything that works for all kinds of C programs and infers all the invariants you might be interested in, but if you refine your question for specific application domains and kinds of invariants, you will have better chances to find relevant answers.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 3 years ago.
Improve this question
I'm doing some Project Euler problems and most of the time, the computations involve large numbers beyond int, float, double etc.
Firstly, I know that I should be looking for more efficient ways of calculation so as to avoid the large number problem. I've heard of the Bignum libraries.
But, for academics interests, I'd like to know how to code my own solution to this problem.
Can any expert please help me out? (My language is C)
You need to store the big numbers in a base that your computer can easily handle with its native types, and then store the digits in a variable length array. I'd suggest that for simplicity you start by storing the numbers in base 10 just to get the hang of how to do this. It will make debugging a lot easier.
Once you have a class that can store the numbers in this form, it's just a matter of implementing the operations add, subtract, multiply, etc. on this class. Each operation will have to iterate over digits of its operands and combine them, being careful to carry correctly so that your digits are never larger than the base. Addition and subtraction are simple. Multiplication requires a bit more work as the naive algorithm requires nested loops. Then once you have that working, you can try implementing exponentiation in an efficient manner (e.g. repeated squaring).
If you are planning to write a serious bignum implementation, base 10 won't cut it. It's wasteful of memory and it will be slow. You should choose a base that is natural for the computer, such as 256 or the word size (2**32). However this will make simple operations more difficult as you will get overflows if you naively add two digits, so you will need to handle that very carefully.
C is not a good choice for Project Euler. The benefits of C are raw speed, machine portability (to an extent, with standard C), language interoperability (if some language communicates with another, C is a popular first choice), sticking close to a specific library or platform's API (because C is common, e.g. OS API), and a stable language & stdlib. None of these benefits apply to solving Project Euler problems. Not even raw speed, because most of the problems aren't about raw computation, but understanding the algorithm required, and you can sit there all day and wait before submission.
If you are attempting Project Euler problems to broaden your experience with C, that's perfectly fine, just realize this experience doesn't necessarily apply to long-lived and real-world C projects you may work on.
For this kind of short, one-off problem those languages commonly described as "scripting languages" will work better, faster (in dev time), and easier. Try Python, it stays close to C in many ways, including a C API, and out of the various popular "scripting languages" is possibly the one for which you will find the most use in conjunction with C projects.
This may become an unpopular answer, but it isn't a rant—plus I really like C and use C/C++ often—and there is an explicit answer here to your problem: "don't use C", with your final large number solution depending on which alternative you choose. Again picking on Python, integers do not have an upper bound (note below), and I use this to naturally code answers to Project Euler problems, where in other languages I have to use a painful-by-comparison alternative number library.
(Python integers: There are two integer types in 2.x, 'int' and 'long' (which have been completely unified in 3.x). The conversion between them is practically seamless, and 'long' allows arbitrarily large values, instead of just being a bigger 'int' type as C's long is.)
A popular bignum library for C/C++ is the GNU MP Bignum Library. I've used it for several Project Euler problems, but fact remains that C isn't a very suitable language for Euler-problems. If performance was more important C would have more to give, but now you're much better off using a language which built in bignum support, such as Ruby (there are lots of others).
A simple way is to think of the number as its string representation in base b. Suppose b=10, simple arithmetic operation like addition on two such strings can be done using the same method we use when adding numbers by pen and paper. The same goes for other simple operations. For better results, you can take a larger base.
A simple bignum implementation like that should be enough for most Project Euler problems (probably all, but I haven't solved much at Euler so can't be sure), but there are ways of using much faster algorithms for operations such as multiplication and division/mod.
Although I recommend writing your own bignum for practice, if you are really stuck you can take ideas from the code of already implemented bigint libraries. For a serious implementation something like gmp is the obvious choice. But you cana also find small bigints coded by other people when solving similar practice problem online (e.g. Abednego's bigint.cpp).
Here's a nice and simple bignum module for C. You can learn from it for ideas. The C code isn't the highest quality, but the algorithm is well implemented and quite common.
For more advanced stuff, look up GMP.
If you want a nice C++ version (I know, you said C, but this is really interesting code), take a look at the internals of CGAL: http://www.cgal.org/
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 5 years ago.
Improve this question
What is the use of finite automata? And all the concepts that we study in the theory of computation. I've never seen their uses yet.
They are the theoretical underpinnings of concepts widely used in computer science and programming, and understanding them helps you better understand how to use them (and what their limits are). The three basic ones you should encounter are, in increasing order of power:
Finite automata, which are equivalent to regular expressions. Regular expressions are widely used in programming for matching strings and extracting text. They are a simple method of describing a set of valid strings using basic characters, grouping, and repitition. They can do a lot, but they can't match balanced sets of parentheses.
Push-down automata, equivalent to context-free grammars. Text/input parsers and compilers use these when regular expressions aren't powerful enough (and one of the things you learn in studying finite automata is what regular expressions can't do, which is crucial to knowing when to write a regular expression and when to use something more complicated). Context-free grammars can describe "languages" (sets of valid strings) where the validity at a certain point in parsing the string does not depend on what else has been seen.
Turing machines, equivalent to general computation (anything you can do with a computer). Some of the things you learn when you cover these enable you to understand the limits of computing itself. A good theory course will teach you about the Halting Problem, which enables you to identify problems for which it is impossible to write a program. Once you've identified such a problem, then you know to stop trying (or refine it to something that is possible).
Understanding the theory and limitations of these various computing mechanisms enable you to better understand problems and programs and think more deeply about programming.
There was a request-for-work published about a year ago on one of the freelance coding exchange sites asking, essentially, for a program which solved the Halting Problem. Several people responded with offers, saying they "understood the requirements" and could "start immediately". It was impossible to write a program which met the requirements. Understanding computing theory enables you to not be that bidder who demonstrates, in public, that he really doesn't understand computing (and doesn't bother to thoroughly investigate a problem before declaring understanding and making an offer).
Finite automata are very useful for communication protocols and for matching strings against regular expressions.
Automatons are used in hardware and software applications. Please read the implementation section here http://en.wikipedia.org/wiki/Finite-state_machine#Implementation
There is also a notion of Automata-based programming. Please check this http://en.wikipedia.org/wiki/Automata-based_programming
cheers
Every GUI, every workflow can be treated as a finite automata. Think of each page as a state and transitions occurring due to certain events. Perhaps you can't proceed to a certain page or the next stage of the workflow until a series of conditions are met.
Finite automata are e.g. used to parse formal languages. This means that finite automata are very usefull in the creation of compiler and interpreter techniques.
Historicaly, the finite state machine showed that a lot of problems can be solved by a very simple automate.
Try taking a compilers course. You will very likely make a compiler or interpreter using a finite state automaton to implement a recursive descent parser.
For example to manage states of some objects with defined life cycle.
As example of this: orders in book shop.
An order can have the following states:
-ordered
-payed
-shipping
-done
and program of the finite automata knows how one state can be changed by other.
The finite automata is a type of state machine (SM). In general SMs are used for parsing formal languages.
You can use as a formal language many entities, not only characters.
And regular language is a type of formal language.
There are some theory that show, what type of the SM is better to parse a regular language:
http://en.wikipedia.org/wiki/Regular_language