I'm currently working on tranforming logical queries for database systems from a DNF form into a CNF form, focussed on queries which have a similar form to
(a and b and c and d and e1) or (a and b and c and d and e2) or (a and b and c and d and e3),
into
a and b and c and d and (e1 or e2 or e3)
I expected there to already be an algorithm for that, but I am currently working with z3 and the only way to form a CNF out of that I saw was with tseitin or similar, which includes extra variables, which are a problem as they can't be used for queries properly.
Using z3 for this purpose is probably not the best option. While there's an internal converter, it's tuned to be used in conjunction with the rest of the solver. If you don't have a need to find a satisfying model, you shouldn't be using a SAT/SMT solver.
Conversion from DNF to CNF can produce exponentially large formulas, and this is the reason why most practical applications will use Tseytin transformation to reduce this complexity; which adds extra variables. If you don't want this, you should use the Quine-McCluskey algorithm, which has worst case exponential behavior. But if your formulas are small enough to start with, it may not matter.
You can code this algorithm yourself, or if you're open to using SymPy, there's an existing implementation (with possible simplifications). See https://docs.sympy.org/latest/modules/logic.html?highlight=to_cnf#sympy.logic.boolalg.to_cnf
Related
I am moving from python to C, in the hope of faster implementation, and trying to learn vectorization in C equivalent to python vectorization. For example, assume that we have binary array Input_Binary_Array, if I want to multiply each element for the index, say, i, by 2**i and then sum all non-zero, in python-vectorization we do the following:
case 1 : Value = (2. ** (np.nonzero(Input_Binary_Array)[0] + 1)).sum()
Or if we do slicing and do elementwise addition/subtraction/multiplication, we do the following:
case 2 : Array_opr= (Input_Binary_Array[size:] * 2**Size -Input_Binary_Array[:-size])
C is a powerful low-level language, so simple for/while loop is quite faster, but I am not sure that there are no equivalent vectorizations like python.
So, my question is, is there an explicit vectorization code for:
1.
multiplying all elements of an array
with a constant number (scalar)
2.
elementwise addition, subtraction, division for 2 given arrays of same size.
3.
slicing, summing, cumulative summing
or, the simple for, while loop is the only faster option to do above operations like python vectorization (case 1, 2)?
The answer is to either use a library to achieve those things, or write one. The C language by itself is fairly minimalist. It's part of the appeal. Some libraries out there include the Intel MLK, and there's gsl, which has that along with huge number of other functions, and more.
Now, with that said, I would recommend that if moving from Python to C is your plan, moving from Python to C++ is the better one. The reason that I say this is because C++ already has a lot of the tools you would be looking for to build what you like syntactically.
Specifically, you want to look at C++ std::vector, iterators, ranges and lambda expressions, all within C++20 and working rather well. I was able to make my own iterator on my own sort of weird collection and then have Linq style functions tacked onto it, with Linq semantics...
So I could say
mycollection<int> myvector = { 1, 2, 4, 5 };
Something like that anyway - the initializer expression rules I forget sometimes.
auto v = mycollection
.where( []( auto& itm ) { itm > 3; } )
.sum( []( auto& itm ) { return itm; } );
and get more or less what I expect.
Since you control the iterator down to every single detail you could ever want (and the std framework already thought of many), you can make it go as fast as you need, use multiple cores and so on.
In fact, I think STL for MS and maybe GCC both actually have swap in parallel algorithms where you just use them.
So C is good, but consider C++, if you are going that "C like" route. Because that's the only way you'll get the performance you want with the syntax you need.
Iterators basically let you wrap the concept of a for loop as an object. So,
So, my question is, is there an explicit vectorization code for:
1.
multiplying all elements of an array
with a constant number (scalar)
The C language itself does not have a syntax for expressing that with a single simple statement. One would ordinarily write a loop to perform the multiplication element by element, or possibly find or write a library that handles it.
Note also that as far as I am aware, the Python language does not have this either. In particular, the product of a Python list and an integer n is not scalar multiplication of the list elements, but rather a list with n times as many elements. Some of your Python examples look like they may be using Numpy, which can provide for that sort of thing, but that's a third-party package, analogous to a library in C.
elementwise addition, subtraction, division for 2 given arrays of same
size.
Same as above. Including this not being in Python, either, at least not as the effect of any built-in Python operator on objects of a built-in type.
slicing, summing, cumulative summing
C has limited array slicing, in that you can access contiguous subarrays of an array more or less as arrays in their own right. The term "slice" is not typically used, however.
C does not have built-in sum() functions for arrays.
or, the simple for, while loop is the only faster option to do above
operations like python vectorization (case 1, 2)?
There are lots and lots of third-party C libraries, including some good ones for linear algebra. The C language and standard library does not have such features built in. One would typically choose among writing explicit loops, writing a custom library, and integrating a third party library based on how much your code relies on such operations, whether you need or can benefit from customization to your particular cases, and whether the operations need to be highly optimized.
I've designed a parser in C that is able to generate AST, but when I begin to implement simplifications it really got messed up. I've successfully implemented rules for the summation below;
x + 0 -> x
x + x -> 2 * x
etc.
But it took huge amount of effort and code to do it. What I did was to search entire tree and try to find a pattern that I can use (lots of recursion) then if there was a cascade of PLUS nodes, I've added them to a list, then worked on that list (summing numbers and combining variables etc.) then I created another tree from that list, and merged it to existing one. It was this paper I used to implement it. In short given the expression 2*x+1+1+x+0 I got 3*x+2. And it was just summation that got me into so much trouble, I can even imagine the advanced stuff. So I realized I was ding something wrong.
I've read this thread but I'm really confused about term rewriting systems (what it really is, how to implement in C).
Is there a more general and effective way to do simplification on AST? Or how to write a term rewriting system in C
Term rewriting is (in simple words) like the 2 examples you provided. (How to convert x + 0 to x in a AST?). It is about pattern matching on AST's, and once there is a match, a conversion of an equivalent expression. It is also called a term rewriting rule.
Note that having a term rewriting rule is not the absolute or general solution of algebraic simplification. The general solution involves having many rewriting rules (you showed two of them), and apply them in a given AST repeatedly until no one success.
Then, the general solution involves the process or coordination on the application of the rewriting rules. i.e. in order to avoid the re-application of a rule that has previously failed, as an example.
There is not a unique way to do it. There are several systems. For proprietary systems it is not known because they keeps it in secrecy, but there are open source systems too, for example Mathomatica is written in C.
I recommend you to check the open system Fōrmulæ. In this, the process of coordination of rewriting rules (which is called "the reduction engine") is relatively simple. It is written in Java. The advantage of this system is that rewriting rules are not hardwired/hardcoded in the system or the reduction engine (they are hot pluggable). Coding a rewriting rule involves the process of pattern matching and conversion, but no when or how it will be called (it follows the Hollywood principle).
In the specific case of Fōrmulæ:
The reduction engine is based (in general terms) on the post-order tree traversal algorithm. so when a node is "visited", its sub-nodes were already visited and (possibly) transformed, but it is possible to alter such that flow (i.e. to solve the unwanted referentiation of a variable in an assignment x <- 5). Note that it is not just a tree traversal, the AST is being actually changed in the process.
In order to efficiently manage the (possibly hundred or thousand) of rewriting rules, every rule has a type of expression where it is applicable, and when a single node is "visited", only the associated rules are checked for a match. For example, your 2 rules can only be applied to "addition" nodes of an AST.
Rewriting rules are not limited to algebraic simplification, they can be used in many other fields such as programming (Fōrmulæ is also its programming language, see examples of Fōrmulæ programs, or in automatic or assisted theorem proving.
I need to solve a linear equation (Ax = b), with a large sparse A and with différents b multiple time. Thus, an LU (or Cholesky if A is symmetric), factorisation is highly preferable.
I'm using the armadillo library and I've heard that one can use the function:
spsolve(x, A, b, "superlu");
In order to solve such a system. I'm not very concerned about retrieving the L and U matrices. However, it is of primal importance that both L and U aren't recomputed every time I call spsolve.
Does spsolve(x, A, b, "superlu") store the LU decomposition and if not, is there a way to retrieve said matrices?
The fact is that armadillo is, essentially, a "wrap-up" for software developed by the other teams; this particularly is correct for the superlu sparse solver. The feature that you are asking for (solving a series of systems of equations with one and the same matrix but different right-hand sides) may not exist in armadillo at all. You should probably use a sparse linear solver directly (not necessarily superlu) that has that feature embedded. In case if you system is very large, so a factorisation-based solver cannot cope with that, an iterative solver may do, and there is an option to go in such case: since modern CPUs are multi-core, several independent solution processes can be run in parallel. One such iterative solver is described in the following blog (you may ask questions and/or participate in discussion there): http://comecau.blogspot.com/2018_09_05_archive.html
I have a task to implement Toom-Cook 3-way multiplication algorithm. I'm following description on wikipedia http://en.wikipedia.org/wiki/Toom%E2%80%93Cook_multiplication , and I managed to store two big numbers into strings and split the strings into smaller ones according to the "Splitting" step on the wikipedia page. The next step is "evaluation", and I have to calculate a new number p0 = m0 + m2 ("Faster evaluation" by Bordrato - found on the same page) where m0 and m2 are the digits which I created by splitting the large number (in the previous step). The problem is that I cannot simply add up m0 and m2, since those two numbers can still be very large and impossible to add together in a standard way. Does this mean that I have to implement my own algorithm for adding large numbers (as well as substracting and dividing, since they are also needed), or am I missing something? If anyone could link me a possible implementation or even a pseudo code, it would be appreciated.
You have to implement your own methods for addition, subtraction, modulo, etc. Sometime ago I was trying to implement a BigInteger library and I have found some resources that may be useful for you.
BigNum Math book (as pointed by the previous answer)
Java OpenJdk
BigInteger implementation, with documentation
Algorithms and data structures The basic toolbox, (I have learned Karatsube of this book).
By the way, I recommend to use base 2 for your numbers(see here.) because you can take advantage of the nature of the computer to make your operations more easy and fast.
LibTomMath is open source and includes a Toom-Cook multiplication; have a look.
I am seeking advice on how to incorporate C or C++ code into my R code to speed up a MCMC program, using a Metropolis-Hastings algorithm. I am using an MCMC approach to model the likelihood, given various covariates, that an individual will be assigned a particular rank in a social status hierarchy by a 3rd party (the judge): each judge (approx 80, across 4 villages) was asked to rank a group of individuals (approx 80, across 4 villages) based on their assessment of each individual's social status. Therefore, for each judge I have a vector of ranks corresponding to their judgement of each individual's position in the hierarchy.
To model this I assume that, when assigning ranks, judges are basing their decisions on the relative value of some latent measure of an individual's utility, u. Given this, it can then be assumed that a vector of ranks, r, produced by a given judge is a function of an unobserved vector, u, describing the utility of the individuals being ranked, where the individual with the kth highest value of u will be assigned the kth rank. I model u, using the covariates of interest, as a multivariate normally distributed variable and then determine the likelihood of the observed ranks, given the distribution of u generated by the model.
In addition to estimating the effect of, at most, 5 covariates, I also estimate hyperparameters describing variance between judges and items. Therefore, for every iteration of the chain I estimate a multivariate normal density approximately 8-10 times. As a result, 5000 iterations can take up to 14 hours. Obviously, I need to run it for much more than 5000 runs and so I need a means for dramatically speeding up the process. Given this, my questions are as follows:
(i) Am I right to assume that the best speed gains will be had by running some, if not all of my chain in C or C++?
(ii) assuming the answer to question 1 is yes, how do I go about this? For example, is there a way for me to retain all my R functions, but simply do the looping in C or C++: i.e. can I call my R functions from C and then do looping?
(iii) I guess what I really want to know is how best to approach the incorporation of C or C++ code into my program.
First make sure your slow R version is correct. Debugging R code might be easier than debugging C code. Done that? Great. You now have correct code you can compare against.
Next, find out what is taking the time. Use Rprof to run your code and see what is taking the time. I did this for some code I inherited once, and discovered it was spending 90% of the time in the t() function. This was because the programmer had a matrix, A, and was doing t(A) in a zillion places. I did one tA=t(A) at the start, and replaced every t(A) with tA. Massive speedup for no effort. Profile your code first.
Now, you've found your bottleneck. Is it code you can speed up in R? Is it a loop that you can vectorise? Do that. Check your results against your gold standard correct code. Always. Yes, I know its hard to compare algorithms that rely on random numbers, so set the seeds the same and try again.
Still not fast enough? Okay, now maybe you need to rewrite parts (the lowest level parts, generally, and those that were taking the most time in the profiling) in C or C++ or Fortran, or if you are really going for it, in GPU code.
Again, really check the code is giving the same answers as the correct R code. Really check it. If at this stage you find any bugs anywhere in the general method, fix them in what you thought was the correct R code and in your latest version, and rerun all your tests. Build lots of automatic tests. Run them often.
Read up about code refactoring. It's called refactoring because if you tell your boss you are rewriting your code, he or she will say 'why didn't you write it correctly first time?'. If you say you are refactoring your code, they'll say "hmmm... good". THIS ACTUALLY HAPPENS.
As others have said, Rcpp is made of win.
A complete example using R, C++ and Rcpp is provided by this blog post which was inspired by a this post on Darren Wilkinson's blog (and he has more follow-ups). The example is also included with recent releases of Rcpp in a directory RcppGibbs and should get you going.
I have a blog post which discusses exactly this topic which I suggest you take a look at:
http://darrenjw.wordpress.com/2011/07/31/faster-gibbs-sampling-mcmc-from-within-r/
(this post is more relevant than the post of mine that Dirk refers to).
I think the best method currently to integrate C or C++ is the Rcpp package of Dirk Eddelbuettel. You can find a lot of information at his website. There is also a talk at Google that is available through youtube that might be interesting.
Check out this project:
https://github.com/armstrtw/rcppbugs
Also, here is a link to the R/Fin 2012 talk:
https://github.com/downloads/armstrtw/rcppbugs/rcppbugs.pdf
I would suggest to benchmark each step of the MCMC sampler and identify the bottleneck. If you put each full conditional or M-H-step into a function, you can use the R compiler package which might give you 5%-10% speed gain. The next step is to use RCPP.
I think it would be really nice to have a general-purpose RCPP function which generates just one single draw using the M-H algorithm given a likelihood function.
However, with RCPP some things become difficult if you only know the R language: non-standard random distributions (especially truncated ones) and using arrays. You have to think more like a C programmer there.
Multivariate Normal is actually a big issue in R. Dmvnorm is very inefficient and slow. Dmnorm is faster, but it would give me NaNs quicker than dmvnorm in some models.
Neither does take an array of covariance matrices, so it is impossible to vectorize code in many instances. As long as you have a common covariance and means, however, you can vectorize, which is the R-ish strategy to speed up (and which is the oppositve of what you would do in C).