I have a function f(x1,…,xN), where variables xi take only binary values, xi = 0 or 1. I.e. the input space is a space with Hamming distance as metric.
I want to optimise this function numerically and find a minimum.
What’s the best numerical method to get this?
Honestly I don’t know where to start, I never tried to optimise a function of discrete variables, as well as a function of binary variables.
Note: input space is dimensionally huge (Number of variables is big) so I need the proper numerical method for this kind of problem.
Note: input space is dimensionally huge (Number of variables is big) so I need a fast numerical method.
Space and speed can often mean different code.
When space is most important, use bool for single instances of your binary variable and unsigned char for arrays.
When speed is most important, use bool for all instances of your binary variable.
When variable space is critical, use unsigned and encode multiple bits in one unsigned.
For a given case, post an mcve.
Related
I want to make a function that returns the roots of a polynomial, as part of a basic exercise. Is there a safe way to return a variable number of elements (0, 1 or 2, in this case), i.e. making sure the receiver knows how many items were returned so they can avoid a segfault ?
This is arguably more a question of API design than pure programming, the roots themselves are trivial but getting them there safely puzzles me.
The case of a polynomial is special, as the caller knows how many roots there are at maximum. So the caller knows how much memory is needed to safely store the return value.
In this case it is easiest to let the caller pass a buffer to your function and store the roots in there. Your roots function would then only return the number of roots it found.
The other case (where the function returns freshly allocated memory) is way more complicated, and should be avoided if possible. You would introduce strong dependencies to your function (memory allocator) and this way fix the memory allocation schema. This is quite restraining (and unnecessary for your and perhaps many other use-cases). [1]
This could work:
int roots( double roots_out[], double coefs_in[], int ord)
This would be better (less error prone, even if one demands roots_out must have ord-1 elements)
int roots( double roots_out[], int nroots_max, double coefs_in[], int ord)
ord is the order of the polynomial (not the degree) and indicates how many elements there are in coefs_in
Note that there are still some questions left:
What to return if there are infinitely many roots? (zero polynomial?)
What to do if the buffer is too small? (Your solver may generate duplicate roots due to numerical inaccuracies. In other words nroots_max = ord-1 may be sae from mathematical point of view, but not numerically)
[1]: Also even if you may think this way one would save some memory as the number of roots may be less than the degree, this comes with a caveat. During root calculation one needs a big buffer anyway as the number of roots is not known beforehand. So memory saving is a two step process anyway, and, in my opinion, this task should be delegated to the caller. This is most efficient I guess.
I would like to generate a nicely-mixed-up integer fingerprint of an arbitrary C string (s). Most C strings will consist of ASCII text characters:
I want very different fingerprints for similar strings, esp such similar strings as "ab" and "ba"
I want it to be difficult to invert back from the fingerprint to the string (well, my string is typically longer than 32 bits, which means that many strings would map into the same integer), which means again that I want similar strings to yield very different codes;
I want to use the 32 bits available to me efficiently in the integer result,
I want the function source to be small
I want the function to be fast.
one usage is security (but not encryption) related. I can ask a user for a text password, convert it into an integer for storage and later test whether this integer is correct. (I know I could store strings, but I don't want to. guessing a 32-bit integer correctly is impossible if my program can slow down incorrect attempts to the point where brute force cannot work faster than password guessing. another use of this function is as the start of a hash index function (mod array length) into an array.)
alas, I am probably reinventing the wheel here. such functions have probably been written a million times, and by people who are much more versed in cryptography. I don't need AES, of course, but something much more lightweight. the use is different.
my first thinking was
mod 64 each character to take advantage of the ASCII text aspect. now I have 6 bits. call this x.
I can place a 6bit string into 5 locations in a 32-bit space, leaving 2 bits over.
take the current string index position (0, 1, 2...), mod5 it to determine where I want to start to place my x into my running integer result code. XOR my x into this running-result integer.
use the remaining 2 bits to increment a counter [mod 4 to prevent overflow] for each character processed.
then I thought that bit operations may be computer-fast but take more source code. I can think of other choices. take each index position i and multiply it by an ascii representation of each character [or the x from above], and call this y[i]. now do the following:
calculate the natural logarithm of the sums of the y (or this sum plus the running result), and just pretend that the first 32 bits of this result [maybe leaving off the first few bits], which are really a double, are an integer representation. I can XOR each bitint(log(y[i])) into the running integer result.
do it even cheaper. just add the y's, and then do the logarithm with 32-bit pickoff just once at the end. alternatively, run a sum-y through srand as a seed and grab a rand.
there are probably a few other ways to do it, too. in sum, the function should map strings into very different integers, be short to code, and be very fast.
Any pointers?
A common method of generating a non-reversible digest or hash of a string is to generate a Cyclic Redundancy Checksum (CRC).
Source for CRC is widely available, in this case you should use a common CRC-32 such as that used by Ethernet. Different CRCs work on the same principle, buy use different polynomials. Do not be tempted to invent your own polynomial; the distribution is likely to be sub-optimal.
What you're looking for is called a "hash". Two examples of hash functions I'm aware of that return short integers are MurmurHash and SipHash. MurmurHash, as I recall, is not designed to be a cryptographic hash, while SipHash, on the other hand, is indeed designed with security in mind, as stated on its homepage. MurmurHash has 2 versions that return a 32-bit and a 64-bit output. SipHash returns a 64-bit output.
I need to use GMPs mpf_t/mpz_t as keys in a hashtable.
Is there any efficient way to access the raw bytes of the number representation so I can run a hash function over them?
I already read the documentation but I don't really feel smarter now. ;)
Thanks!
Regards,
Ethon
Out of curiosity, why use hashing when you can sort by value? Comparison is very quick, as it compares the bytes / limbs from MSB to LSB, returning a result as soon as they differ.
You can access the raw data using the platform-dependent mp_limb_t type. Both mpz_t and mpf_t have an mp_limb_t vector stored at the address specified by _mp_d, with the number of significant limbs given by the absolute value of the _mp_size field. (the definitions are in gmp.h)
Of course, if the hash function depends on an 8-bit byte vector, you will need to convert the limb vector. Fortunately, the number of bits in a mp_limb_t - GMP_LIMB_BITS - is always going to be divisible by 8 on any sane platform.
Just a quick question: What are people's practices when you have to define the (arbitrary) maximum that some array can take in C. So, some people just choose a round number hoping it will be big enough, others the prime number closer to the round number (!), etc., other some more esoteric number, like the prime number closer to... and so on.
I'm wondering, then, what are some best practices for deciding such values?
Thanks.
There is no general rule. Powers of twos work for buffers, I use 1024 quite often for string buffers in C but any other number would work. Prime numbers are useful for hash tables where simple modulo-hashing works well with prime-number sizes. Of course you define the size as a symbolic constant so that you can change it later.
If I can't pin down a reasonable maximum I tend to use malloc and realloc to grow the array as needed. Using a fixed size array when you can't gurantee that it is large enough for the intended purpose is hazardous.
Best practice is to avoid arbitrary limits whenever possible.
It's not always possible, so second-best practice is to take an educated estimate of the largest thing that the array is ever likely to need to hold, and then round up by a healthy margin, at least 25%. I tend to prefer powers of ten when I do this, because it makes it obvious on inspection that the number is an arbitrary limit. (Powers of two also often signify that, but only if the reader recognizes the number as a power of two, and most readers-of-code don't have that table memorized much past 216. If there's a good reason to use a power of two and it needs to be bigger than that, write it in hex. End of digression.) Always document the reasoning behind your estimate of the largest thing the array needs to hold, even if it's as simple as "anyone with a single source file bigger than 2GB needs to rethink their coding style" (actual example)
Don't use a prime number unless you specifically need the properties of a prime number (e.g. as Juho mentions, for hash tables -- but you only need that there if your hash function isn't very good -- but often it is, unfortunately.) When you do, document that you are intentionally using prime numbers and why, because most people do not recognize prime numbers on sight or know why they might be necessary in a particular situation.
If I need to do this I usually go with either a power of two, or for larger data sets, the number of pages required to hold the data. Most of the time though I prefer to allocate a chunk of memory on the heap and then realloc if the buffer size is insufficient later.
I only define a maximum when I have a strong reason for a particular number to be the maximum. Otherwise, I size it dynamically, perhaps with a sanity-check maximum (e.g. a person's name should not be several megabytes long).
Round numbers (powers of 2) are used because they are often easy for things like malloc to use (many implementations keep up with memory in blocks of various power of two sizes), easier for linkers to use (in the case of static or global arrays), and also because you can use bitwise operations to test for limits of them, which are often faster than < and >.
Prime numbers are used because using prime number sized hash tables is supposed to avoid collision.
Many people likely use both prime number and power of two sizes for things in cases where they don't actually provide any benefit, though.
It really isn't possible to predict at the outset what the maximum size could be.
For example, I coded a small cmdline interpreter, where each line of output produced was stored in a char array of size 200. Sufficient for all possible outputs, don't you think?
That was until I issued the env command which had a line with ~ 400 characters(!).
LS_COLORS='no=00:fi=00:di=01;34:ln=01;36:pi=40;33:so=01;35:bd=40;33;01:cd=40;33;01:or=01;
05;37;41:mi=01;05;37;41:ex=01;32:*.cmd=01;32:*.exe=01;32:*.com=01;32:*.btm=01;32:*.bat=01;32:*.sh=01;
32:*.csh=01;32:*.tar=01;31:*.tgz=01;31:*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.zip=01;31:*.z=01;31:*.Z=01;
31:*.gz=01;31:*.bz2=01;31:*.bz=01;31:*.tz=01;31:*.rpm=01;31:*.cpio=01;31:*.jpg=01;35:*.gif=01;35:*.bmp=01;
35:*.xbm=01;35:*.xpm=01;35:*.png=01;35:*.tif=01;35:';
Moral of the story: Try to use dynamic allocation as far as possible.
I have a bit array that can be very dense in some parts and very sparse in others. The array can get as large as 2**32 bits. I am turning it into a bunch of tuples containing offset and length to make it more efficient to deal with in memory. However, this sometimes is less efficient with things like 10101010100011. Any ideas on a good way of storing this in memory?
If I understand correctly, you're using tuples of (offset, length) to represent runs of 1 bits? If so, a better approach would be to use runs of packed bitfields. For dense areas, you get a nice efficient array, and in non-dense areas you get implied zeros. For example, in C++, the representation might look like:
// The map key is the offset; the vector's length gives you the length
std::map<unsigned int, std::vector<uint32_t> >
A lookup would consist of finding the key before the bit position in question, and seeing if the bit falls in its vector. If it does, use the value from the vector. Otherwise, return 0. For example:
typedef std::map<unsigned int, std::vector<uint32_t> > bitmap; // for convenience
typedef std::vector<uint32_t> bitfield; // also convenience
bool get_bit(const bitmap &bm, unsigned int idx) {
unsigned int offset = idx / 32;
bitmap::const_iterator it = bm.upper_bound(offset);
// bm is the element /after/ the one we want
if (it == bm.begin()) {
// but it's the first, so we don't have the target element
return false;
}
it--;
// make offset be relative to this element start
offset -= it.first;
// does our bit fall within this element?
if (offset >= it.second.size())
return false; // nope
unsigned long bf = it.second[offset];
// extract the bit of interest
return (bf & (1 << (offset % 32))) != 0;
}
It would help to know more. By "very sparse/dense," do you mean millions of consecutive zeroes/ones, or do you mean local (how local?) proportions of 0's very close to 0 or 1? Does one or the other value predominate? Are there any patterns that might make run-length encoding effective? How will you use this data structure? (Random access? What kind of distribution of accessed indexes? Are huge chunks never or very rarely accessed?)
I can only guess you aren't going to be randomly accessing and modifying all 4 billion bits at rates of billions of bits/second. Unless it is phenomenally sparse/dense on a local level (such as any million consecutive bits are likely to be the same except for 5 or 10 bits) or full of large scale repetition or patterns, my hunch is that the choice of data structure depends more on how the array is used than on the nature of the data.
How to structure things will be dependent on what is your data. For trying to represent large amounts of data, you will need to have long runs of zeros or ones. This would eliminate the need to have it respresented. If this is not the case and you have approxiately the same amount of one's and zeros, you would be better off with all of the memory.
It might help to think of this as a compression problem. For compression to be effective there has to be a pattern (or a limit set of items used out of an entire space) and an uneven distribution in order for compression to work. If all the elements are used and evenly distributed, compression is hard to do, or could take more space then the actual data.
If there are only runs of zero and ones, (more then just one), using offset and length might make some sense. If there is inconsistent runs, you could just copy the bits as a bit array where you have offset, length, and values.
How efficient the above is will depend upon if you have a large runs of ones or zeros. You will want to be careful to make sure you are not using more memory to reperesent your memory, then just using memory itself, (i.e. your are using more memory to represent the memory then just placing it into memory).
Check out bison source code. Look at biset implementation. It provides several flavors of implementations to deal with bit arrays with different densities.
How many of these do you intend to keep in memory at once?
As far as I can see, 2**32 bits = 512M, only half a gig, which isn't very much memory nowadays. Do you have anything better to do with it?
Assuming your server has enough ram, allocate it all at startup, then keep it in memory, the network handling thread can execute in just a few instructions in constant time - it should be able to keep up with any workload.