Compile time data-specific optimizations - c

Question

In some cases, one knows at compile time what a particular piece of algorithmic data looks like, and as such might wish to convey this information to the compiler. This question is about how one might best achieve that.
By way of example, consider the following example of a sparse matrix multiplication in which the matrix is constant and known at compile time:
matrix = [ 0, 210, 0, 248, 137]
[ 0, 0, 0, 0, 239]
[ 0, 0, 0, 0, 0]
[116, 112, 0, 0, 7]
[ 0, 0, 0, 0, 165]
In such a case, a fully branchless implementation could be written to implement the matrix vector multiplication for an arbitrary input vector:
#include <stdio.h>
#define ARRAY_SIZE 8
static const int matrix[ARRAY_SIZE] = {210, 248, 137, 239, 116, 112, 7, 165};
static const int input_indices[ARRAY_SIZE] = {1, 3, 4, 4, 0, 1, 4, 4};
static const int output_indices[ARRAY_SIZE] = {0, 0, 0, 1, 3, 3, 3, 4};
static void matrix_multiply(int *input_array, int *output_array)
{
for (int i=0; i<ARRAY_SIZE; ++i){
output_array[output_indices[i]] += (
matrix[i] * input_array[input_indices[i]]);
}
}
int main()
{
int test_input[5] = {36, 220, 212, 122, 39};
int output[5] = {0};
matrix_multiply(test_input, output);
for (int i=0; i<5; ++i){
printf("%d\n", output[i]);
}
}
which prints the correct result for the matrix-vector multiplication (81799, 9321, 0, 29089, 6435).
Further optimisations can be envisaged that build on data specific knowledge about the memory locality of reference.
Now, clearly this is an approach which can be used, but it starts getting unwieldy when the size of the data gets big (say ~100MB in my case) and also in any real world situation would depend on meta-programming to generate the associated data dependent knowledge.
Does the general strategy of baking in data specific knowledge have mileage as regards optimisation? If so, what is the best approach to do this?
In the example given, on one level the whole thing than be reduced to knowledge about ARRAY_SIZE with the arrays set at runtime. This leads me to think the approach is limited (and is really a data structures problem), but I'm very interested to know if the general approach of data derived compile-time optimisations is useful in any situation.

Answer 1

I don't think this is a very good answer to this question but I'm going to try offering it anyway. It's also more of a search for the same basic answer.
I work in 3D VFX including raytracing where it's not uncommon to take a fairly modest input with data structures that build in under a second, and then do a monumental amount of processing subsequently to the point where a user might wait hours for a quality production render in a difficult lighting situation.
In theory at least, this could go so much faster if we could make these "data-specific optimizations". Variables could turn into literal constants, significantly less branching could be required, data that is known to always have an upper bound of 45 elements could be allocated on the stack instead of heap or use another form of memory preallocated in advance, locality of reference could be exploited to a greater deal than ever before, vectorization could be applied more easily, achieving both thread-safety and efficiency could be a lot easier, etc.
Where this gets awkward for me is that this requires information about user inputs which can only be provided after the usual notion of "compile-time". So a lot of my interest here relates to code-generation techniques while the application is running.
Now, clearly this is an approach which can be used, but it starts
getting unwieldy when the size of the data gets big (say ~100MB in my
case) and also in any real world situation would depend on
meta-programming to generate the associated data dependent knowledge.
I think beyond that, if the data size gets excessive, then we do often need a good share of branching and variables just to avoid generating so much code that we start becoming bottlenecked by icache misses.
Yet even the ability to turn a dozen variables accessed frequently into compile-time constants and allowing a handful of data structures to exploit greater knowledge of the specified input (and with the aid of an aggressive optimizer) may yield great mileage here, especially considering how well optimizers do provided they have the necessary information provided in advance.
Some of this could be tackled normally with increasingly elaborate and generalized code, metaprogramming techniques, etc, yet there's a peak to how far we can go there: an optimizer can only optimize as much as the information is has available in advance. The difficulty here is providing that information in a practical way. And, as you already guessed, this can quickly get unwieldy, difficult to maintain, and productivity starts to become just as great (if not greater) of a concern than efficiency.
So the most promising techniques to me revolve about code-generation techniques tuned for a specific problem domain, but not for a specific input (optimizing for the specific input will lean more on the optimizer, the code generation is there so that we can provide more of that info needed for the optimizer more easily/appropriately). A modest example that already does something like this is Open Shading Language, where it uses JIT compilation that exploits this idea to a modest level:
OSL uses the LLVM compiler framework to translate shader networks into
machine code on the fly (just in time, or "JIT"), and in the process
heavily optimizes shaders and networks with full knowledge of the
shader parameters and other runtime values that could not have been
known when the shaders were compiled from source code. As a result, we
are seeing our OSL shading networks execute 25% faster than the
equivalent shaders hand-crafted in C! (That's how our old shaders
worked in our renderer.)
While a 25% improvement over handwritten code is modest, that's still a big deal in a production renderer, and it seems like we could go far beyond that.
The use of nodes as a visual programming language also offers a more restrictive environment that helps reduce human errors, allows expressing solutions at a higher-level, seeing the results of changes made on the fly (instant turnaround), etc. -- so it adds not only efficiency but that productivity we need to avoid getting lost in such optimizations. Maintaining and building the code generator could be a little complex, but it only needs to have the minimal amount of code required and doesn't scale in complexity with the amount of code generated using it.
So apologies -- this isn't exactly an answer to your question as a comment, but I think we're searching for a similar thing.