Let's say I have two pointers to two structs a and b. These two structs each contain an enum, which we'll call x. Given any possible a and b, I want to call a specific function based on the values of their x enums.
What is interesting in my case is that the functions that I want to call look like:
X0_to_X1();
X0_to_X2();
...
X1_to_X0();
...
etc
where X0, X1 etc are possible values of the enum x, meaning that there are X_to_Y functions for every and each combination of the values of the x enum.
The obvious "naive" solution to this would be a switch statement which would be quite big (given that x has quite a few possible values):
switch (a->x) {
case X0:
switch (b->x) {
case X1:
X0_to_X1();
break;
// ... and so on and so forth for every possible pair!
My first attempt at solving this a bit more elegantly was to implement a macro that, given two values of x, could form a function call:
#define CALL_FUNCTION(x1, x2) x1 ## _to_ ## x2 ()
This however does not work, as in my code I never can know the actual values of x before runtime, so it ends up looking like:
CALL_FUNCTION(a->x, b->x);
which of course gets converted to:
a->x_to_b->x();
which makes absolutely no sense.
Is there a way to solve this problem more elegantly, or should I just bite the bullet and implement the enormous switch statement instead?
This problem screams for a lookup table, where you store pointers to the various functions and they are keyed by the enumeration values.
If your enum values are sequential (and no two enumeration constants share the same value), then you can build a lookup table as a simple 2D array:
enum x_t { X0, X1, X2, ..., NUM_X };
void (*lookup[NUM_X][NUM_X])(void) = {
{ NULL, X0_to_X1, X0_to_X2, X0_to_X3, ... },
{ X1_to_X0, NULL, X1_to_X2, X1_to_X3, ... },
{ X2_to_X0, X2_to_X1, NULL, X2_to_X3, ... },
...
};
That assumes you don’t have an "identity” function when your x and y are the same.
Then, you call the desired function by indexing into to table like so:
if ( x != y )
lookup[x][y]();
No, it isn’t pretty, but it beats nested switch statements. You can hide that behind a macro or another function call if you wish.
If your enumeration values aren’t sequential then this particular implementation won’t work - you’d have to build your lookup table a different way, using lists or sparse matrices. But while the setup code may be tedious, it will greatly simplify the logic on the calling side.
Related
I have a function M that outputs complex numbers taking an input range of r's. Instead of just outputting a single complex number, I would like to have the function to output two values (the real and imaginary parts separately) for all the output complex vectors. I would prefer the function to be anonymous function.
I tried the following but did not work since I am just getting single output complex values:
r = linspace(1E-10,1.5,100);
%M= (0.5*((1i*r+0.135).* (1i*r+0.651)))./((1i*r+0.0965).* (1i*r+0.4555))
M= #(r)(0.5*((1i*r+0.135).* (1i*r+0.651)))./((1i*r+0.0965).* (1i*r+0.4555))
How do I separate the real and complex parts of a vector?
Create an anonymous function with a different variable to avoid confusion i.e. create M with:
M = #(k)(0.5*((1i*k+0.135).* (1i*k+0.651)))./((1i*k+0.0965).* (1i*k+0.4555));
then create another anonymous function, say N, that extracts real and imag values and then stacks the result.
N = #(k) [real(M(k)); imag(M(k))];
Call this anonymous function with N(r) to get your expected result.
Alternately if you have already calculated M as in your commented out code then you can do:
N = #(k) [real(k); imag(k)];
and then call it with N(M).
So I'm compiling a subset of C to a simple stack VM for learning purposes and I'd like to know how to best compile a switch statement, e.g.
switch (e) {
case 0: { ... }
case 1: { ... }
...
case k: { ... }
}
The book I'm going through offers a simple way to compile it with indexed jumps but the simple scheme described in the book only works for contiguous, ascending ranges of case values.
Right now I'm using symbolic labels for the first pass and during the second pass I'm going to resolve the labels to actual jump targets because having labels simplifies the initial compilation to stack instructions quite a bit. My idea right now is to generalize what's in the book to any sequence of case values in any order with the following scheme. Call the case values c1, c2, ..., cn and the corresponding labels j1, j2, ..., jn then generate the following sequence of instructions assuming the value of e is on top of the stack:
dup, loadc c1, eq, jumpnz j1,
dup, loadc c2, eq, jumpnz j2,
...
dup, loadc cn, eq, jumpnz jn,
pop, jump :default
j1: ...
j2: ...
...
jn: ...
default: ...
The notation is hopefully clear but if not: dup = duplicate value on top of stack, loadc c = push a constant c on top of the stack, eq = compare top two values on the stack and push 0 or 1 based on the comparison, jumpnz j = jump to the label j if the top stack value is not 0, label: = something that will be resolved to an actual address during second compilation pass.
So my question is then what are some other ways of compiling switch statements? My way of doing it is much less compact than an indexed jump table for contiguous ranges of case values but works out pretty well when there are large gaps.
First sort all the cases. Then identify all (large enough to be worthwhile) continguous or near-contiguous sequences, and treat them as a single unit that is handled with a jump table. Then, instead of your linear sequence of compares and jumps, use a balanced binary tree of branches to minimize the average number of jumps. You do this by comparing against the median of the cases or blocks of cases, recursively.
I have a table schema which includes an int array column, and a custom aggregate function which sums the array contents. In other words, given the following:
CREATE TABLE foo (stuff INT[]);
INSERT INTO foo VALUES ({ 1, 2, 3 });
INSERT INTO foo VALUES ({ 4, 5, 6 });
I need a "sum" function that would return { 5, 7, 9 }. The PL/pgSQL version, which works correctly, is as follows:
CREATE OR REPLACE FUNCTION array_add(array1 int[], array2 int[]) RETURNS int[] AS $$
DECLARE
result int[] := ARRAY[]::integer[];
l int;
BEGIN
---
--- First check if either input is NULL, and return the other if it is
---
IF array1 IS NULL OR array1 = '{}' THEN
RETURN array2;
ELSEIF array2 IS NULL OR array2 = '{}' THEN
RETURN array1;
END IF;
l := array_upper(array2, 1);
SELECT array_agg(array1[i] + array2[i]) FROM generate_series(1, l) i INTO result;
RETURN result;
END;
$$ LANGUAGE plpgsql;
Coupled with:
CREATE AGGREGATE sum (int[])
(
sfunc = array_add,
stype = int[]
);
With a data set of about 150,000 rows, SELECT SUM(stuff) takes over 15 seconds to complete.
I then re-wrote this function in C, as follows:
#include <postgres.h>
#include <fmgr.h>
#include <utils/array.h>
Datum array_add(PG_FUNCTION_ARGS);
PG_FUNCTION_INFO_V1(array_add);
/**
* Returns the sum of two int arrays.
*/
Datum
array_add(PG_FUNCTION_ARGS)
{
// The formal PostgreSQL array objects:
ArrayType *array1, *array2;
// The array element types (should always be INT4OID):
Oid arrayElementType1, arrayElementType2;
// The array element type widths (should always be 4):
int16 arrayElementTypeWidth1, arrayElementTypeWidth2;
// The array element type "is passed by value" flags (not used, should always be true):
bool arrayElementTypeByValue1, arrayElementTypeByValue2;
// The array element type alignment codes (not used):
char arrayElementTypeAlignmentCode1, arrayElementTypeAlignmentCode2;
// The array contents, as PostgreSQL "datum" objects:
Datum *arrayContent1, *arrayContent2;
// List of "is null" flags for the array contents:
bool *arrayNullFlags1, *arrayNullFlags2;
// The size of each array:
int arrayLength1, arrayLength2;
Datum* sumContent;
int i;
ArrayType* resultArray;
// Extract the PostgreSQL arrays from the parameters passed to this function call.
array1 = PG_GETARG_ARRAYTYPE_P(0);
array2 = PG_GETARG_ARRAYTYPE_P(1);
// Determine the array element types.
arrayElementType1 = ARR_ELEMTYPE(array1);
get_typlenbyvalalign(arrayElementType1, &arrayElementTypeWidth1, &arrayElementTypeByValue1, &arrayElementTypeAlignmentCode1);
arrayElementType2 = ARR_ELEMTYPE(array2);
get_typlenbyvalalign(arrayElementType2, &arrayElementTypeWidth2, &arrayElementTypeByValue2, &arrayElementTypeAlignmentCode2);
// Extract the array contents (as Datum objects).
deconstruct_array(array1, arrayElementType1, arrayElementTypeWidth1, arrayElementTypeByValue1, arrayElementTypeAlignmentCode1,
&arrayContent1, &arrayNullFlags1, &arrayLength1);
deconstruct_array(array2, arrayElementType2, arrayElementTypeWidth2, arrayElementTypeByValue2, arrayElementTypeAlignmentCode2,
&arrayContent2, &arrayNullFlags2, &arrayLength2);
// Create a new array of sum results (as Datum objects).
sumContent = palloc(sizeof(Datum) * arrayLength1);
// Generate the sums.
for (i = 0; i < arrayLength1; i++)
{
sumContent[i] = arrayContent1[i] + arrayContent2[i];
}
// Wrap the sums in a new PostgreSQL array object.
resultArray = construct_array(sumContent, arrayLength1, arrayElementType1, arrayElementTypeWidth1, arrayElementTypeByValue1, arrayElementTypeAlignmentCode1);
// Return the final PostgreSQL array object.
PG_RETURN_ARRAYTYPE_P(resultArray);
}
This version takes only 800 ms to complete, which is.... much better.
(Converted to a stand-alone extension here: https://github.com/ringerc/scrapcode/tree/master/postgresql/array_sum)
My question is, why is the C version so much faster? I expected an improvement, but 20x seems a bit much. What's going on? Is there something inherently slow about accessing arrays in PL/pgSQL?
I'm running PostgreSQL 9.0.2, on Fedora Core 8 64-bit. The machine is a High-Memory Quadruple Extra-Large EC2 instance.
Why?
why is the C version so much faster?
A PostgreSQL array is its self a pretty inefficient data structure. It can contain any data type and it's capable of being multi-dimensional, so lots of optimisations are just not possible. However, as you've seen it's possible to work with the same array much faster in C.
That's because array access in C can avoid a lot of the repeated work involved in PL/PgSQL array access. Just take a look at src/backend/utils/adt/arrayfuncs.c, array_ref. Now look at how it's invoked from src/backend/executor/execQual.c in ExecEvalArrayRef. Which runs for each individual array access from PL/PgSQL, as you can see by attaching gdb to the pid found from select pg_backend_pid(), setting a breakpoint at ExecEvalArrayRef, continuing, and running your function.
More importantly, in PL/PgSQL every statement you execute is run through the query executor machinery. This makes small, cheap statements fairly slow even allowing for the fact that they're pre-prepared. Something like:
a := b + c
is actually executed by PL/PgSQL more like:
SELECT b + c INTO a;
You can observe this if you turn debug levels high enough, attach a debugger and break at a suitable point, or use the auto_explain module with nested statement analysis. To give you an idea of how much overhead this imposes when you're running lots of tiny simple statements (like array accesses), take a look at this example backtrace and my notes on it.
There is also a significant start-up overhead to each PL/PgSQL function invocation. It isn't huge, but it's enough to add up when it's being used as an aggregate.
A faster approach in C
In your case I would probably do it in C, as you have done, but I'd avoid copying the array when called as an aggregate. You can check for whether it's being invoked in aggregate context:
if (AggCheckCallContext(fcinfo, NULL))
and if so, use the original value as a mutable placeholder, modifying it then returning it instead of allocating a new one. I'll write a demo to verify that this is possible with arrays shortly... (update) or not-so-shortly, I forgot how absolute horrible working with PostgreSQL arrays in C is. Here we go:
// append to contrib/intarray/_int_op.c
PG_FUNCTION_INFO_V1(add_intarray_cols);
Datum add_intarray_cols(PG_FUNCTION_ARGS);
Datum
add_intarray_cols(PG_FUNCTION_ARGS)
{
ArrayType *a,
*b;
int i, n;
int *da,
*db;
if (PG_ARGISNULL(1))
ereport(ERROR, (errmsg("Second operand must be non-null")));
b = PG_GETARG_ARRAYTYPE_P(1);
CHECKARRVALID(b);
if (AggCheckCallContext(fcinfo, NULL))
{
// Called in aggregate context...
if (PG_ARGISNULL(0))
// ... for the first time in a run, so the state in the 1st
// argument is null. Create a state-holder array by copying the
// second input array and return it.
PG_RETURN_POINTER(copy_intArrayType(b));
else
// ... for a later invocation in the same run, so we'll modify
// the state array directly.
a = PG_GETARG_ARRAYTYPE_P(0);
}
else
{
// Not in aggregate context
if (PG_ARGISNULL(0))
ereport(ERROR, (errmsg("First operand must be non-null")));
// Copy 'a' for our result. We'll then add 'b' to it.
a = PG_GETARG_ARRAYTYPE_P_COPY(0);
CHECKARRVALID(a);
}
// This requirement could probably be lifted pretty easily:
if (ARR_NDIM(a) != 1 || ARR_NDIM(b) != 1)
ereport(ERROR, (errmsg("One-dimesional arrays are required")));
// ... as could this by assuming the un-even ends are zero, but it'd be a
// little ickier.
n = (ARR_DIMS(a))[0];
if (n != (ARR_DIMS(b))[0])
ereport(ERROR, (errmsg("Arrays are of different lengths")));
da = ARRPTR(a);
db = ARRPTR(b);
for (i = 0; i < n; i++)
{
// Fails to check for integer overflow. You should add that.
*da = *da + *db;
da++;
db++;
}
PG_RETURN_POINTER(a);
}
and append this to contrib/intarray/intarray--1.0.sql:
CREATE FUNCTION add_intarray_cols(_int4, _int4) RETURNS _int4
AS 'MODULE_PATHNAME'
LANGUAGE C IMMUTABLE;
CREATE AGGREGATE sum_intarray_cols(_int4) (sfunc = add_intarray_cols, stype=_int4);
(more correctly you'd create intarray--1.1.sql and intarray--1.0--1.1.sql and update intarray.control. This is just a quick hack.)
Use:
make USE_PGXS=1
make USE_PGXS=1 install
to compile and install.
Now DROP EXTENSION intarray; (if you already have it) and CREATE EXTENSION intarray;.
You'll now have the aggregate function sum_intarray_cols available to you (like your sum(int4[]), as well as the two-operand add_intarray_cols (like your array_add).
By specializing in integer arrays a whole bunch of complexity goes away. A bunch of copying is avoided in the aggregate case, since we can safely modify the "state" array (the first argument) in-place. To keep things consistent, in the case of non-aggregate invocation we get a copy of the first argument so we can still work with it in-place and return it.
This approach could be generalised to support any data type by using the fmgr cache to look up the add function for the type(s) of interest, etc. I'm not particularly interested in doing that, so if you need it (say, to sum columns of NUMERIC arrays) then ... have fun.
Similarly, if you need to handle dissimilar array lengths, you can probably work out what to do from the above.
PL/pgSQL excels as server-side glue for SQL elements. Procedural elements and lots of assignments are not among its strengths. Assignments, tests or looping are comparatively expensive and only warranted if they help to take shortcuts one could not achieve with just SQL. The same logic implemented in C will always be faster, but you seem to be well aware of that ...
Typically, a pure SQL solution is faster. Compare this simple, equivalent solution in your test setup:
SELECT array_agg(a + b)
FROM (
SELECT unnest('{1, 2, 3 }'::int[]) AS a
, unnest('{4, 5, 6 }'::int[]) AS b
) x;
You can wrap it into a simple SQL function. Or, for best performance, integrate it into your big query directly:
SELECT tbl_id, array_agg(a + b)
FROM (
SELECT tbl_id
, unnest(array1) AS a
, unnest(array2) AS b
FROM tbl
ORDER BY tbl_id
) x
GROUP BY tbl_id;
Set-returning functions only work in parallel in a SELECT if the number of returned rows is identical. I.e.: works only for arrays of equal length. The behavior has eventually been sanitized in Postgres 10. See:
What is the expected behaviour for multiple set-returning functions in SELECT clause?
It's generally best to test with a current version of Postgres. At least update to the latest point release (9.0.15 at the time of writing). Might be part of the explanation for the big difference in performance.
Postgres 9.4
Performance improvements for array handling.
There is a cleaner solution for unnesting in parallel now:
Unnest multiple arrays in parallel
Is there any way to "vector" assign an array of struct.
Currently I can
edges(1000000) = struct('weight',1.0); //This really does not assign the value, I checked on 2009A.
for i=1:1000000; edges(i).weight=1.0; end;
But that is slow, I want to do something more like
edges(:).weight=[rand(1000000,1)]; //with or without the square brackets.
Any ideas/suggestions to vectorize this assignment, so that it will be faster.
Thanks in advance.
This is much faster than deal or a loop (at least on my system):
N=10000;
edge(N) = struct('weight',1.0); % initialize the array
values = rand(1,N); % set the values as a vector
W = mat2cell(values, 1,ones(1,N)); % convert values to a cell
[edge(:).weight] = W{:};
Using curly braces on the right gives a comma separated value list of all the values in W (i.e. N outputs) and using square braces on the right assigns those N outputs to the N values in edge(:).weight.
You can try using the Matlab function deal, but I found it requires to tweak the input a little (using this question: In Matlab, for a multiple input function, how to use a single input as multiple inputs?), maybe there is something simpler.
n=100000;
edges(n)=struct('weight',1.0);
m=mat2cell(rand(n,1),ones(n,1),1);
[edges(:).weight]=deal(m{:});
Also I found that this is not nearly as fast as the for loop on my computer (~0.35s for deal versus ~0.05s for the loop) presumably because of the call to mat2cell. The difference in speed is reduced if you use this more than once but it stays in favor of the for loop.
You could simply write:
edges = struct('weight', num2cell(rand(1000000,1)));
Is there something requiring you to particularly use a struct in this way?
Consider replacing your array of structs with simply a separate array for each member of the struct.
weights = rand(1, 1000);
If you have a struct member which is an array, you can make an extra dimension:
matrices = rand(3, 3, 1000);
If you just want to keep things neat, you could put these arrays into a struct:
edges.weights = weights;
edges.matrices = matrices;
But if you need to keep an array of structs, I think you can do
[edges.weight] = rand(1, 1000);
The reason that the structs in your example don't get initialized properly is that the syntax you're using only addresses the very last element in the struct array. For a nonexistent array, the rest of them get implicitly filled in with structs that have the default value [] in all their fields.
To make this behavior clear, try doing a short array with clear edges; edges(1:3) = struct('weight',1.0) and looking at each of edges(1), edges(2), and edges(3). The edges(3) element has 1.0 in its weight like you want; the others have [].
The syntax for efficiently initializing an array of structs is one of these.
% Using repmat and full assignment
edges = repmat(struct('weight', 1.0), [1 1000]);
% Using indexing
% NOTE: Only correct if variable is uninitialized!!!
edges(1:1000) = struct('weight', 1.0); % QUESTIONABLE
Note the 1:1000 instead of just 1000 when indexing in to the uninitialized edges array.
There's a problem with the edges(1:1000) form: if edges is already initialized, this syntax will just update the values of selected elements. If edges has more than 1000 elements, the others will be left unchanged, and your code will be buggy. Or if edges is a different type, you could get an error or weird behavior depending on its existing datatype. To be safe, you need to do clear edges before initializing using the indexing syntax. So it's better to just do full assignment with the repmat form.
BUT: Regardless of how you initialize it, an array-of-structs like this is always going to be inherently slow to work with for larger data sets. You can't do real "vectorized" operations on it because your primitive arrays are all broken up in to separate mxArrays inside each struct element. That includes the field assignment in your question – it is not possible to vectorize that. Instead, you should switch a struct-of-arrays like Brian L's answer suggests.
You can use a reverse struct and then do all operations without any errors
like this
x.E(1)=1;
x.E(2)=3;
x.E(2)=8;
x.E(3)=5;
and then the operation like the following
x.E
ans =
3 8 5
or like this
x.E(1:2)=2
x =
E: [2 2 5]
or maybe this
x.E(1:3)=[2,3,4]*5
x =
E: [10 15 20]
It is really faster than for_loop and you do not need other big functions to slow your program.
I am newbie in C and new to stackoveflow too. I have some problems in coding first oder formula like
forall([X],implies(X,f(X)))
Here x is a variable, implies is predicate and f is function. It sounds like for all x, x implies function of x i'e f(x).
using C. Any kind of suggestions and help will be appreciated.
First order formulas have boolean propositional parts (in your example, "implies(x,f(x))") and quantifiers ("Forall x").
You should already know that coding a function call "f(x)" is coded exactly that way in C.
You code the propositional part as boolean C code using logic connectives. For your example, "implication" isn't a native C operator so you have to substitute slightly different code for it. In c, the "?" operator does the trick. "a?b:c" produces "b" if "a" is true, and "c" otherwise. For your example:
x?f(x):false
Quantifiers mean you have to enumerate the set of possible values of the quantified variable, which always has some abstract type. In logic, this set might be infinite, but it computers it is not. In your case, you need to enumerate the set of values which could be "x"s. To do that, you need a way to represent a set; a cheesy way to do that in C is to use an array to hold the set members X, and to iterate through the array:
type_of_x set_of_x[1000];
... fill x somehow ...
for(i=1;i<number_of_set_elements;i++)
{ x= set_of_x[i];
... evaluate formula ...
}
Since a "forall" is false if any propositional instance is false, you need to exit the enumeration when you find a false example:
boolean set_of_x[1000]; // in your example, x must be a boolean variable
// forall x
... fill x somehow ...
final_value=true;
for (i=1;i<number_set_elements; i++)
{ x= set_of_x[i];
if (x?f(x):false)
{ final_value=false;
break;
}
}
... final_value set correctly here...
"exists" is true if any propositional instance is true, so you need to exit the enumeration when you find a true result:
// exists x
... fill x somehow ...
final_value=false;
for (i=1;i<number_set_elements; i++)
{ x= set_of_x[i];
if (x?f(x):false)
{ final_value=true;
break;
}
}
... final_value set correctly here...
If you have multiple quantifiers, you'll end up with nested loops, one loop for each quantifier. If your formula is complex, you'll likely need several intermediate boolean variables to compute the values of the various parts.
You'll also end up with a variety of "sets" (some arrays, some linked lists, some hash tables) so you'll need to learn how to use these data structures. Also, your quantified values might not be booleans, but that's OK;
you can still pass them to functions that compute boolean values. To compute the FOL for:
forall p:Person old(p) and forall f:Food ~likes(p,f)
the following code skeleton would be used (details left to the reader):
person array_of_persons[...];
foods array_of_foods[...]
for (i=...
{ p=array_of_persons[i];
is_old = old(p);
for(j=...
{ f=array_of_foods[j];
...
if (is_old && !likes(p,f)) ...
}
}
C is an imperative programming language. "Imperative" here means that execution happens via the programmer specifically telling the computer what to do.
For what you want, Prolog is more appropriate. It is based on first-order predicate logic, and execution happens by trying to "find a resolution refutation of the negated query" that's specified by the user's goal. This approach is very unlike C since execution is much more implicit and the expression of intent looks much different.
If you have lots of time, you can write your own constraint solver or Prolog interpreter in C, but by default, C has no first-class support for what you're looking for.