dbt macros, how to use table in query - snowflake-cloud-data-platform

I am trying to write the macro below that creates a checksum column less specified values - but what values should be input for table as the below does not work ( in context of jinja/macros/dbt)
{% macro generate_checksum(source) %}
SELECT HASH(*) AS hash_value
FROM (SELECT * EXCLUDE (a, b, f, c, d, e, z, g)
from {{source}} )
{% endmacro %}

You have a few options:
You could pass a Relation as an argument to your macro. You could leave your macro as-is, and then call it in your model file with:
{{ generate_checksum(ref('some_model')) }}
{# or use source() or this in place of ref() above #}
You could pass a string into your macro, and make the call to ref inside your macro. This would lock you into a specific type of relation (e.g., only ref, not source or this). Your macro becomes:
{% macro generate_checksum(model_name) %}
SELECT HASH(*) AS hash_value
FROM (SELECT * EXCLUDE (a, b, f, c, d, e, z, g)
from {{ ref(model_name) }} )
{% endmacro %}
and you call it in your model file with a string literal:
{{ generate_checksum('some_model') }}

Related

How to loop multiple decay rate in multiple columns in pyspark

I try to pass a list in parameter of my function.
My list is composed of different coefficients to be apply to lag numerous columns.
However, I only manage to generate the columns in my dataframe for the first value of my list.
this is my actual result :
"col1", "col2", "col1_0.2", "col2_0.2"
what is expected :
"col1", "col2", "col1_0.2", "col2_0.2", "col1_0.4", "col2_0.4", "col1_0.6", "col2_0.6"
I must have missed a few things in my loop ?
selected_col = col_selector(df, ["col1", "col2"])
w = Window.partitionBy("student").orderBy("date")
coef = (.1,.4,.6)
def custom_coef(col, w, coef):
for x in coef:
return sum(
pow(i, x) * F.lag(F.col(col), i, default=0).over(w)
for i in range(1)
).alias(col +"_"+str(x))
new_df = df.select(
F.col("*"),
*[custom_coef(col, w, coef) for col in selected_col]
)
thanks
The return statement in the custom_coef function ends the function after the first execution of loop over coef. This means that custom_coef will always return the first column definition, and this is the column definition for coef 0.1. As the function is called once per column in selected_col you get the result that you are describing.
One way to fix the problem without changing the structure of the code is to replace return with yield. This way custom_coef creates one generator per element of selected_col. These generators can be chained with itertools.chain and this result can be used as parameter of the select statement:
def custom_coef(col, w, coef):
for x in coef:
yield sum( #use yield instead of return
pow(i, x) * F.lag(F.col(col), i, default=0).over(w)
for i in range(1)
).alias(col +"_"+str(x))
new_df = df.select(
F.col("*"),
*chain(*[custom_coef(col, w, coef) for col in selected_col]) #chain the generators
)
new_df.show()

Angular.js square root for this calculator

I have searched far and wide looking for a simplified square root short code version and found plenty of versions in JavaScript, Java, jQuery ... but nothing in Angular.js.
Below is my code:
<body align"center"><form><h1 align="center">Quick 2 number Angular.js Calculator </h1></center>
<center><h6>- Teaching aide only -</h6></center>
<div ng-app="">
<input type="number" ng-model="numb1" placeholder="number1" size="5px"><br />
<input type="number" ng-model="numb2" placeholder="number2" size="5px"><br />
<b style="color:#0000FF"><button disabled>+</button>
<button disabled>-</button>
<button disabled>X</button>
<button disabled>/</button></b>
<button style="background-color:lime;color:green" disabled>ENT</button>
<button style="background-color:orange;color:red">CLR</button>
<center><h2>Answers Below</h2></center>
<p>My first expression: {{ numb1 }} ; {{ numb2 }}</p>
<p>Addition: {{ numb1 + numb2 }}</p>
<p>Subtraction: {{ numb1 - numb2 }}</p>
<p>Multiplication: {{ numb1 * numb2 }}</p>
<p>Division: {{ numb1 / numb2 }}</p>
<p>Square of {{ numb1 }}<small><sup>2</sup></small> is {{ numb1 * numb1 }}<br>Square of {{ numb2 }}<small><sup>2</sup></small> is {{ numb2 * numb2 }}</p>
<p>Cube of {{ numb1 }}<small><sup>3</sup></small> is {{ numb1 * numb1 * numb1 }}<br>Cube of {{ numb2 }}<small>
<sup>3</sup></small> is {{ numb2 * numb2 * numb2 }}</p>
</form>
The best way is to use filter for this:
angular.module('app', []).filter('sqrt', function() {
return function(number) {
return Math.sqrt(number);
};
});
<div ng-app="app">
<input type="number" ng-model="numb1" placeholder="number1" size="5px"><br />
<div>{{ numb1 | sqrt }}</div>
</div>
<script src="https://ajax.googleapis.com/ajax/libs/angularjs/1.7.2/angular.min.js"></script>
In this case, javascript has what you want. You need to pass an argument to the Math.sqrt function for this. Here is an example from MDN. You could create a service or a directive to wrap around this function, but angularjs itself doesn't have this function.
function calcHypotenuse(a, b) {
return(Math.sqrt((a * a) + (b * b)));
}
console.log(calcHypotenuse(3, 4));
// expected output: 5
console.log(calcHypotenuse(5, 12));
// expected output: 13
console.log(calcHypotenuse(0, 0));
// expected output: 0
If you have to create a custom solution, I would recommend looking at the javascript code for the function itself to see how it's implemented. You can go to Firefox for example, to download their implementation.
I was able to find the algorithm, but not the code there after I downloaded it.
Square Root
A simple iterative algorithm is used to compute the greatest integer
less than or equal to the square root. Essentially, this is Newton's
linear approximation, computed by finding successive values of the
equation:
x[k]^2 - V x[k+1] = x[k] - ------------
2 x[k]
...where V is the value for which the square root is being sought. In
essence, what is happening here is that we guess a value for the
square root, then figure out how far off we were by squaring our guess
and subtracting the target. Using this value, we compute a linear
approximation for the error, and adjust the "guess". We keep doing
this until the precision gets low enough that the above equation
yields a quotient of zero. At this point, our last guess is one
greater than the square root we're seeking.
The initial guess is computed by dividing V by 4, which is a heuristic
I have found to be fairly good on average. This also has the
advantage of being very easy to compute efficiently, even for large
values.
So, the resulting algorithm works as follows:
x = V / 4 /* compute initial guess */
loop t = (x * x) - V /* Compute absolute error */ u = 2 * x /* Adjust by tangent slope */ t = t / u
/* Loop is done if error is zero */ if(t == 0)
break
/* Adjust guess by error term */ x = x - t
end
x = x - 1
The result of the computation is the value of x.
In Html :
call the cube and square function defined in angular controller
<input type="number" ng-model="numb1" placeholder="number1" ng-change="cube(numb1)" size="5px"><br />
<p>Square of {{ numb1 }}<small><sup>2</sup></small> is {{ cubeResult }}</p>
In Controller(*.js) :
define a function that returns square and cube of particular value
$scope.cubeResult =0;
$scope.square= function(value){
$scope.cubeResult = value * value;
};
$scope.cube = function(value){
$scope.cubeResult = value * value * value;
};

Special meaning of <> and anonymous arrays inside regex in Perl 6

Outside regex, <> behaves more or less like single quotes. My shallow understanding seems to tell me that, inside regex, <> allows evaluation and interpolation of codes:
# Outside regex, <> acts like single quotes:
> my $x = <{"one"}>
{"one"}
> $x.WHAT
(Str)
# Inside regex, <> evaluates and interpolates:
> my $b="one";
one
> say "zonez" ~~ m/ <{$b}> / # Evaluates {$b} then quotes: m/ one /
「one」
> say "zonez" ~~ m/ <$b> / # Interpolates and quotes without {}
「one」
Because an array variable is allowed inside a regex, I suspect that the Perl 6 regex engine expends the array into OR's when there is <> inside regex surrounding the array.
I also suspect that in a user-defined character class, <[ ]>, the array [] inside <> more or less works like an anonymous array in a way similar to #a below, and the contents of the array (chars in the character class) are expended to OR's.
my #a = $b, "two";
[one two]
> so "zonez" ~~ m/ #a /;
True
> say "ztwoz" ~~ m/ <{[$b, "two"]}> / # {} to eval array, then <> quotes
「two」
> say "ztwoz" ~~ m/ <{#a}> /
「two」
> say "ztwoz" ~~ m/ <#a> /
「two」
> say "ztwoz" ~~ m/ one || two / # expands #a into ORs: [||] #a;
# [||] is a reduction operator;
「two」
And char class expansion:
> say "ztwoz" ~~ m/ <[onetw]> / # like [||] [<o n e t w>];
「t」
> say "ztwoz" ~~ m/ o|n|e|t|w /
「t」
> my #m = < o n e t w >
[o n e t w]
> say "ztwoz" ~~ m/ #m /
「t」
I have not looked into the Rakudo source code, and my understanding is limited. I have not been able to construct anonymous arrays inside regex to prove that <> indeed constructs arrays inside regex.
So, is <> inside regex something special? Or should I study the Rakudo source code (which I really try not to do at this time)?
Outside of a regex <> acts like qw<>, that is it quotes and splits on spaces.
say <a b c>.perl;
# ("a", "b", "c")
It can be expanded to
q :w 'a b c'
Q :q :w 'a b c'
Q :single :words 'a b c'
I recommend reading Language: Quoting Constructs as this is a more broad topic than can be discussed here.
This has almost nothing to do with what <> does inside of a regex.
The use of <> in regexes is not useful in base Perl 6 code, and qw is not that useful in regexes. So these characters are doing double duty, mainly because there are very few non-letter and non-number characters in ASCII. The only time it acts like qw is if the character immediately following < is a whitespace character.
Inside of a regex it can be thought of as injecting some code into the regex; sort of like a macro, or a function call.
/<{ split ';', 'a;b;c' }>/;
/ [ "a" | "b" | "c" ] /;
( Note that | tries all alternations at the same time while || tries the leftmost one first, followed by the next one, etc. That is || basically works the way | does in Perl 5 and PCRE. )
/<:Ll - [abc]>/
/ [d|e|f|g|h|i|j|k|l|m|n|o|p|q|r|s|t|u|v|w|x|y|z] / # plus other lowercase letters
/ <#a> /
/ [ "one" | "two" ] /
Note that / #a / also dissolves into the same construct.
/ <?{ 1 > 0 }> /
# null regex always succeeds
/ [ '' ] /
/ <?{ 1 == 0 }> /
# try to match a character after the end of the string
# (can never succeed)
/ [ $ : . ] /
Those last two aren't quite accurate, but may be a useful way to think about it.
It is also used to call regex "methods".
grammar Foo {
token TOP { <alpha> } # calls Grammar.alpha and uses it at that point
}
If you noticed I always surrounded the substitution with [] as it always acts like an independent sub expression.
Technically none of these are implemented in the way I've shown them, it is just a theoretical model that is easier to explain.
Within regex <> are used for what I tend to call "generalized assertions". Whenever you match something with regex, you're making a series of assertions about what the string should look like. If all of the assertions are true, the entire regex matches. For example, / foo / asserts that the string "foo" appears within the string being matched; / f o* / asserts that the string should contain an "f" followed by zero or more "o", etc.
In any case, for generalized assertions, Rakudo Perl 6 uses the character immediately after the < to determine what kind of assertion is being made. If the character after < is alphabetic (e.g. <foo>) it is taken to mean a named subrule; if the character after < is {, it's an assertion that contains code that is to be interpolated into the pattern (e.g., <{ gen_some_regex(); }>); if the character after < is a [, it's a character class; if the character after < is a : then it expects to match an Unicode property (e.g., <:Letter>); if the character after < is a ? or !, you get positive and negative zero-width assertions respectively; etc.
And finally, outside of regex, <> act as "quote words". If the character immediately following the < is a whitespace character, within regex, it will also act as a kind of "quote words":
> "I'm a bartender" ~~ / < foo bar > /
「bar」
This is matched as if it were an alternation, that is < foo bar > will match one of foo or bar as if you'd written foo | bar.

angularjs date filter ignores apostrophe

I would like to have date formatted like here on SO: "Feb 6, '14".
I cannot put the apostrophe there before the number 14.
I tried the following formatter strings:
{{model.since | date:'MMM d, \'yy'}}
{{model.since | date:'MMM d, 'yy'}}
{{model.since | date:"MMM d, 'yy"}}
{{model.since | date:"MMM d, 'yy"}}
The apostrophe is not displayed.
How can I put the apostrophe there?
From the documentation on date:
In order to output single quote, use two single quotes in a sequence (e.g. "h 'o''clock'").
Since a string literal must also be enclosed in single quotes, you need four single quotes in a row. And since you're using single quotes in your format, you need to enclose the entire format string with double quotes, rather than single quotes.
Thus, your formatter string should be {{model.since | date:"MMM d, ''''yy"}}

In a SELECT, take one group of fields or another depending on a field value

What is wrong with the commented out part in the SELECT below ?
If I uncomment the whole CASE...END, the SELECT becomes invalid.
What I want is, depending on wether there is a delivery address or not, take one group of address fields or another, ideally WITHOUT repeating the condition or using a COALESCE for each field. I am using SQL Server 2008 R2.
Thanks !
SELECT a, b, c,
-- CASE x is null
-- THEN d, e, f,
-- ELSE g, h, i,
-- END
k, l, m
FROM sometable
You're missing a WHEN, and you're only allowed to return a single value from a CASE expression:
SELECT a, b, c,
CASE WHEN x is null
THEN d
ELSE g
END,
CASE WHEN x is null
THEN e
ELSE h
END,
CASE WHEN x is null
THEN f
ELSE i
END,
k, l, m
FROM sometable
This also assumes the types of d and g are compatible, as are (e,h) and (f,i)
This query will give you the right answer and it's also easy to modify
select
s.a, s.b, s.c,
a.f1, a.f2, a.f3,
s.k, s.l, s.m
from sometable as s
outer apply (
select s.d as f1, s.e as f2, s.f as f3 where s.x is null
union all
select s.g as f1, s.h as f2, s.i as f3 where s.x is not null
) as a
SQL FIDDLE EXAMPLE
consider to put the condition for x in the where clause and then union
select a,b,c,d,e,f from table where x is null
union
select a,b,c,g,h,i from table where x is not null
the case thing needs a WHEN. I always use this structure
CASE ...
WHEN ...
THEN ...
ELSE ...
END AS Fieldname
all 5 must be present. it can be used to define one ouputfield. within it you could of course coallesce, but it will not give back an array of fields
EDIT: the ELSE is in fact not necessary, read informed comment below

Resources