We use DBT for ELT in snowflake. Want to add comments to each column in Snowflake. Either using COMMENT or ALTER command after every full refresh.
Decided to add macros with commands and call it under on-run-end hook.
{% macro comment_transactions_master() %}
{% if execute %}
(COMMENT ON COLUMN
"DEV_SCHEMA"."DBT_TEMP"."TR_MASTER"."TR_ID" IS 'testing comment';)
{% endif %}
{% endmacro %}
Since there are 100+ columns and I am new to DBT, is there a better way to perform this operation?
I don't know about snowflake but I know in other databases, you can add comments to multiple columns in a table like so:
comment on column schema.table (
a is 'just a comment',
b is 'just another comment'
)
So for this you can use this macro:
{% macro snowflake__alter_column_comment(relation, column_dict) %}
COMMENT on COLUMN {{ relation }} (
{% for column_name in column_dict %}
{% set comment = column_dict[column_name]['description'] %}
{{ column_name }} is '{{ comment }}'{%- if not loop.last %}, {% endif -%}
{% endfor %}
)
{% endmacro %}
And add this to snowflakes persist_docs macro:
{% macro snowflake__persist_docs(relation, model, for_relation, for_columns) -%}
{# -- Override the persist_docs default behaviour to add the short descriptions --#}
.........
{# Add the new macro here #}
{% if for_columns and config.persist_column_docs() and model.columns %}
{% do run_query(alter_column_comment(relation, model.columns)) %}
{% endif %}
{% endmacro %}
Persist_docs is in almost every materialization so you should be fine. Let me know if this helps.
I am pretty new to DBT myself but I think I found a simpler way that works at least on Snowflake, and which doesn't involve defining macros.
It's described in full detail here; https://docs.getdbt.com/reference/resource-configs/persist_docs
Essentially, you write a models/schema.yml containing descriptions for your relations (tables or views) and each of their columns. And then in a model's own config block you add persist_docs={"relation"=true, "columns"=true}
Related
Using HubL (as I'm building a module in HubSpot), I have two arrays:
topics : Which is a list of topics.
all_tags: Which is an array of all the blog tags in the system.
If I dump out these arrays, this is what it will return:
{{ topics }} prints the following: [Data, Accounting, Insight]
{{ all_tags }} prints the following: [Accounting, Press, Data]
So essentially, {{ topics }} has a tag ("Insight") that doesn't exist in the system yet.
What I'm trying to do is to create a third array, which will contain matching results from the two above arrays. For example, topics_final, once returned, should print [Data, Accounting].
But, when printing {{ topics_final }}, the array is empty.
What I've tried:
<!-- this gets all tags -->
{% set all_tags = blog_topics( blog_id , 250) %}
<!-- create arrays -->
{% set topics = [] %}
{% set topics_final = [] %}
<!-- append topic data to the array -->
{% for item in module.add_topics.topics %}
{% set topic_option = item|striptags %}
{% do topics.append( topic_option ) %}
{% endfor %}
<!-- check if topic tags exists in HubSpot -->
{% for topics in all_tags %}
{% if topics in all_tags %}
{{ topics }}
<!-- results with above
Data, Accounting, Insight
-->
{% else %}
else
{% endif %}
{% endfor %}
With the above, it just prints out the {{ topics }}, even though Insight isn't in the all_tags array.
Note: Tagging Jinja2 as the syntax is similar
A combination of the filter reject and the built in test in could help you achieve this.
Sadly, it seems the reject filter does not accept negating a test, still, you can reject all the elements of topics that are not in all_tags then reject the said elements from the topics list.
So that ends with:
{{ topics | reject('in', topics | reject('in', all_tags) | list) | list }}
Switch yields:
[
"Data",
"Accounting"
]
I have a products object:
assign rel_products = collection.products
How can I check for a specific product within this object? By id preferably. I'm doing this check so I can continue in a for loop:
{% if rel_products contains related_product.id %}
{% continue %}
{% endif %}
This code above isn't working.
I am not very familiar with liquid but the problem in your code snippet looks like you are assigning the collection to the variable rel_products, so you will have to loop through that and then check the id in each iteration. Possibly something like this?
{% for product in rel_products %}
{% if product contains 'Id you want to check' %}
{% continue %}
{% endif %}
{% endfor %}
I think this might be the solution you needed.
{% assign rel_products = collection.products %}
{% for product in rel_products %}
{% if product.id == 12345789 %} //123456789 is just an example
<!-- Something you want to do -->
{% endif %}
{% endfor %}
I've been developing a website for my company g7g20.com and we are interested in creating a most popular articles or top 10 articles panel and was wondering if there was a way of accessing the analytics to display this?
We already have a latest and related and just want to mix it up a bit. Any thoughts would be most welcome.
The following code will give you all products which sold more than 5 units during the last 100 days.
{% assign date = globals.site.dateNow | date_add: -100, "day" -%}
{module_data resource="products" version="v3" fields="id" limit="100" where="\{'orders.units':\{'$gt':'5'\},'orders.createDate':\{'$gt':'{{date}}'\}\}" collection="units"}
<pre>{{units|json}}</pre>
You can then use the product IDs like so and render the products
{% for item in units.items -%}
{module_data resource="catalogproducts" version="v3" limit="1" fields="productId,catalogueId" where="\{'product.id':'{{item.id}}'\}" order="catalogueId" collection="products"}
{% for item in products.items -%}
{% assign CATID = {{item.catalogueId}} -%}
{% assign PROID = {{item.productId}} -%}
{module_product,{{CATID}},{{PROID}}}
{% endfor -%}
{% endfor -%}
I'm trying to make a simple IF function that checks if the date of an item is equal to today or not. Unfortunately I can't get it to work. I basically decides that the statement is false and doesn't show any content, even when it show. I am not getting any error either.
The code I use is following:
{% if item.valid_until.date == now.date %}
<div id="what_i_want_to_show">
CONTENT
</div>
{% endif %}
The content of valid_until is a DateTimeProperty from a Google App Engine app. Normally working with this in Django template doesn't cause any problems. Solutions from similar questions on SO didn't work so far. Am I missing something obvious?
UPDATE 1:
This statement runs in a loop on the result of a database query. Therefore doing the comparison in the view didn't work as far as I could see, since I would have to send the variable with each item.
There are 2 aproach on this case:
1st:
you can add a #property on model
Model:
from datetime import date
#property
def is_past_due(self):
return timezone.now() > self.valid_until # if valid until default is timezone.now else change it
Template:
{% if item.is_past_due %}
<!--In the past-->
{% else %}
{{ item.valid_until.date|date:"Y-m-d" }}
{% endif %}
2nd:
declare a today date with format on template
{% now "Y-m-d" as todays_date %}
{% if todays_date < item.valid_until.date|date:"Y-m-d" %}
<div id="what_i_want_to_show">
CONTENT
</div>
{% endif %}
I made a very minor mod to the GqlQuery to retrieve only specified records using the
'where' keyword. The output, however, displays all entries from the guestbook db!
(I need to filter the data by author)
Guestbook5_datastore code:
#greetings = db.GqlQuery("SELECT * FROM Greeting ORDER BY date DESC LIMIT 10")
greetings = db.GqlQuery("SELECT * FROM Greeting where greeting.author='mike'")
index.html code:
{% for greeting in greetings %}
{% if greeting.author %}
<b>{{ greeting.author.nickname }}</b> wrote:
{% else %}
An Anonymous person wrote:
{% endif %}
<blockquote>{{ greeting.content|escape }}</blockquote>
{% endfor %}
Your author property is not a string, so I don't think you can do
greeting.author='mike'
I'm surprised that you wouldn't get an error telling you that though, rather than it returning them all!
You're attempting to filter based on a property of another entity, which would require a join. This isn't supported in App Engine.