I have a JSON file and when I create an external table in Snowflake and query it the result is each row is a JSON record.
Can I flatten the structure when creating the external table to put each field in a col and row format instead?
Thanks
Yes. You'd define each column in your external table definition. There are some good parquet examples here: https://docs.snowflake.com/en/sql-reference/sql/create-external-table.html which would be very similar to your JSON use-case.
I would also recommend using an MV over that external table to improve performance.
Related
I have a scenario where I need to put my CSV files into multiple tables. These table names will be different or unknown until run time. How can I create the SSIS package to load data into a unknown table name. To be specific, I have a table A and table B. Structure of both these tables are same. Today they maybe called Table A and Table B, but they may change to table C and table D with same structure. Now, I want to write a ssis package to load data into these 2 tables, but since the table names change all the time, I do not know on how to accomplish this
Thanks!
I do not know where to begin from
Can I create multi columns partition in snowflake external table
partition by ( date, time)?
also is it possible to create partition table from table columns( such name, create date etc). all the document is indicating use metdata$filename column to create the partition column.
A partition column definition is an expression that parses the column metadata in the internal (hidden) METADATA$EXTERNAL_TABLE_PARTITION column. As of now if you want to use any column from your data files, you need to define the column in the form of Metadata$external_table.
https://docs.snowflake.com/en/sql-reference/sql/create-external-table.html#partitioning-parameters
You can create multiple partitions like data and time however, you need to be selective to make sure that the partition you are specifying is more performant.
External table partition is based on the file paths, which is stored as metadata$filename to the external table.
Yes, you can define multiple columns as partitions for Snowflake's external tables like "partition by (date, time)", but they need to be from the metadata, not the actual data column in the external table file, because they are not used to partition the actual files.
I have created a lookup table for an existing table and now I want to undo what I did and bring the original table to its original form.
I, unfortunately, don't know how to do so.
Creating a lookup table generates a table and a sequence. The format is <column_name>_LOOKUP (for the table) and <column_name>_LOOKUP_SEQ (for the sequence).
They can both be dropped in the object browser.
I want to know the way how postgresql manage columns of table.
Say for e.g
I have created one table that contains 2 fields, so how postgresql manage these columns, table? In how many tables postgresql create entry for a single column ?
I would like to know the structure how the postgresql manage table and it's fields.
I only about pg_attribute table.
It would be good if anyone can share useful links.
Any help would be really appriciated.
Tables (and indexes) are organized in 8KB blocks in files in the data directory.
The column definitions are only in pg_attribute.
A table row with all its columns is stored together in one table block, and a table block can contain several such rows. In other words, PostgreSQL uses the traditional row oriented storage model.
Details can be read in the documentation.
Note: Don't use PostgreSQL 9.1 any more.
When I create a full text index on a temporary table #Table in a query? I got Invalid name object #Table.
Is creating full text index possible in sql server?
According to the documentation, no it is not possible:
A full-text index must be defined on a base table; it cannot be
defined on a view, system table, or temporary table.
This should be clarified to point out that since the version that documentation was written for, indexed views were added to SQL Server, and documentation there states that:
one full-text index is allowed per table or indexed view