I've an ADX table with 50+ columns and the data in the table is hundreds of millions records. I want to iterate each column in the table and fetch all the columns that doesn't have any data (All column is empty).
I was able to achieve using the IsEmpty() on a single column.
MyAdxTable
| where isempty(columnName) == false
| count
However, I need to execute this on all the columns of this table and other tables in my database.
Is there a way to iterate each column of the table programmatically, so this could be achieved by writing an adx function which accepts a table name and the function iterates all the columns programmatically and return the response of all the columns that has empty values.
Related
I have two tables.
1) Staging table with multiple columns and Date being one of them.
Date
9/1/2018
2) A Date Dimension Table which has only one column called as Date
Date
1/1/2018
2/1/2018
3/1/2018
4/1/2018
I am writing a logic in SSIS, where it checks the staging table with the dimension table and inserts the missing dates in the dimension table.
To do this, I use the following logic.
The Lookup component has the correct 1 row input from staging table and returns a value of NULL. So, the insertion fails due to constrains.
I do have re-direct non matched rows to no match output enabled inside the Lookup on screen 1.
Kindly help me with this.
The solution is to change the Lookup operation to add as a New Column :-
That's not the problem. The problem is when a date gets past the lookup and is a duplicate.
Run it through an Aggregate [group by] on the date column before inserting into the dimension.
Make sure you are using the correct date. There will be no lookup matches for the records you want to insert (therefore that column is null by default). You shouldn't even add any columns from lookup for this use.
I have two tables, A and B. Both are related by a field. What I want to do is update a field of a subset of data from table B. This subset is filtered by a table A data. The data to be set is also taken from table B but from another different subset of data also filtered by another data from table A. Can this be done in DB2 using a direct query?
Thank you
This shows the details of the table where it has single column and having multiple rows .I have to get the data in such a way that the rows after each DATE valued row has to be selected into different columns. for example in the table ,first row is the DATETIME followed by 3 rows with TEXT value ,then I need DATE in one column and remaining 3 rows in other columns . Totally one row wth 4 different columns. Similarly we have other rows which has one DATE row followed by single char rows or two chars rows. I had tried few queries by splitting each row in to different tables based on comman character and querying into single table using below link:
http://sqlhints.com/2013/09/07/joining-two-tables-without-any-common-column-between-them-sql-server/
However the data is not corresponding to original data..
Please help me with login por query.
I am new to SSIS and I hope someone can point me in the right direction!
I need to move data from one database to another. I have written a query that takes data from a number of tables (SOURCE). I then use a conditional split (Condition: Id = id) to a number of tables in the destination database. Here is my problem, I need another table populating which takes the ‘id’ value from the three tables and uses them in a fourth table as attributes, along with additional data from SOURCE.
I think I need to pass the id values to parameters but there does not seem a way to do this when inserting to ADO NET Destination.
Fourth table will have inserted id values(auto incremented) from table1, table2 and table3.
Am I going about this correctly or is there a better way?
Thanks in advance!
I know of no way to get the IDENTITY values of rows inserted in a Dataflow destination for use in the same Dataflow.
Probably the way to do what you want to do is to make a fourth branch in your dataflow inserting the columns that you have into the fourth table, and leaving the foreign keys (the ids from the other 3 tables) blank.
Then after the Dataflow, use an ExecuteSQL task to call a stored procedure that populates the missing columns in the fourth table by looking up their ids in the other three tables.
If your fourth table doesn't have the values you need to lookup the ids in the other three tables, then you can have the dataflow go to a staging table that does have those values, and populate the fourth table from the staging table while looking up the ids from the corresponding values.
I have a table with few columns Name,stage1, stage2, stage3, stage4, stage5. Now I want to insert values into these columns, but every time when a new row is being inserted into the table, the number of stages to be entered under each row is undefined For Example:-
Suppose in row 1 values for stage1 and stage2 are defined, and for row 2 values for stage1, stage2, stage3 and stage4 are defined and so on
Problem
I am not able to insert a new row in the table because of the uneven distribution of values for each name.
You basically want to use a Relational database for unstructured data. I assume this because you tagged SQL Server.
This is what document dbs or noSQL dbs were designed for.
However, you can emulate this if you want by storing the data in a single column. Within that column you can store either JSON or XML. JSON is going to be hard to query, but SQL Server does support an XML field type that you can query with XPath.
The other option is to rotate your data 90 degrees. Instead of each stage being a column stage1, stage2, stage3.... create a row for each stage. Each row would have a stageNumber field or some such. You could later pivot this data to display it with stages as columns in Excel or some other pivot table or whatever.