I have a flat file and used conditional split to filter the record into a single row. For example, RecordType == "2" retrieves single row with record having multiple columns say A,B,C,D and E. I want to pass the result of Column C value to a variable. And then to use it to update the table like:
Update tablename
Set A = that variable
Where A is null
Could you please help me in find out the solution.
I would not use the variable but use a Ole DB Command object.
You set the connection.
Then add your SQL from above:
Update tablename Set A = ? Where A is null
The map to Col C.
However, what I might guess you are trying to do is add a column to your other record set that has the detail but no key.
I would use a script component to do this:
Similar to this example:
Importing Grouped Report Data to Database
Related
I need to read a column on a db on ADF and use all it´s values as parameters in a foreach.
I tried reading the column using a dataflow and a cache sink to then in a pipeline use Set Variable and then the foreach...but instead of an array of values I get an array with one value that contains all the others I want (but i cant iterate over)
I am using:
#array(activity('myDataflow').output.runStatus.output.columName
Any help is appreciated, seems simple enough (column to array) but I am stuck
Use a lookup activity to get the data from SQL server, and run for each loop on the output of the lookup.
Example:
Create a new pipeline
Add a lookup activity
Choose your source dataset (in this example, an Azure SQL database)
Remove the checkbox from “First row only”
Choose a table, stored procedure or type in a query
SELECT 1 AS result UNION ALL
SELECT 2 AS result UNION ALL
SELECT 3 AS result UNION ALL
SELECT 4 AS result
Add a foreach activity
In the foreach activity, under settings tab: “Items” -
#activity('Lookup SQL query').output.value – where 'Lookup SQL
query' is the name of the lookup activity
Inside the foreach loop, add a wait activity
In the settings tab, “Wait time in seconds” : #item().result .
item() is the current loop, and result is the name of the SQL column
debug the pipeline. You can see that the foreach activity iterates 4 times, for every row returned from the sql query.
You can use append variable activity also, inside ForEach after lookup.
First create an array variable in the pipeline.
Then use append variable activity inside ForEach and give
#item.<your_column_name>
Result variable stored in a sample variable:
Result:
I tried reading the column using a dataflow and a cache sink to then in a pipeline use Set Variable and then the foreach
If you want to do it with dataflows instead of lookup, use the same above procedure and give the below dynamic content in the ForEach.
#activity('Data flow1').output.runStatus.output.sink1.value
I'm writing an SSIS package to load data from a .csv into a db.
There's a column in the csv file that is supposed to have a count, but the records sometimes have text, so I can't just load the data in as an integer. It looks something like this:
I want the data to land in the db destination as an integer instead of a string. I want the transformation to change any text to a 1, any blank value to a 1, and leave all the other numbers as-is.
My attempts have so far included using the Derived Column functionality, which I couldn't get the right expression(s) for it seems, and creating a temp table to run a sql query through the data, which kept breaking my data flow.
There are three approaches you can follow.
(1) Using a derived column
You should add a derived column with the following expression to check if the values are numeric or not:
(DT_I4)[count] == (DT_I4)[count] ? [count] : 1
Then in the derived column editor, go to the error output configuration and set the error handling event to Ignore failure.
Now add another derived column to replace null values with 1 :
REPLACENULL([count_derivedcolumn],1)
You can refer to the following article for a step-by-step guide:
Validate Numeric or Non-Numeric Data in SQL Server Integration Services without the Script Task
(2) Using a script component
If you know C# or Visual Basic.NET, you can add a script component to check if the value is numeric and replace nulls and string values with 1
(3) Update data in SQL
You can stage data in its initial form into the SQL database and use an update query to replace nulls and string values with 1 as follows:
UPDATE [staging_table]
SET [count] = 1
WHERE [count] IS NULL or ISNUMERIC([count]) = 0
I did not get much help from Snowflake documentation about how I can take give column names using snowflake functions.
I have automated report which will do calculation for the given dates. here it is
select
sum(case when logdate = to_date(dateadd('day', - 10, '2019-11-14')) then eng_fees + data_fees end) AS to_date(dateadd('day', - 10, '2019-11-14'))
from myTable
where logdate = '2019-11-04'
I am getting following output for my column name below
to_date(dateadd('day', - 10, '2019-11-14'))
100
My expected output for my column name
2019-11-04
100
how can I print expected date as column name in Snowflake?
Your statement is using an AS to name your column. Snowflake will treat that as a literal, not a calculation. In order to do what you're requesting inside a function, you'll need to use a Javascript Function, I think. This will allow you to dynamically build the SQL Statement with your calculated column name predefined.
https://docs.snowflake.net/manuals/sql-reference/udf-js.html
You can't have dynamic column names without using external functionality (or TASK scheduling).
You can create a JavaScript Stored Procedure that generates a VIEW where the column names can be set by dynamic parameters / expressions.
The normal way of handling this is to use a reporting tool that can display your fixed column result set with dynamic headers or run dynamic SQL altogether.
I see 2 paths of getting close to what you want:
1) you can simply use UNION to have your headers displayed as the first row and change the actual column aliases into 1....N - so that they provide the number of column - this is going to be the fastest and the cheapest
2) you can use dynamic SQL to generate a query that will have your alias names dynamically filled and then simply run it (and you can use to example PIVOT to construct such query)
I have 1 .csv file that I import. Because of the layout of the file I need to import it again to get a certain value that needs to be used for the next Data Flow task that will use that value for every row.
Below is an example of the file:
I need STATEMENT DATE for all records.
This is how the package looks like:
Inside Get Statement Date task I have the following Data Flow:
I don't really want to export the column I just want it to go into a variable that I can use in the next task And I am not sure were this should be done.
I am still learning how SSIS works.
Any help would be much appreciated.
Use a script component as Destination and assign that column to your variable.
Keep in mind the variable Datatype should be compatible.
Declare a variable in class ScriptMain to store your value:
public DateTime t;
On Input0_ProcessInputRow(Input0Buffer Row):
t = Row.StatementDate;
On PostExecute:
Variables.teste = t;
It will assign it the number of rows you actually have.(it's not a problem since you are only retrieving one row I guess). Only the last row will actually be saved to your variable.
Use RecordSet Destination. This can be used to store temporary results which will be stored in a variable of type object. You can use this along with Foreach loop to process each record. you can refer this link for more info.
RecordSet Destination
I am attempting to replace specific values in one column with a new value but it does not work (no errors, no replacement of values, nothing happen).
UPDATE Components
SET Unit='kg'
WHERE Unit='КГ'
How can I replace all values "КГ" with "kg" in column Unit?
I thing that your Unit column is NVarChar() data type. Try following query:
UPDATE Components
SET Unit=N'kg'
WHERE Unit=N'КГ'
Another reason: If you have instead of update trigger on Components table and not update this column on it, your update not affected and no raise error too.
I suggest to use Quotename to resolve this, it is used for this type of string.
UPDATE Components
SET Unit=QUOTENAME('kg')
WHERE Unit=QUOTENAME('КГ')
It is simple and direct query which you run, then its ok. Other wise as #Mehdi said, I also fever that statement.
http://www.c-sharpcorner.com/Blogs/7515/quotename-function-in-sql-server.aspx