I'm using 2 lookups in ssis for 2 tables. If a column is null in the other table, im going to use the other value and vice versa.
how to implement this one?
Here is one idea how to do it:
Lookup component:
Add the column of the other table as a new column (like 'column2').
After the lookup put a derived column:
Derived column name will be your initial column, set it to replace the original column.
In the expression put:
REPLACENULL(column,column2)
Use your look up component, and add the new column (Correspondant value in Lookup Table)
Assuming that your columns names are LookupColumn1 and LookupColumn2 And OutColumn is the expected output column:
First Method
Add a script component
Mark LookupColumn1 and LookupColumn2 as Input
Add an Output Column OutColumn ( DataType: DT_STR or DT_WSTR)
In your Script Write the following Code (inside ProcessInputRow Sub) :
Public Overrides Sub InputBuffer0_ProcessInputRow(ByVal Row As InputBuffer0)
If Not Row.LookUpColumn2_IsNull AndAlso _
Not String.IsnullOrEmpty(Row.LookUpColumn2.Trim) Then
Row.OutColumn = Row.LookUpColumn2.Trim
ElseIf Not Row.LookupColumn1_IsNull AndAlso _
Not String.IsnullOrEmpty(Row.LookupColumn1.Trim) Then
Row.OutColumn = Row.LookupColumn1.Trim
Else
Row.OutColumn_IsNull = True
End If
End Sub
Script Logic: If LookupColumn2 is not null or empty, it's value is assigned to OutColumn , if LookupColumn2 is null or empty we checks the value of LookupColumn1 : if it is not null or empty, it's value is assigned to OutColumn , else OutColumn is NULL
Second Method
Create a derrived column with the following expression
REPLACENULL(LookupColumn2,LookupColumn1)
Read more about REPLACENULL In this MSDN article
Related
output--
I have written the query in snowflake to generate Json file, from the query output want to remove fields which has NULL. OBJECT_CONSTRUCT is not working properly for some column its not passing NULL value where else for some column its giving null value as result.
Input-
Json remove any field which has value NULL or blank.
{"DIFID":122,"DIF_FLAG":"NULL","DIF_TYPE":"asian/white","FOCAL_COUNT":2370,"REFERENCE_COUNT":17304},
Required Output-
Json remove any field which has value NULL or blank.
{"DIFID":122,"DIF_TYPE":"asian/white","FOCAL_COUNT":2370,"REFERENCE_COUNT":17304},
query-
select distinct ITEMSTATID,object_construct(
'DIFID',DIFID,
'DIF_TYPE',DIF_TYPE,
'DIF_FLAG',DIF_FLAG,
'FOCAL_COUNT',FOCAL_COUNT::integer,
'REFERENCE_COUNT',REFERENCE_COUNT::integer,
'DIF_METHOD',DIF_METHOD,
'DIF_VALUE',DIF_VALUE)
DIF
from DEV_IPM.STAGEVAULT.DIF_STATISTICS;
For string column and 'NULL' as string literal column's value is not skipped:
CREATE OR REPLACE TABLE DIF_STATISTICS
AS
SELECT 1 AS ITEMSTATID,
122 AS DIFID,
'NULL' AS DIF_FLAG, -- here
'asian/white' AS DIF_TYPE,
2370 AS FOCAL_COUNT,
17304 AS REFERENCE_COUNT;
Output:
The value is definitely stored as TEXT:
SELECT null AS DIF_FLAG, 'NULL' AS DIF_FLAG;
On the left: true NULL on the right: NULL string
If it the case then it should be nullified NULLIF(DIF_FLAG, 'NULL') before passing to OBJECT_CONSTRUCT function:
SELECT ITEMSTATID,
object_construct(
'DIFID',DIFID,
'DIF_TYPE',DIF_TYPE,
'DIF_FLAG',NULLIF(DIF_FLAG, 'NULL'),
'FOCAL_COUNT',FOCAL_COUNT::integer,
'REFERENCE_COUNT',REFERENCE_COUNT::integer) AS DIF
FROM DIF_STATISTICS;
Previous answer before column details were provided (also plausible):
It is working as intended:
NULL Values
Snowflake supports two types of NULL values in semi-structured data:
SQL NULL: SQL NULL means the same thing for semi-structured data types as it means for structured data types: the value is missing or unknown.
JSON null (sometimes called “VARIANT NULL”): In a VARIANT column, JSON null values are stored as a string containing the word “null” to distinguish them from SQL NULL values.
OBJECT_CONSTRUCT
If the key or value is NULL (i.e. SQL NULL), the key-value pair is omitted from the resulting object. A key-value pair consisting of a not-null string as key and a JSON NULL as value (i.e. PARSE_JSON(‘NULL’)) is not omitted.
For true SQL NULL values, that column is ommitted:
CREATE OR REPLACE TABLE DIF_STATISTICS
AS
SELECT 1 AS ITEMSTATID,
122 AS DIFID,
NULL AS DIF_FLAG,
'asian/white' AS DIF_TYPE,
2370 AS FOCAL_COUNT,
17304 AS REFERENCE_COUNT;
SELECT ITEMSTATID,
object_construct(
'DIFID',DIFID,
'DIF_TYPE',DIF_TYPE,
'DIF_FLAG',DIF_FLAG,
'FOCAL_COUNT',FOCAL_COUNT::integer,
'REFERENCE_COUNT',REFERENCE_COUNT::integer) AS DIF
FROM DIF_STATISTICS;
Output:
Probably the data type of the column DIFID is VARIANT/OBJECT:
CREATE OR REPLACE TABLE DIF_STATISTICS
AS
SELECT 1 AS ITEMSTATID,
122 AS DIFID,
PARSE_JSON('NULL') AS DIF_FLAG, -- here
'asian/white' AS DIF_TYPE,
2370 AS FOCAL_COUNT,
17304 AS REFERENCE_COUNT;
Output:
I have defined an expression in one of my cells from my dataset1. My Design window and I have to repeat for each month's cells but I'm getting an #ERROR when I click on the PREVIEW tab in SSRS.
My thought is if the ActivityMonth value = 1 and the Type value = "PIF" then display the value in the Data column.
=IIF(Fields!Activity_Month.Value = 1 AND Fields!Type.Value = "PIF", Fields!Data.Value, 0)
I got this WARNING from SSRS:
[rsRuntimeErrorInExpression] The Value expression for the textrun ‘Textbox1471.Paragraphs[0].TextRuns[0]’ contains an error: Input string was not in a correct format.
But it ran successfully.
From the comments and the edit history, it looks like you have used & mark which is used to concatenate strings instead of AND keyword. After editing the expression - for me - the following expression looks great:
=IIF(Fields!Activity_Month.Value = 1 AND Fields!Type.Value = "PIF", Fields!Data.Value, 0)
But i have two remarks:
It may cause an error due to the different data types returned by the expression (0 is integer, Data.Value has another data type:
If Fields!Data.Value is of type string then use the following expression:
=IIF(Fields!Activity_Month.Value = 1 AND Fields!Type.Value = "PIF", Fields!Data.Value, "0")
Another thing to mention is that if value contains null it may throw an exception, so you have to check if field contains null:
SSRS expression replace NULL with another field value
isnull in SSRS expressions
Using SSIS I am bringing in raw text files that contain this in the output:
I use this data later to report on. The Key columns get pivoted. However, I don't want to show all those columns individually, I only want to show the total.
To accomplish this my idea was calculate the Sum on insert using a trigger, and then insert the sum as a new row into the data.
The output would look something like:
Is what I'm trying to do possible? Is there a better way to do this dynamically on pivot? To be clear I'm not just pivoting these rows for a report, there are other ones that don't need the sum calculated.
Using derived column and Script Component
You can achieve this by following these steps:
Add a derived column (name: intValue) with the following expression:
(DT_I4)(RIGHT([Value],2) == "GB" ? SUBSTRING([Value],1,FINDSTRING( [Value], " ", 1)) : "0")
So if the value ends with GB then the number is taken else the result is 0.
After that add a script component, in the Input and Output Properties, click on the Output and set the SynchronousInput property to None
Add 2 Output Columns outKey , outValue
In the Script Editor write the following script (VB.NET)
Private SumValues As Integer = 0
Public Overrides Sub PostExecute()
MyBase.PostExecute()
Output0Buffer.AddRow()
Output0Buffer.outKey = ""
Output0Buffer.outValue = SumValues.ToString & " GB"
End Sub
Public Overrides Sub Input0_ProcessInputRow(ByVal Row As Input0Buffer)
Output0Buffer.AddRow()
Output0Buffer.outKey = Row.Key
Output0Buffer.outValue = Row.Value
SumValues += Row.intValue
End Sub
I am going to show you a way but I don't recommend adding total to the end of the detail data. If you are going to report on it show it as a total.
After source add a data transformation:
C#
Add two columns to your data flow: Size int and type string
Select Value as readonly
Here is the code:
string[] splits = Row.value.ToString().Split(' '); //Make sure single quote for char
int goodValue;
if(Int32.TryParse(splits[0], out goodValue))
{
Row.Size = goodValue;
Row.Type = "GB";
}
else
{
Row.Size = 0;
Row.Type="None";
}
Now you have the data with the proper data types to do arithmatic in your table.
If you really want the data in your format. Add a multicast and an aggregate and SUM(Size) and then merge back into your original flow.
I was able to solve my problem in another way using a trigger.
I used this code:
INSERT INTO [Table] (
[Filename]
, [Type]
, [DeviceSN]
, [Property]
, [Value]
)
SELECT ms.[Filename],
ms.[Type],
ms.[DeviceSN],
'Memory Device.Total' AS [Key],
CAST(SUM(CAST(left(ms.[Value], 2) as INT)) AS VARCHAR) + ' GB' as 'Value'
FROM [Table] ms
JOIN inserted i ON i.Row# = ms.Row#
WHERE ms.[Value] like '%GB'
GROUP BY ms.[filename],
ms.[type],
ms.[devicesn]
I have an SqlDataReader that is declared like this:
Dim myReader As SqlDataReader
myReader = SqlHelper.ExecuteReader(ConnectionString, "storedProcedure1", CInt(myTextBox.Text))
Later I use the results like this:
If myReader.HasRows Then
While myReader.Read()
Row = Table1.NewRow()
Row.Item("REF") = myReader.GetString(0)
Row.Item("CD") = myReader.GetString(1)
Row.Item("NAME") = myReader.GetString(2)
Row.Item("KEY") = myReader.GetDecimal(3)
Row.Item("STRING") = myReader.GetString(0) & " - " & myReader.GetString(1) & " - " & myReader.GetString(2).ToString().Replace("'", "") & " - " & myReader.GetString(4).ToString().Replace("'", "")
Table1.Rows.Add(Row)
'Fill Drop Down
drpMenu.Items.Add(New ListItem(myReader.GetString(0) & " - " & myReader.GetString(1) & " - " & myReader.GetString(2).ToString().Replace("'", "") & " - " & myReader.GetString(4).ToString().Replace("'", "")))
End While
End If
myTextBox is a textbox that the user enters a possible location number that gets searched for using the stored procedure. If the user enters a valid location number, this works great and I have no problems. If they enter a non-existent location number, I get an exception:
System.IndexOutOfRangeException: Index was outside the bounds of the array.
I would think that the If myReader.HasRows line would keep me from trying to read and manipulate results that don't exist but there must be something I'm missing. myTextBox is already being validated elsewhere in the code to make sure the user typed in an integer without any wacky characters so bad input doesn't seem to be the problem either.
How do I find out whether the location number exists before calling SqlHelper.ExecuteReader()? Or maybe the better question is how do I gracefully handle this exception and tell the user the location wasn't found?
EDIT: Here's the stored procedure.
ALTER PROCEDURE [dbo].[storedProcedure]
-- Add the parameters for the stored procedure here
#MBR as integer
AS
BEGIN
EXEC ('{CALL RM#IMLIB.spGETLOC( ?)}', #MBR) at AS400
END
EDIT #2: When I run the stored procedure in SMS and pass a valid location, it returns what I expect. If I pass an invalid location number, it returns nothing.
First. create a local variable...and cast the textbox value to the local variable...to make sure that isn't the error.
dim myValue as Int32
myValue = '' convert textbox value to an int.
Second...typically, my datareader code looks like this.
If (Not ( reader Is Nothing) ) then
If reader.HasRows Then
Do While reader.Read()
Console.WriteLine(reader.GetInt32(0) _
& vbTab & reader.GetString(1))
Loop
End If
End If
(You'll have to adjust the GetInt32 or GetString and the ordinal number to your specific case of course).
My guess is that your child-procedure (RM#IMLIB.spGETLOC) has logic in it that does NOT return a row if there isn't a match.
You can still return a result....that has no rows in it. ** (Read that again).
For example
Select ColA, ColB from dbo.MyTable where 0=1
This will return a result, with no rows. That is different from not returning a(ny) select statement..(Read that again)
My guess is that this little nuance is where you get an issue.
APPEND:
If you cannot change the child-stored procedure....
Create a #TempTable...
Populate the #TempTable with the child-stored procedure.
Do a select from the #TempTable.
Here is a generic example:
IF OBJECT_ID('tempdb..#TempOrders') IS NOT NULL
begin
drop table #TempOrders
end
CREATE TABLE #TempOrders
(
ColumnA int
, [ColumnB] nchar(5)
)
/* Note, your #temp table must have the exact same columns as returned by the child procedure */
INSERT INTO #TempOrders ( ColumnA, ColumnB )
exec dbo.uspChildProcedure ParameterOne
Select ColumnA, ColumnB from #TempOrders
IF OBJECT_ID('tempdb..#TempOrderDetails') IS NOT NULL
begin
drop table #TempOrderDetails
end
I finally figured out my issue.
Later in my code, I had a DataRow array that was empty if the location couldn't be found. I was getting the exception because it was trying to grab an Item from array(0) which makes complete sense.
I missed it initially because I thought it was a DataRow, not a DataRow array. Man, I hate fixing up code I didn't write...
I have a working copy of an application that will open workbooks/sheets and copy data succesffully between the two then saves but I need to parse some data as I copy it into another cell.
I was thinking..
~ create array
~ get all values in xlSourceFile.worksheets("sheet1") and store into an array
~ parse through the array extracting the data I need (text-to-column programatically)
~ write the array data to two specific columns in excel worksheet
the data I am trying to parse is Firstname / Lastname - Email and I want this as a result:
Joe Shmoe to go into one column // Joe Shmoes Email to another column.
I am writing this in vb.net using the imports Microsoft.Office.Interop to manipulate Excel.
Excuse the formatting, I'm new to SO. This is VBA but I believe the general logic will work. It assumes that the email address has no space padding after it. It searches backward on the raw combined string for the first blank space and flags that as the start of the email address (end of the name).
It loops out when the next cell is empty.
The data is assumed to look like this:
"First Name Last Name myaddress#example.com"
For Each cell In Worksheets("Sheet1").Range("A:A")
i = i + 1
If cell = "" Then GoTo loopout
rawstring = cell.Value
'rawString = "First Name Last Name myaddress#example.com"
emailStartPosition = InStrRev(rawstring, " ")
myname = Left(rawstring, emailStartPosition)
myemail = Right(rawstring, Len(rawstring) - emailStartPosition)
Worksheets("Sheet1").Range("B" & i).Value = myname
Worksheets("Sheet1").Range("C" & i).Value = myemail
Next
loopout:
End Sub
Column B will have the name and Column C will have the email address.