Using SSIS I am bringing in raw text files that contain this in the output:
I use this data later to report on. The Key columns get pivoted. However, I don't want to show all those columns individually, I only want to show the total.
To accomplish this my idea was calculate the Sum on insert using a trigger, and then insert the sum as a new row into the data.
The output would look something like:
Is what I'm trying to do possible? Is there a better way to do this dynamically on pivot? To be clear I'm not just pivoting these rows for a report, there are other ones that don't need the sum calculated.
Using derived column and Script Component
You can achieve this by following these steps:
Add a derived column (name: intValue) with the following expression:
(DT_I4)(RIGHT([Value],2) == "GB" ? SUBSTRING([Value],1,FINDSTRING( [Value], " ", 1)) : "0")
So if the value ends with GB then the number is taken else the result is 0.
After that add a script component, in the Input and Output Properties, click on the Output and set the SynchronousInput property to None
Add 2 Output Columns outKey , outValue
In the Script Editor write the following script (VB.NET)
Private SumValues As Integer = 0
Public Overrides Sub PostExecute()
MyBase.PostExecute()
Output0Buffer.AddRow()
Output0Buffer.outKey = ""
Output0Buffer.outValue = SumValues.ToString & " GB"
End Sub
Public Overrides Sub Input0_ProcessInputRow(ByVal Row As Input0Buffer)
Output0Buffer.AddRow()
Output0Buffer.outKey = Row.Key
Output0Buffer.outValue = Row.Value
SumValues += Row.intValue
End Sub
I am going to show you a way but I don't recommend adding total to the end of the detail data. If you are going to report on it show it as a total.
After source add a data transformation:
C#
Add two columns to your data flow: Size int and type string
Select Value as readonly
Here is the code:
string[] splits = Row.value.ToString().Split(' '); //Make sure single quote for char
int goodValue;
if(Int32.TryParse(splits[0], out goodValue))
{
Row.Size = goodValue;
Row.Type = "GB";
}
else
{
Row.Size = 0;
Row.Type="None";
}
Now you have the data with the proper data types to do arithmatic in your table.
If you really want the data in your format. Add a multicast and an aggregate and SUM(Size) and then merge back into your original flow.
I was able to solve my problem in another way using a trigger.
I used this code:
INSERT INTO [Table] (
[Filename]
, [Type]
, [DeviceSN]
, [Property]
, [Value]
)
SELECT ms.[Filename],
ms.[Type],
ms.[DeviceSN],
'Memory Device.Total' AS [Key],
CAST(SUM(CAST(left(ms.[Value], 2) as INT)) AS VARCHAR) + ' GB' as 'Value'
FROM [Table] ms
JOIN inserted i ON i.Row# = ms.Row#
WHERE ms.[Value] like '%GB'
GROUP BY ms.[filename],
ms.[type],
ms.[devicesn]
Related
I'm using 2 lookups in ssis for 2 tables. If a column is null in the other table, im going to use the other value and vice versa.
how to implement this one?
Here is one idea how to do it:
Lookup component:
Add the column of the other table as a new column (like 'column2').
After the lookup put a derived column:
Derived column name will be your initial column, set it to replace the original column.
In the expression put:
REPLACENULL(column,column2)
Use your look up component, and add the new column (Correspondant value in Lookup Table)
Assuming that your columns names are LookupColumn1 and LookupColumn2 And OutColumn is the expected output column:
First Method
Add a script component
Mark LookupColumn1 and LookupColumn2 as Input
Add an Output Column OutColumn ( DataType: DT_STR or DT_WSTR)
In your Script Write the following Code (inside ProcessInputRow Sub) :
Public Overrides Sub InputBuffer0_ProcessInputRow(ByVal Row As InputBuffer0)
If Not Row.LookUpColumn2_IsNull AndAlso _
Not String.IsnullOrEmpty(Row.LookUpColumn2.Trim) Then
Row.OutColumn = Row.LookUpColumn2.Trim
ElseIf Not Row.LookupColumn1_IsNull AndAlso _
Not String.IsnullOrEmpty(Row.LookupColumn1.Trim) Then
Row.OutColumn = Row.LookupColumn1.Trim
Else
Row.OutColumn_IsNull = True
End If
End Sub
Script Logic: If LookupColumn2 is not null or empty, it's value is assigned to OutColumn , if LookupColumn2 is null or empty we checks the value of LookupColumn1 : if it is not null or empty, it's value is assigned to OutColumn , else OutColumn is NULL
Second Method
Create a derrived column with the following expression
REPLACENULL(LookupColumn2,LookupColumn1)
Read more about REPLACENULL In this MSDN article
I have an SqlDataReader that is declared like this:
Dim myReader As SqlDataReader
myReader = SqlHelper.ExecuteReader(ConnectionString, "storedProcedure1", CInt(myTextBox.Text))
Later I use the results like this:
If myReader.HasRows Then
While myReader.Read()
Row = Table1.NewRow()
Row.Item("REF") = myReader.GetString(0)
Row.Item("CD") = myReader.GetString(1)
Row.Item("NAME") = myReader.GetString(2)
Row.Item("KEY") = myReader.GetDecimal(3)
Row.Item("STRING") = myReader.GetString(0) & " - " & myReader.GetString(1) & " - " & myReader.GetString(2).ToString().Replace("'", "") & " - " & myReader.GetString(4).ToString().Replace("'", "")
Table1.Rows.Add(Row)
'Fill Drop Down
drpMenu.Items.Add(New ListItem(myReader.GetString(0) & " - " & myReader.GetString(1) & " - " & myReader.GetString(2).ToString().Replace("'", "") & " - " & myReader.GetString(4).ToString().Replace("'", "")))
End While
End If
myTextBox is a textbox that the user enters a possible location number that gets searched for using the stored procedure. If the user enters a valid location number, this works great and I have no problems. If they enter a non-existent location number, I get an exception:
System.IndexOutOfRangeException: Index was outside the bounds of the array.
I would think that the If myReader.HasRows line would keep me from trying to read and manipulate results that don't exist but there must be something I'm missing. myTextBox is already being validated elsewhere in the code to make sure the user typed in an integer without any wacky characters so bad input doesn't seem to be the problem either.
How do I find out whether the location number exists before calling SqlHelper.ExecuteReader()? Or maybe the better question is how do I gracefully handle this exception and tell the user the location wasn't found?
EDIT: Here's the stored procedure.
ALTER PROCEDURE [dbo].[storedProcedure]
-- Add the parameters for the stored procedure here
#MBR as integer
AS
BEGIN
EXEC ('{CALL RM#IMLIB.spGETLOC( ?)}', #MBR) at AS400
END
EDIT #2: When I run the stored procedure in SMS and pass a valid location, it returns what I expect. If I pass an invalid location number, it returns nothing.
First. create a local variable...and cast the textbox value to the local variable...to make sure that isn't the error.
dim myValue as Int32
myValue = '' convert textbox value to an int.
Second...typically, my datareader code looks like this.
If (Not ( reader Is Nothing) ) then
If reader.HasRows Then
Do While reader.Read()
Console.WriteLine(reader.GetInt32(0) _
& vbTab & reader.GetString(1))
Loop
End If
End If
(You'll have to adjust the GetInt32 or GetString and the ordinal number to your specific case of course).
My guess is that your child-procedure (RM#IMLIB.spGETLOC) has logic in it that does NOT return a row if there isn't a match.
You can still return a result....that has no rows in it. ** (Read that again).
For example
Select ColA, ColB from dbo.MyTable where 0=1
This will return a result, with no rows. That is different from not returning a(ny) select statement..(Read that again)
My guess is that this little nuance is where you get an issue.
APPEND:
If you cannot change the child-stored procedure....
Create a #TempTable...
Populate the #TempTable with the child-stored procedure.
Do a select from the #TempTable.
Here is a generic example:
IF OBJECT_ID('tempdb..#TempOrders') IS NOT NULL
begin
drop table #TempOrders
end
CREATE TABLE #TempOrders
(
ColumnA int
, [ColumnB] nchar(5)
)
/* Note, your #temp table must have the exact same columns as returned by the child procedure */
INSERT INTO #TempOrders ( ColumnA, ColumnB )
exec dbo.uspChildProcedure ParameterOne
Select ColumnA, ColumnB from #TempOrders
IF OBJECT_ID('tempdb..#TempOrderDetails') IS NOT NULL
begin
drop table #TempOrderDetails
end
I finally figured out my issue.
Later in my code, I had a DataRow array that was empty if the location couldn't be found. I was getting the exception because it was trying to grab an Item from array(0) which makes complete sense.
I missed it initially because I thought it was a DataRow, not a DataRow array. Man, I hate fixing up code I didn't write...
At my selection Screen you can choose with a radio button Group with which kind of number you want to select the information. (material number, construction contract or customer order).
After choosing a kind, the User have to fill in the number(s) into the corresponding select option.
With this information I select the information I need an move them to the itab t_marc. This table has the same fields as marc.
When the User choose material number at the selection screen everything works fine and the values to every number the user wrote down is displayed at the ALV-List.
When selecting by one of the other numbers, the values in output are also right, but only the information to the last denoted number will be edited.
How can I move all numbers to my itabs?
PARAMETERS: p_mat RADIOBUTTON GROUP radi.
PARAMETERS: p_auf RADIOBUTTON GROUP radi.
PARAMETERS: p_vbl RADIOBUTTON GROUP radi.
SELECT-OPTIONS: s_matnr FOR marc-matnr.
SELECT-OPTIONS: s_aufnr FOR aufk-aufnr.
SELECT-OPTIONS: s_vbeln FOR vbap-vbeln.
start-of-selection
IF p_mat = 'X'.
SELECT * FROM marc
INTO TABLE t_marc
WHERE matnr IN s_matnr
AND werks = p_werks.
ELSEIF p_auf = 'X'.
SELECT * FROM afpo
INTO TABLE t_afpo
WHERE aufnr IN s_aufnr.
LOOP AT t_afpo.
SELECT * FROM marc
INTO TABLE t_marc
WHERE matnr = t_afpo-matnr
AND werks = p_werks.
ENDLOOP.
ELSEIF p_vbl = 'X'.
SELECT * FROM vbap
INTO TABLE t_vbap
WHERE vbeln = s_vbeln-low
AND posnr IN s_posnr.
LOOP AT t_vbap.
SELECT * FROM marc
INTO TABLE t_marc
WHERE matnr = t_vbap-matnr
AND werks = p_werks.
ENDLOOP.
You're overwriting the records every time in this loop (and the similar loops).
LOOP AT t_afpo.
SELECT * FROM marc
INTO TABLE t_marc
WHERE matnr = t_afpo-matnr
AND werks = p_werks.
ENDLOOP.
"INTO TABLE" overwrites every time. You could switch to "APPENDING TABLE." Alternatively, I would use a for all entries select instead (no loop).
SELECT * FROM marc
INTO TABLE t_marc
FOR ALL ENTRIES IN t_afpo
WHERE matnr = t_afpo-matnr
AND werks = p_werks.
Always make sure there are records in the driver table (in this case, t_afpo) or you will have performance issues.
I have a correct MDX-query:
SELECT NON EMPTY { [Measures].[IssueOpened] } ON COLUMNS,
NON EMPTY { ([Projects].[Id].[Id].ALLMEMBERS * [Priorities].[Id].[Id].ALLMEMBERS ) } ON ROWS
FROM [Reports]
WHERE [CreatedOn].[Date].&[2010-01-01T00:00:00]:[CreatedOn].[Date].&[2010-02-01T00:00:00]
I need to create SSRS-report with filter on CreatedOn dimension.
Here is my non-working solution:
I transform query to:
SELECT NON EMPTY { [Measures].[IssueOpened] } ON COLUMNS,
NON EMPTY { ([Projects].[Id].[Id].ALLMEMBERS * [Priorities].[Id].[Id].ALLMEMBERS ) } ON ROWS
FROM (SELECT (STRTOSET(#CreatedOnDate, CONSTRAINED) ) ON COLUMNS
FROM [Reports])
CreatedOnDate parameter (type = datetime)
Set value of CreatedOnDate parameter to value:
="[CreatedOn].[Date].[" + Format(CDate(Parameters!CreatedOnDate.Value), "yyyy-MM-dd") + "T00:00:00]"
But when I run the report I get:
The restriction imposed by the CONSTRAINED flag in the STRTOSET function were violated
Somehow the parameter is not what you think. CONSTRAINED flag will force generating an error if the string is not a member when using StrToSet MDX function (check the failing example) :
You can try without the CONSTRAINED flag or find out the value of your parameter :
WITH
MEMBER myParam AS "[CreatedOn].[Date].[" +
Format(CDate(Parameters!CreatedOnDate.Value), "yyyy-MM-dd")
+ "T00:00:00]"
SELECT
[Measures].[myParam] on 0
FORM [Reports]
Playing a bit with this should make easier spotting the issue.
It should work if you use STRTOMEMBER for each side of the range
STRTOMEMBER("[CreatedOn].[Date].&[2010-01-01T00:00:00]", constrained):STRTOMEMBER("[CreatedOn].[Date].&[2010-02-01T00:00:00]", constrained)
I'm using Linq to Sql and have a stored proc that won't generate a class. The stored proc draws data from multiple tables into a flat file resultset.
The amount of data returned must be as small as possible, the number of round trips to the Sql Server need to be limited, and the amount of server-side processing must be limited as this is for an ASP.NET MVC project.
So, I'm trying to write a Linq to Sql Query however am struggling to both replicate and limit the data returned.
Here's the stored proc that I'm trying to convert:
SELECT AdShops.shop_id as ID, Users.image_url_75x75, AdShops.Advertised,
Shops.shop_name, Shops.title, Shops.num_favorers as hearts, Users.transaction_sold_count as sold,
(select sum(L4.num_favorers) from Listings as L4 where L4.shop_id = L.shop_id) as listings_hearts,
(select sum(L4.views) from Listings as L4 where L4.shop_id = L.shop_id) as listings_views,
L.title AS listing_title, L.price as price, L.listing_id AS listing_id, L.tags, L.materials, L.currency_code,
L.url_170x135 as listing_image_url_170x135, L.url AS listing_url, l.views as listing_views, l.num_favorers as listing_hearts
FROM AdShops INNER JOIN
Shops ON AdShops.shop_id = Shops.shop_id INNER JOIN
Users ON Shops.user_id = Users.user_id INNER JOIN
Listings AS L ON Shops.shop_id = L.shop_id
WHERE (Shops.is_vacation = 0 AND
L.listing_id IN
(
SELECT listing_id
FROM (SELECT l2.user_id , l2.listing_id, RowNumber = ROW_NUMBER() OVER (PARTITION BY l2.user_id ORDER BY NEWID())
FROM Listings l2
INNER JOIN (
SELECT user_id
FROM Listings
GROUP BY
user_id
HAVING COUNT(*) >= 3
) cnt ON cnt.user_id = l2.user_id
) l2
WHERE l2.RowNumber <= 3 and L2.user_id = L.user_id
)
)
ORDER BY Shops.shop_name
Now, so far I can return a flat file but am not able to limit the number of listings. Here's where I'm stuck:
Dim query As IEnumerable = From x In db.AdShops
Join y In (From y1 In db.Shops
Where y1.Shop_name Like _Search + "*" AndAlso y1.Is_vacation = False
Order By y1.Shop_name
Select y1) On y.Shop_id Equals x.shop_id
Join z In db.Users On x.user_id Equals z.User_id
Join l In db.Listings On l.Shop_id Equals y.Shop_id
Select New With {
.shop_id = y.Shop_id,
.user_id = z.user_id,
.listing_id = l.Listing_id
} Take 24 ' Fields ommitted for briefity...
I assume to select a random set of 3 listings per shop, I'd need to use a lambda expression however am not sure how to do this. Also, need to add in somewhere consolidated totals for listing fieelds against individual shops...
Anyone have any thoughts?
UPDATE:
Here's the current solution that I'm looking at:
Result class wrapper:
Public Class NewShops
Public Property Shop_id As Integer
Public Property listing_id As Integer
Public Property tl_listing_hearts As Integer?
Public Property tl_listing_views As Integer?
Public Property listing_creation As Date
End Class
Linq + code:
Using db As New Ads.DB(Ads.DB.Conn)
Dim query As IEnumerable(Of IGrouping(Of Integer, NewShops)) =
(From x In db.AdShops
Join y In (From y1 In db.Shops
Where (y1.Shop_name Like _Search + "*" AndAlso y1.Is_vacation = False)
Select y1
Skip ((_Paging.CurrentPage - 1) * _Paging.ItemsPerPage)
Take (_Paging.ItemsPerPage))
On y.Shop_id Equals x.shop_id
Join z In db.Users On x.user_id Equals z.User_id
Join l In db.Listings On l.Shop_id Equals y.Shop_id
Join lt In (From l2 In db.Listings _
Group By id = l2.Shop_id Into Hearts = Sum(l2.Num_favorers), Views = Sum(l2.Views), Count() _
Select New NewShops With {.tl_listing_views = Views,
.tl_listing_hearts = Hearts,
.Shop_id = id})
On lt.Shop_id Equals y.Shop_id
Select New NewShops With {.Shop_id = y.Shop_id,
.tl_listing_views = lt.tl_listing_views,
.tl_listing_hearts = lt.tl_listing_hearts,
.listing_creation = l.Creation,
.listing_id = l.Listing_id
}).GroupBy(Function(s) s.Shop_id).OrderByDescending(Function(s) s(0).tl_listing_views)
Dim Shops as New Dictionary(Of String, List(Of NewShops))
For Each item As IEnumerable(Of NewShops) In query
Shops.Add(item(0).shop_name, (From i As NewShops In item
Order By i.listing_creation Descending
Select i Take 3).ToList)
Next
End Using
Anyone have any other suggestions?
From the looks of that SQL and code, I'd not be turning it into LINQ queries. It'll just obfuscate the logic and probably take you days to get it correct.
If SQLMetal doesn't generate it properly, have you considered using the ExecuteQuery method of the DataContext to return a list of the items you're after?
Assuming that your sproc you're trying to convert is called sp_complicated, and takes in one parameter, something like the following should do the trick
Protected Class TheResults
Public Property ID as Integer
Public Property image_url_75x75 as String
'... and so on and so forth for all the returned columns. Be careful with nulls
End Class
'then, when you want to use it
Using db As New Ads.DB(Ads.DB.Conn)
dim results = db.ExecuteQuery(Of TheResults)("exec sp_complicated {0}", _Search)
End Using
Before you freak out, that's not susceptible to SQL Injection. L2SQL uses proper SQLParameters, as long as you use the squigglies and don't just concatenate the strings yourself.