I have created a lookup for a list of IDs and a subsequent Foreach loop to run an sql stmt for each ID.
My variable for catching the list of IDs is called MissingRecordIDs and is of type Object. In the Foreach container I map each value to a variable called RecordID of type Int32. No fancy scripts - I followed these instructions: https://www.simple-talk.com/sql/ssis/implementing-foreach-looping-logic-in-ssis-/ (without the file loading part - I am just running an SQL stmt).
It runs fine from within SSIS, but when I deploy it my Integration Services Catalogue in MSSQL it fails.
This is the error I get when running from SQL Mgt Studio:
I thought I could just put a Precendence Constraint after MissingRecordsIDs get filled to check for NULL and skip the Foreach loop if necessary - but I can't figure out how to check for NULL in an Object variable?
Here is the Variable declaration and Object getting enumerated:
And here is the Variable mapping:
The SQL stmt that is in 'Lookup missing Orders':
select distinct cast(od.order_id as int) as order_id
from invman_staging.staging.invman_OrderDetails_cdc od
LEFT OUTER JOIN invman_staging.staging.invman_Orders_cdc o
on o.order_id = od.order_id and o.BatchID = ?
where od.BatchID = ?
and o.order_id is null
and od.order_id is not null
In the current environment this query returns nothing - there are no missing Orders, so I don't want to go into the 'Foreach Order Loop' at all.
This is a known issue Microsoft is aware of: https://connect.microsoft.com/SQLServer/feedback/details/742282/ssis-2012-wont-assign-null-values-from-sql-queries-to-variables-of-type-string-inside-a-foreach-loop
I would suggest to add an ISNULL(RecordID, 0) to the query as well as set an expression to the component "Load missing Orders" in order to enable it only when RecordID != 0.
In my case it wasn't NULL causing the problem, the ID value which I loaded from database was stored as nvarchar(50), even if it was a integer, I attempted to use it as integer in SSIS and it kept giving me the same error message, this worked for me:
SELECT CAST(id as INT) FROM dbo.Table
Related
I am aggregating errors in order to track total number of errors being logged. I am currently trying to create a query that finds/groups several records that have similar value in order to record them as one error and not several individual ones. The only difference is an id or 2, which is why they are being grouped. The database gets injected with errors from our system via app insights and analytic stream. I have a table that holds the "template" error that will be used to find and group those specific errors. Not all errors being recorded need a template because they are being grouped appropriately because they are exactly the same. When trying to use like to find the errors, it seems to be having a hard time with the dashes. I am having a hard time finding info to help me with this issue. I've tried to use the replace for removing the dashes, but that doesn't work because the errors are too long.
Sample template:
Auto Resubmit for % failed with ' Object reference not set to an instance of an object. '
Sample error:
Auto Resubmit for 004e9e2d-3704-4cfd-a90d-42520203df79 - 18723191 failed with ' Object reference not set to an instance of an object. '
Auto Resubmit for 0130e64e-64e6-4a23-88a4-51fba823705b - 18734821 failed with ' Object reference not set to an instance of an object. '
Auto Resubmit for 11809bf5672f4e98987119dbd06e5d78 - 17359076 failed with ' Object reference not set to an instance of an object. '
Sample Query:
select top 1000 * from errorTable where error like 'Auto Resubmit for % failed with '' Object reference not set to an instance of an object.'
If the only thing that's differing is the parameter values, you could normalize the string before storing/checking it. There are lots of utilities around that can do that. One example is the ExtractSQLTemplate function in our free SDU Tools. You don't need to use the toolkit. Just grab the code for that function as an example. You can see it here: https://youtu.be/yX5q00m_uCA
The other option is to use a full text index on the string instead. You can use functions like FREETEXTTABLE and/or FREETEXT to find similarities between the strings.
One of the main reason that you are not able to use Template Table is that you have basically fail the very purpose of Template by storing error in this manner.
In other word, your table are not Normalize nor any relation is define between
2 tables.
Auto Resubmit for 004e9e2d-3704-4cfd-a90d-42520203df79 - 18723191 failed with
' Object reference not set to an instance of an object. '
Your error table design should be like,
TemplateID -- id column of Template table
Module -- Name of module from where error originated
SubmoduleName-- Method name
ErrorMessage-- Original error message like "Object reference not set to an instance of an object".
CustomError -- Auto Resubmit for 004e9e2d-3704-4cfd-a90d-42520203df79 - 18723191 failed with
This way you can easily join Template Table with Error Table using column ErrorMessage.
ErrorMessage : It should always contain original message thrown by `exception`.
You can customize your Error Table as per your requirement.It will take only little effort.
It would be very nice if you can store TemplateID (FK) in Error table.
All problem will vanish at once.
In short keep table in such a manner that it is easy to join them without string manipulation.
It is not clear, in what pattern % will be replace.
create table #Template(Templateid int identity(1,1),Template varchar(500))
insert into #Template values ('Auto Resubmit for % failed with ''Object reference not set to an instance of an object.''')
--select * from #Template
create table #ErrorLog(Errorid int identity(1,1),ErrorMessage varchar(500))
insert into #ErrorLog values
('Auto Resubmit for 004e9e2d-3704-4cfd-a90d-42520203df79 - 18723191 failed with ''Object reference not set to an instance of an object.''')
,('Auto Resubmit for 0130e64e-64e6-4a23-88a4-51fba823705b - 18734821 failed with ''Object reference not set to an instance of an object.''')
,('Auto Resubmit for 11809bf5672f4e98987119dbd06e5d78 - 17359076 failed with ''Object reference not set to an instance of an object.''' )
,('Auto Resubmit for itemid2 - 18137385 failed with '' Access Denied: userId=''12602174''' )
Use any split string function to split Template table by '%'
and store it in #temp table
create table #temp(Errorid int identity(1,1),ErrMsg varchar(500),rownum int)
insert into #temp
select value,row_number()over(order by (Select null))rn from #Template t
cross apply(select replace(value,'failed with','')value from string_split(t.Template,'%'))ca
select t1.* from
(
select el.ErrorMessage,c.ErrMsg,c.rownum from #ErrorLog EL
inner join #temp c on el.ErrorMessage like '%' +ErrMsg+'%' and c.rownum=1
)t1
inner join #temp c1 on t1.ErrorMessage like '%' +ltrim(c1.ErrMsg)+'%' and c1.rownum=2
select * from #temp
drop table #Template,#ErrorLog,#temp
In fact,for writing near perfect query 1 need to analyze data and write as many Test cases for it.
There is this script in the SQL Server that needs to be converted in PostgreSQL.
Here is the script:
UPDATE Categories_convert SET Categories_convert.ParentID =
Categories_convert_1.CategoryID
FROM Categories_convert LEFT OUTER JOIN Categories_convert AS
Categories_convert_1 ON Categories_convert.MainGroupID_FK =
Categories_convert_1.MainGroupID
WHERE (((Categories_convert.Level)=2));
Then I tried to convert it to postgres. Here is the script:
UPDATE x_tmp_categories_convert SET orig.parentid =
cat2.categoryid
FROM x_tmp_categories_convert LEFT OUTER JOIN x_tmp_categories_convert AS
cat2 ON x_tmp_categories_convert.maingroupid_fk =
x_tmp_categories_convert.maingroupid
WHERE (((cat.level)=2));
Note that I have already created the table Categories_convert of SQLServer to Postgresql and renamed it to x_tmp_categories_convert .
All the fields in postgresql is in lowercase.
Now the problem is when i execute the converted script to postgresql, an error will occur:
ERROR: table name "x_tmp_categories_convert" specified more than once
SQL state: 42712
What I do wrong in the conversion?
UPDATE:
I have tried #a_horse_with_no_name 's answer but it didn't update the records at all. The parentid field is still empty. It is supposed to map all the parentid of that categor based on its maingoupid_fk.
Below is a snapshot of the records after executing the suggested script.
I have opted out the name for disclosure reasons.
Records snapshot link
UPDATE v2:
I am using php to migrate the data so pardon me for the variables used.
Here are the 2 insert statements used before the questioned update script is executed:
INSERT INTO x_tmp_categories_convert(maingroupid,name,vendorid,level,parentid)
VALUES ($id,'$mainGroup',$vendorID,1,Null);
INSERT INTO x_tmp_categories_convert(subgroupid,maingroupid_fk,name,vendorid,level)
VALUES ($id,'$mainGroupId','$subGroup',$vendorID,2);
Also, this is the table definition of the x_tmp_categories_convert table:
CREATE TABLE x_tmp_categories_convert
(
categoryid serial NOT NULL,
parentid double precision,
name character varying(255),
level double precision,
vendorid double precision,
maingroupid integer,
subgroupid integer,
maingroupid_fk integer,
pageid integer,
subgroupid_fk integer,
CONSTRAINT code_pk PRIMARY KEY (categoryid)
)
WITH (
OIDS=FALSE
);
UPDATE v3:
Already SOLVED by a_horse_with_no_name. Thank you
I have edited #a_horse_with_no_name 's answer and to make it work. I guess it is just a copy paste error.
Here is the final script:
UPDATE x_tmp_categories_convert
SET parentid = cat.categoryid
FROM x_tmp_categories_convert AS cat
WHERE x_tmp_categories_convert.maingroupid_fk = cat.maingroupid
AND x_tmp_categories_convert.level = 2;
For #a_horse_with_no_name, I am very grateful to you as I cannot come up with the solution without your help. +1 for you man. Thank you
In a SQL Server 2000 stored procedure, I'm trying to perform an update statement to a table variable. It is currently giving me the error: "The column prefix 'WST' does not match with a table name or alias name used in the query."
UPDATE WST
SET
WST.QtySold = (SELECT SUM(II.QtyShipped)
FROM #InvoiceItems II
WHERE II.InvoiceDate >= WST.StartDate
AND II.InvoiceDate <= WST.EndDate),
WST.TotalSales = (SELECT SUM(ISNULL(II.QtyShipped, 0) * ISNULL(II.UnitPrice, 0))
FROM #InvoiceItems II
WHERE II.InvoiceDate >= WST.StartDate
AND II.InvoiceDate <= WST.EndDate),
WST.TotalCost = (SELECT SUM(ISNULL(II.QtyShipped, 0) * ISNULL(II.UnitCost, 0))
FROM #InvoiceItems II
WHERE II.InvoiceDate >= WST.StartDate
AND II.InvoiceDate <= WST.EndDate)
FROM #WeeklySalesTrend WST
WHERE WST.WeekNo = 1
This error only appeared after I created the temp table #InvoiceItems and replaced the Inner Join of two tables with the temp table. Why would changing the two-table inner join out for a temp table cause this error and how do I fix/get around it?
I don't have a SQL 2000 box to test this on but I think your query syntax is correct. I suspect there may be invisible control characters that's messing up the parsing. This used to happen from time to time but I haven't seen the issue for a while. Try typing the query into a new query window from scratch.
The answer boils down to this, SQL Query Analyzer gave me the wrong location for the error. The error was actually because of an Insert statement 17 lines higher in the code where I was trying to use WST when I never aliased it up there. I don't know if this was a glitch/bug on Query Analyzer's side or if the stored procedure optimized to more lines and there its location was off. Either way, the problem has been fixed. Thanks for the suggestions and quick responses though and sorry for wasting your time.
I m trying to write if statement to give error message if user try to add existing ID number.When i try to enter existing id i get error .untill here it s ok but when i type another id no and fill the fields(name,adress etc) it doesnt go to database.
METHOD add_employee.
DATA: IT_EMP TYPE TABLE OF ZEMPLOYEE_20.
DATA:WA_EMP TYPE ZEMPLOYEE_20.
Data: l_count type i value '2'.
SELECT * FROM ZEMPLOYEE_20 INTO TABLE IT_EMP.
LOOP AT IT_EMP INTO WA_EMP.
IF wa_emp-EMPLOYEE_ID eq pa_id.
l_count = l_count * '0'.
else.
l_count = l_count * '1'.
endif.
endloop.
If l_count eq '2'.
WA_EMP-EMPLOYEE_ID = C_ID.
WA_EMP-EMPLOYEE_NAME = C_NAME.
WA_EMP-EMPLOYEE_ADDRESS = C_ADD.
WA_EMP-EMPLOYEE_SALARY = C_SAL.
WA_EMP-EMPLOYEE_TYPE = C_TYPE.
APPEND wa_emp TO it_emp.
INSERT ZEMPLOYEE_20 FROM TABLE it_emp.
CALL FUNCTION 'POPUP_TO_DISPLAY_TEXT'
EXPORTING
TITEL = 'INFO'
TEXTLINE1 = 'Record Added Successfully.'.
elseif l_count eq '0'.
CALL FUNCTION 'POPUP_TO_DISPLAY_TEXT'
EXPORTING
TITEL = 'INFO'
TEXTLINE1 = 'Selected ID already in database.Please type another ID no.'.
ENDIF.
ENDMETHOD.
I'm not sure I'm getting your explanation. Why are you trying to re-insert all the existing entries back into the table? You're just trying to insert C_ID etc if it doesn't exist yet? Why do you need all the existing entries for that?
If so, throw out that select and the loop completely, you don't need it. You have a few options...
Just read the table with your single entry
SELECT SINGLE * FROM ztable INTO wa WITH KEY ID = C_ID etc.
IF SY-SUBRC = 0.
"this entry exists. popup!
ENDIF.
Use a modify statement
This will overwrite duplicate entries with new data (so non key fields may change this way), it wont fail. No need for a popup.
MODIFY ztable FROM wa.
Catch the SQL exceptions instead of making it dump
If the update fails because of an exception, you can always catch it and deal with exceptional situations.
TRY .
INSERT ztable FROM wa.
CATCH sapsql_array_insert_duprec.
"do your popup, the update failed because of duplicate records
ENDTRY.
I think there's a bug when appending in internal table 'IT_EMP' and inserting in 'ZEMPLOYEE_20' table.
Suppose you append the first time and then you insert. But when you append the second time you will have 2 records in 'IT_EMP' that are going to be inserted in 'ZEMPLOYEE_20'. That is because you don't refresh or clear the internal table and there you will have a runtime error.
According to SAP documentation on 'Inserting Lines into Tables ':
Inserting Several Lines
To insert several lines into a database table, use the following:
INSERT FROM TABLE [ACCEPTING DUPLICATE KEYS] . This
writes all lines of the internal table to the database table in
one single operation. The same rules apply to the line type of
as to the work area described above. If the system is able to
insert all of the lines from the internal table, SY-SUBRC is set to 0.
If one or more lines cannot be inserted because the database already
contains a line with the same primary key, a runtime error occurs.
Maybe the right direction here is trying to insert the work area directly but before you must check if record already exists using the primary key.
Check the SAP documentation on this issue clicking the link before.
On the other hand, once l_count is zero because of l_count = l_count * '0'. that value will never change to any other number making that you won't append or insert again.
why are you retrieving all entries from zemployee_20 ?
You can directly check wether the 'id' exists already or not by using select single. If exists, then show message, if not, add.
It is recommended to retrieve only one field when its needed and not the entire table with asterisc *
SELECT single employee_id FROM ZEMPLOYEE_20 where employee_id = p_id INTO v_id. ( or field in structure )
if sy-subrc = 0. "exists
"show message
else. "not existing id
"populate structure and then add record to Z table
endif.
Furthermore, l_count is not only unnecessary but also bad implemented.
You can directly use the insert query,if the sy-subrc is unsuccessful raise the error message.
WA_EMP-EMPLOYEE_ID = C_ID.
WA_EMP-EMPLOYEE_NAME = C_NAME.
WA_EMP-EMPLOYEE_ADDRESS = C_ADD.
WA_EMP-EMPLOYEE_SALARY = C_SAL.
WA_EMP-EMPLOYEE_TYPE = C_TYPE.
INSERT ZEMPLOYEE_20 FROM WA_EMP.
If sy-subrc <> 0.
Raise the Exception.
Endif.
On the query below I keep getting this error:
Cannot read the next data row for the dataset DataSetProject. (rsErrorReadingNextDataRow)
It appears to be the where clause, if I take it out it seems to work. So I added a cast to the where clause with no luck. Is there something special I need to do in the where clause to get this to work? Just an FYI this is in a report that is pulling an id from the url.
SELECT new_projects.new_projectsId AS ProjectId
, new_projects.new_name AS ProjectName
, new_projects.new_Description AS ProjectDescription
FROM
new_projects
LEFT OUTER JOIN new_projectsteps
ON new_projects.new_projectsId = new_projectsteps.new_ProjectSteps2Id
LEFT OUTER JOIN Task
ON new_projectsteps.new_projectstepsId = Task.RegardingObjectId
WHERE
(new_projects.new_projectsId = cast(#id AS UNIQUEIDENTIFIER))
Thanks!
EDIT:
The id in SQL is a Unique Identifier, the value of #id is being pulled from the querystring(url). So it would look like: &id='BC02ABC0-A6A9-E111-BCAD-32B731EEDD84'
Sorry for the missing info.
I suspect the single quotes are coming through. So either don't have them there by stripping them out before being passed to your parameter or use:
WHERE new_projects.new_projectsId = CONVERT(UNIQUEIDENTIFIER, REPLACE(#id, '''', ''));
If you try a direct comparison when the GUID contains other characters, you should get:
Msg 8169, Level 16, State 2, Line 1 Conversion failed when
converting from a character string to uniqueidentifier.
If this is not what's happening, then don't say "the id in SQL is a Unique Identifier" - show ALL of the code so we can try to reproduce the problem.